* CORDA-4034 Reduce forkEvery to 15 to attempt to mitigate memory leak.
* ENT-5679 Disable test which triggers OOM
* Run tests on two Jenkins agents
* Fixed processing JUnit test results by Allure
* Add timeouts to VaultObserverExceptionTest
* Revert "CORDA-3845: Update BC, log4j, slf4j (#6699)" to eliminate introduced memory leaks
Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
* CORDA-3845: Update BC to 1.64
* CORDA-3845: Upgraded log4j to 2.12.1
* We can remove the use of Manifests from the logging package so that when _it_ logs it doesn't error on the fact the stream was already closed by the default Java logger.
* Remove the logging package as a plugin
* latest BC version
* Remove old test
* Fix some rebased changes to log file handling
* Update slf4j too
Co-authored-by: Ryan Fowler <fowlerrr@users.noreply.github.com>
Co-authored-by: Adel El-Beik <adel.el-beik@r3.com>
* TM-88 only write the test to the file if it passes otherwise retry
* TM-88 1 failing test for testing purposes
* TM-88 fail an integration test instead of unit test
* TM-88 include failed test for retry
* TM-88 new logic for retrying failed tests
* TM-88 refactored to use retry class when dealing with test failures + copy results even after pod failure
* TM-88 remaining pods as set to prevent duplicates, limit retries to prevent crashed build and a bit more logging
* TM-88 atomic int instead of int array
* TM-88 moving atomic int outside retry loops
* TM-88 removing second retry as it is not needed
* TM-88 small fix to final copy
* TM-88 while loop to prevent destruction of results on test retry
* TM-88 removing shutdown hook
* TM-88 reverting to previous implementation
* TM-88 copying xml files before retry
* TM-88 removing fail
* Quick and dirty change to stop "Unable to start notaries." error message (#5686)
"Unable to start notaries. A required port might be bound already" is
returned whenever a startup error occurs while starting the notary nodes
in driver tests. This hides the real error.
This change prints the actual error to std_err and read from file
at a later point. This means the real error is not lost and will be
shown in failed builds.
* Suppress detekt warnings
* CORDA-3350: Increase size of constraints column (#5639)
* Detekt
* Update api file with new threshold
* Add check in transaction builder
* Revert "Add check in transaction builder"
This reverts commit ca3128f44c.
* Add check for max number of keys
* Update api file
* Address Tudor's comments
* Remove check for pre-5 and add test for EC keys
* fix typo and rename liquibase script
* updated docs with measurement numbers for composite keys
* Make detekt happy again
Reapply change that was lost during merge - Adjust resolution of byteman jar to use java 11 compatible mechanism. - a1077092
Manual cherry pick of these changes (a1077092 + 3fafbe55)
Byteman is absent for most of the integration tests and long stacktrace is seen in the log
presently that un-necessarily attracts attention and consumes logging space.
(cherry picked from commit 2b6e59e7bd)
* CORDA-3194 Replay start flow events when responding flow fails initial checkpoint commit
Logic has already been added to recover from initial checkpoint commit
failures on the initiating flow side but this did not suffice for
the same failure occurring on the responding flow's side.
The same idea has been added to resolve the responding flow's issue.
`ExternalMessageEvent` now has a `flowId` that is maintained on the
event. Messages can then be replayed to start/restart the flow, while
the event provides the flow id to each flow start.
Each `ExternalMessageEvent` implementation generates a random `flowId`
when constructed.
Events are stored in Artemis. This allows the solution to recover across
node restarts as the events will be pulled from artemis again when
restarting.
In the future `flowId`s will probably moved off of the events and
generated purely on the responding flow's node.
* CORDA-3194 Add test to verify that errors removing a responding flow are recoverable
* Added a new way for environment variables to be loaded, which allows for underscore based separation.
* Moved test to its own kotlin file.
* Added case insensitivity support.
* The corda. prefix is now case insensitive too.
* Removed unused variable.
* Added env variables support for driverDSL. Shadowing corda. properties raises an exception.
* Driver api stability fix.
* Changed type of cordapps param to reflect the real one, rather than what IntelliJ auto completed.
* Some detekt issue fixes. Spread operator removed, baselined api stability constructors and buggy line.
* Fixed misspelled variable.
* Reverted unintentional changes.
* Added suppress instead of changing baseline.
* Reworked logic to handle previously defined CORDA_ starting properties and handle accordingly. Fixed a bug where wrong class was used for reflection walking.
* Fix for detekt issues.
* Changed message to a more understandable one.
* Changelog + doc note, console error grammar.
* Changes according to PR review.
* CORDA-3232: Make backward compatible RPC client changes
Such that it will be able to talk to new and old server versions.
* CORDA-3232: Make backward compatible RPC server changes
Such that it will be able to talk to new and old client versions.
* CORDA-3232: Trick Detekt
* CORDA-3232: Integration test for multi-interface communication.
* CORDA-3232: Add legacy mode test.
* CORDA-3232: Making Detekt happier
* CORDA-3232: Fix Detekt baseline after merge with `4.3` branch
* CORDA-3232: Incrementing Platform version
As discussed with @lockathan
* CORDA-3232: Fix legacy test post platform version increment
* CORDA-3232: Use recursive logic to establish complete population of method names
* Revert "CORDA-3232: Incrementing Platform version"
This reverts commit d75f48aa
* CORDA-3232: Remove logic that conditions on PLATFORM_VERSION
* CORDA-3232: Making Detekt happier
* CORDA-3232: Few more changes after conversation with @mnesbit
* CORDA-3232: Make a strict match to `CordaRPCOps` on client side
Or else will fail:
net.corda.tools.shell.InteractiveShellIntegrationTest.dumpCheckpoints creates zip with json file for suspended flow
Flagging that `InternalCordaRPCOps.dumpCheckpoints` cannot be called.
* CORDA-3232: Address PR comments by @rick-r3
* CORDA-3232: Address further review input from @rick-r3
* Change the way how methods stored in the map;
* Extend test to make sure that `CordaRPCOps` can indeed be mixed with other RPC interfaces.
* Do a proper check for whether the quasar agent is active instead of sniffing JVM args
* Use JavaAgent.isActive instead of SuspendableHelper.isJavaAgentActive for clarity
* Removed IdentityServiceInternal as it is no longer used.
* Removed externalIdForPublicKey API from KMS and added it to IdentityService.
Added a registerKeyToExternalId API on IdentityService.
* Fix remaining compile errors.
* Removed "registerKeyToParty" and in its place added a new registerKey method which takes a PublicKey, Party and optionally a UUID.
Added a cache to the "PersistentIdentityService" to store other node's public keys.
Added the cache and new hibernate entity to all teh places where one needs to add them.
New keys created by teh node now automatically get associated entries in the KEY -> PARTY map and optionally the KEy -> EXT ID map.
Added a test.
* Removed old comments and TODOs.
* Fixed broken test.
Added comments/explanations for what's going on in IdentityService.
Updated kdocs.
* First try at Implementing publicKeysForExternalId.
* Fixed broken test.
* Added migration.
Amended existing persistent identity service migration to handle new migration.
Addressed some review comments.
* Fixed broken test - whoops!
* Implemented mock identity service methods.
* Added back exception when remapping a key to a different party.
* Fixed compile errors.
Fixed broken tests.
* Use set instead of first entry in ourNames.
Allow node operators to blacklist signing keys (using blacklistedAttachmentSigningKeys config option). These blacklisted keys prevent attachments that are received over the network from being trusted. The docs have been updated to detail how to generate the key hashes that the config requires.
A new shell command attachments trustRoots has been added to see what attachments exist on the node along with information about their trust and where it comes from.
run dumpCheckpoints has been replaced by checkpoints dump as InternalCordaRPCOps needed to change to prevent a function that is meant to be internal from being visible on the shell.
* Split integration tests
* add simple example of printing all methods annotated with @Test
* add docker plugin to root project
remove docker plugin from child projects
add Dockerfile for image to use when testing
add task to build testing image to root project
* add comment describing proposed testing workflow
* simple attempt at running tests in docker container
* add my first k8s interaction script
* add fabric8 as dependnency to buildSrc
* before adding classpath
* collect reports from containers and run through testReports
* re-enable kubes backed testing
* for each project
1. add a list tests task
2. use this list tests task to modify the included tests
3. add a parallel version of the test task
* tweak logic for downloading test report XML files
* use output of parallel testing tasks in report tasks to determine build resultCode
* prepare for jenkins test
* prepare for jenkins test
* make docker reg password system property
* add logging to print out docker reg creds
* enable docker build
* fix gradle build file
* gather xml files into root project
* change log level for gradle modification
* stop printing gradle docker push passwd
* tidy up report generation
* fix compilation errors
* split signature constraints test into two
* change Sig constraint tests type hierarchy
* tidy up build.gradle
* try method based test includes
* add unit test for test listing
* fix bug with test slicing
* stop filtering ignored tests to make the numbers match existing runs
* change log level to ensure print out
* move all plugin logic to buildSrc files
* tidy up test modification
add comments to explain what DistributedTesting plugin does
* move new plugins into properly named packages
* tidy up runConfigs
* fix compile errors due to merge with slow-integration-test work
* add system parameter to enable / disable build modification
* add -Dkubenetise to build command
* address review comments
* type safe declaration of parameters in KubesTest
* migrate PersistentIdentityService to use key.toShortString()
update definition of PublicKeyToExternalId mapping to allow fast lookup by externalId/publicKey
* fix misspelled table name
* add test of migration script
* add design document for proposal to move IdentityService to using the correct PK.toStringShort() method for hashing a publickey
* add enterprise testing considerations to design
* address review comments
* fix compilation errors
* modify PublicKeyToOwningIdentityCache to use toStringShort() as it's lookup key
* address syzmon's code review comments
* CORDA-3139: Cater for port already bound scenario during port allocation
Also moved `SharedMemoryIncremental` into a separate file as it getting bigger
and improved readability of logic and added some logging.
* CORDA-3139: Fix the unit test
* CORDA-3139: Improve logging when failing
* CORDA-3139: Improve stability of the test
* [ENT-3801] Store transactions in the database during transaction resolution (#2305)
* ENT-3801: Store downloaded txns as part of the backchain resolution into the db rather than the checkpoint
It's very inefficient to store the downloaded backchain in the checkpoint as more of it downloaded. Instead, if a threshold is reached (which currently defaults at 0) then the backchain is stored in the transactions table as unverified. A new is_verified column has been added to track this. Initially testing on the OS codebase has been very promising but unfortunately this current code is not quite ready. I had to quickly port it to ENT as this is meant to be an ENT-only optimisation.
To that effect, there is a TransactionResolver abstraction with two implementations: an in-memory one which has the old behaviour, and which will be the behaviour for OS, and a db one.
DBTransactionStorage hasn't been fully updated and I had to comment out the optimistic path for now.
Most of these changes will need to be ported to OS to keep the merge conflicts in check, but obviously not DbTransactionsResolver and the "is_verified" changes in DBTransactionStorage. DBTransactionStorage does have other refactoring which will make sense to port though.
* [ENT-3801] Start work on allowing modifications in AppendOnlyPersistentMap
* [ENT-3801] Add transaction resolver tests
* [ENT-3801] Adjust suspendable annotations
* [ENT-3801] Fix the ResolveTransactionFlow tests
* [ENT-3801] Update ResolveTransactionsFlow tests
* [ENT-3801] Add a liquibase migration script for isVerified
* [ENT-3801] Ensure the migration runs in the correct place
* [ENT-3801] Handle resolution of already present transactions
* [ENT-3801] Fix compile error in performance test app
* [ENT-3801] Logging and comment updates, plus a test case
* [ENT-3801] Add a notary change resolution test
* [ENT-3801] Add a contract upgrade transaction test
* [ENT-3801] Change new column to be a character based status
* [ENT-3801] Migration script type change
* [ENT-3801] Address first round of review comments
* [ENT-3801] Update variable names in AppendOnlyPersistentMap
* [ENT-3801] Another variable name clarification
* [ENT-3801] Fix missing name changes
* [ENT-3801] Make the signature list immutable when constructing cache value
* [ENT-3801] Add a locking strategy for unverified transactions
* [ENT-3801] Address tidying up review comments
* [ENT-3801] First attempt at ensuring locks are released after commit
* [ENT-3801] Remove references to old cache name
* [ENT-3801] Update locking logic
* [ENT-3801] Fix potential deadlock with read/write transaction locks
* [ENT-3801] Remove read locks, and ensure minimal extra suspends
* [ENT-3801] Fix build issues in tests
* [ENT-3801] Use the correct clock when calculating sleep durations
* [ENT-3801] Add a pessimism flag for writing verified transactions
* [ENT-3801] Change logging statement to debug
(cherry picked from commit 8ab6a55e17)
* [NOTICK] Fix up imports for some changed files
* [NOTICK] Fix transaction resolution tests
* [NOTICK] Reinstate the DBTransactionsResolver
* [NOTICK] Add the topological sort back to recordTransactions
* [NOTICK] Adjust test case to remove dependency on query ordering
* [NOTICK] Make test code match that in ENT
* Fix test execution flakiness on fast hardware: set node info polling interval to 1 second in DriverDSL Node Startup (was only being set to 1 sec in Node Registration). Follow-up to https://github.com/corda/corda/pull/5240
* Diagnostics: add additional logging to NodeInfoFilersCopier.
* Diagnostics: add additional logging to NodeInfoWatcher
* Downgrade logging level severity to debug.
* Make atomic counter unique across instances.
* Moved core.flows and core.contract tests
* move internal tests
* move node, schemas and serialization tests
* Move transactions tests
* Move test access helper into test package
* Move remaining tests, and create access helpers to test internals.
* Fix test dependencies.
* Delete some cruft that flagged up on the API checker
* Add readme explaing where tests need to go.
* fix typo
* add a shared memory port allocator to allow multiple processes to share a single allocation pool
* remove dangerous reset function on port allocator
* set forkCount = 2 in node integration test
* only allow one build of a cordapp at any given time for Driver tests
* make all portallocation requests use same starting point
* globally set forks to 6
* tweak forking parameters to allow parallel builds
* tweak unit test parallelism
* 2 workers for integrationTest
* some more tweaks for parallel builds
* some more tweaks for parallel builds
* seems that 49K is not the start of ephemeral ports on all kernels
* tweak parallel settings
* try fix RPC shutdown test in parallel env
* add some logging for RPC shutdown test
* added some logging around PortAllocation tests - try figure out where they are getting stuck
* added some logging around PortAllocation tests - try figure out where they are getting stuck
* fix api-scanner tests
* minimize api changes
* revert to complying with existing API
* add the AtomicInteger for api compatibility reasons
* make sizing script executable
* address review comments pt1
* address review comments pt2
* fix compile errors after review comments
* return to using home dir as temp dir seemed to interact badly with gradle
* Initial commit with a test that can detect a logger that isn't initialised with by lazy that gets called during startup.
* Test improvement. Test fix for api stability.
* Added explanatory comment for the test.
* Changes according to PR review. Added empty file check to make test more robust.
