* [ENT-3801] Store transactions in the database during transaction resolution (#2305)
* ENT-3801: Store downloaded txns as part of the backchain resolution into the db rather than the checkpoint
It's very inefficient to store the downloaded backchain in the checkpoint as more of it downloaded. Instead, if a threshold is reached (which currently defaults at 0) then the backchain is stored in the transactions table as unverified. A new is_verified column has been added to track this. Initially testing on the OS codebase has been very promising but unfortunately this current code is not quite ready. I had to quickly port it to ENT as this is meant to be an ENT-only optimisation.
To that effect, there is a TransactionResolver abstraction with two implementations: an in-memory one which has the old behaviour, and which will be the behaviour for OS, and a db one.
DBTransactionStorage hasn't been fully updated and I had to comment out the optimistic path for now.
Most of these changes will need to be ported to OS to keep the merge conflicts in check, but obviously not DbTransactionsResolver and the "is_verified" changes in DBTransactionStorage. DBTransactionStorage does have other refactoring which will make sense to port though.
* [ENT-3801] Start work on allowing modifications in AppendOnlyPersistentMap
* [ENT-3801] Add transaction resolver tests
* [ENT-3801] Adjust suspendable annotations
* [ENT-3801] Fix the ResolveTransactionFlow tests
* [ENT-3801] Update ResolveTransactionsFlow tests
* [ENT-3801] Add a liquibase migration script for isVerified
* [ENT-3801] Ensure the migration runs in the correct place
* [ENT-3801] Handle resolution of already present transactions
* [ENT-3801] Fix compile error in performance test app
* [ENT-3801] Logging and comment updates, plus a test case
* [ENT-3801] Add a notary change resolution test
* [ENT-3801] Add a contract upgrade transaction test
* [ENT-3801] Change new column to be a character based status
* [ENT-3801] Migration script type change
* [ENT-3801] Address first round of review comments
* [ENT-3801] Update variable names in AppendOnlyPersistentMap
* [ENT-3801] Another variable name clarification
* [ENT-3801] Fix missing name changes
* [ENT-3801] Make the signature list immutable when constructing cache value
* [ENT-3801] Add a locking strategy for unverified transactions
* [ENT-3801] Address tidying up review comments
* [ENT-3801] First attempt at ensuring locks are released after commit
* [ENT-3801] Remove references to old cache name
* [ENT-3801] Update locking logic
* [ENT-3801] Fix potential deadlock with read/write transaction locks
* [ENT-3801] Remove read locks, and ensure minimal extra suspends
* [ENT-3801] Fix build issues in tests
* [ENT-3801] Use the correct clock when calculating sleep durations
* [ENT-3801] Add a pessimism flag for writing verified transactions
* [ENT-3801] Change logging statement to debug
(cherry picked from commit 8ab6a55e177e868bcc47e891fae7f91cc99d8174)
* [NOTICK] Fix up imports for some changed files
* [NOTICK] Fix transaction resolution tests
* [NOTICK] Reinstate the DBTransactionsResolver
* [NOTICK] Add the topological sort back to recordTransactions
* [NOTICK] Adjust test case to remove dependency on query ordering
* [NOTICK] Make test code match that in ENT
* CORDA-2617: Add failover listeners to terminate node process
This is a backport of changes done in Corda Enterprise.
It will be triggered in case of:
a) Loss of connectivity to in-built Artemis for Bridge Control;
b) Loss of connectivity to in-built Artemis for P2P connectivity.
Note on merge to CE: Disregard these changes and take whatever CE already has.
* CORDA-2617: Update documentation on stability of Corda Node
* CORDA-2617: Documentation update after discussion with @mnesbit
* parallelize download of nodeInfos
* actually call new list based addNodes method
* address review comments
fix NetworkMapUpdaterTest
* ensure threadpools are shutdown after network-map download is completed
* use NamedThreadFactory instead of re-implementing it.
* fix imports after rebase
* address review comments
* remove extra whitespace
* Initial commit based on experimental kryo hook agent.
* WIP
* Added documentation.
* Additional improvements and documentation following more testing.
* Added field level instrumentation + basic type handlers for String, byteArray, charArray, primitive types.
* Working version (without array type handling)
* Missing build.gradle file.
* Handle display of Arrays and String.
Pruning output to avoid repetition (by loop depth, object count).
* Added configurable StackDepth (for display purposes) and filter out ProgressTracker stacks.
* Further array handling (Object arrays, 2D, 3D), improved display and general code cleanup.
* Various fixes and improvements following demo to RP.
* Clean-up
* Further clean-up
* Set checkpoint id before deserialization.
* Update documentation
* Final clean-up.
* Minor documentation fixes.
* Updates following PR review feedback.
* Add changelog entry.
* Fix test execution flakiness on fast hardware: set node info polling interval to 1 second in DriverDSL Node Startup (was only being set to 1 sec in Node Registration). Follow-up to https://github.com/corda/corda/pull/5240
* Diagnostics: add additional logging to NodeInfoFilersCopier.
* Diagnostics: add additional logging to NodeInfoWatcher
* Downgrade logging level severity to debug.
* Make atomic counter unique across instances.
Change to `SingleThreadedStateMachineManager`.
Instead of throwing an exception when a flow's fiber cannot be found,
just log the message at info level.
As a general purpose API, allPersisted should not be loading the entire contents of the database table into memory. Instead now it returns a Stream for processing of elements.
* CORDA-3021: Introduce `SignOnlyCryptoService` and use it whenever possible
Also modify `CryptoServiceFactory` to show how sign only implementation can be created.
* CORDA-3021: Undo some of my earlier changes
Which after discussion with @dimosr and @fowlerrr proven to be contradictory.
Dumps all the node's checkpoints as JSON into a single zip file in the node's directory. The output contains:
* All the fields for the top-level flow
* The current sub-flow call stack, along with the current progress tracker step for each sub-flow
* The event that suspended the flow, which if it's a send or sendAndReceive will show the payload that was sent
* Low level information on the active sessions with other peers
Party and AnonymousParty have been retrofitted to implement this interface and are currently the only supported types. A new FlowLogic.initiateFlow(Destination) method overload to easily support the addition of new destination types in future versions.
To prevent making `dumpCheckpoints` part of the public API a new
interface, `InternalCordaRPCOps` has been created and the function
has been moved there. `InternalCordaRPCOps` inherits from
`CordaRPCOps`.
`CordaRPCOpsImpl` now implements `InternalCordaRPCOps`.
`RunShellCommand` and `StringToMethodCallParser` required additional
changes due to issues handling inherited functions. This has only been
raised now due to `InternalCordaRPCOps` inheriting from `CordaRPCOps`.
Many classes have had references to `CordaRPCOps` changed to
`InternalCordaRPCOps`.
* ENT-3642: move the crypto service builder method to node-api
* ENT-3642: add arg for different crypto services
* ENT-3642: add arg for cryptoservice config
* ENT-3642: add null check for BCC requirement
* ENT-3642: remove double-bang operator
* ENT-3642: cryptoservice factory method moved to its own class, improve API
* ENT-3642: remove import
Store the flow dump zip in the logs directory.
Only allow one dump to be created at a time. This is protected by
using a `AtomicInteger` lock and checking for the zip's existence.
* Added periodic message logged from a timer which is switched on and off by the node drain events.
* Timer also spawns on start() to ensure that if a node is shutdown in draining mode and started with it, it will continue logging messages.
Dumps all the node's checkpoints as JSON into a single zip file in the node's directory. The output contains:
* All the fields for the top-level flow
* The current sub-flow call stack, along with the current progress tracker step for each sub-flow
* The event that suspended the flow, which if it's a send or sendAndReceive will show the payload that was sent
* Low level information on the active sessions with other peers
* CORDA-2838 Set Artemis memory config.
* CORDA-2838 Cannot have page size larger than max size.
* CORDA-2838 Use real slow consumers. Need to see if the old config settings can work with a global limit to avoid this.
