On node start, load CordaServices before starting the NotaryService,
so that the NotaryService can check that the services it requires are
available when starting.
Resolves#6172.
* CORDA-3762: Integration test exposing the problem reported
* CORDA-3726: Additional logging
* CORDA-3726: Prevent thread leaks
* CORDA-3726: New `journalBufferTimeout` parameter
* CORDA-3726: Override `journalBufferTimeout` parameter
* CORDA-3726: Making Detekt happier
* CORDA-3276: Account for extra thread user in MockNetwork
For real node this does not matter as `shutdown` can safely be called multiple times, which is not true for server thread provided by MockNetwork
* CORDA-3276: Do not make SMM shutdown "executor" as it belongs to AbstractNode
* CORDA-3276: Address input from @rick-r3
* CORDA-3276: Fix test after rebase
* adding blocked functions ro RestrictedEntityManager and creating RestrictedConnection class
* adding flow tests and fixing issues regarding the review
* adding quasar util to gradle
* updating flow tests
* adding space before } at .isThrownBy()
* adding spaces
* [EG-503] Spent state audit tool
Fixes
* Refinements to notary query interfaces. Feature complete.
* EG-503: Introduce optional `notaryService` in `ServiceHubCoreInternal`
* Remove redundant logic following change to use extensions API
Co-authored-by: Viktor Kolomeyko <viktor.kolomeyko@r3.com>
* CORDA-3696: Temporary update to enable JDK11 build and test. Will eventually be switchable.
* CORDA-3696: Filter out the Nashorn warning.
* CORDA-3696: Add JDK11 classifier.
* CORDA-3696: Updated match string to cope with JDK11.
* CORDA-3696: Filtering out SPHINCS256_SHA256 where failing due to JDK11.
* CORDA-3696: Now remove SPHINCS256_SHA256 only if JDK11.
* CORDA-3696: Fix test failure - switch to regex matching.
* CORDA-3696: Hide the illegal access warnings.
* CORDA-3696: Check for Java11 when disabling Java11 warnings.
* CORDA-3696: Fix unneccessary non null check.
* CORDA-3696: Reverting build env to JDK8
* CORDA-3696: Revert hiding of illegal access warnings via Unsafe class.
* CORDA-3696: Remove internal access warnings and new JDK11 version checker.
* CORDA-3696: Updated build file for OS
* CORDA-3696: Removed typo
* CORDA-3696: Fixed space typo.
* CORDA-3696: Open modules to remove the illegal access warnings.
Co-authored-by: Adel El-Beik <adelel-beik@19LDN-MAC108.local>
* CORDA-3691 Delete checkpoint when flow finishes
The checkpoint and its related records in joined tables should be deleted
when a flow finishes.
Keeping these flows around will be completed in the future.
* CORDA-3691 Ignore some flow metadata tests
Ignore tests around recording the finish time of flow metadata records
since we are not currently keeping COMPLETED flows in the database.
Flows that are kept for overnight observation:
- Save their Checkpoint.status as 'HOSPITALIZED' in the database
- Save the error that caused the hospitalization in the database
A new Event was added for this reason. Whenever the hospital determines
a flow for hospitalization, it adds this Event in the flow's fiber queue.
When processed it creates a new DB transaction, stores the checkpoint status along with
the error, and it adds a 'FlowContinuation.ProcessEvents' continuation so that the fiber keeps
processing events (effectively since there are no more events in the fiber's channel, the fiber will suspend).
Flows that error:
- Their checkpoints are kept in the database
- Save their Checkpoint.status as 'FAILED'
- Save the error that caused the error in the database
Upon erroring, the flow's Checkpoint.status gets updated('FAILED') and the checkpoint is stored
in the database instead of getting removed. The flow then propagates the error to counterparties,
sets its future with the error and gets removed from memory.
* ENT-4967: Require no classifier for corda-node-djvm, corda-deserializers-djvm.
* Also remove classifiers from core, serialization and finance-contracts.
* Compile corda-serialization-djvm for Java 8 and remove its classifier.
Added a new field Completed to the in-memory object FlowState.
FlowState.Completed is corresponds to flow_state=Null in the DB.
This change will save disk space.
* Run serialisation tests with both in-process and out-of-process nodes.
* Add custom serialisers and whitelists to Driver's AMQPServerSerializationScheme.
* Run serialisation tests with both in-process and out-of-process nodes.
* Add custom serialisers and whitelists to Driver's AMQPServerSerializationScheme.
* CORDA-3601 Record a flow's finish time
Record a flow's finish time by updating its metadata record. It is set
in `updateCheckpoint` by checking the status of the checkpoint. If it is
`COMPLETED` it will set the `finishInstant` on the metadata object and
update it.
* CORDA-3601 Record flow finish time for all finished statuses
Update the flow finish time for the following statuses:
- COMPLETED
- KILLED
- FAILED
* CORDA-3601 Use platform clock in `DBCheckpointStorage`
* CORDA-3669 Do not execute `ExecuteAsyncOperation` multiple times
When a `FlowExternalOperation` or `FlowExternalAsyncOperation` executes
and completes a flag (`isFlowResumed`) is switched to true.
This flag was used inside of `DoRemainingWorkTransition` to decide
whether to skip over the execution of an event.
Since this flag was being switched to true when the external operation's
future completed, it was possible for _unexpected_ events to be placed
in the fiber's queue that would retrigger the
`FlowIORequest.ExecuteAsyncOperation`, that is held as the checkpoint's
next `FlowIORequest`to process.
By using the existing `StateMachineState.isTransactionTracked` (and
renaming it to `isWaitingForFuture`) we can decide to not process the
`FlowIORequest.ExecuteAsyncOperation` if it has already been called
before. This moves this code path in line with
`FlowIORequest.WaitForLedgerCommit`.
Random `DoRemainingWork` events can now be pushed to the fiber's queue
without causing the `FlowIORequest.ExecuteAsyncOperation` to execute
again.
* CORDA-3596 Record flow metadata
Record flow metadata during the zero'th checkpoint that occurs before
calling the flow's `call` function.
This required adding an RPC call's arguments to the `InvocationContext`
that gets created. These arguments are then accessible within the
statemachine and from the `Checkpoint` class. The arguments are then
extracted when recording a flow's metadata inside of
`DBCheckpointStorage`.
Updated the size of the started by column to 128 since it was not long
enough to hold the fully qualified class of a service that started a
flow.
* CORDA-3596 Remove arguments from in-memory checkpoint
When executing a flows first real suspend (from flow code) the arguments
contained in the `InvocationContext` are removed. This saves holding
these arguments for the whole lifecyle of a flow.
* CORDA-3596 Increase `cordapp_name` column to 128
* CORDA-3596 Join metadata by `flow_id`
Due to changes in where metadata is recorded, there is no need for
having `invocation_id` as the metadata table's primary key. The
`flow_id` is now the primary key of the table and is used to join to the
main checkpoints table.
The `invocation_id` has been removed from the checkpoints table since it
is not needed for the join anymore.
* CORDA-3596 Remove `received_time` from metadata table
* CORDA-3596 Remove unused `StartReason` enum
* CORDA-3596 Simple `DBCheckpointStorageTests` for metadata
* CORDA-3596 Truncate really long flow names
* CheckpointStorage.getAllCheckpoints will not fetch COMPLETED, FAILED and KILLED flows by default
* Rename getAllCheckpoints to getAllRunnableCheckpoints for clarity
* Fix Detekt issue
* Rename getAllRunnableCheckpoints to getRunnableCheckpoints
* Minor kdoc update
* Bring back in CheckpointStorage.getAllCheckpoints to co-exist with getRunnableCheckpoints
* Add progress tracker information to checkpoint
The checkpoint Datebase is updated when the statemachine suspends
with the progress trackers current step name. This is truncated if
it is longer than the Database column.
* Minor rename in statemachine for clarity
* Set/ Reset Checkpoint.status to RUNNABLE after when suspending
* Removing/ Moving comment as it makes no longer sense to be there since, we now always create a new Checkpoint object in SingleThreadedStateMachineManager.createFlowFromCheckpoint through tryDeserializeCheckpoint
* Set -in memory- Checkpoint.status to RUNNABLE when a flow is retrying from Checkpoint
Due to a change in how messaging works, `ActionExecutorImpl
.executeSendInitial` was no longer being called. Changing the byteman
script to throw exception on hits to `ActionExecutorImpl
.executeSendMultiple` allowed the tests to pass.
When a flow is finished do not delete the checkpoint from the DB.
Instead, the FlowStatus is marked as Completed in the DB.
Updated numerous tests which relied on the flow being removed
when finished.
* Update Checkpoint DB to update flow io request
* Modify flow monitor to update Checkpoint DB with waiting flows
This happens periodically.
* Refactored code to avoid looping twice and updated tests
* Fix tests after rebasing
* Fix MR comments (non-functional refactor of tests + FlowMonitor).
* Made visible for testing method private in DBCheckpointStorage
This is not needed anymore.
* Explicity check if ioRequestType has changed in update method
* Fix shadowing warning
* Import non deprecated Assert into test
* Use AssertEquals not assert in test
* Address more comments (minor refactor) of DBCheckpointStorage
* Minor fix use it instead of referencing object explicitly
* Add null check to DBCheckpointStorage
* Revert changes to Flow Monitor.
We will instead store the information in the main thread of the
state machine.
* Remove now uneeded API and make statemachine update ioRequest
* Add Integration Test to check statemachine updates DB on Recieve
* Use simpleName in checkpoint storage instead of class.
Hibernate was previously resetting the class field this is now
set to null (when getting checkpoint form DB) and a new method
for getting back the simple name as a string.
* Update StateMachineState to store simple name.
* Fix after rebase broke stuff + renamed test
* Fix Detekt issue
* Remove uneeded null assertion
