* CORDA-3722 withEntityManager can rollback its session
## Summary
Improve the handling of database transactions when using
`withEntityManager` inside a flow.
Extra changes have been included to improve the safety and
correctness of Corda around handling database transactions.
This focuses on allowing flows to catch errors that occur inside an
entity manager and handle them accordingly.
Errors can be caught in two places:
- Inside `withEntityManager`
- Outside `withEntityManager`
Further changes have been included to ensure that transactions are
rolled back correctly.
## Catching errors inside `withEntityManager`
Errors caught inside `withEntityManager` require the flow to manually
`flush` the current session (the entity manager's individual session).
By manually flushing the session, a `try-catch` block can be placed
around the `flush` call, allowing possible exceptions to be caught.
Once an error is thrown from a call to `flush`, it is no longer possible
to use the same entity manager to trigger any database operations. The
only possible option is to rollback the changes from that session.
The flow can continue executing updates within the same session but they
will never be committed. What happens in this situation should be handled
by the flow. Explicitly restricting the scenario requires a lot of effort
and code. Instead, we should rely on the developer to control complex
workflows.
To continue updating the database after an error like this occurs, a new
`withEntityManager` block should be used (after catching the previous
error).
## Catching errors outside `withEntityManager`
Exceptions can be caught around `withEntityManager` blocks. This allows
errors to be handled in the same way as stated above, except the need to
manually `flush` the session is removed. `withEntityManager` will
automatically `flush` a session if it has not been marked for rollback
due to an earlier error.
A `try-catch` can then be placed around the whole of the
`withEntityManager` block, allowing the error to be caught while not
committing any changes to the underlying database transaction.
## Savepoints / Transactionality
To make `withEntityManager` blocks work like mini database transactions,
save points have been utilised. A new savepoint is created when opening
a `withEntityManager` block (along with a new session). It is then used
as a reference point to rollback to if the session errors and needs to
roll back. The savepoint is then released (independently from
completing successfully or failing).
Using save points means, that either all the statements inside the
entity manager are executed, or none of them are.
## Some implementation details
- A new session is created every time an entity manager is requested,
but this does not replace the flow's main underlying database session.
- `CordaPersistence.transaction` can now determine whether it needs
to execute its extra error handling code. This is needed to allow errors
escape `withEntityManager` blocks while allowing some of our exception
handling around subscribers (in `NodeVaultService`) to continue to work.
On node start, load CordaServices before starting the NotaryService,
so that the NotaryService can check that the services it requires are
available when starting.
Resolves#6172.
* Retired the rst doc source structure under /docs and updated the /docs/README.md
*Rollback of /example-code and /whitepaper dirs back under /docs dir until new code example process is in place
* CORDA-3762: Integration test exposing the problem reported
* CORDA-3726: Additional logging
* CORDA-3726: Prevent thread leaks
* CORDA-3726: New `journalBufferTimeout` parameter
* CORDA-3726: Override `journalBufferTimeout` parameter
* CORDA-3726: Making Detekt happier
* CORDA-3276: Account for extra thread user in MockNetwork
For real node this does not matter as `shutdown` can safely be called multiple times, which is not true for server thread provided by MockNetwork
* CORDA-3276: Do not make SMM shutdown "executor" as it belongs to AbstractNode
* CORDA-3276: Address input from @rick-r3
* CORDA-3276: Fix test after rebase
* CORDA-3701 Fix bugs in some iterator checkpoint serializers
* Added some more tests and tidied up implementation some more.
* Fix imports to be detekt compliant
* Add timeouts to tests
* adding blocked functions ro RestrictedEntityManager and creating RestrictedConnection class
* adding flow tests and fixing issues regarding the review
* adding quasar util to gradle
* updating flow tests
* adding space before } at .isThrownBy()
* adding spaces
* CORDA-3716: Fix SandboxEnumSerializer to handle enums that override toString().
* Remove more uses of Enum.toString() from the Corda serializer.
* Add test coverage for this case to standard enum serializer.
* Increase maxWaitTimeout in IRSDemoTest to 150 seconds.
* CORDA-3716: Fix SandboxEnumSerializer to handle enums that override toString().
* Remove more uses of Enum.toString() from the Corda serializer.
* Add test coverage for this case to standard enum serializer.
* Increase maxWaitTimeout in IRSDemoTest to 150 seconds.
* [EG-503] Spent state audit tool
Fixes
* Refinements to notary query interfaces. Feature complete.
* EG-503: Introduce optional `notaryService` in `ServiceHubCoreInternal`
* Remove redundant logic following change to use extensions API
Co-authored-by: Viktor Kolomeyko <viktor.kolomeyko@r3.com>
* CORDA-3696: Temporary update to enable JDK11 build and test. Will eventually be switchable.
* CORDA-3696: Filter out the Nashorn warning.
* CORDA-3696: Add JDK11 classifier.
* CORDA-3696: Updated match string to cope with JDK11.
* CORDA-3696: Filtering out SPHINCS256_SHA256 where failing due to JDK11.
* CORDA-3696: Now remove SPHINCS256_SHA256 only if JDK11.
* CORDA-3696: Fix test failure - switch to regex matching.
* CORDA-3696: Hide the illegal access warnings.
* CORDA-3696: Check for Java11 when disabling Java11 warnings.
* CORDA-3696: Fix unneccessary non null check.
* CORDA-3696: Reverting build env to JDK8
* CORDA-3696: Revert hiding of illegal access warnings via Unsafe class.
* CORDA-3696: Remove internal access warnings and new JDK11 version checker.
* CORDA-3696: Updated build file for OS
* CORDA-3696: Removed typo
* CORDA-3696: Fixed space typo.
* CORDA-3696: Open modules to remove the illegal access warnings.
Co-authored-by: Adel El-Beik <adelel-beik@19LDN-MAC108.local>
* CORDA-3691 Delete checkpoint when flow finishes
The checkpoint and its related records in joined tables should be deleted
when a flow finishes.
Keeping these flows around will be completed in the future.
* CORDA-3691 Ignore some flow metadata tests
Ignore tests around recording the finish time of flow metadata records
since we are not currently keeping COMPLETED flows in the database.
Flows that are kept for overnight observation:
- Save their Checkpoint.status as 'HOSPITALIZED' in the database
- Save the error that caused the hospitalization in the database
A new Event was added for this reason. Whenever the hospital determines
a flow for hospitalization, it adds this Event in the flow's fiber queue.
When processed it creates a new DB transaction, stores the checkpoint status along with
the error, and it adds a 'FlowContinuation.ProcessEvents' continuation so that the fiber keeps
processing events (effectively since there are no more events in the fiber's channel, the fiber will suspend).
Flows that error:
- Their checkpoints are kept in the database
- Save their Checkpoint.status as 'FAILED'
- Save the error that caused the error in the database
Upon erroring, the flow's Checkpoint.status gets updated('FAILED') and the checkpoint is stored
in the database instead of getting removed. The flow then propagates the error to counterparties,
sets its future with the error and gets removed from memory.
* ENT-4967: Require no classifier for corda-node-djvm, corda-deserializers-djvm.
* Also remove classifiers from core, serialization and finance-contracts.
* Compile corda-serialization-djvm for Java 8 and remove its classifier.
the current code doesn't forward the exception to the logging system, this means that any cause inside the exception is lost as with all of the stacktraces (both the one of the thrown exception and the one belonging to its cause).
The correct way to log an exception is to pass both the message and the exception to the logging system.
* CORDA-3690: Changed algorithm name used in signature from ECDSA to EC. JDK11 checks with in key generation.
* CORDA-3690: Remove the SHA512WITHSPHINCS256 signature scheme from the generate key pair and sign test.
* CORDA-3690: Algorithm in SignatureScheme has changed to EC from ECDSA so change test to match.
Co-authored-by: Adel El-Beik <adelel-beik@19LDN-MAC108.local>
