* ENT-6588 Restrict database operations platform flag
Put the restricting of database operations in `RestrictedConnection` and
`RestrictedEntityManager` behind a platform version flag.
`RESTRICTED_DATABASE_OPERATIONS = 7` was added to signify this.
If the version is less than 7, then the database operations will not be
restricted. A warning is logged to indicate that they are using
potentially dangerous methods.
If the version is 7 or greater, then the database operations are
restricted and throw an error if called.
Co-authored-by: Dan Newton <dan.newton@r3.com>
* CORDA-3755: Switched attachments map to a WeakHashMap (#6214)
* Bump OS release version 4.6
* CORDA-3755: Switched attachments map to a WeakHashMap
* CORDA-3755: Added explicit strong references to map key.
* CORDA-3755: Keeping detekt happy.
* CORDA-3755: Test a gc in verify.
* CORDA-3755: Making detekt happy.
* CORDA-3755: Suppress warnings for weak reference test.
* CORDA-3755: Fixing build failure with attachments.
* CORDA-3755: Rewrite based on Ricks input - now handles attachment already existing in map!
* CORDA-3755: Refactor WeakReference behaviour into AttachmentsHolderImpl and provide alternate version of this class for core-deterministic.
* CORDA-3755: Added more tests for WeakHashMap.
* CORDA-3755: Ignore the tests using System.gc keep for local testing only
* CORDA-3755: Adding comment to explain the ignored tests.
* Make AttachmentsHolderImpl package-private inside core-deterministic, just like it is inside core.
* CORDA-3755: Update assertions following review comments.
* CORDA-3755: Removing import
* CORDA-3755: Removed unused var.
* CORDA-3755: Reverting files that somehow got changed in rebase.
Co-authored-by: nargas-ritu <ritu.gupta@r3.com>
Co-authored-by: Chris Rankin <chris.rankin@r3.com>
* CORDA-3769: Switched attachments class loader cache to use caffeine (#6326)
* CORDA-3769: Switched attachments class loader cache to use caffeine with original implementation used by determinstic core.
* CORDA-3769: Removed default ctor arguments.
* CORDA-3769: Switched mapping function to Function type to avoid synthetic method being generated.
* CORDA-3769: Now using a cache created from NamedCacheFactory for the attachments class loader cache.
* CORDA-3769: Making detekt happy.
* CORDA-3769: The finality tests now check for UntrustedAttachmentsException which will actually happen in reality.
* CORDA-3769: Refactored after review comments.
* CORDA-3769: Removed the AttachmentsClassLoaderSimpleCacheImpl as DJVM does not need it. Also updated due to review comments.
* CORDA-3769: Removed the generic parameters from AttachmentsClassLoader.
* CORDA-3769: Removed unused imports.
* CORDA-3769: Updates from review comments.
* CORDA-3769: Updated following review comments. MigrationServicesForResolution now uses cache factory. Ctor updated for AttachmentsClassLoaderSimpleCacheImpl.
* CORDA-3769: Reduced max class loader cache size
* CORDA-3769: Fixed the attachments class loader cache size to a fixed default
* CORDA-3769: Switched attachments class loader size to be reduced by fixed value.
* CORDA-4125: Parameter has been added to a private ctor.
Co-authored-by: nargas-ritu <ritu.gupta@r3.com>
Co-authored-by: Chris Rankin <chris.rankin@r3.com>
* CORDA-3722 withEntityManager can rollback its session
Improve the handling of database transactions when using
`withEntityManager` inside a flow.
Extra changes have been included to improve the safety and
correctness of Corda around handling database transactions.
This focuses on allowing flows to catch errors that occur inside an
entity manager and handle them accordingly.
Errors can be caught in two places:
- Inside `withEntityManager`
- Outside `withEntityManager`
Further changes have been included to ensure that transactions are
rolled back correctly.
Errors caught inside `withEntityManager` require the flow to manually
`flush` the current session (the entity manager's individual session).
By manually flushing the session, a `try-catch` block can be placed
around the `flush` call, allowing possible exceptions to be caught.
Once an error is thrown from a call to `flush`, it is no longer possible
to use the same entity manager to trigger any database operations. The
only possible option is to rollback the changes from that session.
The flow can continue executing updates within the same session but they
will never be committed. What happens in this situation should be handled
by the flow. Explicitly restricting the scenario requires a lot of effort
and code. Instead, we should rely on the developer to control complex
workflows.
To continue updating the database after an error like this occurs, a new
`withEntityManager` block should be used (after catching the previous
error).
Exceptions can be caught around `withEntityManager` blocks. This allows
errors to be handled in the same way as stated above, except the need to
manually `flush` the session is removed. `withEntityManager` will
automatically `flush` a session if it has not been marked for rollback
due to an earlier error.
A `try-catch` can then be placed around the whole of the
`withEntityManager` block, allowing the error to be caught while not
committing any changes to the underlying database transaction.
To make `withEntityManager` blocks work like mini database transactions,
save points have been utilised. A new savepoint is created when opening
a `withEntityManager` block (along with a new session). It is then used
as a reference point to rollback to if the session errors and needs to
roll back. The savepoint is then released (independently from
completing successfully or failing).
Using save points means, that either all the statements inside the
entity manager are executed, or none of them are.
- A new session is created every time an entity manager is requested,
but this does not replace the flow's main underlying database session.
- `CordaPersistence.transaction` can now determine whether it needs
to execute its extra error handling code. This is needed to allow errors
escape `withEntityManager` blocks while allowing some of our exception
handling around subscribers (in `NodeVaultService`) to continue to work.
* CORDA-3755: Switched attachments map to a WeakHashMap (#6214)
* CORDA-3772: Now specify source and target of 8 when compiling contract classes.
* CORDA-3651: addManifest now uses separate files for reading and writing. (#6026)
* CORDA-3651: addManifest now uses separate files for reading and writing.
* CORDA-3651: The jar scanning loader now closes itsself.
Co-authored-by: Adel El-Beik <adelel-beik@19LDN-MAC108.local>
Co-authored-by: Adel El-Beik <adelel-beik@19LDN-MAC108.local>
* Bump OS release version 4.6
* CORDA-3755: Switched attachments map to a WeakHashMap
* CORDA-3755: Added explicit strong references to map key.
* CORDA-3755: Keeping detekt happy.
* CORDA-3755: Test a gc in verify.
* CORDA-3755: Making detekt happy.
* CORDA-3755: Suppress warnings for weak reference test.
* CORDA-3755: Fixing build failure with attachments.
* CORDA-3755: Rewrite based on Ricks input - now handles attachment already existing in map!
* CORDA-3755: Refactor WeakReference behaviour into AttachmentsHolderImpl and provide alternate version of this class for core-deterministic.
* CORDA-3755: Added more tests for WeakHashMap.
* CORDA-3755: Ignore the tests using System.gc keep for local testing only
* CORDA-3755: Adding comment to explain the ignored tests.
* Make AttachmentsHolderImpl package-private inside core-deterministic, just like it is inside core.
* CORDA-3755: Update assertions following review comments.
* CORDA-3755: Removing import
* CORDA-3755: Removed unused var.
* CORDA-3755: Reverting files that somehow got changed in rebase.
Co-authored-by: nargas-ritu <ritu.gupta@r3.com>
Co-authored-by: Chris Rankin <chris.rankin@r3.com>
* CORDA-3715: When loading cordapps now check that contract classes have class version between 49 and 52
* CORDA-3715: Now check class version when contract verification takes place.
* CORDA-3715: Making detekt happy with number of levels in func
* CORDA-3715: Make use of new ClassGraph release which provides class file major version number.
* CORDA-3715: Changed package name in test jars
* CORDA-3715: Use ClassGraph when loading attachments.
* CORDA-3715: Reverted file to 4.5 version
* CORDA-3715: Updating method to match non deterministic version.
* CORDA-3715: Added in default param.
* CORDA-3715: Adjusted min JDK version to 1.1
* CORDA-3715: Switching check to JDK 1.2
* CORDA-3715: Now version check SerializationWhitelist classes.
* CORDA-3715: Switched default to null for range.
* CORDA-3291 `isKilled` flag and session errors for killed flows
## Summary
Two major improvements have been worked on:
- A new flag named `isKilled` has been added to `FlowLogic` to allow
developers to break out of loops without suspension points.
- Killed flows now send session errors to their counter parties allowing
their flows to also terminate without further coordination.
Achieving these changes required a __fundamental__ change to how flows are
killed as well as how they sleep.
## `isKilled` flag
The addition of `FlowLogic.isKilled` allows flows to check if the
current flow has been killed. They can then throw an exception to lead
to the flow's termination (following the standard error pathway). They
can also perform some extra logic or not throw an exception if they
really wanted to.
No matter what, once the flag is set, the flow will terminate. Due to
timing, a killed flow might successfully process its next suspension
event, but it will then process a killed transition and terminate.
## Send session errors when killing a flow
A flow will now send session errors to all of its counter parties. They
are transferred as `UnexpectedFlowEndException`s. This allows initiated
flows to handle these errors as they see fit, although they should
probably just terminate.
## How flows are killed
### Before
Originally we were relying on Quasar to interrupt a flow's fiber, we
could then handle the resulting `InterruptedException`. The problem with
this solution is that it only worked when a flow was already suspended
or when a flow moved into suspension. Flows stuck in loops did not work.
### After
We now *do not* use Quasar to interrupt a flow's fiber. Instead, we
switch `FlowStateMachine.isKilled` to true and schedule a new event.
Any event that is processed after switching this flag will now cause a
`KilledFlowTransition`. This transition follows similar logic to how
error propagation works. Note, the extra event allows a suspended flow
to be killed without waiting for the event that it was _really_ waiting
for.
This allows a lot of the tidy up code in `StateMachineManager.killFlow`
to be removed as tidy up is executed as part of removing a flow.
Deleting a flow's checkpoint and releasing related soft locks is still
handled manually in case of infinite loops but also triggered as part
of the actions executed in a transition.
This required flow sleeping to be changed as we no longer rely on
quasar.
## How flows now sleep
The reliance on Quasar to make a flow sleep has been removed.
Instead, when a flow sleeps we create a `ScheduledFuture` that is
delayed for the requested sleep duration. When the future executes it
schedules a `WakeUpFromSleep` event that wakes up the flow... Duh.
`FlowSleepScheduler` handles the future logic. It also uses the same
scheduled thread pool that timed flows uses.
A future field was added to `StateMachineState`. This removes the
need for concurrency control around flow sleeps as the code path does
not need to touch any concurrent data structures.
To achieve this:
- `StateMachineState.future` added as a `var`
- When the `ScheduledFuture` is created to wake up the flow the passed
in `StateMachineState` has its `future` value changed
- When resumed `future` and `isWaitingForFuture` are set to `null` and
`false` respectively
- When cancelling a sleeping flow, the `future` is cancelled and nulled
out. `isWaitingForFuture` is not changed since the flow is ending anyway
so really the value of the field is not important.
* CORDA-3722 withEntityManager can rollback its session
## Summary
Improve the handling of database transactions when using
`withEntityManager` inside a flow.
Extra changes have been included to improve the safety and
correctness of Corda around handling database transactions.
This focuses on allowing flows to catch errors that occur inside an
entity manager and handle them accordingly.
Errors can be caught in two places:
- Inside `withEntityManager`
- Outside `withEntityManager`
Further changes have been included to ensure that transactions are
rolled back correctly.
## Catching errors inside `withEntityManager`
Errors caught inside `withEntityManager` require the flow to manually
`flush` the current session (the entity manager's individual session).
By manually flushing the session, a `try-catch` block can be placed
around the `flush` call, allowing possible exceptions to be caught.
Once an error is thrown from a call to `flush`, it is no longer possible
to use the same entity manager to trigger any database operations. The
only possible option is to rollback the changes from that session.
The flow can continue executing updates within the same session but they
will never be committed. What happens in this situation should be handled
by the flow. Explicitly restricting the scenario requires a lot of effort
and code. Instead, we should rely on the developer to control complex
workflows.
To continue updating the database after an error like this occurs, a new
`withEntityManager` block should be used (after catching the previous
error).
## Catching errors outside `withEntityManager`
Exceptions can be caught around `withEntityManager` blocks. This allows
errors to be handled in the same way as stated above, except the need to
manually `flush` the session is removed. `withEntityManager` will
automatically `flush` a session if it has not been marked for rollback
due to an earlier error.
A `try-catch` can then be placed around the whole of the
`withEntityManager` block, allowing the error to be caught while not
committing any changes to the underlying database transaction.
## Savepoints / Transactionality
To make `withEntityManager` blocks work like mini database transactions,
save points have been utilised. A new savepoint is created when opening
a `withEntityManager` block (along with a new session). It is then used
as a reference point to rollback to if the session errors and needs to
roll back. The savepoint is then released (independently from
completing successfully or failing).
Using save points means, that either all the statements inside the
entity manager are executed, or none of them are.
## Some implementation details
- A new session is created every time an entity manager is requested,
but this does not replace the flow's main underlying database session.
- `CordaPersistence.transaction` can now determine whether it needs
to execute its extra error handling code. This is needed to allow errors
escape `withEntityManager` blocks while allowing some of our exception
handling around subscribers (in `NodeVaultService`) to continue to work.
* [EG-503] Spent state audit tool
Fixes
* Refinements to notary query interfaces. Feature complete.
* EG-503: Introduce optional `notaryService` in `ServiceHubCoreInternal`
* Remove redundant logic following change to use extensions API
Co-authored-by: Viktor Kolomeyko <viktor.kolomeyko@r3.com>
* CORDA-3696: Temporary update to enable JDK11 build and test. Will eventually be switchable.
* CORDA-3696: Filter out the Nashorn warning.
* CORDA-3696: Add JDK11 classifier.
* CORDA-3696: Updated match string to cope with JDK11.
* CORDA-3696: Filtering out SPHINCS256_SHA256 where failing due to JDK11.
* CORDA-3696: Now remove SPHINCS256_SHA256 only if JDK11.
* CORDA-3696: Fix test failure - switch to regex matching.
* CORDA-3696: Hide the illegal access warnings.
* CORDA-3696: Check for Java11 when disabling Java11 warnings.
* CORDA-3696: Fix unneccessary non null check.
* CORDA-3696: Reverting build env to JDK8
* CORDA-3696: Revert hiding of illegal access warnings via Unsafe class.
* CORDA-3696: Remove internal access warnings and new JDK11 version checker.
* CORDA-3696: Updated build file for OS
* CORDA-3696: Removed typo
* CORDA-3696: Fixed space typo.
* CORDA-3696: Open modules to remove the illegal access warnings.
Co-authored-by: Adel El-Beik <adelel-beik@19LDN-MAC108.local>
* ENT-4967: Require no classifier for corda-node-djvm, corda-deserializers-djvm.
* Also remove classifiers from core, serialization and finance-contracts.
* Compile corda-serialization-djvm for Java 8 and remove its classifier.
* CORDA-3690: Changed algorithm name used in signature from ECDSA to EC. JDK11 checks with in key generation.
* CORDA-3690: Remove the SHA512WITHSPHINCS256 signature scheme from the generate key pair and sign test.
* CORDA-3690: Algorithm in SignatureScheme has changed to EC from ECDSA so change test to match.
Co-authored-by: Adel El-Beik <adelel-beik@19LDN-MAC108.local>
* Stop capturing 'FlowLogic' references in flowAsyncOperation;
Creating concrete classes removes the implicit reference to FlowLogic (as this) being included in the anonymous object
* Modify test code so that lambdas no longer get implicit references to their enclosing 'FlowLogic'
* Stop capturing 'FlowLogic' references in flowAsyncOperation;
Creating concrete classes removes the implicit reference to FlowLogic (as this) being included in the anonymous object
* Modify test code so that lambdas no longer get implicit references to their enclosing 'FlowLogic'
* CORDA-3668: Prevent AttachmentURLConnection from assigning ALL_PERMISSIONS to classes inside an attachment.
* Strengthen the comment warning about AttachmentURLConnection.getPermission.
* [CORDA-3628] - Implement sendAll API
* detekt
* Some minor refactorings and docs
* Eliminate warnings
* Address Rick's comments
* Switch sendAll to use a set
