* ENT-6588 Restrict database operations platform flag
Put the restricting of database operations in `RestrictedConnection` and
`RestrictedEntityManager` behind a platform version flag.
`RESTRICTED_DATABASE_OPERATIONS = 7` was added to signify this.
If the version is less than 7, then the database operations will not be
restricted. A warning is logged to indicate that they are using
potentially dangerous methods.
If the version is 7 or greater, then the database operations are
restricted and throw an error if called.
Co-authored-by: Dan Newton <dan.newton@r3.com>
Co-authored-by: Dan Newton <dan.newton@r3.com>
* CORDA-3755: Switched attachments map to a WeakHashMap (#6214)
* Bump OS release version 4.6
* CORDA-3755: Switched attachments map to a WeakHashMap
* CORDA-3755: Added explicit strong references to map key.
* CORDA-3755: Keeping detekt happy.
* CORDA-3755: Test a gc in verify.
* CORDA-3755: Making detekt happy.
* CORDA-3755: Suppress warnings for weak reference test.
* CORDA-3755: Fixing build failure with attachments.
* CORDA-3755: Rewrite based on Ricks input - now handles attachment already existing in map!
* CORDA-3755: Refactor WeakReference behaviour into AttachmentsHolderImpl and provide alternate version of this class for core-deterministic.
* CORDA-3755: Added more tests for WeakHashMap.
* CORDA-3755: Ignore the tests using System.gc keep for local testing only
* CORDA-3755: Adding comment to explain the ignored tests.
* Make AttachmentsHolderImpl package-private inside core-deterministic, just like it is inside core.
* CORDA-3755: Update assertions following review comments.
* CORDA-3755: Removing import
* CORDA-3755: Removed unused var.
* CORDA-3755: Reverting files that somehow got changed in rebase.
Co-authored-by: nargas-ritu <ritu.gupta@r3.com>
Co-authored-by: Chris Rankin <chris.rankin@r3.com>
* CORDA-3769: Switched attachments class loader cache to use caffeine (#6326)
* CORDA-3769: Switched attachments class loader cache to use caffeine with original implementation used by determinstic core.
* CORDA-3769: Removed default ctor arguments.
* CORDA-3769: Switched mapping function to Function type to avoid synthetic method being generated.
* CORDA-3769: Now using a cache created from NamedCacheFactory for the attachments class loader cache.
* CORDA-3769: Making detekt happy.
* CORDA-3769: The finality tests now check for UntrustedAttachmentsException which will actually happen in reality.
* CORDA-3769: Refactored after review comments.
* CORDA-3769: Removed the AttachmentsClassLoaderSimpleCacheImpl as DJVM does not need it. Also updated due to review comments.
* CORDA-3769: Removed the generic parameters from AttachmentsClassLoader.
* CORDA-3769: Removed unused imports.
* CORDA-3769: Updates from review comments.
* CORDA-3769: Updated following review comments. MigrationServicesForResolution now uses cache factory. Ctor updated for AttachmentsClassLoaderSimpleCacheImpl.
* CORDA-3769: Reduced max class loader cache size
* CORDA-3769: Fixed the attachments class loader cache size to a fixed default
* CORDA-3769: Switched attachments class loader size to be reduced by fixed value.
* CORDA-4125: Parameter has been added to a private ctor.
Co-authored-by: nargas-ritu <ritu.gupta@r3.com>
Co-authored-by: Chris Rankin <chris.rankin@r3.com>
* Upgrade of Bouncy Castle to resolve security issue
* Changed default signature scheme
* Reverted default change in draft
* Key conversion for BC with ed25519
* Initializing BC provider to use X509EdDSAEngine for Signature.Ed25519
* removed unsude imports
Co-authored-by: Nick Dunstone <nick.a.dunstone@gmail.com>
Do not let a user reattach to a flow started by another user.
Reattaching to a flow using startFlowWithClientId for a flow not
started by the current user throws a PermissionException
Reattaching to a flow using reattachFlowWithClientId for a flow not
started by the current user returns null.
finishedFlowsWithClientIds does not return flows started by other
users.
Normal rpc permissions around startFlowWithClientId and
startFlowDynamicWithClientId has also been added.
To allow admins to remove client ids as well as be able to see all the
client ids on the node, admin versions have been added that bypass the
user restrictions. These can be permitted via rpc to only provide
their usage to admins.
* INFRA-424 linux1 jenkinsfile
* INFRA-424 full run
* INFRA-424 bigger heap size
* Upgraded DJVM to handle BC - latest version of BC is a multirelease JAR.
When reading JKS keystore if a BC EdDSAPrivateKey is returned then swap for a net.i2p EdDSA private key.
* Temporary downgrade of BC
* Removed the BC EdDSA conversion
* INFRA-424 bigger heap size
* Upgrading Quasar to handle openJ9 different fields.
* INFRA-424: Handle lack of SUPPRESSED_SENTINEL in openj9.
* INFRA-424: If BCEdDSA public or private key is generated convert to net.i2p EdDSA form.
* INFRA-424 bigger heap size
* INFRA-424: On openJ9 only getting upto milli resolution.
* INFRA-424: Handle keystore returning a BCEdDSAPrivateKey.
* INFRA-424: Disable test on JDK11, as it requires the custom cordapp to generate JDK8 contract code, which we now check for.
* INFRA-424: Truncated time test to resolution of millis for openj9.
* INFRA-424 disabling log intensive tests until a fix is developed
* INFRA-424 one more test disabled
* INFRA-424: Disabled a couple of tests failing on openj9.
* INFRA-424: Disabling failing openj9 tests.
* INFRA-424: Disabling test failing on openj9.
* INFRA-424: Ignoring another flaky sleep test on openj9.
* INFRA-424 run integrationTests
* INFRA-424 set timeout to 4 hours
* INFRA-424: Cope with exception message from openj9.
* INFRA-424: Handle the coloured text characters openj9 adds.
* INFRA-424: Disabling test as it is generating JDK11 contract code under JDK11. Currently on JDK8 contract code allowed.
* INFRA-424: Commenting test out for openj9. Output of the processs thats read by the test is sometimes garbled.
* INFRA-424 switching to smoke tests
* INFRA-424 switching to slow integration tests
* INFRA-424 full run
* INFRA-424 moving jenkinsfile
* INFRA-424 removing references
* INFRA-424: Created common IS_OPENJ9 func for ignoring tests.
Co-authored-by: Schife <razvan.codreanu@r3.com>
Add `CordaRPCOps.reattachFlowWithClientId` to allow clients to reattach
to an existing flow by only providing a client id. This behaviour is the
same as calling `startFlowDynamicWithClientId` for an existing
`clientId`. Where it differs is `reattachFlowWithClientId` will return
`null` if there is no flow running or finished on the node with the same
client id.
Return `null` if record deleted from race-condition
Making statemachine not remove COMPLETED flows' checkpoints from the database
if they are started with a clientId, instead they are getting persisted and retained within
the database along with their result (`DBFlowResult`).
On flow start with a client id (`startFlowDynamicWithClientId`), if the client id maps to
a flow that was previously started with the same client id and the flow is now finished,
then fetch the `DBFlowResult` from the database to construct a
`FlowStateMachineHandle` done future and return it back to the client.
Object stored as results must abide by the storage serializer rules. If they fail to do so
the result will not be stored and an exception is thrown to the client to indicate this.
* CORDA-3717: Apply custom serializers to checkpoints
* Remove try/catch to fix TooGenericExceptionCaught detekt rule
* Rename exception
* Extract method
* Put calls to the userSerializer on their own lines to improve readability
* Remove unused constructors from exception
* Remove unused proxyType field
* Give field a descriptive name
* Explain why we are looking for two type parameters when we only use one
* Tidy up the fetching of types
* Use 0 seconds when forcing a flow checkpoint inside test
* Add test to check references are restored correctly
* Add CheckpointCustomSerializer interface
* Wire up the new CheckpointCustomSerializer interface
* Use kryo default for abstract classes
* Remove unused imports
* Remove need for external library in tests
* Make file match original to remove from diff
* Remove maySkipCheckpoint from calls to sleep
* Add newline to end of file
* Test custom serializers mapped to interfaces
* Test serializer configured with abstract class
* Move test into its own package
* Rename test
* Move flows and serializers into their own source file
* Move broken map into its own source file
* Delete comment now source file is simpler
* Rename class to have a shorter name
* Add tests that run the checkpoint serializer directly
* Check serialization of final classes
* Register as default unless the target class is final
* Test PublicKey serializer has not been overridden
* Add a broken serializer for EdDSAPublicKey to make test more robust
* Split serializer registration into default and non-default registrations. Run registrations at the right time to preserve Cordas own custom serializers.
* Check for duplicate custom checkpoint serializers
* Add doc comments
* Add doc comments to CustomSerializerCheckpointAdaptor
* Add test to check duplicate serializers are logged
* Do not log the duplicate serializer warning when the duplicate is the same class
* Update doc comment for CheckpointCustomSerializer
* Sort serializers by classname so we are not registering in an unknown or random order
* Add test to serialize a class that references itself
* Store custom serializer type in the Kryo stream so we can spot when a different serializer is being used to deserialize
* Testing has shown that registering custom serializers as default is more robust when adding new cordapps
* Remove new line character
* Remove unused imports
* Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt
* Remove comment
* Update comment on exception
* Make CustomSerializerCheckpointAdaptor internal
* Revert "Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt"
This reverts commit b835de79bd.
* Restore "Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt""
This reverts commit 718873a4e9.
* Pass the class loader instead of the context
* Do less work in test setup
* Make the serialization context unique for CustomCheckpointSerializerTest so we get a new Kryo pool for the test
* Rebuild the Kryo pool for the given context when we change custom serializers
* Rebuild all Kryo pools on serializer change to keep serializer list consistent
* Move the custom serializer list into CheckpointSerializationContext to reduce scope from global to a serialization context
* Remove unused imports
* Make the new checkpointCustomSerializers property default to the empty list
* Delegate implementation using kotlin language feature
Introducing a new flow start method (`startFlowDynamicWithClientId`) passing in a `clientId`.
Once `startFlowDynamicWithClientId` gets called, the `clientId` gets injected into `InvocationContext` and also pushed to the logging context.
If a new flow starts with this method, then a < `clientId` to flow > pair is kept on node side, even after the flow's lifetime. If `startFlowDynamicWithClientId` is called again with the same `clientId` then the node identifies that this `clientId` refers to an existing < `clientId` to flow > pair and returns back to the rpc client a `FlowStateMachineHandle` future, created out of that pair.
`FlowStateMachineHandle` interface was introduced as a thinner `FlowStateMachine`. All `FlowStateMachine` properties used by call sites are moved into this new interface along with `clientId` and then `FlowStateMachine` extends it.
Introducing an acknowledgement method (`removeClientId`). Calling this method removes the < `clientId` to flow > pair on the node side and frees resources.
* CORDA-3769: Switched attachments class loader cache to use caffeine with original implementation used by determinstic core.
* CORDA-3769: Removed default ctor arguments.
* CORDA-3769: Switched mapping function to Function type to avoid synthetic method being generated.
* CORDA-3769: Now using a cache created from NamedCacheFactory for the attachments class loader cache.
* CORDA-3769: Making detekt happy.
* CORDA-3769: The finality tests now check for UntrustedAttachmentsException which will actually happen in reality.
* CORDA-3769: Refactored after review comments.
* CORDA-3769: Removed the AttachmentsClassLoaderSimpleCacheImpl as DJVM does not need it. Also updated due to review comments.
* CORDA-3769: Removed the generic parameters from AttachmentsClassLoader.
* CORDA-3769: Removed unused imports.
* CORDA-3769: Updates from review comments.
* CORDA-3769: Updated following review comments. MigrationServicesForResolution now uses cache factory. Ctor updated for AttachmentsClassLoaderSimpleCacheImpl.
* CORDA-3769: Reduced max class loader cache size
* CORDA-3769: Fixed the attachments class loader cache size to a fixed default
* CORDA-3769: Switched attachments class loader size to be reduced by fixed value.
* CORDA-3722 withEntityManager can rollback its session
Improve the handling of database transactions when using
`withEntityManager` inside a flow.
Extra changes have been included to improve the safety and
correctness of Corda around handling database transactions.
This focuses on allowing flows to catch errors that occur inside an
entity manager and handle them accordingly.
Errors can be caught in two places:
- Inside `withEntityManager`
- Outside `withEntityManager`
Further changes have been included to ensure that transactions are
rolled back correctly.
Errors caught inside `withEntityManager` require the flow to manually
`flush` the current session (the entity manager's individual session).
By manually flushing the session, a `try-catch` block can be placed
around the `flush` call, allowing possible exceptions to be caught.
Once an error is thrown from a call to `flush`, it is no longer possible
to use the same entity manager to trigger any database operations. The
only possible option is to rollback the changes from that session.
The flow can continue executing updates within the same session but they
will never be committed. What happens in this situation should be handled
by the flow. Explicitly restricting the scenario requires a lot of effort
and code. Instead, we should rely on the developer to control complex
workflows.
To continue updating the database after an error like this occurs, a new
`withEntityManager` block should be used (after catching the previous
error).
Exceptions can be caught around `withEntityManager` blocks. This allows
errors to be handled in the same way as stated above, except the need to
manually `flush` the session is removed. `withEntityManager` will
automatically `flush` a session if it has not been marked for rollback
due to an earlier error.
A `try-catch` can then be placed around the whole of the
`withEntityManager` block, allowing the error to be caught while not
committing any changes to the underlying database transaction.
To make `withEntityManager` blocks work like mini database transactions,
save points have been utilised. A new savepoint is created when opening
a `withEntityManager` block (along with a new session). It is then used
as a reference point to rollback to if the session errors and needs to
roll back. The savepoint is then released (independently from
completing successfully or failing).
Using save points means, that either all the statements inside the
entity manager are executed, or none of them are.
- A new session is created every time an entity manager is requested,
but this does not replace the flow's main underlying database session.
- `CordaPersistence.transaction` can now determine whether it needs
to execute its extra error handling code. This is needed to allow errors
escape `withEntityManager` blocks while allowing some of our exception
handling around subscribers (in `NodeVaultService`) to continue to work.
* CORDA-3755: Switched attachments map to a WeakHashMap (#6214)
* CORDA-3772: Now specify source and target of 8 when compiling contract classes.
* CORDA-3651: addManifest now uses separate files for reading and writing. (#6026)
* CORDA-3651: addManifest now uses separate files for reading and writing.
* CORDA-3651: The jar scanning loader now closes itsself.
Co-authored-by: Adel El-Beik <adelel-beik@19LDN-MAC108.local>
Co-authored-by: Adel El-Beik <adelel-beik@19LDN-MAC108.local>
Added command-line option: `--pause-all-flows` to the Node to control this.
This mode causes all checkpoints to be set to status PAUSED when the
state machine starts up (in StartMode.Safe mode).
Changed the state machine so that PAUSED checkpoints are loaded into
memory (the checkpoint is deserialised but the flow state is left serialised)
but not started.
Messages from peers are queued whilst the flow is paused and processed
once the flow is resumed.
* Bump OS release version 4.6
* CORDA-3755: Switched attachments map to a WeakHashMap
* CORDA-3755: Added explicit strong references to map key.
* CORDA-3755: Keeping detekt happy.
* CORDA-3755: Test a gc in verify.
* CORDA-3755: Making detekt happy.
* CORDA-3755: Suppress warnings for weak reference test.
* CORDA-3755: Fixing build failure with attachments.
* CORDA-3755: Rewrite based on Ricks input - now handles attachment already existing in map!
* CORDA-3755: Refactor WeakReference behaviour into AttachmentsHolderImpl and provide alternate version of this class for core-deterministic.
* CORDA-3755: Added more tests for WeakHashMap.
* CORDA-3755: Ignore the tests using System.gc keep for local testing only
* CORDA-3755: Adding comment to explain the ignored tests.
* Make AttachmentsHolderImpl package-private inside core-deterministic, just like it is inside core.
* CORDA-3755: Update assertions following review comments.
* CORDA-3755: Removing import
* CORDA-3755: Removed unused var.
* CORDA-3755: Reverting files that somehow got changed in rebase.
Co-authored-by: nargas-ritu <ritu.gupta@r3.com>
Co-authored-by: Chris Rankin <chris.rankin@r3.com>
