* CORDA-4098 assumption test (#6871) (#6873)
* Upgrade of Bouncy Castle to resolve security issue
* Changed default signature scheme
* Reverted default change in draft
* Key conversion for BC with ed25519
* Initializing BC provider to use X509EdDSAEngine for Signature.Ed25519
* removed unsude imports
Co-authored-by: Nick Dunstone <nick.a.dunstone@gmail.com>
Co-authored-by: Alexey Kadyrov <67952405+alexey-kadyrov-r3@users.noreply.github.com>
* CORDA-4130: Move checkNotaryWhitelisted call to run under attachmentsClassLoader (#6890)
* CORDA-4130: Move checkNotaryWhitelisted call to run under attachmentsClassLoader for normal transactions.
* CORDA-4130: Reverted API change.
* CORDA-4130: Further simplication. Removed protected method.
* CORDA-4130: Remove unused import.
* Revert "CORDA-4130: Remove unused import."
This reverts commit d0836bda81.
* Revert "CORDA-4130: Further simplication. Removed protected method."
This reverts commit 3023a2e1ac.
Co-authored-by: Nick Dunstone <49945179+nickdunstone13@users.noreply.github.com>
Co-authored-by: Alexey Kadyrov <67952405+alexey-kadyrov-r3@users.noreply.github.com>
* Upgrade of Bouncy Castle to resolve security issue
* Changed default signature scheme
* Reverted default change in draft
* Key conversion for BC with ed25519
* Initializing BC provider to use X509EdDSAEngine for Signature.Ed25519
* removed unsude imports
Co-authored-by: Nick Dunstone <nick.a.dunstone@gmail.com>
Co-authored-by: Alexey Kadyrov <67952405+alexey-kadyrov-r3@users.noreply.github.com>
Co-authored-by: Alexey Kadyrov <67952405+alexey-kadyrov-r3@users.noreply.github.com>
* CORDA-4105 Add public API to allow custom serialization schemes (#6848)
* CORDA-4105 Add public API to allow custom serialization schemes
* Fix Detekt
* Suppress warning
* Fix usused import
* Improve API to use generics
This does not break Java support (only Intelij gets confused).
* Add more detailed documentation to public interfaces
* Change internal variable name after rename public API
* Update Public API to use ByteSquence instead of SerializedBytes
* Remove unused import
* Fix whitespace.
* Add added public API to .ci/api-current.txt
* Improve public interfaces
Rename CustomSchemeContext to SerializationSchemeContext to improve
clarity and move to it's own file. Improve kdoc to make things less
confusing.
* Update API current with changed API
* CORDA-4104 Implement custom serialization scheme discovery mechanism (#6854)
* CORDA-4104 Implement CustomSerializationScheme Discovery Mechanism
Discovers a single CustomSerializationScheme implementation inside
the drivers dir using a system property.
* Started MockNetwork test
* Add driver test of Custom Serialization Scheme
* Fix detekt and minor style error
* Respond to review comments
Allow non-single arg constructors (there must be one no args
constructor), move code from SerializationEnviroment into its
own file, improve exceptions to be more user friendly.
* Fix minor bug in Scheme finding code + improve error messages
* CORDA-4104 Improve test coverage of custom serialization scheme discovery (#6855)
* CORDA-4104 Add test of classloader scanning for CustomSerializationSchemes
* Fix Detekt
* NOTICK Clarify KDOC on SerializationSchemeContext (#6865)
* CORDA-4111 Change Component Group Serialization to use contex when the lazy map is constructed (#6856)
Currently the component group will recheck the thread local (global)
serialization context when component groups are serialized lazily.
Instead store the serialization context when the lazy map is constructed
and use that latter when doing serialization lazily.
* CORDA-4106 Test wire transaction can still be written to the ledger (#6860)
* Add test that writes transaction to the Database
* Improve test check serialization scheme in test body
* CORDA-4119 Minor changes to serialisation injection for transaction building (#6868)
* CORDA-4119 Minor changes to serialisation injection for transaction building
Scan the CorDapp classloader instead of the drivers classloader.
Add properties map to CustomSerialiaztionContext (copied from SerializationContext).
Change API to let a user pass in the serialization context in TransactionBuilder.toLedgerTransaction
* Improve KDOC + fix shawdowing issue in CordaUtils
* Pass only the properties map into theTransactionBuilder.toWireTransaction
Not the entire serializationContext
* Revert change to CordaUtils
* Improve KDOC explain pitfalls of setting properties
* wip
* wip
* wip (need to review IEE comments)
* wip
* wip
* Small refactoring, fixed network-verifier's TestNotaryFlow
* Added command line option to explicitly enable hash agility support
* wip-do-not-push
* wip
* wip
* wip
* aligned merkletree/transaction hash algorithms
* wip
* Added mixed algorithm support for nodes vs leaves and corrected mixed algorithm tests
* moved global computeNonce and componentHash to DigestService
* added comment for failing test to fix
* wip
* Minor cleanups, added deprecated componentHash/computeNonce
* restored exploratory changes to failing SignedTransaction test
* cleaned up and minor rafactoring
* Fixed some tests with hardcoded hash algorithm
* some changes and cleanups following code review
* WIP commit before large change
* WIP Fixed 3 tests
* WIP removed direct references to randomSHA256() and sha256()
* Updated/added liquibase migrations to support larger hash algorithms
* Reviewed, cleanups, comments, fixes
* removing direct references to sha256()
* WIP verifying obligations test errors
* reviewing obligation/attachment issues with sha3_256
* Full review before PR - intermediate commits
* Reviewed and cleaned up
* Futher cleanup
* Fixed partial tree backward compatible json and cleanups
* all tests passing
* Removed couple of unused imports
* Reworked global componentHash function to avoid deprecated warnings
* replaced SHA3s with some alternate SHA2s
* Removed SHA3-256 and SHA3-512 references
* fixed some tests using non ubiquitous hash algorithms
* Fixed ABI compatibility (not for TransactionBuilder)
* Fixed ABI compatibility to TransactionBuilder
* couple of fixes
* fixed DigestService's randomHash
* Removed constructor with loosely typed args for private constructor of LedgerTransaction class (API removal)
* re-introduced LedgerTransaction deprecated ctor for deserialization
* Add possibility to load CustomMessageDigest bypassing JCA (#6798)
* Change api-current for DigestAlgorithm
* disable flaky tests
* addressed liquibase migration script versions
* Removed TODOs and cleanups
* relaxed privacy salt validation
* Fixed privacy salt test to comply with relaxed validation
* detekt and privacySalt validation
* diff cleanup
* diff cleanup
* removed unused import
* removed PrivacySalt's validateFor method and references
* removed invalid character
Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
* wip
* wip
* wip (need to review IEE comments)
* wip
* wip
* Small refactoring, fixed network-verifier's TestNotaryFlow
* Added command line option to explicitly enable hash agility support
* wip-do-not-push
* wip
* wip
* wip
* aligned merkletree/transaction hash algorithms
* wip
* Added mixed algorithm support for nodes vs leaves and corrected mixed algorithm tests
* moved global computeNonce and componentHash to DigestService
* added comment for failing test to fix
* wip
* Minor cleanups, added deprecated componentHash/computeNonce
* restored exploratory changes to failing SignedTransaction test
* cleaned up and minor rafactoring
* Fixed some tests with hardcoded hash algorithm
* some changes and cleanups following code review
* WIP commit before large change
* WIP Fixed 3 tests
* WIP removed direct references to randomSHA256() and sha256()
* Updated/added liquibase migrations to support larger hash algorithms
* Reviewed, cleanups, comments, fixes
* removing direct references to sha256()
* WIP verifying obligations test errors
* reviewing obligation/attachment issues with sha3_256
* Full review before PR - intermediate commits
* Reviewed and cleaned up
* Futher cleanup
* Fixed partial tree backward compatible json and cleanups
* all tests passing
* Removed couple of unused imports
* Reworked global componentHash function to avoid deprecated warnings
* replaced SHA3s with some alternate SHA2s
* Removed SHA3-256 and SHA3-512 references
* fixed some tests using non ubiquitous hash algorithms
* Fixed ABI compatibility (not for TransactionBuilder)
* Fixed ABI compatibility to TransactionBuilder
* couple of fixes
* fixed DigestService's randomHash
* Removed constructor with loosely typed args for private constructor of LedgerTransaction class (API removal)
* re-introduced LedgerTransaction deprecated ctor for deserialization
* Add possibility to load CustomMessageDigest bypassing JCA (#6798)
* Change api-current for DigestAlgorithm
* disable flaky tests
Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
Do not let a user reattach to a flow started by another user.
Reattaching to a flow using startFlowWithClientId for a flow not
started by the current user throws a PermissionException
Reattaching to a flow using reattachFlowWithClientId for a flow not
started by the current user returns null.
finishedFlowsWithClientIds does not return flows started by other
users.
Normal rpc permissions around startFlowWithClientId and
startFlowDynamicWithClientId has also been added.
To allow admins to remove client ids as well as be able to see all the
client ids on the node, admin versions have been added that bypass the
user restrictions. These can be permitted via rpc to only provide
their usage to admins.
* INFRA-424 linux1 jenkinsfile
* INFRA-424 full run
* INFRA-424 bigger heap size
* Upgraded DJVM to handle BC - latest version of BC is a multirelease JAR.
When reading JKS keystore if a BC EdDSAPrivateKey is returned then swap for a net.i2p EdDSA private key.
* Temporary downgrade of BC
* Removed the BC EdDSA conversion
* INFRA-424 bigger heap size
* Upgrading Quasar to handle openJ9 different fields.
* INFRA-424: Handle lack of SUPPRESSED_SENTINEL in openj9.
* INFRA-424: If BCEdDSA public or private key is generated convert to net.i2p EdDSA form.
* INFRA-424 bigger heap size
* INFRA-424: On openJ9 only getting upto milli resolution.
* INFRA-424: Handle keystore returning a BCEdDSAPrivateKey.
* INFRA-424: Disable test on JDK11, as it requires the custom cordapp to generate JDK8 contract code, which we now check for.
* INFRA-424: Truncated time test to resolution of millis for openj9.
* INFRA-424 disabling log intensive tests until a fix is developed
* INFRA-424 one more test disabled
* INFRA-424: Disabled a couple of tests failing on openj9.
* INFRA-424: Disabling failing openj9 tests.
* INFRA-424: Disabling test failing on openj9.
* INFRA-424: Ignoring another flaky sleep test on openj9.
* INFRA-424 run integrationTests
* INFRA-424 set timeout to 4 hours
* INFRA-424: Cope with exception message from openj9.
* INFRA-424: Handle the coloured text characters openj9 adds.
* INFRA-424: Disabling test as it is generating JDK11 contract code under JDK11. Currently on JDK8 contract code allowed.
* INFRA-424: Commenting test out for openj9. Output of the processs thats read by the test is sometimes garbled.
* INFRA-424 switching to smoke tests
* INFRA-424 switching to slow integration tests
* INFRA-424 full run
* INFRA-424 moving jenkinsfile
* INFRA-424 removing references
* INFRA-424: Created common IS_OPENJ9 func for ignoring tests.
Co-authored-by: Schife <razvan.codreanu@r3.com>
* CORDA-3960: Port MultiRPCClient to OS
* CORDA-3960: Carefully restore serialisation environment in `MultiRpcClientTest` to eliminate side effects on other tests
* CORDA-3960: Move ThreadDumpUtils.kt to `core-utils`
* CORDA-3657 Extract information from state machine
`FlowReadOperations` interface provides functions that extract
information about flows from the state machine manager.
`FlowOperator` implements this interface (along with another currenly
empty interface).
* CORDA-3657 Rename function and use set
* initial test is passing
* wip
* done tests
* additional tests to cover more FlowIORequest variations
* completed tests
* The quasar.jar should nat have been changed
* Fixed issues reported by detekt
* got rid of sync objects, instead relying on nodes being offline
* Added extra grouping test and minor simplification
* Hospital test must use online node which fails on otherside
* Added additional information required for the ENT
* Added tests to cover SEND FlowIORequests
* using node name constants from the core testing module
* Changed flow operator to the query pattern
* made query fields mutable to simply building query
* fixed detekt issue
* Fixed test which had dependency on the order int the result (failed for windows)
* Fixed recommendations in PR
* Moved WrappedFlowExternalOperation and WrappedFlowExternalAsyncOperation to FlowExternalOperation.kt as per PR comment
* Moved extension to FlowAsyncOperation
* removed unnecessarily brackets
Co-authored-by: LankyDan <danknewton@hotmail.com>
Add `CordaRPCOps.reattachFlowWithClientId` to allow clients to reattach
to an existing flow by only providing a client id. This behaviour is the
same as calling `startFlowDynamicWithClientId` for an existing
`clientId`. Where it differs is `reattachFlowWithClientId` will return
`null` if there is no flow running or finished on the node with the same
client id.
Return `null` if record deleted from race-condition
Making statemachine not remove COMPLETED flows' checkpoints from the database
if they are started with a clientId, instead they are getting persisted and retained within
the database along with their result (`DBFlowResult`).
On flow start with a client id (`startFlowDynamicWithClientId`), if the client id maps to
a flow that was previously started with the same client id and the flow is now finished,
then fetch the `DBFlowResult` from the database to construct a
`FlowStateMachineHandle` done future and return it back to the client.
Object stored as results must abide by the storage serializer rules. If they fail to do so
the result will not be stored and an exception is thrown to the client to indicate this.
* CORDA-3717: Apply custom serializers to checkpoints
* Remove try/catch to fix TooGenericExceptionCaught detekt rule
* Rename exception
* Extract method
* Put calls to the userSerializer on their own lines to improve readability
* Remove unused constructors from exception
* Remove unused proxyType field
* Give field a descriptive name
* Explain why we are looking for two type parameters when we only use one
* Tidy up the fetching of types
* Use 0 seconds when forcing a flow checkpoint inside test
* Add test to check references are restored correctly
* Add CheckpointCustomSerializer interface
* Wire up the new CheckpointCustomSerializer interface
* Use kryo default for abstract classes
* Remove unused imports
* Remove need for external library in tests
* Make file match original to remove from diff
* Remove maySkipCheckpoint from calls to sleep
* Add newline to end of file
* Test custom serializers mapped to interfaces
* Test serializer configured with abstract class
* Move test into its own package
* Rename test
* Move flows and serializers into their own source file
* Move broken map into its own source file
* Delete comment now source file is simpler
* Rename class to have a shorter name
* Add tests that run the checkpoint serializer directly
* Check serialization of final classes
* Register as default unless the target class is final
* Test PublicKey serializer has not been overridden
* Add a broken serializer for EdDSAPublicKey to make test more robust
* Split serializer registration into default and non-default registrations. Run registrations at the right time to preserve Cordas own custom serializers.
* Check for duplicate custom checkpoint serializers
* Add doc comments
* Add doc comments to CustomSerializerCheckpointAdaptor
* Add test to check duplicate serializers are logged
* Do not log the duplicate serializer warning when the duplicate is the same class
* Update doc comment for CheckpointCustomSerializer
* Sort serializers by classname so we are not registering in an unknown or random order
* Add test to serialize a class that references itself
* Store custom serializer type in the Kryo stream so we can spot when a different serializer is being used to deserialize
* Testing has shown that registering custom serializers as default is more robust when adding new cordapps
* Remove new line character
* Remove unused imports
* Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt
* Remove comment
* Update comment on exception
* Make CustomSerializerCheckpointAdaptor internal
* Revert "Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt"
This reverts commit b835de79bd.
* Restore "Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt""
This reverts commit 718873a4e9.
* Pass the class loader instead of the context
* Do less work in test setup
* Make the serialization context unique for CustomCheckpointSerializerTest so we get a new Kryo pool for the test
* Rebuild the Kryo pool for the given context when we change custom serializers
* Rebuild all Kryo pools on serializer change to keep serializer list consistent
* Move the custom serializer list into CheckpointSerializationContext to reduce scope from global to a serialization context
* Remove unused imports
* Make the new checkpointCustomSerializers property default to the empty list
* Delegate implementation using kotlin language feature
Introducing a new flow start method (`startFlowDynamicWithClientId`) passing in a `clientId`.
Once `startFlowDynamicWithClientId` gets called, the `clientId` gets injected into `InvocationContext` and also pushed to the logging context.
If a new flow starts with this method, then a < `clientId` to flow > pair is kept on node side, even after the flow's lifetime. If `startFlowDynamicWithClientId` is called again with the same `clientId` then the node identifies that this `clientId` refers to an existing < `clientId` to flow > pair and returns back to the rpc client a `FlowStateMachineHandle` future, created out of that pair.
`FlowStateMachineHandle` interface was introduced as a thinner `FlowStateMachine`. All `FlowStateMachine` properties used by call sites are moved into this new interface along with `clientId` and then `FlowStateMachine` extends it.
Introducing an acknowledgement method (`removeClientId`). Calling this method removes the < `clientId` to flow > pair on the node side and frees resources.
* CORDA-3769: Switched attachments class loader cache to use caffeine with original implementation used by determinstic core.
* CORDA-3769: Removed default ctor arguments.
* CORDA-3769: Switched mapping function to Function type to avoid synthetic method being generated.
* CORDA-3769: Now using a cache created from NamedCacheFactory for the attachments class loader cache.
* CORDA-3769: Making detekt happy.
* CORDA-3769: The finality tests now check for UntrustedAttachmentsException which will actually happen in reality.
* CORDA-3769: Refactored after review comments.
* CORDA-3769: Removed the AttachmentsClassLoaderSimpleCacheImpl as DJVM does not need it. Also updated due to review comments.
* CORDA-3769: Removed the generic parameters from AttachmentsClassLoader.
* CORDA-3769: Removed unused imports.
* CORDA-3769: Updates from review comments.
* CORDA-3769: Updated following review comments. MigrationServicesForResolution now uses cache factory. Ctor updated for AttachmentsClassLoaderSimpleCacheImpl.
* CORDA-3769: Reduced max class loader cache size
* CORDA-3769: Fixed the attachments class loader cache size to a fixed default
* CORDA-3769: Switched attachments class loader size to be reduced by fixed value.
Added command-line option: `--pause-all-flows` to the Node to control this.
This mode causes all checkpoints to be set to status PAUSED when the
state machine starts up (in StartMode.Safe mode).
Changed the state machine so that PAUSED checkpoints are loaded into
memory (the checkpoint is deserialised but the flow state is left serialised)
but not started.
Messages from peers are queued whilst the flow is paused and processed
once the flow is resumed.
