Commit Graph

2208 Commits

Author SHA1 Message Date
Jose Coll
2c775bcc41
ENT-9924 Update recording of transaction flow recovery metadata into Send/Receive transaction flows. (#7374) 2023-06-02 16:05:28 +01:00
Shams Asari
5706f89639 Merge branch 'release/os/4.8' into shams-4.9-frwd-merge-a817218b
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/ArtemisTcpTransport.kt
#	node-api/src/test/kotlin/net/corda/nodeapi/internal/revocation/RevocationTest.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/CertificateRevocationListNodeTests.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/ProtonWrapperTests.kt
#	node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt
#	node/src/main/kotlin/net/corda/node/internal/artemis/BrokerJaasLoginModule.kt
#	node/src/main/kotlin/net/corda/node/internal/artemis/CertificateChainCheckPolicy.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/ArtemisMessagingServer.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/NodeNettyAcceptorFactory.kt
2023-06-01 16:37:57 +01:00
Shams Asari
ed08b2c5de ENT-9793: Added Page.previousPageAnchor to allow detection of vault changes whilst pages are loaded 2023-05-26 12:52:06 +01:00
Jose Coll
2e29e36e01
ENT-9923 Ledger Recovery: split out recovery metadata into own database schema. (#7364) 2023-05-24 09:42:09 +01:00
Shams Asari
c065021328 ENT-8827: The ordering of vault query results is clobbered by ServiceHub.loadStates 2023-05-18 15:59:25 +01:00
Jose Coll
c3e39a7052
ENT-9842 Re-factor 2PF to support issuance transactions (no notarisation) with observers. (#7349)
Re-factor 2PF to support issuance transactions (no notarisation) with observers.
2023-04-27 16:58:17 +01:00
Rick Parker
9ba3919980
ENT-9583 Public key caching of encoded form (OS) (#7332) 2023-04-26 17:49:52 +01:00
Rick Parker
f4917e08e1
ENT-9837 Use a cache to avoid resolving contract class name for a contract state repeatedly. (#7348) 2023-04-26 12:34:57 +01:00
Jose Coll
1d4feedc62
ENT-9147 Propagate and handle Notary Error (Part 2) (#7346) 2023-04-26 09:06:32 +01:00
Jose Coll
0bd4364653
ENT-9823 Rename handleDoubleSpend -> propagateDoubleSpendErrorToPeers (#7338) 2023-04-20 15:34:46 +01:00
Jose Coll
fffc3e4c5d
ENT-9822 Performance optimisation: use getNodeByLegalIdentity() backed by cache. (#7336) 2023-04-20 09:16:55 +01:00
Jose Coll
7bd3f5dd33
ENT-9147 Remove un-notarised transactions upon Double Spend. (#7324) 2023-04-19 15:31:47 +01:00
Jose Coll
18690dba94 Update JavaDoc. 2023-03-30 09:08:09 +01:00
Jose Coll
4beeb470df Additional signature verification and validation: recordTransactions() 2023-03-28 12:48:33 +01:00
Jose Coll
b4983597e2
ENT-6875 Two Phase Finality - CLEAN-UP (#7321)
* Remove completed TODOs

* Prevent mis-leading progress tracker message.
2023-03-24 08:55:37 +00:00
Connel McGovern
fda5cd07d2 INFRA-1955: Merging forward from 4.10 to 4.11 2023-03-22 11:24:31 +00:00
Jose Coll
d2900d54ab
ENT-6875 Two Phase Finality Flow - improve ledger consistency & recoverability (#7290) 2023-03-22 10:47:51 +00:00
Chris Cochrane
f63e049bcb
ENT-8924 Corrected JavaDoc/KDoc comments for ServiceLifecycleEvent (#7294) 2023-02-22 18:04:22 +00:00
Jose Coll
f8896ef706 Bump platform version. 2023-01-24 11:42:34 +00:00
Adel El-Beik
2e724cb82f ENT-8823: Removed unused import. 2022-12-08 16:57:45 +00:00
Adel El-Beik
044b83508e ENT-8823: Added copyBaggageToTags configuration option to control if baggage is copied to tags.
Switched SimplLogTelemetryComponent to info logs.
2022-12-08 16:54:38 +00:00
Adel El-Beik
50a1a4ce09 ENT-8823: StartEnd events now dont get mangled when parent span is a user span. Also other refactorings like using OT propagator now. 2022-12-06 19:24:19 +00:00
Adel El-Beik
e46b7bdd5a
ENT-6893: Added interface for clients to grab openetelemetry handle. (#7264)
* ENT-6893: Added interface for clients to grab openetelemetry handle.

* ENT-6893: Make detekt happy.

* ENT-6893: Fix warnings.

* ENT-6893: Make detekt happy.

* ENT-6893: Now shutdown opentelemetry when node stops or client is closed.

* ENT-6893: OpenTelemetryDriver is not not a singleton.
2022-11-24 13:34:08 +00:00
Chris Cochrane
e3d22dadbc
ENT-8796 - Fwd-merge from OS 4.9 2022-11-21 10:56:58 +00:00
Chris Cochrane
0c4c5bd138
Merge branch 'release/os/4.8' into cc/merge/os-4.8/os-4.9/16-Nov-2022
Fwd-merge from OS 4.8.
2022-11-16 17:35:33 +00:00
Rick Parker
43168387b4
ENT-8794 Delay closing of attachment class loaders (#7267)
* Delay closing of attachment class loaders until all SerializationContext that refer to them (from BasicVerifier) have gone out of scope.

More comments

* Avoid any concurrency issues with queue processing

* Better concurrency behaviour

* Stop re-using attachment URLs as it turns out we can't close the URLClassLoaders when URLs are shared

* Refactor to use a ReferenceQueue.

Co-authored-by: Chris Rankin <chris.rankin@r3.com>
2022-11-15 09:25:54 +00:00
Adel El-Beik
7a133f687c
ENT-6893: First cut of telemetry integration. (#7247)
First cut of telemetry integration.

Open telemetry can be enabled in two ways, first is via an opentelemetry java agent specified on the command line. With this way you get the advantage of spans created from other libraries, like hibernate. The java agent does byte code rewriting to insert spans.
The second way is with the open telemetry driver (that links with the opentelemetry sdk). This is a fat jar provided with this project and needs to go into the node drivers directory.
2022-10-28 14:41:39 +01:00
Rick Parker
bdcd25477d
ENT-7039: Notary signature checking requires non-interned SecureHashes. (#7254)
There was a mistake made when we first introduced notary request signature checking, in that we didn't wrap it in SerializedBytes so it always got deserialized as part of the flow message payload. So to check the signature, it has to be re-serialized. This means for cross-version compatibility we can never change the serialized format of NotarisationRequest. In this case we need make sure that every SecureHash mentioned in that data structure is a distinct instance, even if the values are repeated / identical, as that is how it was in Corda 1.

With the introduction of interning of SecureHash, this ceased to be true once again, including undoing the attempts to force it on the sending side that had been introduced in previous versions of Corda. So here we introduce a way to force it, and consolidate the forcing to distinct SecureHash instances in the NotarisationRequest itself, rather than leaving to the caller of the constructor to remember to do it, so that serialized form will always be as per Corda 1.
2022-10-28 14:03:47 +01:00
Rick Parker
b29713d7b9
ENT-6947 Intern common types to reduce heap footprint (#7239)
ENT-6947: Implement interning for SecureHash, CordaX500Name, PublicKey, AsbtractParty and SignatureAttachmentConstraint, including automatic detection of internable types off companion objects in AMQP & Kyro deserialization.  In some cases, add new factory methods to companion objects, and make main code base use them.

Performance tested in performance cluster with no negative impact visible (so default concurrency setting seems okay).

Testing suggests 5-6x memory saving for tokens in TokensSDK in memory selector.  Should see approx. 1 million tokens per GB or better (1.5 million for the tokens we tested with).
2022-10-18 09:28:41 +01:00
Chris Cochrane
0ab4ac94c8
ENT-6903 - address security issues reported by Snyk (#7244)
ENT-6903: Update versions of BC, Shiro, Netty okhttp
Also updated platform version to 12.
2022-09-30 17:49:21 +01:00
Chris Cochrane
14e2ef3478
Updated KDoc for quick RPCs 2022-08-18 13:02:20 +01:00
Chris Cochrane
03fa529292
ENT-6791 New service lifecycle event published just before starting the state machine (#7164) 2022-06-10 19:56:02 +01:00
Chris Rankin
47ec57186e Merge commit '91ef44dc12b8cc921b9ec0a37b44de75c256d92d' into chrisr3-os-4.8-merge 2022-05-23 12:27:49 +01:00
Chris Rankin
57ddafa148 ENT-6532: Close any deserialization classloader that is evicted from the cache.
Also revert disabling caches when reading URLs.
2022-05-20 11:51:01 +01:00
Ulrik Rasmussen
af1dbf4c6d Do not emit warning when StatePointer.isResolved = false
The warning is misleading because it is emitted in the case where the
ServiceHub reference is not null but one or more state pointers were
configured to not be automatically resolved by the transaction
builder.
2022-04-28 15:13:31 +02:00
Adel El-Beik
ba8cd1b80d Merge remote-tracking branch 'origin/release/os/4.8' into adel/merge-from-4.8-21Feb 2022-02-21 12:52:29 +00:00
Dan Newton
fa739fc467
ENT-6588 Restrict database operations platform flag (#7053)
* ENT-6588 Restrict database operations platform flag

Put the restricting of database operations in `RestrictedConnection` and
`RestrictedEntityManager` behind a platform version flag.

`RESTRICTED_DATABASE_OPERATIONS = 7` was added to signify this.

If the version is less than 7, then the database operations will not be
restricted. A warning is logged to indicate that they are using
potentially dangerous methods.

If the version is 7 or greater, then the database operations are
restricted and throw an error if called.
2022-02-08 10:06:44 +00:00
Chris Rankin
758a69f904
Validate LedgerTransaction deserialised from AttachmentsClassLoader. (#7049) (#7052) 2022-01-31 09:40:01 +00:00
Chris Rankin
c955093ce8
Validate LedgerTransaction deserialised from AttachmentsClassLoader. (#7049) 2022-01-28 17:05:59 +00:00
Adel El-Beik
9444c9b637
ENT-6567: Bump platform version to 11 (#7043)
* ENT-6567: Bump platform version to 11

* ENT-6567: Bump platform version to 11
2022-01-26 11:37:22 +00:00
Ramzi El-Yafi
831f6d60cc Merge remote-tracking branch 'origin/release/os/4.8' into ramzi/os-48-49-merge 2022-01-21 10:55:29 +00:00
Ramzi El-Yafi
caa4de47d3
ENT-6533 Update validating notary flow to be non-idempotent (#7030) 2022-01-19 13:15:51 +00:00
Chris Rankin
552317f2b3 Merge branch 'release/os/4.8' into chrisr3-os48-merge 2022-01-17 11:02:11 +00:00
Chris Rankin
4f1a07cbcc
ENT-6548: Ensure LazyMappedList is realised with correct SerializationContext. (#7028) 2022-01-17 09:57:33 +00:00
Dan Newton
044202550d
ENT-6421 Public version of FlowManagerRPCOps (#7006)
A public version of `FlowManagerRPCOps` which does not live in an
internal package has been added. This new interface shares the same name
as the internal one.

Because of the name sharing, the internal version has been
`@Deprecated`.

`FlowManagerRPCOpsImpl` implements both the new and old interfaces. This
 allows for backwards compatibility, allowing old shells or clients to
 call the old interface on newer nodes without breaking.
2022-01-07 16:40:29 +00:00
Walter Oggioni
efaf1549a9
CORDA-4173 Obfuscated zib bombs used for unit tests, so that antivirus software stop complaining about them (#6989) 2021-12-21 14:08:02 +00:00
Chris Rankin
e93e7c2846
ENT-6412: Upgrade ClassGraph 4.8.90 -> 4.8.135. (#6973) 2021-11-23 09:43:41 +00:00
Chris Rankin
caeafb8201
ENT-6357: Deserialize LedgerTransaction elements for each Contract.verify(). (#6962)
* ENT-6357: Deserialize LedgerTransaction elements for each Contract.verify().

* Lock the LedgerTransaction and NetworkParameters objects down for contract verification.

* Refactor BasicVerifier to be package private instead of public.

* Simplify verifyConstraints() operation.

* Review fixes: replace HashSet with LinkedHashSet, and add signing parties to commands via mapIndexed.

* Ensure tests also run notary nodes "out of process".

* Streamline SerializationContext switching.

* Cache deserialised cryptographic instances during contract verification.

* Invoke Class.forName() instead of ClassLoader.loadClass() to reduce contention on the system classloader's lock.

* Deserialization cache key now pre-computes its hash code.

* Allow AttachmentsClassLoader to be used concurrently.

* Cache all Envelope objects for reuse during contract verification.

* Generate CertPathProxy hash code using conventional algorithm.

* Adjust CustomSerializer.Proxy to allow better access to SerializationContext.
2021-11-10 16:38:40 +00:00
Walter Oggioni
883e794853
ENT-6330 Fixed reading jar entries in memory (#6960)
* ENT-6330 Fixed reading jar entries in memory

This is a trivial fix that is however enough to allow to send zip bombs as attachments without the node crashing, a size limit could be added for increased reliability

* added attachment cumulative size check

* added compression ratio check

* added unit test and moved the code to a standalone verifier object

* removed attachment check from AttachmentClassLoader to minimize performance impact
2021-10-13 09:46:20 +01:00
Ramzi El-Yafi
9146228b0f
NAAS-295 Fix notary flow retries after ETA message sent (#6965) 2021-10-04 17:37:22 +01:00
Agnieszka Szczepanska
0b9513ffa3 Merge branch 'release/os/4.7' into aga/make-recordDependencies-suspendable-4.8 2021-04-27 15:08:13 +01:00
Agnieszka Szczepanska
9aa4389902 resolve merge conflicts 2021-04-26 11:35:17 +01:00
Agnieszka Szczepanska
832751d41d delete redundant lines at the end of file 2021-04-21 14:24:23 +02:00
Agnieszka Szczepanska
88f584cac5 fast forward merge 2021-04-21 12:16:22 +02:00
Agnieszka Szczepanska
c48f32215c Merge branch 'release/os/4.4' into aga/make-recordDependencies-suspendable-4.5 2021-04-20 16:25:47 +02:00
Agnieszka Szczepanska
8dd9dee9c4
NOTICK - make recordDependencies suspendable (#6896)
* add suspendable annotation to recordDependencies

* add suspendable annotation to recordDependencies implementation
2021-04-14 08:30:53 +01:00
Edoardo Ierina
241170ffa4
Backward compatible fix for Merkle tree case where a single leaf becomes the tree root (#6895) 2021-04-12 11:09:39 +01:00
Edoardo Ierina
bde2d2a8c7
updated inxonsistent comments (#6892) 2021-03-22 14:38:22 +00:00
Adel El-Beik
bf9f4a5c4a
NOTICK: Merge from OS 4.7 (#6891)
* CORDA-4098 assumption test (#6871) (#6873)

* Upgrade of Bouncy Castle to resolve security issue

* Changed default signature scheme

* Reverted default change in draft

* Key conversion for BC with ed25519

* Initializing BC provider to use X509EdDSAEngine for Signature.Ed25519

* removed unsude imports

Co-authored-by: Nick Dunstone <nick.a.dunstone@gmail.com>

Co-authored-by: Alexey Kadyrov <67952405+alexey-kadyrov-r3@users.noreply.github.com>

* CORDA-4130: Move checkNotaryWhitelisted call to run under attachmentsClassLoader (#6890)

* CORDA-4130: Move checkNotaryWhitelisted call to run under attachmentsClassLoader for normal transactions.

* CORDA-4130: Reverted API change.

* CORDA-4130: Further simplication. Removed protected method.

* CORDA-4130: Remove unused import.

* Revert "CORDA-4130: Remove unused import."

This reverts commit d0836bda81.

* Revert "CORDA-4130: Further simplication. Removed protected method."

This reverts commit 3023a2e1ac.

Co-authored-by: Nick Dunstone <49945179+nickdunstone13@users.noreply.github.com>
Co-authored-by: Alexey Kadyrov <67952405+alexey-kadyrov-r3@users.noreply.github.com>
2021-03-19 10:21:29 +00:00
Adel El-Beik
1470d14795
CORDA-4130: Move checkNotaryWhitelisted call to run under attachmentsClassLoader (#6890)
* CORDA-4130: Move checkNotaryWhitelisted call to run under attachmentsClassLoader for normal transactions.

* CORDA-4130: Reverted API change.

* CORDA-4130: Further simplication. Removed protected method.

* CORDA-4130: Remove unused import.

* Revert "CORDA-4130: Remove unused import."

This reverts commit d0836bda81.

* Revert "CORDA-4130: Further simplication. Removed protected method."

This reverts commit 3023a2e1ac.
2021-03-18 14:24:30 +00:00
Edoardo Ierina
a17263124f
Bumped platformVersion from 9 to 10 (#6882) 2021-02-22 10:58:39 +00:00
Adel El-Beik
9cd02dc62d Merge remote-tracking branch 'origin/release/os/4.4' into adel/merge-from-4.4-19-Feb-21 2021-02-19 13:55:04 +00:00
Adel El-Beik
360b3f8d0c
CORDA-4125: Backport AttachmentClassLoader updates to 4.4 to enable ENT-6152 backport (#6878)
* CORDA-3755: Switched attachments map to a WeakHashMap (#6214)

* Bump OS release version 4.6

* CORDA-3755: Switched attachments map to a WeakHashMap

* CORDA-3755: Added explicit strong references to map key.

* CORDA-3755: Keeping detekt happy.

* CORDA-3755: Test a gc in verify.

* CORDA-3755: Making detekt happy.

* CORDA-3755: Suppress warnings for weak reference test.

* CORDA-3755: Fixing build failure with attachments.

* CORDA-3755: Rewrite based on Ricks input - now handles attachment already existing in map!

* CORDA-3755: Refactor WeakReference behaviour into AttachmentsHolderImpl and provide alternate version of this class for core-deterministic.

* CORDA-3755: Added more tests for WeakHashMap.

* CORDA-3755: Ignore the tests using System.gc keep for local testing only

* CORDA-3755: Adding comment to explain the ignored tests.

* Make AttachmentsHolderImpl package-private inside core-deterministic, just like it is inside core.

* CORDA-3755: Update assertions following review comments.

* CORDA-3755: Removing import

* CORDA-3755: Removed unused var.

* CORDA-3755: Reverting files that somehow got changed in rebase.

Co-authored-by: nargas-ritu <ritu.gupta@r3.com>
Co-authored-by: Chris Rankin <chris.rankin@r3.com>

* CORDA-3769: Switched attachments class loader cache to use caffeine (#6326)

* CORDA-3769: Switched attachments class loader cache to use caffeine with original implementation used by determinstic core.

* CORDA-3769: Removed default ctor arguments.

* CORDA-3769: Switched mapping function to Function type to avoid synthetic method being generated.

* CORDA-3769: Now using a cache created from NamedCacheFactory for the attachments class loader cache.

* CORDA-3769: Making detekt happy.

* CORDA-3769: The finality tests now check for UntrustedAttachmentsException which will actually happen in reality.

* CORDA-3769: Refactored after review comments.

* CORDA-3769: Removed the AttachmentsClassLoaderSimpleCacheImpl as DJVM does not need it. Also updated due to review comments.

* CORDA-3769: Removed the generic parameters from AttachmentsClassLoader.

* CORDA-3769: Removed unused imports.

* CORDA-3769: Updates from review comments.

* CORDA-3769: Updated following review comments. MigrationServicesForResolution now uses cache factory. Ctor updated for AttachmentsClassLoaderSimpleCacheImpl.

* CORDA-3769: Reduced max class loader cache size

* CORDA-3769: Fixed the attachments class loader cache size to a fixed default

* CORDA-3769: Switched attachments class loader size to be reduced by fixed value.

* CORDA-4125: Parameter has been added to a private ctor.

Co-authored-by: nargas-ritu <ritu.gupta@r3.com>
Co-authored-by: Chris Rankin <chris.rankin@r3.com>
2021-02-19 13:38:22 +00:00
Nick Dunstone
42bb25462d
CORDA-4098 assumption test (#6871) (#6873) (#6874)
* Upgrade of Bouncy Castle to resolve security issue

* Changed default signature scheme

* Reverted default change in draft

* Key conversion for BC with ed25519

* Initializing BC provider to use X509EdDSAEngine for Signature.Ed25519

* removed unsude imports

Co-authored-by: Nick Dunstone <nick.a.dunstone@gmail.com>

Co-authored-by: Alexey Kadyrov <67952405+alexey-kadyrov-r3@users.noreply.github.com>

Co-authored-by: Alexey Kadyrov <67952405+alexey-kadyrov-r3@users.noreply.github.com>
2021-02-17 17:14:05 +00:00
Nick Dunstone
d41f608e4e
CORDA-4098 assumption test (#6871) (#6873)
* Upgrade of Bouncy Castle to resolve security issue

* Changed default signature scheme

* Reverted default change in draft

* Key conversion for BC with ed25519

* Initializing BC provider to use X509EdDSAEngine for Signature.Ed25519

* removed unsude imports

Co-authored-by: Nick Dunstone <nick.a.dunstone@gmail.com>

Co-authored-by: Alexey Kadyrov <67952405+alexey-kadyrov-r3@users.noreply.github.com>
2021-02-17 12:53:30 +00:00
Alexey Kadyrov
ae2bb9992f
CORDA-4098 assumption test (#6871)
* Upgrade of Bouncy Castle to resolve security issue

* Changed default signature scheme

* Reverted default change in draft

* Key conversion for BC with ed25519

* Initializing BC provider to use X509EdDSAEngine for Signature.Ed25519

* removed unsude imports

Co-authored-by: Nick Dunstone <nick.a.dunstone@gmail.com>
2021-02-17 10:27:45 +00:00
William Vigor
20dbbf008d
CORDA-4103 Feature Branch: Serialization injection for transaction building (#6867)
* CORDA-4105 Add public API to allow custom serialization schemes (#6848)

* CORDA-4105 Add public API to allow custom serialization schemes

* Fix Detekt

* Suppress warning

* Fix usused import

* Improve API to use generics

This does not break Java support (only Intelij gets confused).

* Add more detailed documentation to public interfaces

* Change internal variable name after rename public API

* Update Public API to use ByteSquence instead of SerializedBytes

* Remove unused import

* Fix whitespace.

* Add added public API to .ci/api-current.txt

* Improve public interfaces

Rename CustomSchemeContext to SerializationSchemeContext to improve
clarity and move to it's own file. Improve kdoc to make things less
confusing.

* Update API current with changed API

* CORDA-4104 Implement custom serialization scheme discovery mechanism (#6854)

* CORDA-4104 Implement CustomSerializationScheme Discovery Mechanism

Discovers a single CustomSerializationScheme implementation inside
the drivers dir using a system property.

* Started MockNetwork test

* Add driver test of Custom Serialization Scheme

* Fix detekt and minor style error

* Respond to review comments

Allow non-single arg constructors (there must be one no args
constructor), move code from SerializationEnviroment into its
own file, improve exceptions to be more user friendly.

* Fix minor bug in Scheme finding code  + improve error messages

* CORDA-4104 Improve test coverage of custom serialization scheme discovery (#6855)

* CORDA-4104 Add test of classloader scanning for CustomSerializationSchemes

* Fix Detekt

* NOTICK Clarify KDOC on SerializationSchemeContext (#6865)

* CORDA-4111 Change Component Group Serialization to use contex when the lazy map is constructed (#6856)

Currently the component group will recheck the thread local (global)
serialization context when component groups are serialized lazily.
Instead store the serialization context when the lazy map is constructed
and use that latter when doing serialization lazily.

* CORDA-4106 Test wire transaction can still be written to the ledger (#6860)

* Add test that writes transaction to the Database

* Improve test check serialization scheme in test body

* CORDA-4119 Minor changes to serialisation injection for transaction building (#6868)

* CORDA-4119 Minor changes to serialisation injection for transaction building

Scan the CorDapp classloader instead of the drivers classloader.
Add properties map to CustomSerialiaztionContext (copied from SerializationContext).
Change API to let a user pass in the serialization context in TransactionBuilder.toLedgerTransaction

* Improve KDOC + fix shawdowing issue in CordaUtils

* Pass only the properties map into theTransactionBuilder.toWireTransaction

Not the entire serializationContext

* Revert change to CordaUtils

* Improve KDOC explain pitfalls of setting properties
2021-02-11 15:27:03 +00:00
Edoardo Ierina
d7c9f20019
CORDA-4100: Updates to DigestAlgorithm interface (#6849)
Updated DigestAlgorithm interface to separate componentHash and computeNonce hashing functions.
2021-02-01 10:52:47 +00:00
Alexey Koren
56df286410
Add ability to set custom initiating flows for NotaryService (#6832) 2021-01-26 13:56:42 +00:00
Stefan Iliev
2c25a5c768
AG-323 Flow startup queue metric (#6811) 2020-11-30 16:49:08 +00:00
Edoardo Ierina
494654cea6
CORDA-3823: hash agility updates for rc03 (#6800)
* wip

* wip

* wip (need to review IEE comments)

* wip

* wip

* Small refactoring, fixed network-verifier's TestNotaryFlow

* Added command line option to explicitly enable hash agility support

* wip-do-not-push

* wip

* wip

* wip

* aligned merkletree/transaction hash algorithms

* wip

* Added mixed algorithm support for nodes vs leaves and corrected mixed algorithm tests

* moved global computeNonce and componentHash to DigestService

* added comment for failing test to fix

* wip

* Minor cleanups, added deprecated componentHash/computeNonce

* restored exploratory changes to failing SignedTransaction test

* cleaned up and minor rafactoring

* Fixed some tests with hardcoded hash algorithm

* some changes and cleanups following code review

* WIP commit before large change

* WIP Fixed 3 tests

* WIP removed direct references to randomSHA256() and sha256()

* Updated/added liquibase migrations to support larger hash algorithms

* Reviewed, cleanups, comments, fixes

* removing direct references to sha256()

* WIP verifying obligations test errors

* reviewing obligation/attachment issues with sha3_256

* Full review before PR - intermediate commits

* Reviewed and cleaned up

* Futher cleanup

* Fixed partial tree backward compatible json and cleanups

* all tests passing

* Removed couple of unused imports

* Reworked global componentHash function to avoid deprecated warnings

* replaced SHA3s with some alternate SHA2s

* Removed SHA3-256 and SHA3-512 references

* fixed some tests using non ubiquitous hash algorithms

* Fixed ABI compatibility (not for TransactionBuilder)

* Fixed ABI compatibility to TransactionBuilder

* couple of fixes

* fixed DigestService's randomHash

* Removed constructor with loosely typed args for private constructor of LedgerTransaction class (API removal)

* re-introduced LedgerTransaction deprecated ctor for deserialization

* Add possibility to load CustomMessageDigest bypassing JCA (#6798)

* Change api-current for DigestAlgorithm

* disable flaky tests

* addressed liquibase migration script versions

* Removed TODOs and cleanups

* relaxed privacy salt validation

* Fixed privacy salt test to comply with relaxed validation

* detekt and privacySalt validation

* diff cleanup

* diff cleanup

* removed unused import

* removed PrivacySalt's validateFor method and references

* removed invalid character

Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
2020-11-12 17:03:43 +00:00
Denis Rekalov
14e545826c
CORDA-4076: Fix SecureHash compatibility with previous versions (#6801) 2020-11-12 09:39:57 +00:00
Denis Rekalov
d9f905cb81
Disable enhanced validation, rename SQL patches (#6799) 2020-11-06 12:32:41 +00:00
Edoardo Ierina
82a114a329
[DRAFT] feat/CORDA-3823-hash-agility-qa-ready (#6789)
* wip

* wip

* wip (need to review IEE comments)

* wip

* wip

* Small refactoring, fixed network-verifier's TestNotaryFlow

* Added command line option to explicitly enable hash agility support

* wip-do-not-push

* wip

* wip

* wip

* aligned merkletree/transaction hash algorithms

* wip

* Added mixed algorithm support for nodes vs leaves and corrected mixed algorithm tests

* moved global computeNonce and componentHash to DigestService

* added comment for failing test to fix

* wip

* Minor cleanups, added deprecated componentHash/computeNonce

* restored exploratory changes to failing SignedTransaction test

* cleaned up and minor rafactoring

* Fixed some tests with hardcoded hash algorithm

* some changes and cleanups following code review

* WIP commit before large change

* WIP Fixed 3 tests

* WIP removed direct references to randomSHA256() and sha256()

* Updated/added liquibase migrations to support larger hash algorithms

* Reviewed, cleanups, comments, fixes

* removing direct references to sha256()

* WIP verifying obligations test errors

* reviewing obligation/attachment issues with sha3_256

* Full review before PR - intermediate commits

* Reviewed and cleaned up

* Futher cleanup

* Fixed partial tree backward compatible json and cleanups

* all tests passing

* Removed couple of unused imports

* Reworked global componentHash function to avoid deprecated warnings

* replaced SHA3s with some alternate SHA2s

* Removed SHA3-256 and SHA3-512 references

* fixed some tests using non ubiquitous hash algorithms

* Fixed ABI compatibility (not for TransactionBuilder)

* Fixed ABI compatibility to TransactionBuilder

* couple of fixes

* fixed DigestService's randomHash

* Removed constructor with loosely typed args for private constructor of LedgerTransaction class (API removal)

* re-introduced LedgerTransaction deprecated ctor for deserialization

* Add possibility to load CustomMessageDigest bypassing JCA (#6798)

* Change api-current for DigestAlgorithm

* disable flaky tests

Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
2020-11-05 22:05:29 +00:00
Dan Newton
6b5e76f87a Merge branch 'release/os/4.6' into dan/os-4.6-to-4.7-merge-2020-10-30 2020-10-30 16:08:15 +00:00
Denis Rekalov
c9056f171b
CORDA-4062: Bump platform version to 9 for safe identity key rotation (#6777) 2020-10-20 12:31:47 +03:00
Denis Rekalov
4193adf6fd
CORDA-3979: Support for multiple trust roots (#6772) 2020-10-20 12:18:00 +03:00
Denis Rekalov
551b3f0811
CORDA-4054: combine different identities of the same notary after its key rotation (#6734) 2020-10-16 13:53:04 +01:00
Adel El-Beik
e7df5818e4
INFRA-424 CORDA-4058: Disable flaky tests on openj9 (#6774) 2020-10-16 13:03:25 +01:00
Ross Nicoll
1ccd84bbcb Merge remote-tracking branch 'origin/release/os/4.6' into rni/notick/merge-46-47 2020-10-14 15:52:56 +01:00
Denis Rekalov
cdd725e79c
CORDA-3972: Support for node identity rotation in IdentityService (#6752) 2020-10-12 18:01:32 +03:00
Andrius Dagys
cfd57faf5f
NOTICK: Update network parameter maxMessageSize API doc (#6334)
This network parameter is now enforced contrary to the description.
2020-09-25 11:25:51 +01:00
Ryan Fowler
ed9f6f0ce1 Merge branch 'release/os/4.6' into rfowler-os-4.6-ent-4.6-20200922 2020-09-22 11:42:00 +01:00
Dan Newton
bd7b96e816
ENT-5768 startFlowWithClientId permissions (#6708)
Do not let a user reattach to a flow started by another user.

Reattaching to a flow using startFlowWithClientId for a flow not
started by the current user throws a PermissionException

Reattaching to a flow using reattachFlowWithClientId for a flow not
started by the current user returns null.

finishedFlowsWithClientIds does not return flows started by other
users.

Normal rpc permissions around startFlowWithClientId and
startFlowDynamicWithClientId has also been added.

To allow admins to remove client ids as well as be able to see all the
client ids on the node, admin versions have been added that bypass the
user restrictions. These can be permitted via rpc to only provide
their usage to admins.
2020-09-16 16:11:06 +01:00
Ross Nicoll
0c16957622 Merge remote-tracking branch 'origin/release/os/4.6' into rni/notick/46-47-merge-2020-09-04 2020-09-04 14:15:47 +01:00
Adel El-Beik
9962c9085d
INFRA-424: Merge openj9 updates into 4.6 (#6683)
* INFRA-424 linux1 jenkinsfile

* INFRA-424 full run

* INFRA-424 bigger heap size

* Upgraded DJVM to handle BC - latest version of BC is a multirelease JAR.
When reading JKS keystore if a BC EdDSAPrivateKey is returned then swap for a net.i2p EdDSA private key.

* Temporary downgrade of BC

* Removed the BC EdDSA conversion

* INFRA-424 bigger heap size

* Upgrading Quasar to handle openJ9 different fields.

* INFRA-424: Handle lack of SUPPRESSED_SENTINEL in openj9.

* INFRA-424: If BCEdDSA public or private key is generated convert to net.i2p EdDSA form.

* INFRA-424 bigger heap size

* INFRA-424: On openJ9 only getting upto milli resolution.

* INFRA-424: Handle keystore returning a BCEdDSAPrivateKey.

* INFRA-424: Disable test on JDK11, as it requires the custom cordapp to generate JDK8 contract code, which we now check for.

* INFRA-424: Truncated time test to resolution of millis for openj9.

* INFRA-424 disabling log intensive tests until a fix is developed

* INFRA-424 one more test disabled

* INFRA-424: Disabled a couple of tests failing on openj9.

* INFRA-424: Disabling failing openj9 tests.

* INFRA-424: Disabling test failing on openj9.

* INFRA-424: Ignoring another flaky sleep test on openj9.

* INFRA-424 run integrationTests

* INFRA-424 set timeout to 4 hours

* INFRA-424: Cope with exception message from openj9.

* INFRA-424: Handle the coloured text characters openj9 adds.

* INFRA-424: Disabling test as it is generating JDK11 contract code under JDK11. Currently on JDK8 contract code allowed.

* INFRA-424: Commenting test out for openj9. Output of the processs thats read by the test is sometimes garbled.

* INFRA-424 switching to smoke tests

* INFRA-424 switching to slow integration tests

* INFRA-424 full run

* INFRA-424 moving jenkinsfile

* INFRA-424 removing references

* INFRA-424: Created common IS_OPENJ9 func for ignoring tests.

Co-authored-by: Schife <razvan.codreanu@r3.com>
2020-09-02 14:35:30 +01:00
Joseph Zuniga-Daly
8fd4fd1bf8 Merge OS 4.6 into OS 4.7 2020-08-26 11:09:02 +01:00
Walter Oggioni
49f598308b
added checkpoints debug shell command (#6574) 2020-08-25 10:10:25 +01:00
Viktor Kolomeyko
9fc896beb3
CORDA-3960: Port MultiRPCClient to OS (#6644)
* CORDA-3960: Port MultiRPCClient to OS

* CORDA-3960: Carefully restore serialisation environment in `MultiRpcClientTest` to eliminate side effects on other tests

* CORDA-3960: Move ThreadDumpUtils.kt to `core-utils`
2020-08-18 10:35:19 +01:00
Dan Newton
854e6638ff
CORDA-3881 Get all finished flows with client ids (#6580)
Return map of `clientId` -> success/fail
2020-08-17 10:27:32 +01:00
Viktor Kolomeyko
b81eb1559d
CORDA-3959: Eliminate InternalCordaRPCOps (#6600)
* CORDA-3959: Make `ThreadContextAdjustingRpcOpsProxy` flexible for `RPCOps` it uses

* CORDA-3959: More changes towards supporting multiple `RPCOps` implementations

* CORDA-3959: Detekt baseline update

* CORDA-3959: Integration test compilation fix

* CORDA-3959: Introduce `CheckpointRPCOpsImpl` and wire it on

* CORDA-3959: Use multiple RPCOps interfaces in the shell commands

* CORDA-3959: Detekt baseline update

* CORDA-3959: Update RPCPermissionsTests

* CORDA-3959: Update RPCSecurityManagerTest

* CORDA-3959: Remove deprecated marker and rename the property

* CORDA-3959: Detekt baseline

* CORDA-3959: Introduce AttachmentTrustInfoRPCOpsImpl and wire it on

* CORDA-3959: Delete `InternalCordaRPCOps`

* CORDA-3959: Detekt baseline update

* CORDA-3959: Rename `CheckpointRPCOps` to `FlowManagerRPCOps`
2020-08-13 14:41:52 +01:00
Alexey Kadyrov
0b6b69bbda
CORDA-3657/5459 inspect waiting flows (#6540)
* CORDA-3657 Extract information from state machine

`FlowReadOperations` interface provides functions that extract
information about flows from the state machine manager.

`FlowOperator` implements this interface (along with another currenly
empty interface).

* CORDA-3657 Rename function and use set

* initial test is passing

* wip

* done tests

* additional tests to cover more FlowIORequest variations

* completed tests

* The quasar.jar should nat have been changed

* Fixed issues reported by detekt

* got rid of sync objects, instead relying on nodes being offline

* Added extra grouping test and minor simplification

* Hospital test must use online node which fails on otherside

* Added additional information required for the ENT

* Added tests to cover SEND FlowIORequests

* using node name constants from the core testing module

* Changed flow operator to the query pattern

* made query fields mutable to simply building query

* fixed detekt issue

* Fixed test which had dependency on the order int the result (failed for windows)

* Fixed recommendations in PR

* Moved WrappedFlowExternalOperation and  WrappedFlowExternalAsyncOperation to FlowExternalOperation.kt as per PR comment

* Moved extension to FlowAsyncOperation

* removed unnecessarily brackets

Co-authored-by: LankyDan <danknewton@hotmail.com>
2020-08-12 10:14:05 +01:00
Dan Newton
3f31aeaa5f
CORDA-3822 Add CordaRPCOps.reattachFlowWithClientId (#6579)
Add `CordaRPCOps.reattachFlowWithClientId` to allow clients to reattach
to an existing flow by only providing a client id. This behaviour is the
same as calling `startFlowDynamicWithClientId` for an existing
`clientId`. Where it differs is `reattachFlowWithClientId` will return
`null` if there is no flow running or finished on the node with the same
client id.

Return `null` if record deleted from race-condition
2020-08-06 11:42:02 +01:00
Kyriakos Tharrouniatis
8a37108a58 Merge branch 'release/os/4.6' into os_4.6-feature_pass_in_client_id_when_starting_a_flow-merge 2020-07-30 10:34:03 +01:00
Kyriakos Tharrouniatis
ad61d7b3f4
CORDA-3692 - Store result information in checkpoint results table (#6473)
Making statemachine not remove COMPLETED flows' checkpoints from the database 
if they are started with a clientId, instead they are getting persisted and retained within 
the database along with their result (`DBFlowResult`).

On flow start with a client id (`startFlowDynamicWithClientId`), if the client id maps to
a flow that was previously started with the same client id and the flow is now finished, 
then fetch the `DBFlowResult` from the database to construct a 
`FlowStateMachineHandle` done future and return it back to the client.

Object stored as results must abide by the storage serializer rules. If they fail to do so
the result will not be stored and an exception is thrown to the client to indicate this.
2020-07-29 15:57:00 +01:00
Joseph Zuniga-Daly
c33720c73d
CORDA-3717: Apply custom serializers to checkpoints (#6392)
* CORDA-3717: Apply custom serializers to checkpoints

* Remove try/catch to fix TooGenericExceptionCaught detekt rule

* Rename exception

* Extract method

* Put calls to the userSerializer on their own lines to improve readability

* Remove unused constructors from exception

* Remove unused proxyType field

* Give field a descriptive name

* Explain why we are looking for two type parameters when we only use one

* Tidy up the fetching of types

* Use 0 seconds when forcing a flow checkpoint inside test

* Add test to check references are restored correctly

* Add CheckpointCustomSerializer interface

* Wire up the new CheckpointCustomSerializer interface

* Use kryo default for abstract classes

* Remove unused imports

* Remove need for external library in tests

* Make file match original to remove from diff

* Remove maySkipCheckpoint from calls to sleep

* Add newline to end of file

* Test custom serializers mapped to interfaces

* Test serializer configured with abstract class

* Move test into its own package

* Rename test

* Move flows and serializers into their own source file

* Move broken map into its own source file

* Delete comment now source file is simpler

* Rename class to have a shorter name

* Add tests that run the checkpoint serializer directly

* Check serialization of final classes

* Register as default unless the target class is final

* Test PublicKey serializer has not been overridden

* Add a broken serializer for EdDSAPublicKey to make test more robust

* Split serializer registration into default and non-default registrations. Run registrations at the right time to preserve Cordas own custom serializers.

* Check for duplicate custom checkpoint serializers

* Add doc comments

* Add doc comments to CustomSerializerCheckpointAdaptor

* Add test to check duplicate serializers are logged

* Do not log the duplicate serializer warning when the duplicate is the same class

* Update doc comment for CheckpointCustomSerializer

* Sort serializers by classname so we are not registering in an unknown or random order

* Add test to serialize a class that references itself

* Store custom serializer type in the Kryo stream so we can spot when a different serializer is being used to deserialize

* Testing has shown that registering custom serializers as default is more robust when adding new cordapps

* Remove new line character

* Remove unused imports

* Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt

* Remove comment

* Update comment on exception

* Make CustomSerializerCheckpointAdaptor internal

* Revert "Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt"

This reverts commit b835de79bd.

* Restore "Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt""

This reverts commit 718873a4e9.

* Pass the class loader instead of the context

* Do less work in test setup

* Make the serialization context unique for CustomCheckpointSerializerTest so we get a new Kryo pool for the test

* Rebuild the Kryo pool for the given context when we change custom serializers

* Rebuild all Kryo pools on serializer change to keep serializer list consistent

* Move the custom serializer list into CheckpointSerializationContext to reduce scope from global to a serialization context

* Remove unused imports

* Make the new checkpointCustomSerializers property default to the empty list

* Delegate implementation using kotlin language feature
2020-07-22 17:31:59 +01:00
Dimos Raptis
7261fa690f
CORDA-3506 - Implement session close operations (#6357) 2020-07-21 13:26:11 +01:00
Kyriakos Tharrouniatis
22d92d5ef0
CORDA-3809 Expose client side unique RPC ID for flow starts (#6307)
Introducing a new flow start method (`startFlowDynamicWithClientId`) passing in a `clientId`.

Once `startFlowDynamicWithClientId` gets called, the `clientId` gets injected into `InvocationContext` and also pushed to the logging context.

If a new flow starts with this method, then a < `clientId` to flow > pair is kept on node side, even after the flow's lifetime. If `startFlowDynamicWithClientId` is called again with the same `clientId` then the node identifies that this `clientId` refers to an existing < `clientId` to flow > pair and returns back to the rpc client a `FlowStateMachineHandle` future, created out of that pair.

`FlowStateMachineHandle` interface was introduced as a thinner `FlowStateMachine`. All `FlowStateMachine` properties used by call sites are moved into this new interface along with `clientId` and then `FlowStateMachine` extends it.

Introducing an acknowledgement method (`removeClientId`). Calling this method removes the < `clientId` to flow > pair on the node side and frees resources.
2020-07-16 10:52:08 +01:00
Adel El-Beik
2fa6b5a208
CORDA-3769: Switched attachments class loader cache to use caffeine (#6326)
* CORDA-3769: Switched attachments class loader cache to use caffeine with original implementation used by determinstic core.

* CORDA-3769: Removed default ctor arguments.

* CORDA-3769: Switched mapping function to Function type to avoid synthetic method being generated.

* CORDA-3769: Now using a cache created from NamedCacheFactory for the attachments class loader cache.

* CORDA-3769: Making detekt happy.

* CORDA-3769: The finality tests now check for UntrustedAttachmentsException which will actually happen in reality.

* CORDA-3769: Refactored after review comments.

* CORDA-3769: Removed the AttachmentsClassLoaderSimpleCacheImpl as DJVM does not need it. Also updated due to review comments.

* CORDA-3769: Removed the generic parameters from AttachmentsClassLoader.

* CORDA-3769: Removed unused imports.

* CORDA-3769: Updates from review comments.

* CORDA-3769: Updated following review comments. MigrationServicesForResolution now uses cache factory. Ctor updated for AttachmentsClassLoaderSimpleCacheImpl.

* CORDA-3769: Reduced max class loader cache size

* CORDA-3769: Fixed the attachments class loader cache size to a fixed default

* CORDA-3769: Switched attachments class loader size to be reduced by fixed value.
2020-07-16 09:58:36 +01:00
Chris Rankin
57e5f27961
CORDA-3906: Allow usage of SchedulableState in deterministic CorDapps. (#6457) 2020-07-15 09:09:28 +01:00