Commit Graph

898 Commits

Author SHA1 Message Date
Shams Asari
11d0054fcc
ENT-11055: Basic external verification (#7545)
* ENT-11055: Basic external verification

Introduction of the external transaction verifier, a separate JVM process for verifying `SignedTransaction`s. The end goal is for this verifier to be built with Kotlin 1.2 so that it creates a compatible verification environment for transactions with 4.11 contracts. For now however the verifier is built against Kotlin 1.8, same as the node.

External verification is enabled when the the system property `net.corda.node.verification.external` is set to `true`. When enabled, all verification requests made via `SignedTransaction.verify` are sent to the external verifier, regardless of the transaction content. It will do the vast bulk of the verification and then send the result back, namely if an exception occurred. If it did, then it's re-thrown in the node.

The external verifier is a stateless process, with no connection to the node's database. All transaction resolution information needed to create the relevant ledger transaction object are made to the node, which waits in a loop servicing these requests until it receives the result. The verifier Jar is embedded in the Corda node Jar, and is extracted and run when needed for the first time. The node opens up a local port for the verifier to communicate with, which is specified to the verifier in the process command line. This all means there is no extra configuration or deployment required to support external verification.

The existing code had some initial attempts and abstractions to support a future external verification feature. However,
they were either incorrect or didn't quite fit. One such example was `TransactionVerifierService`. It incorrectly operated on the `LedgerTransaction` level, which doesn't work since the transaction needs to be first serialised. Instead a new abstraction, `VerificationSupport` has been introduced, which represents all the operations needed to resolve and verify a `SignedTransaction`, essentially replacing `ServicesForResolution` (a lot of the changes are due to this). The external verifier implements this with a simple RPC mechanism, whilst the node needed a new (internal) `ServiceHub` abstraction, `VerifyingServiceHub`. `ServicesForResolution` hasn't been deleted since it's public API, however all classes implementing it must also implement `VerifyingServiceHub`. This is possible to do without breaking compatibility since `ServicesForResolution` is annotated with  `@DoNotImplement`.

Changes to `api-current.txt` were made due to the removal of `TransactionVerifierService`, which was clearly indicated as an internal class, and returning `TransactionBuilder.toLedgerTransactionWithContext` back to an internal method.

* Address review comments

* One bulk load states method

* Merge fix
2023-12-07 11:29:27 +00:00
Shams Asari
199e167639
ENT-11192: Migrate usage of @Test.expected annotation parameter (#7593)
Replaced usage of `@Test.expected` annotation parameter with more specific exception assertions. This is also needed to migrate away from the explicit timeouts in every tests.
2023-12-06 16:45:51 +00:00
Shams Asari
4cf5fe55dd
ENT-11099: Update Java compile source & target to 17 (#7594)
And removed unused `jdkClassifier` in build files.
2023-12-06 09:46:29 +00:00
Arshad Mahmood
1b4189b2b3 Added explicit publishing configuration for projects bundled as jars, this change was required when upgrading to kotlin 1.9.0 2023-11-08 10:48:10 +00:00
Arshad Mahmood
6dd33fb8f7 Upgrade to gradle 7.6, kotlin 1.8 and jdk 17
Major changes due to JDK 17:
1. JDK17 JCE Provider now has built-in support for eddsas, corda uses
   the bouncycastle (i2p) implementation. This PR removes the conflicting
   algorithms from the built-in JCE provider.

2. JavaScript scripting has been removed from the JDK, the corda log4j config was using
   scripting to conditionally output additional diagnostic info if the MDC
   was populated. This PR has removed the scripting.

3. The artifactory plug-ins used are now deprecated, this PR has removed them
   and uses the same code as Corda 5 for publishing to artifactory.

4. Javadoc generation has been modified to use the latest dokka plug-ins.

5. Gradle 7.6 has implemented an incredibly annoying change where transitive
   dependencies are not put on the compile classpath, so that they have to be
   explicitly added as dependencies to projects.

6. Mockito has been updated, which sadly meant that quite a few source files
   have to changes to use the new (org.mockito.kotlin) package name. This makes
   this PR appear much larger than it is.

7. A number of tests have been marked as ignored to get a green, broadly they fall
   into 3 classes.

   The first is related to crypto keypair tests, it appears some logic
   in the JDK prefers to use the SunJCE implementation and we prefer to use
   bouncycastle. I believe this issue can be fixed with better test setup.

   The second group is related to our use of a method called "uncheckedCast(..)",
   the purpose of this method was to get rid of the annoying unchecked cast compiler
   warning that would otherwise exist. It looks like the Kotlin 1.9 compiler type
   inference differs and at runtime sometimes the type it infers is "Void" which causes
   an exception at runtime. The simplest solution is to use an explicit cast instead of
   unchecked cast, Corda 5 have removed unchecked cast from their codebase.

   The third class are a number of ActiveMQ tests which appear to have a memory leak somewhere.
2023-11-06 10:24:17 +00:00
Jose Coll
be433c1fd0
ENT-10100 Filter out UNVERIFIED txns + move TransactionStatus package. (#7543) 2023-10-24 17:01:37 +01:00
Jose Coll
bc718088fe
ENT-10100 Changes required to support recovery of IN_FLIGHT transactions. (#7541) 2023-10-24 10:38:46 +01:00
r3-build
67837523ab Merging forward updates from release/os/4.10 to release/os/4.11 - 2023-10-06 2023-10-06 16:44:41 +00:00
r3-build
de4eda5a35 Merging forward updates from release/os/4.9 to release/os/4.10 - 2023-10-06 2023-10-06 13:47:28 +00:00
r3-build
1b332bb2e0 Merging forward updates from release/os/4.8 to release/os/4.9 - 2023-10-05 2023-10-05 14:18:04 +00:00
r3-build
4325131c27 Merging forward updates from release/os/4.7 to release/os/4.8 - 2023-10-05 2023-10-05 10:17:26 +00:00
Rick Parker
3cdb8901b4
Don't bind unless shared memory says it is free (#7520) 2023-10-05 11:16:12 +01:00
Jose Coll
b16c93e76b
ENT-6875 Deserialisation exception on output state can cause ledger inconsistency (#7476) 2023-09-20 08:44:56 +01:00
Jose Coll
32c3b0c517 Merge remote-tracking branch 'origin/release/os/4.11' into colljos/encryption_receiver_self_recovery 2023-08-22 12:31:29 +01:00
Shams Asari
4a7a4eb5bb
ENT-9876: Encrypting the ledger recovery participant distribution list (#7423) 2023-08-22 11:14:37 +01:00
Jose Coll
a6786769e5
ENT-10306 Determine whether to use 2PF based on the execution CorDapp TPV. (#7447) 2023-08-21 09:11:00 +01:00
Jose Coll
4a6e99556b Incorporating PR review feedback. 2023-08-18 17:22:42 +01:00
Jose Coll
daea257214 Merge remote-tracking branch 'origin/release/os/4.11' into colljos/encryption_receiver_self_recovery 2023-08-16 17:38:34 +01:00
Shams Asari
e0e4f51ba2
ENT-10285: Remove experimental DJVM support (#7442) 2023-08-09 08:44:32 +01:00
Shams Asari
de67ab7377 ENT-9876: Encrypting the ledger recovery participant distribution list 2023-07-25 14:59:43 +01:00
Ronan Browne
991aa9fff9 ES-853: Merging forward updates from 4.10 to 4.11 (2023-07-17) 2023-07-17 14:35:59 +01:00
Shams Asari
03a9a95216 Merge branch 'release/os/4.10' into shams-4.11-merge-6b655c0f
# Conflicts:
#	.github/workflows/check-pr-title.yml
#	node/src/main/kotlin/net/corda/node/internal/ServicesForResolutionImpl.kt
#	node/src/main/kotlin/net/corda/node/services/vault/NodeVaultService.kt
2023-07-17 10:17:38 +01:00
Connel McGovern
b01f85fb11 Merge branch 'release/os/4.9' into connelm/merge-4.9-4.10-2023-07-14
# Conflicts:
#	build.gradle
2023-07-14 18:19:55 +01:00
Tom Stark
58ecce1713
ENT-9875: New network parameters (#7398)
* ENT-9875: Added new network parameters

- Added `transactionRecoveryPeriod`
- Added `confidentialIdentityPreGenerationPeriod`

These new parameters are currently set to be nullable meaning they can be ignored and the duration if not specified will be null rather than, e.g., 0. This currently allows for nothing changing/breaking in the node-api

_Note: if these params can stay as nullable then the deprecated constructor might not even be needed (since the existing one will still work), needs to be discussed._
2023-07-14 12:02:21 +01:00
Ronan Browne
befae353ac ES-853: Merging forward updates from 4.8 to 4.9 (2023-07-13) 2023-07-13 15:25:55 +01:00
Shams Asari
3a6deeefa7 Merge branch 'release/os/4.9' into shams-4.10-merge-e6a80822
# Conflicts:
#	.github/workflows/check-pr-title.yml
#	.snyk
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/AMQPClient.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/AMQPClientSslErrorsTest.kt
#	node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt
2023-07-13 10:53:30 +01:00
Ronan Browne
6bcc843f19 ES-853: Merging forward updates from 4.7 to 4.8 (2023-07-13) 2023-07-13 07:07:17 +01:00
Ronan Browne
010a9a6c08 ES-853: Merging forward updates from 4.6 to 4.7 (2023-07-12) 2023-07-12 20:02:32 +01:00
Ronan Browne
b410cd2a5d
ES-853: update Artifactory refrences to new public location (#7416)
* ES-853: update artifactory references to new public location
2023-07-12 17:36:57 +01:00
Shams Asari
671ce9ef97 Merge branch 'os-4.8-merge-point' into shams-4.9-merge-0b8536b9
# Conflicts:
#	.github/workflows/check-pr-title.yml
#	node/src/main/kotlin/net/corda/node/services/messaging/NodeNettyAcceptorFactory.kt
2023-07-07 15:20:43 +01:00
Adel El-Beik
602190095c Merge remote-tracking branch 'origin/release/os/4.10' into adel/merge-from-os4.10-23jun-23 2023-06-23 11:39:02 +01:00
Shams Asari
0bfce451ea ENT-10013: Vault service refactoring backport 2023-06-06 16:17:58 +01:00
nargas-ritu
5510cb151b
Merge pull request #7375 from corda/shams-vault-page-overlap
ENT-9793: Added Page.previousPageAnchor to allow detection of vault changes whilst pages are loaded
2023-06-06 11:07:45 +01:00
Jose Coll
f791adf442
ENT-9924 (Follow-up) Update recording of transaction flow recovery metadata into Send/Receive transaction flows. (#7382) 2023-06-05 16:59:06 +01:00
Jose Coll
2c775bcc41
ENT-9924 Update recording of transaction flow recovery metadata into Send/Receive transaction flows. (#7374) 2023-06-02 16:05:28 +01:00
Shams Asari
5706f89639 Merge branch 'release/os/4.8' into shams-4.9-frwd-merge-a817218b
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/ArtemisTcpTransport.kt
#	node-api/src/test/kotlin/net/corda/nodeapi/internal/revocation/RevocationTest.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/CertificateRevocationListNodeTests.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/ProtonWrapperTests.kt
#	node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt
#	node/src/main/kotlin/net/corda/node/internal/artemis/BrokerJaasLoginModule.kt
#	node/src/main/kotlin/net/corda/node/internal/artemis/CertificateChainCheckPolicy.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/ArtemisMessagingServer.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/NodeNettyAcceptorFactory.kt
2023-06-01 16:37:57 +01:00
Adel El-Beik
d37df1572e Merge remote-tracking branch 'origin/release/os/4.9' into adel/merge-from-4.9 2023-05-31 12:27:41 +01:00
Adel El-Beik
e15f92b526
Merge pull request #7355 from corda/shams-crl-netty-threads-blocked
ENT-9806: Netty threads no longer blocked if CRL endpoints are unresponsive
2023-05-27 10:43:41 +01:00
Shams Asari
ed08b2c5de ENT-9793: Added Page.previousPageAnchor to allow detection of vault changes whilst pages are loaded 2023-05-26 12:52:06 +01:00
Jose Coll
2e29e36e01
ENT-9923 Ledger Recovery: split out recovery metadata into own database schema. (#7364) 2023-05-24 09:42:09 +01:00
Shams Asari
e6336666c2 Merge branch 'release/os/4.10' into shams-4.11-fwrd-merge-046ed0ac
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/crypto/X509Utilities.kt
2023-05-22 09:55:19 +01:00
Shams Asari
c065021328 ENT-8827: The ordering of vault query results is clobbered by ServiceHub.loadStates 2023-05-18 15:59:25 +01:00
Chris Cochrane
6e06630980
Upgraded sshd-common compile-time dependency 2023-05-18 11:25:46 +01:00
Shams Asari
0a617097be ENT-9806: Prevent Netty threads being blocked due to unresponsive CRL endpoints 2023-05-15 12:59:24 +01:00
Shams Asari
a37d09486f Merge branch 'release/os/4.9' into shams-4.10-fwrd-merge-7b7e3847 2023-05-15 09:55:31 +01:00
Shams Asari
151dc16a0e Merge branch 'release/os/4.8' into shams-4.9-frwd-merge-31a34e5a
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/ArtemisTcpTransport.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/NodeNettyAcceptorFactory.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/P2PMessagingClient.kt
2023-05-12 14:34:33 +01:00
Shams Asari
31a34e5a5c
ENT-9941: Improved Netty logging, especially of the embedded broker (#7365) 2023-05-12 10:11:09 +01:00
Jose Coll
c3e39a7052
ENT-9842 Re-factor 2PF to support issuance transactions (no notarisation) with observers. (#7349)
Re-factor 2PF to support issuance transactions (no notarisation) with observers.
2023-04-27 16:58:17 +01:00
Chris Cochrane
e12ab8194f
Merge branch 'release/os/4.7' into cc/merge/os-4.7-os-4.8/25-apr-2023
Fwd-merge from OS 4.7.
2023-04-25 14:54:21 +01:00
Chris Cochrane
91c38fa84b
Merge branch 'release/os/4.6' into cc/merge/os-4.6-os-4.7/24-apr-2023
Fwd-merge from OS 4.6.
2023-04-24 20:13:50 +01:00