Commit Graph

617 Commits

Author SHA1 Message Date
Shams Asari
b3265314ce ENT-11445: Support legacy contract CorDapp dependencies
The `TransactionBuilder` has been updated to look for any missing dependencies to legacy contract attachments, in the same way it does for missing dependencies for CorDapps in the "cordapps" directory,

Since `TransactionBuilder` does verification on the `WireTransaction` and not a `SignedTransaction`, much of the verification logic in `SignedTransaction` had to moved to `WireTransaction` to allow the external verifier to be involved. The external verifier receives a `CoreTransaction` to verify instead of a `SignedTransaction`. `SignedTransaction.verify` does the signature checks first in-process, before then delegating the reset of the verification to the `CoreTransaction`.

A legacy contract dependency is defined as an attachment containing the missing class which isn't also a non-legacy Cordapp (i.e. a CorDapp which isn't in the "cordapp" directory).
2024-03-13 10:58:11 +00:00
Shams Asari
0091807c2f ENT-11101: Fix all crypto issues introduced by Java 17 upgrade
The various crypto tests that were previously ignored have been re-enabled.

The abandoned i2p EdDSA library has been replaced with native support that was added in Java 15.

Java 17 (via the `SunEC` provider) does not support the secp256k1 curve (one of the two ECDSA curves supported in Corda). This would not normally have been an issue as secp256k1 is already taken care of by Bouncy Castle. However, this only works if the `Crypto` API is used or if `”BC”` is explicitly specified as the provider (e.g. `Signature.getInstance(“SHA256withECDSA”, “BC”)`). If no provider is specified, which is what is more common, and actually what the Java docs recommend, then this doesn’t work as the `SunEC` provider is selected. To resolve this, a custom provider was created, installed just in front of `SunEC`, which “augments” `SunEC` by delegating to Bouncy Castle if keys or parameters for secp256k1 are encountered.

`X509Utilities.createCertificate` now calls `X509Certificate.verify()` to verify the created certificate, rather than using the Bouncy Castle API. This is more representative of how certificates will be verified (e.g. during SSL handshake) and weeds out other issues (such as unsupported curve error for secp256k1).

`BCCryptoService` has been renamed to `DefaultCryptoService` as it no longer explicitly uses Bouncy Castle but rather uses the installed security providers. This was done to fix a failing test. Further, `BCCryptoService` was already relying on the installed providers in some places.

The hack to get Corda `SecureRandom` working was also resolved. Also, as an added bonus, tests which ignored `SPHINCS256_SHA256` have been reinstated.

Note, there is a slightly inconsistency between how EdDSA and ECDSA keys are handled (and also RSA). For the later, Bouncy Castle is preferred, and methods such as `toSupportedKey*` will convert any JDK class to Bouncy Castle. For EdDSA the preference is the JDK (`SunEC`). However, this is simply a continuation of the previous preference of the i2p library over Bouncy Castle.
2024-03-04 13:29:49 +00:00
Shams Asari
200333b198 ENT-11355: Backwards compatibility with older nodes via new attachments component group 2024-02-19 17:19:49 +00:00
Shams Asari
f30ba33929 ENT-11255: Scan attachments to determine if they are Kotlin 1.2 or later
The node now sends a transaction to the verifier if any of its attachments were compiled with Kotlin 1.2 (the net.corda.node.verification.external system property has been removed). It uses kotlinx-metadata to read the Kotlin metadata in the attachment to determine this. For now this scanning is done each time the attachment is loaded from the database.

The existing external verification integration tests were converted into smoke tests so that 4.11 nodes could be involved. This required various improvements to NodeProcess.Factory. A new JAVA_8_HOME environment variable, pointing to JDK 8, is required to run these tests.

There is still some follow-up work that needs to be done:

Sending transactions from a 4.11 node to a 4.12 node works, but not the other way round. A new WireTransaction component group needs to be introduced for storing 4.12 attachments so that they can be safely ignored by 4.11 nodes, and the 4.12 node needs to be able to load both 4.11 and 4.12 versions of the same contracts CorDapp so that they can be both attached to the transaction.
Even though attachments are cached when retrieved from the database, the Kotlin metadata version should be stored in the attachments db table, rather than being scanned each time.
Finally, VerificationService was refactored into NodeVerificationSupport and can be passed into SignedTransaction.verifyInternal, instead of needing the much heavier VerifyingServiceHub. This makes it easier for internal tools to verify transactions and spawn the verifier if necessary.
2024-01-22 11:31:51 +00:00
Jose Coll
0bcebd3dc1 Detekt. 2023-12-19 10:02:12 +00:00
Jose Coll
b47d5ec5c7 Merge branch 'release/os/4.12' into merge-release/os/4.11-release/os/4.12-2023-12-15-79 2023-12-19 08:46:52 +00:00
Jose Coll
c1fe0e739a Resolve conflicts. 2023-12-19 08:46:26 +00:00
r3-build
a475c275db Merging forward updates from release/os/4.10 to release/os/4.11 - 2023-12-15 2023-12-15 11:41:19 +00:00
r3-build
27759ea74f Merging forward updates from release/os/4.9 to release/os/4.10 - 2023-12-14 2023-12-14 17:51:03 +00:00
r3-build
40a96c5760 Merging forward updates from release/os/4.8 to release/os/4.9 - 2023-12-14 2023-12-14 17:06:04 +00:00
r3-build
0dbdd08984 Merging forward updates from release/os/4.7 to release/os/4.8 - 2023-12-14 2023-12-14 15:30:28 +00:00
Maciej Swierad
97310a8895 ENT-11281/Activating forward merge logic 2023-12-14 14:57:13 +00:00
Arshad Mahmood
91d4c33513 ENT-11264 Fixed initialization of field serializer 2023-12-13 16:34:44 +00:00
Shams Asari
11d0054fcc
ENT-11055: Basic external verification (#7545)
* ENT-11055: Basic external verification

Introduction of the external transaction verifier, a separate JVM process for verifying `SignedTransaction`s. The end goal is for this verifier to be built with Kotlin 1.2 so that it creates a compatible verification environment for transactions with 4.11 contracts. For now however the verifier is built against Kotlin 1.8, same as the node.

External verification is enabled when the the system property `net.corda.node.verification.external` is set to `true`. When enabled, all verification requests made via `SignedTransaction.verify` are sent to the external verifier, regardless of the transaction content. It will do the vast bulk of the verification and then send the result back, namely if an exception occurred. If it did, then it's re-thrown in the node.

The external verifier is a stateless process, with no connection to the node's database. All transaction resolution information needed to create the relevant ledger transaction object are made to the node, which waits in a loop servicing these requests until it receives the result. The verifier Jar is embedded in the Corda node Jar, and is extracted and run when needed for the first time. The node opens up a local port for the verifier to communicate with, which is specified to the verifier in the process command line. This all means there is no extra configuration or deployment required to support external verification.

The existing code had some initial attempts and abstractions to support a future external verification feature. However,
they were either incorrect or didn't quite fit. One such example was `TransactionVerifierService`. It incorrectly operated on the `LedgerTransaction` level, which doesn't work since the transaction needs to be first serialised. Instead a new abstraction, `VerificationSupport` has been introduced, which represents all the operations needed to resolve and verify a `SignedTransaction`, essentially replacing `ServicesForResolution` (a lot of the changes are due to this). The external verifier implements this with a simple RPC mechanism, whilst the node needed a new (internal) `ServiceHub` abstraction, `VerifyingServiceHub`. `ServicesForResolution` hasn't been deleted since it's public API, however all classes implementing it must also implement `VerifyingServiceHub`. This is possible to do without breaking compatibility since `ServicesForResolution` is annotated with  `@DoNotImplement`.

Changes to `api-current.txt` were made due to the removal of `TransactionVerifierService`, which was clearly indicated as an internal class, and returning `TransactionBuilder.toLedgerTransactionWithContext` back to an internal method.

* Address review comments

* One bulk load states method

* Merge fix
2023-12-07 11:29:27 +00:00
Shams Asari
755c7b73b0
ENT-11111: Reverted exposure of internal ConcurrencyUtils method (#7586) 2023-12-06 09:55:35 +00:00
Connel McGovern
2adb502405
Merge branch 'release/os/4.11' into merge-release/os/4.10-release/os/4.11-2023-11-20-9 2023-11-21 09:16:06 +00:00
r3-build
5e9fdb7b2b Merging forward updates from release/os/4.9 to release/os/4.10 - 2023-11-20 2023-11-20 14:29:12 +00:00
Connel McGovern
7c681e025d ES-1131: Re-add authenticateGradleWrapper 2023-11-20 11:06:46 +00:00
Connel McGovern
fa899c55c2 ES-1131: Re-add authenticateGradleWrapper 2023-11-20 11:05:54 +00:00
Ronan Browne
18d98760b1
Merge branch 'release/os/4.9' into merge-release/os/4.8-release/os/4.9-2023-11-19-6 2023-11-19 19:56:09 +00:00
Adel El-Beik
548242e3cd
Merge branch 'release/os/4.8' into merge-release/os/4.7-release/os/4.8-2023-11-17-6 2023-11-18 18:10:07 +00:00
Connel McGovern
5b846406c8
ES-1331: Enable Gradle Enterprise build scans & remote caching (#7574)
* ES-1131: Enable Gradle Enterprise build scans & remote caching
2023-11-17 17:41:45 +00:00
Arshad Mahmood
1614bd5a63 Updated api definitions due to format change, 2023-11-06 10:45:00 +00:00
Arshad Mahmood
6dd33fb8f7 Upgrade to gradle 7.6, kotlin 1.8 and jdk 17
Major changes due to JDK 17:
1. JDK17 JCE Provider now has built-in support for eddsas, corda uses
   the bouncycastle (i2p) implementation. This PR removes the conflicting
   algorithms from the built-in JCE provider.

2. JavaScript scripting has been removed from the JDK, the corda log4j config was using
   scripting to conditionally output additional diagnostic info if the MDC
   was populated. This PR has removed the scripting.

3. The artifactory plug-ins used are now deprecated, this PR has removed them
   and uses the same code as Corda 5 for publishing to artifactory.

4. Javadoc generation has been modified to use the latest dokka plug-ins.

5. Gradle 7.6 has implemented an incredibly annoying change where transitive
   dependencies are not put on the compile classpath, so that they have to be
   explicitly added as dependencies to projects.

6. Mockito has been updated, which sadly meant that quite a few source files
   have to changes to use the new (org.mockito.kotlin) package name. This makes
   this PR appear much larger than it is.

7. A number of tests have been marked as ignored to get a green, broadly they fall
   into 3 classes.

   The first is related to crypto keypair tests, it appears some logic
   in the JDK prefers to use the SunJCE implementation and we prefer to use
   bouncycastle. I believe this issue can be fixed with better test setup.

   The second group is related to our use of a method called "uncheckedCast(..)",
   the purpose of this method was to get rid of the annoying unchecked cast compiler
   warning that would otherwise exist. It looks like the Kotlin 1.9 compiler type
   inference differs and at runtime sometimes the type it infers is "Void" which causes
   an exception at runtime. The simplest solution is to use an explicit cast instead of
   unchecked cast, Corda 5 have removed unchecked cast from their codebase.

   The third class are a number of ActiveMQ tests which appear to have a memory leak somewhere.
2023-11-06 10:24:17 +00:00
r3-build
2f6d226f9d Merging forward updates from release/os/4.11 to release/os/4.12 - 2023-10-27 2023-10-27 12:29:00 +00:00
Jose Coll
be433c1fd0
ENT-10100 Filter out UNVERIFIED txns + move TransactionStatus package. (#7543) 2023-10-24 17:01:37 +01:00
r3-build
5ae5ca95ea Merging forward updates from release/os/4.11 to release/os/4.12 - 2023-10-17 2023-10-17 06:05:08 +00:00
Jose Coll
6a2bad8077
ENT-10110 Back-port changes from ENT + additional clean-up (#7532) 2023-10-17 07:03:49 +01:00
Ronan Browne
1fdc23692a ES-1351: set up 4.12 branch 2023-09-14 11:13:15 +01:00
Waldemar Zurowski
7175d56f6c ES-1313: Merging forward updates from release/os/4.10 to release/os/4.11
Conflicts:
* .ci/dev/forward-merge/Jenkinsfile
2023-09-08 09:09:58 +01:00
Waldemar Zurowski
9f395b5ec3 ES-1313: Merging forward updates from release/os/4.9 to release/os/4.10
Conflicts:
* .ci/dev/forward-merge/Jenkinsfile
2023-09-07 20:36:14 +01:00
Waldemar Zurowski
a7274c4ec2 ES-1313: Merging forward updates from release/os/4.8 to release/os/4.9
Conflicts:
* .ci/dev/forward-merge/Jenkinsfile
2023-09-07 18:30:28 +01:00
Waldemar Zurowski
fd55e4c8d4 ES-1313: Merging forward updates from release/os/4.7 to release/os/4.8
Conflicts:
* .ci/dev/forward-merge/Jenkinsfile
2023-09-07 16:08:20 +01:00
Waldemar Zurowski
1c903b0ff2 ES-1313: Merging forward updates from release/os/4.6 to release/os/4.7
Conflicts:
* .ci/dev/forward-merge/Jenkinsfile
2023-09-07 14:06:10 +01:00
Waldemar Żurowski
dfe0ed294e
ES-1313: changes for better future forward merges (#7486)
* add more comments as a form of documentation
* move variable names away from calling pipeline for better experience
of future forward merges if pipeline parameters were to change
2023-09-07 14:39:02 +02:00
Waldemar Zurowski
658e7217a3 ES-1313: Merging forward updates from release/os/4.6 to release/os/4.7
Conflicts:

* .ci/dev/forward-merge/Jenkinsfile
2023-09-07 11:27:57 +01:00
Ronan Browne
8e2115293a
ES-1313: Update Jenkinsfile for slack channel (#7484) 2023-09-07 11:49:33 +02:00
Waldemar Żurowski
d7a5428665
Updated for new branch 2023-09-04 12:08:52 +01:00
r3-build
33798df51d Merging forward updates from release/os/4.10 to release/os/4.11 - 2023-09-04 2023-09-04 10:21:44 +00:00
Waldemar Żurowski
8a01b25684
updated forward merger for current branch
* updated originBranch to match current branch
* updated targetBranch to match branch with the next version
19:11
2023-09-01 22:50:29 +02:00
r3-build
612cfcdb32 Merging forward updates from release/os/4.9 to release/os/4.10 - 2023-09-01 2023-09-01 20:47:47 +00:00
Waldemar Żurowski
3a093ef324
updated forward merged to new branch 2023-09-01 18:25:46 +02:00
r3-build
313d342b02 Merging forward updates from release/os/4.8 to release/os/4.9 - 2023-09-01 2023-09-01 16:23:11 +00:00
Waldemar Żurowski
3d8636c716
updated for new branch 2023-09-01 14:55:04 +02:00
r3-build
fe9483d331 Merging forward updates from release/os/4.7 to release/os/4.8 - 2023-09-01 2023-09-01 12:47:45 +00:00
Waldemar Żurowski
267b500c93
updated forward merged to new branch 2023-09-01 11:56:24 +02:00
r3-build
8349675a23 Merging forward updates from release/os/4.6 to release/os/4.7 - 2023-09-01 2023-09-01 09:51:40 +00:00
Waldemar Żurowski
b0f86e26f9
ES-412: added automatic forward merge for Corda 4 (#7469)
* uses an already existing shared pipeline
* will start automatic PR creation for changes between 4.6 and 4.7
2023-09-01 10:50:47 +01:00
Waldemar Zurowski
849f8666a6 INFRA-1804: Merging forward updates from release/os/4.7 to release/os/4.8 2023-08-30 16:40:20 +01:00
Waldemar Zurowski
5daa091fdc INFRA-1804: Merging forward updates from release/os/4.6 to release/os/4.7 2023-08-30 14:30:31 +01:00