Commit Graph

3356 Commits

Author SHA1 Message Date
Shams Asari
a7d0684fe7
ENT-11384: Cleanup JarScanningCordappLoader (#7664)
* It uses URLs when in fact CorDapps are jar files, and so should being Path. It also does URL equality, which is not recommended
* Address (very old) TODO of removing RestrictedURL, which is not needed

Also, back-ported some minor changes from https://github.com/corda/enterprise/pull/5057.
2024-01-29 13:44:14 +00:00
Adel El-Beik
d642ebfbd7
Merge pull request #7662 from corda/arshadm/ent-6914-disable-node-metadata
ENT-6914 Disabled module metadata generation for the node capsule
2024-01-25 14:10:38 +00:00
Shams Asari
63f8e220c8
ENT-11251: Upgrade to Kotlin language version 1.9 (#7660) 2024-01-25 13:51:19 +00:00
Arshad Mahmood
4ea42c4d75 ENT-6914 Disabled module metadata generation for the node capsule as it was generating invalid json 2024-01-25 12:11:47 +00:00
Chris Cochrane
975500d878
ENT-11351 - Compiler warnings pass 3 (#7659)
* More compiler warnings fixed

* Amended deprecation suppression annotations, as per review comments
2024-01-25 10:18:58 +00:00
Adel El-Beik
3abb218bca
Merge pull request #7654 from corda/shams-external-verifier-analyse-txs
ENT-11255: Scan attachments to determine if they are Kotlin 1.2 or later
2024-01-23 16:56:47 +00:00
Chris Cochrane
f15e6ec56a
ENT-11351 - Compiler warnings pass 2 (#7655)
* Addressed compiler warnings

* Removed unchecked cast fixes - not for this PR

* Sorted out detekt issues
2024-01-23 10:19:03 +00:00
Shams Asari
f30ba33929 ENT-11255: Scan attachments to determine if they are Kotlin 1.2 or later
The node now sends a transaction to the verifier if any of its attachments were compiled with Kotlin 1.2 (the net.corda.node.verification.external system property has been removed). It uses kotlinx-metadata to read the Kotlin metadata in the attachment to determine this. For now this scanning is done each time the attachment is loaded from the database.

The existing external verification integration tests were converted into smoke tests so that 4.11 nodes could be involved. This required various improvements to NodeProcess.Factory. A new JAVA_8_HOME environment variable, pointing to JDK 8, is required to run these tests.

There is still some follow-up work that needs to be done:

Sending transactions from a 4.11 node to a 4.12 node works, but not the other way round. A new WireTransaction component group needs to be introduced for storing 4.12 attachments so that they can be safely ignored by 4.11 nodes, and the 4.12 node needs to be able to load both 4.11 and 4.12 versions of the same contracts CorDapp so that they can be both attached to the transaction.
Even though attachments are cached when retrieved from the database, the Kotlin metadata version should be stored in the attachments db table, rather than being scanned each time.
Finally, VerificationService was refactored into NodeVerificationSupport and can be passed into SignedTransaction.verifyInternal, instead of needing the much heavier VerifyingServiceHub. This makes it easier for internal tools to verify transactions and spawn the verifier if necessary.
2024-01-22 11:31:51 +00:00
Arshad Mahmood
e5355d9e75 ENT-6914 Fix generated pom 2024-01-22 10:22:08 +00:00
Chris Cochrane
1ff853b421
ENT-11351 - Compiler warnings pass 1 (#7652)
* Removed warnings - pass 1

* Resolve detekt errors

* Properly compare X500 distinguished names
2024-01-19 10:26:50 +00:00
Balwant Kothari
795e61807d
ENT-11113 Fixed review comments 2024-01-18 00:34:59 +05:30
Balwant Kothari
13e13fd236
ENT-11113 Updating test case to user overrridden Sysout instead of mock 2024-01-18 00:15:46 +05:30
Balwant Kothari
49f35aa5ea
ENT-11113 Updating test case for accessing modifier as per JDK17 compatibility 2024-01-15 15:26:10 +05:30
Adel El-Beik
d9e0172bf7
Merge pull request #7643 from corda/bk/os_fixes_412
ENT-11113 Update instant time resolution in test cases
2024-01-12 18:26:25 +00:00
Balwant Kothari
22e96f1bda
ENT-11113 Instant default time resolution is nano but HashedDistributionList.PublicHeader default derialisation happens at millis resolution to passing time in millis resolution as input 2024-01-11 18:15:19 +05:30
Shams Asari
ccc605493d WIP 2024-01-10 10:47:32 +00:00
Shams Asari
2e63ca6264
ENT-11065: Remove the need for JVM flags in client code (#7635) 2024-01-03 11:22:03 +00:00
Shams Asari
406f7ff292
ENT-11056: Compile the external verifier using Kotlin 1.2 (#7622)
This requires Kotlin 1.2 versions of core and serialization (core-1.2 and serialization-1.2 respectively), which are just "shell" modules and which compile the existing source code with Kotlin 1.2. The 1.2 plugin does not work with the current version of Gradle and so the 1.2 compiler has to be called directly.

Now with two versions of Kotlin in the code base, each module needs to have its version manually specified to ensure a clean separation. Otherwise, the default Kotlin version can override 1.2 when needed.

Some of the code was tidied-up or improved to enable it to be cross-compiled. For post-1.2 APIs being used, they have been copied into core-1.2 with the same method signatures. OpenTelemetryComponent was moved to node-api, along with the dependency, to avoid also having a 1.2 version for the opentelemetry module.
2024-01-02 17:02:20 +00:00
Adel El-Beik
4791f0d84f
Merge pull request #7630 from corda/merge-release/os/4.11-release/os/4.12-2023-12-21-86
ENT-11118: Merging forward updates from release/os/4.11 to release/os/4.12 - 2023-12-21
2023-12-22 12:23:02 +00:00
Arshad Mahmood
26861ffd05 ENT-11295 ActiveMQ behaviour has changed so that CREATE_ADDRESS is perforned before CREATE_DURABLE_QUEUE in this situation 2023-12-21 15:03:09 +00:00
Balwant Kothari
d235e887fe
ENT-11113 Ignored JDK 17 related fixes for byteman related issues (#7626)
* Updated mockito version and removed ignored annotation to relevant test cases
2023-12-21 11:11:55 +00:00
r3-build
2eb1e9f426 Merging forward updates from release/os/4.11 to release/os/4.12 - 2023-12-21 2023-12-21 03:50:35 +00:00
Adel El-Beik
e815b381ef
Merge pull request #7617 from corda/merge-release/os/4.11-release/os/4.12-2023-12-15-79
ENT-11281: Merging forward updates from release/os/4.11 to release/os/4.12 - 2023-12-15
2023-12-19 13:15:51 +00:00
Adel El-Beik
7cbfc71a74
Merge pull request #7621 from corda/feature/arshad/ent-11294-fix-checkpoint-failing-test
ENT-11294  Fix failing tests in - FlowCheckpointVersionNodeStartupCheckTest
2023-12-19 11:26:10 +00:00
Jose Coll
5a807e1eed Detekt. 2023-12-19 09:25:43 +00:00
Jose Coll
b47d5ec5c7 Merge branch 'release/os/4.12' into merge-release/os/4.11-release/os/4.12-2023-12-15-79 2023-12-19 08:46:52 +00:00
Jose Coll
c1fe0e739a Resolve conflicts. 2023-12-19 08:46:26 +00:00
Arshad Mahmood
2a149e3aee ENT-11294 Handle kryo exception wrapped in KryoException 2023-12-18 16:12:34 +00:00
Adel El-Beik
422786dccc
Merge pull request #7608 from corda/shams-verification-service
ENT-11267: Introducing VerificationService, which implements VerificationSupport in terms of node-based services
2023-12-18 13:47:32 +00:00
Shams Asari
61a05a90eb
ENT-11155: Remove internal Kotlin utilities which have since been added after 1.2 (#7585)
This is mostly the `Path` extension functions in `PathUtils.kt`.
2023-12-18 12:05:08 +00:00
r3-build
16ade601ad Merging forward updates from release/os/4.10 to release/os/4.11 - 2023-12-16 2023-12-16 19:18:14 +00:00
r3-build
1cf66d4a8f Merging forward updates from release/os/4.9 to release/os/4.10 - 2023-12-15 2023-12-15 20:02:41 +00:00
Tom Stark
71e187dddb ENT-11118: Netty SSL handshake fix (#5028)
* ENT-11118: Added fix for netty SSL handshake fail
2023-12-15 17:16:15 +00:00
Shams Asari
a34932e887 ENT-11267: Introducing VerificationService, which implements VerificationSupport in terms of node-based services 2023-12-12 15:01:48 +00:00
Arshad Mahmood
c94f1d730c ENT-11271 Publish dependencies in the maven pom.xml 2023-12-12 13:24:15 +00:00
Adel El-Beik
6c4ab9a440
Merge pull request #7575 from corda/merge-release/os/4.11-release/os/4.12-2023-11-14-36
ENT-11130: Merging forward updates from release/os/4.11 to release/os/4.12 - 2023-11-14
2023-12-07 17:54:21 +00:00
Shams Asari
e2bcd0499e ENT-11263: Remove TooGenericExceptionCaught detekt rule 2023-12-07 13:36:21 +00:00
Shams Asari
11d0054fcc
ENT-11055: Basic external verification (#7545)
* ENT-11055: Basic external verification

Introduction of the external transaction verifier, a separate JVM process for verifying `SignedTransaction`s. The end goal is for this verifier to be built with Kotlin 1.2 so that it creates a compatible verification environment for transactions with 4.11 contracts. For now however the verifier is built against Kotlin 1.8, same as the node.

External verification is enabled when the the system property `net.corda.node.verification.external` is set to `true`. When enabled, all verification requests made via `SignedTransaction.verify` are sent to the external verifier, regardless of the transaction content. It will do the vast bulk of the verification and then send the result back, namely if an exception occurred. If it did, then it's re-thrown in the node.

The external verifier is a stateless process, with no connection to the node's database. All transaction resolution information needed to create the relevant ledger transaction object are made to the node, which waits in a loop servicing these requests until it receives the result. The verifier Jar is embedded in the Corda node Jar, and is extracted and run when needed for the first time. The node opens up a local port for the verifier to communicate with, which is specified to the verifier in the process command line. This all means there is no extra configuration or deployment required to support external verification.

The existing code had some initial attempts and abstractions to support a future external verification feature. However,
they were either incorrect or didn't quite fit. One such example was `TransactionVerifierService`. It incorrectly operated on the `LedgerTransaction` level, which doesn't work since the transaction needs to be first serialised. Instead a new abstraction, `VerificationSupport` has been introduced, which represents all the operations needed to resolve and verify a `SignedTransaction`, essentially replacing `ServicesForResolution` (a lot of the changes are due to this). The external verifier implements this with a simple RPC mechanism, whilst the node needed a new (internal) `ServiceHub` abstraction, `VerifyingServiceHub`. `ServicesForResolution` hasn't been deleted since it's public API, however all classes implementing it must also implement `VerifyingServiceHub`. This is possible to do without breaking compatibility since `ServicesForResolution` is annotated with  `@DoNotImplement`.

Changes to `api-current.txt` were made due to the removal of `TransactionVerifierService`, which was clearly indicated as an internal class, and returning `TransactionBuilder.toLedgerTransactionWithContext` back to an internal method.

* Address review comments

* One bulk load states method

* Merge fix
2023-12-07 11:29:27 +00:00
Balwant Kothari
74ca2c6734
ENT-10560 JDK 17 Test Cases Fixes (#7598)
* Updated mockito version and removed ignored annotation to relevant test cases

* Updated mockito version and removed ignored annotation to relevant test cases
2023-12-07 10:46:56 +00:00
Shams Asari
199e167639
ENT-11192: Migrate usage of @Test.expected annotation parameter (#7593)
Replaced usage of `@Test.expected` annotation parameter with more specific exception assertions. This is also needed to migrate away from the explicit timeouts in every tests.
2023-12-06 16:45:51 +00:00
Shams Asari
4cf5fe55dd
ENT-11099: Update Java compile source & target to 17 (#7594)
And removed unused `jdkClassifier` in build files.
2023-12-06 09:46:29 +00:00
Rick Parker
59ff476a40
ENT-11050: Support flag to indicate if in transaction back chain resolution, and use it to drop deserialization errors to warn (#7591)
* Support flag to indicate if in transaction back chain resolution, and use it to drop deserialization errors to warn
* Fix and rename test that has different outcome now
2023-11-28 11:15:21 +00:00
Chris Cochrane
d8b19de46e
Renamed index to keep Oracle happy 2023-11-21 11:48:37 +00:00
r3-build
45bd9c95d8 Merging forward updates from release/os/4.11 to release/os/4.12 - 2023-11-14 2023-11-14 13:58:20 +00:00
Chris Cochrane
9065c1703a
Updated changeset to match Enterprise 2023-11-13 17:07:49 +00:00
Chris Cochrane
974d81cf41
Revert netty version 2023-11-10 12:41:44 +00:00
Jose Coll
e94e02fc4b Update index name (to meet Oracle max length limitation of 30 chars). 2023-11-09 11:25:56 +00:00
Arshad Mahmood
1b4189b2b3 Added explicit publishing configuration for projects bundled as jars, this change was required when upgrading to kotlin 1.9.0 2023-11-08 10:48:10 +00:00
Jose Coll
c445c72d3a
ENT-10110 Store Ledger Recovery records only if the transaction was locally stored in the first place. (#7564) 2023-11-08 08:52:25 +00:00
Chris Cochrane
3c34caabc3
ENT-10888,ENT-11108 - Security vulnerabilties (#7556)
* Security vulnerabilties
* Upgraded netty
* ENT-11108: Upgraded tcnative depedency to 2.2.224.
* Added architecture classifiers for netty-tcnative
2023-11-07 10:46:03 +00:00
Arshad Mahmood
6dd33fb8f7 Upgrade to gradle 7.6, kotlin 1.8 and jdk 17
Major changes due to JDK 17:
1. JDK17 JCE Provider now has built-in support for eddsas, corda uses
   the bouncycastle (i2p) implementation. This PR removes the conflicting
   algorithms from the built-in JCE provider.

2. JavaScript scripting has been removed from the JDK, the corda log4j config was using
   scripting to conditionally output additional diagnostic info if the MDC
   was populated. This PR has removed the scripting.

3. The artifactory plug-ins used are now deprecated, this PR has removed them
   and uses the same code as Corda 5 for publishing to artifactory.

4. Javadoc generation has been modified to use the latest dokka plug-ins.

5. Gradle 7.6 has implemented an incredibly annoying change where transitive
   dependencies are not put on the compile classpath, so that they have to be
   explicitly added as dependencies to projects.

6. Mockito has been updated, which sadly meant that quite a few source files
   have to changes to use the new (org.mockito.kotlin) package name. This makes
   this PR appear much larger than it is.

7. A number of tests have been marked as ignored to get a green, broadly they fall
   into 3 classes.

   The first is related to crypto keypair tests, it appears some logic
   in the JDK prefers to use the SunJCE implementation and we prefer to use
   bouncycastle. I believe this issue can be fixed with better test setup.

   The second group is related to our use of a method called "uncheckedCast(..)",
   the purpose of this method was to get rid of the annoying unchecked cast compiler
   warning that would otherwise exist. It looks like the Kotlin 1.9 compiler type
   inference differs and at runtime sometimes the type it infers is "Void" which causes
   an exception at runtime. The simplest solution is to use an explicit cast instead of
   unchecked cast, Corda 5 have removed unchecked cast from their codebase.

   The third class are a number of ActiveMQ tests which appear to have a memory leak somewhere.
2023-11-06 10:24:17 +00:00
Jose Coll
361cd678cb
ENT-10531 Add additional index on distribution records tables. Sync table names. (#7548) 2023-11-02 18:31:43 +00:00
Jose Coll
be433c1fd0
ENT-10100 Filter out UNVERIFIED txns + move TransactionStatus package. (#7543) 2023-10-24 17:01:37 +01:00
Jose Coll
bc718088fe
ENT-10100 Changes required to support recovery of IN_FLIGHT transactions. (#7541) 2023-10-24 10:38:46 +01:00
Jose Coll
6a2bad8077
ENT-10110 Back-port changes from ENT + additional clean-up (#7532) 2023-10-17 07:03:49 +01:00
Jose Coll
a66042ec09
ENT-10110 Ledger Recovery database table tweaks (#7528) 2023-10-10 17:41:49 +01:00
Jose Coll
fdf8d5344b
ENT-10110 Ledger Recovery tweaks (#7519) 2023-10-05 16:53:20 +01:00
Jose Coll
4e355b953b
ENT-10110 timeDiscriminator must also be shared for DR synchronisation. (#7517) 2023-10-02 17:28:44 +01:00
Adel El-Beik
b7e484c3d8 ENT-9943: Missed test that had assumed party id was still a long, now updated to string. 2023-09-29 16:14:10 +01:00
Adel El-Beik
3a23f60199
ENT-9943: Now use SecureHash to represent CordaX500Name in distributi… (#7501) 2023-09-29 09:03:00 +01:00
Adel El-Beik
238608224b
ENT-10607: Reduced length of distribution records table name to under… (#7505)
* ENT-10607: Reduced length of distribution records table name to under 30 chars for Oracle.

* ENT-10607: Need to update the entity classes as well.
2023-09-27 14:01:44 +01:00
Shams Asari
ea4bcd5369 ENT-10811: Removed index on primary key column in node_aes_encryption_keys table
The index is redundant on the primary key, and causes an issue in the schema migration for some databases, such as Oracle.
2023-09-22 15:09:33 +01:00
Jose Coll
b16c93e76b
ENT-6875 Deserialisation exception on output state can cause ledger inconsistency (#7476) 2023-09-20 08:44:56 +01:00
r3-build
13e32ae978 Merging forward updates from release/os/4.10 to release/os/4.11 - 2023-09-08 2023-09-08 16:48:35 +00:00
r3-build
17f5cd5524 Merging forward updates from release/os/4.9 to release/os/4.10 - 2023-09-08 2023-09-08 14:56:56 +00:00
Shams Asari
fb4247d5c9
ENT-10756: Use normal select count query for totalStatesAvailable (#7492)
Some DBs do not like the use of `distinct` in the count query for the `Page.totalStatesAvailable` result. It turns out it's not needed anyway and a non-distinct select count works as well.
2023-09-08 15:55:23 +01:00
r3-build
33798df51d Merging forward updates from release/os/4.10 to release/os/4.11 - 2023-09-04 2023-09-04 10:21:44 +00:00
r3-build
612cfcdb32 Merging forward updates from release/os/4.9 to release/os/4.10 - 2023-09-01 2023-09-01 20:47:47 +00:00
Adel El-Beik
134ff60af8 ENT-10694: Switch the noisy SSL handshake to trace logging. But allow it to be enabled again with a system property. 2023-08-31 18:33:34 +01:00
Adel El-Beik
3ae6db8c04
ENT-10122: Now also added consuming transaction id in the resolveAndMakeUpdate code path. (#7459) 2023-08-30 11:33:01 +01:00
Shams Asari
6edb1b779c Placate Detekt 2023-08-23 15:03:08 +01:00
Shams Asari
fe416879cd Merge branch 'release/os/4.10' into shams-4.11-merge-53fe0e6c
# Conflicts:
#	node/src/main/kotlin/net/corda/node/services/vault/NodeVaultService.kt
2023-08-23 14:56:46 +01:00
Jose Coll
4fef01a5b0 Clean-up. 2023-08-22 16:03:13 +01:00
Jose Coll
32c3b0c517 Merge remote-tracking branch 'origin/release/os/4.11' into colljos/encryption_receiver_self_recovery 2023-08-22 12:31:29 +01:00
Shams Asari
46914a2b35 Merge branch 'release/os/4.9' into shams-4.10-merge-c8135331 2023-08-22 11:24:38 +01:00
Shams Asari
4a7a4eb5bb
ENT-9876: Encrypting the ledger recovery participant distribution list (#7423) 2023-08-22 11:14:37 +01:00
Shams Asari
fb303a20bf Merge branch 'release/os/4.8' into shams-4.9-merge-5cdbec9d
# Conflicts:
#	node/src/main/kotlin/net/corda/node/services/vault/NodeVaultService.kt
2023-08-21 14:18:09 +01:00
Shams Asari
5cdbec9ddf
ENT-6876: Optimised vault query to not query for total state count if the first page isn't full (#7449) 2023-08-21 10:30:42 +01:00
Jose Coll
a6786769e5
ENT-10306 Determine whether to use 2PF based on the execution CorDapp TPV. (#7447) 2023-08-21 09:11:00 +01:00
Jose Coll
6a7e9000a4 Detekt 2023-08-18 17:26:22 +01:00
Jose Coll
4a6e99556b Incorporating PR review feedback. 2023-08-18 17:22:42 +01:00
Jose Coll
06e43eb9e2 Fixes following merge. 2023-08-17 08:47:58 +01:00
Jose Coll
f565232f36 Fix compilation errors following merge. 2023-08-16 18:05:18 +01:00
Jose Coll
9b7affa6b3 Fix compilation errors following merge. 2023-08-16 17:40:33 +01:00
Jose Coll
daea257214 Merge remote-tracking branch 'origin/release/os/4.11' into colljos/encryption_receiver_self_recovery 2023-08-16 17:38:34 +01:00
Jose Coll
492373d180 Introduction of Sender and Receiver Distribution Lists to support receiver self-recovery mode. 2023-08-16 17:02:58 +01:00
Jose Coll
0130914c89
ENT-9927 Ledger Recovery: synchronise changes from ENT -> OS. (#7445) 2023-08-15 15:32:00 +01:00
Shams Asari
e0e4f51ba2
ENT-10285: Remove experimental DJVM support (#7442) 2023-08-09 08:44:32 +01:00
Shams Asari
32af6f5c2d
ENT-10416: Rename ledger recovery tx_id columns to transaction_id (#7444)
This is so that the node archiving service, which scans for tables containing "transaction_id" column, can automatically archive the sender and receiver distribution record information with the transaction.
2023-08-09 08:43:21 +01:00
Adel El-Beik
eccb9b4af6
Merge pull request #7430 from corda/adel/ENT-10122
ENT-10122: Add consuming transaction id to vault states table.
2023-08-07 13:35:04 +01:00
Adel El-Beik
f543e47652 ENT-10371: Fix unit test failure in CordaServiceLifecycleTests. 2023-08-02 13:39:04 +01:00
Adel El-Beik
3465917a93 ENT-10122: Fixed detekt issue. 2023-08-01 15:48:34 +01:00
Adel El-Beik
c614b21a2a ENT-10122: Added annotation for backwards compatibility and added test. 2023-08-01 15:11:21 +01:00
Shams Asari
4b7e2a3995 ENT-10045: Fix vault query bug on externalId and mapping with multiple keys 2023-07-26 12:46:27 +01:00
Shams Asari
de67ab7377 ENT-9876: Encrypting the ledger recovery participant distribution list 2023-07-25 14:59:43 +01:00
Jose Coll
7d1d2297e7
ENT-10289 Ensure Sender and Receiver Distribution records share the same timestamp (#7437) 2023-07-20 09:55:44 +01:00
Adel El-Beik
aa9e41c7c2 ENT-10122: Updated tests to include consuming transaction id in the Vault.Update check. 2023-07-19 16:36:39 +01:00
Adel El-Beik
60bb4c58f2 ENT-10122: Made the consuming tx id field nullable, added missing changelog file. 2023-07-18 17:45:53 +01:00
Adel El-Beik
669d6590af ENT-10122: Add consuming transaction id to vault states table. 2023-07-17 17:58:31 +01:00
Shams Asari
03a9a95216 Merge branch 'release/os/4.10' into shams-4.11-merge-6b655c0f
# Conflicts:
#	.github/workflows/check-pr-title.yml
#	node/src/main/kotlin/net/corda/node/internal/ServicesForResolutionImpl.kt
#	node/src/main/kotlin/net/corda/node/services/vault/NodeVaultService.kt
2023-07-17 10:17:38 +01:00