Commit Graph

816 Commits

Author SHA1 Message Date
Arshad Mahmood
6dd33fb8f7 Upgrade to gradle 7.6, kotlin 1.8 and jdk 17
Major changes due to JDK 17:
1. JDK17 JCE Provider now has built-in support for eddsas, corda uses
   the bouncycastle (i2p) implementation. This PR removes the conflicting
   algorithms from the built-in JCE provider.

2. JavaScript scripting has been removed from the JDK, the corda log4j config was using
   scripting to conditionally output additional diagnostic info if the MDC
   was populated. This PR has removed the scripting.

3. The artifactory plug-ins used are now deprecated, this PR has removed them
   and uses the same code as Corda 5 for publishing to artifactory.

4. Javadoc generation has been modified to use the latest dokka plug-ins.

5. Gradle 7.6 has implemented an incredibly annoying change where transitive
   dependencies are not put on the compile classpath, so that they have to be
   explicitly added as dependencies to projects.

6. Mockito has been updated, which sadly meant that quite a few source files
   have to changes to use the new (org.mockito.kotlin) package name. This makes
   this PR appear much larger than it is.

7. A number of tests have been marked as ignored to get a green, broadly they fall
   into 3 classes.

   The first is related to crypto keypair tests, it appears some logic
   in the JDK prefers to use the SunJCE implementation and we prefer to use
   bouncycastle. I believe this issue can be fixed with better test setup.

   The second group is related to our use of a method called "uncheckedCast(..)",
   the purpose of this method was to get rid of the annoying unchecked cast compiler
   warning that would otherwise exist. It looks like the Kotlin 1.9 compiler type
   inference differs and at runtime sometimes the type it infers is "Void" which causes
   an exception at runtime. The simplest solution is to use an explicit cast instead of
   unchecked cast, Corda 5 have removed unchecked cast from their codebase.

   The third class are a number of ActiveMQ tests which appear to have a memory leak somewhere.
2023-11-06 10:24:17 +00:00
Chris Cochrane
7f867a0a49
ENT-11038 - Redirect Liquibase logging via Slf4j (#7552) 2023-10-30 15:41:21 +00:00
Shams Asari
4a7a4eb5bb
ENT-9876: Encrypting the ledger recovery participant distribution list (#7423) 2023-08-22 11:14:37 +01:00
Shams Asari
825a970b92
ENT-6750: Checkpoint serialisation to support primitive void.class (#7448) 2023-08-21 09:49:39 +01:00
Shams Asari
e0e4f51ba2
ENT-10285: Remove experimental DJVM support (#7442) 2023-08-09 08:44:32 +01:00
Shams Asari
03a9a95216 Merge branch 'release/os/4.10' into shams-4.11-merge-6b655c0f
# Conflicts:
#	.github/workflows/check-pr-title.yml
#	node/src/main/kotlin/net/corda/node/internal/ServicesForResolutionImpl.kt
#	node/src/main/kotlin/net/corda/node/services/vault/NodeVaultService.kt
2023-07-17 10:17:38 +01:00
Shams Asari
3a6deeefa7 Merge branch 'release/os/4.9' into shams-4.10-merge-e6a80822
# Conflicts:
#	.github/workflows/check-pr-title.yml
#	.snyk
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/AMQPClient.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/AMQPClientSslErrorsTest.kt
#	node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt
2023-07-13 10:53:30 +01:00
Shams Asari
671ce9ef97 Merge branch 'os-4.8-merge-point' into shams-4.9-merge-0b8536b9
# Conflicts:
#	.github/workflows/check-pr-title.yml
#	node/src/main/kotlin/net/corda/node/services/messaging/NodeNettyAcceptorFactory.kt
2023-07-07 15:20:43 +01:00
Chris Cochrane
0f2312a201
ENT-8983 - Postgres migration failure (#7408)
* Set DB transaction isolation level only if its going to change; upgraded dependencies

* Removed duplicate changeset
2023-07-07 13:29:35 +01:00
Shams Asari
5b3180bf7b ENT-10016: Give all node threads descriptive names 2023-06-09 11:17:26 +01:00
Shams Asari
4dcd9245d3 ENT-9806: Using Artemis SSLContextFactory service to pass in custom TrustManagerFactory. This removes the need to copy code from NettyAcceptor. 2023-06-02 15:10:58 +01:00
Shams Asari
5706f89639 Merge branch 'release/os/4.8' into shams-4.9-frwd-merge-a817218b
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/ArtemisTcpTransport.kt
#	node-api/src/test/kotlin/net/corda/nodeapi/internal/revocation/RevocationTest.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/CertificateRevocationListNodeTests.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/ProtonWrapperTests.kt
#	node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt
#	node/src/main/kotlin/net/corda/node/internal/artemis/BrokerJaasLoginModule.kt
#	node/src/main/kotlin/net/corda/node/internal/artemis/CertificateChainCheckPolicy.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/ArtemisMessagingServer.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/NodeNettyAcceptorFactory.kt
2023-06-01 16:37:57 +01:00
Shams Asari
a817218b08
ENT-9806: Added peer info to SSL handshake logging, and other changes for ENT merge (#7380) 2023-06-01 15:51:58 +01:00
Adel El-Beik
e15f92b526
Merge pull request #7355 from corda/shams-crl-netty-threads-blocked
ENT-9806: Netty threads no longer blocked if CRL endpoints are unresponsive
2023-05-27 10:43:41 +01:00
Shams Asari
9ebcfd3176 Merge fix 2023-05-22 10:00:03 +01:00
Shams Asari
e6336666c2 Merge branch 'release/os/4.10' into shams-4.11-fwrd-merge-046ed0ac
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/crypto/X509Utilities.kt
2023-05-22 09:55:19 +01:00
Shams Asari
0cc3ffe1d6
ENT-9941: Moved new connector factory to node-api (#7369) 2023-05-16 08:45:01 +01:00
Shams Asari
0a617097be ENT-9806: Prevent Netty threads being blocked due to unresponsive CRL endpoints 2023-05-15 12:59:24 +01:00
Shams Asari
a37d09486f Merge branch 'release/os/4.9' into shams-4.10-fwrd-merge-7b7e3847 2023-05-15 09:55:31 +01:00
Shams Asari
13ca00deea ENT-9941: Moved new connector factory to node-api 2023-05-12 17:42:27 +01:00
Shams Asari
151dc16a0e Merge branch 'release/os/4.8' into shams-4.9-frwd-merge-31a34e5a
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/ArtemisTcpTransport.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/NodeNettyAcceptorFactory.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/P2PMessagingClient.kt
2023-05-12 14:34:33 +01:00
Shams Asari
31a34e5a5c
ENT-9941: Improved Netty logging, especially of the embedded broker (#7365) 2023-05-12 10:11:09 +01:00
Rick Parker
9ba3919980
ENT-9583 Public key caching of encoded form (OS) (#7332) 2023-04-26 17:49:52 +01:00
Shams Asari
2556fde971 Merge branch 'release/os/4.10' into shams-4.11-frwd-merge-fc531761 2023-04-13 10:55:03 +01:00
Shams Asari
33f7fea8d2 Fix compile issues 2023-04-12 16:30:37 +01:00
Shams Asari
7b6fc51710 Merge branch 'release/os/4.9' into shams-4.10-fwrd-merge-ceb76c56
# Conflicts:
#	common/logging/src/main/kotlin/net/corda/common/logging/Constants.kt
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/AMQPClient.kt
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/SSLHelper.kt
2023-04-12 16:23:59 +01:00
Shams Asari
724a4e5a36 Merge branch 'release/os/4.8' into shams-4.9-fwrd-merge-d140cb59
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/ArtemisTcpTransport.kt
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/AMQPChannelHandler.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/CertificateRevocationListNodeTests.kt
#	node/src/main/kotlin/net/corda/node/internal/artemis/CertificateChainCheckPolicy.kt
2023-04-11 11:54:26 +01:00
Shams Asari
ddb979d693 ENT-8898: Revert change made to externalCrlSource checking
It's now moved back to createPKIXRevocationChecker(). RevocationConfig.enrichExternalCrlSource has also been removed, and moved to where it's called.
2023-04-06 14:42:38 +01:00
Shams Asari
1e6ccfdb60
ENT-8898: Replaced JDK cert revocation with custom plugable implementation (#7322) 2023-04-03 10:26:01 +01:00
Shams Asari
0213861d22
ENT-9569: Apply the 60s SSL handshake timeout to the embedded Artemis server (#7315) 2023-03-22 13:22:12 +00:00
Mahmoud Almahroum
021c70143b
ENT-8826 Upgrade Liquibase to latest version - more review feedback (#7279) 2023-03-10 14:28:40 +00:00
Mahmoud Almahroum
1a0d354903
ENT-8983 Upgrade H2 and liquibase to latest version (#7298) 2023-03-03 15:10:25 +00:00
Adel El-Beik
79babc999c ENT-6993: Upgrade BC to 1.72. 2022-11-02 15:32:18 +00:00
Adel El-Beik
7a133f687c
ENT-6893: First cut of telemetry integration. (#7247)
First cut of telemetry integration.

Open telemetry can be enabled in two ways, first is via an opentelemetry java agent specified on the command line. With this way you get the advantage of spans created from other libraries, like hibernate. The java agent does byte code rewriting to insert spans.
The second way is with the open telemetry driver (that links with the opentelemetry sdk). This is a fat jar provided with this project and needs to go into the node drivers directory.
2022-10-28 14:41:39 +01:00
Rick Parker
b29713d7b9
ENT-6947 Intern common types to reduce heap footprint (#7239)
ENT-6947: Implement interning for SecureHash, CordaX500Name, PublicKey, AsbtractParty and SignatureAttachmentConstraint, including automatic detection of internable types off companion objects in AMQP & Kyro deserialization.  In some cases, add new factory methods to companion objects, and make main code base use them.

Performance tested in performance cluster with no negative impact visible (so default concurrency setting seems okay).

Testing suggests 5-6x memory saving for tokens in TokensSDK in memory selector.  Should see approx. 1 million tokens per GB or better (1.5 million for the tokens we tested with).
2022-10-18 09:28:41 +01:00
Chris Cochrane
5ca5b8d096
Do not black-list AMQP targets that suffer a handshake failure 2022-09-13 11:41:19 +01:00
Chris Cochrane
03fa529292
ENT-6791 New service lifecycle event published just before starting the state machine (#7164) 2022-06-10 19:56:02 +01:00
Chris Cochrane
500be9cbcd
Post-review updates; added integration tests 2022-06-08 13:36:50 +01:00
Adel El-Beik
5a41348679 Merge branch 'release/os/4.8' into adel/merge-from-4.8-1stApr 2022-04-01 11:10:58 +01:00
Adel El-Beik
20f3b25d4d Merge remote-tracking branch 'origin/release/os/4.7' into adel/merge-from-4.7-31March 2022-03-31 16:19:54 +01:00
Adel El-Beik
6c11184eae CORDA-4239: Regenerate test certificates with 10yr validity. 2022-03-30 16:35:40 +01:00
Adel El-Beik
4594c76259 ENT-6637: Targeted exclude of the jgroups dependency. 2022-03-04 11:52:14 +00:00
Viktor Kolomeyko
162f76f710
ENT-6693: Change the way how message sizes are determined (#7090)
Especially relevant for `AMQPLargeMessage`.
2022-02-24 13:54:27 +00:00
Adel El-Beik
ba8cd1b80d Merge remote-tracking branch 'origin/release/os/4.8' into adel/merge-from-4.8-21Feb 2022-02-21 12:52:29 +00:00
Adel El-Beik
c11eb382ce
ENT-6588 Restrict database operations platform flag (#7053) (#7073) (#7074) (#7076)
* ENT-6588 Restrict database operations platform flag

Put the restricting of database operations in `RestrictedConnection` and
`RestrictedEntityManager` behind a platform version flag.

`RESTRICTED_DATABASE_OPERATIONS = 7` was added to signify this.

If the version is less than 7, then the database operations will not be
restricted. A warning is logged to indicate that they are using
potentially dangerous methods.

If the version is 7 or greater, then the database operations are
restricted and throw an error if called.

Co-authored-by: Dan Newton <dan.newton@r3.com>
2022-02-18 09:42:59 +00:00
Adel El-Beik
76366398fe
ENT-6584: Move sending of events to finally block. This makes sure it… (#4383) (#7057)
* ENT-6584: Move sending of events to finally block. This makes sure it gets executed in event of an exception.
2022-02-08 10:32:12 +00:00
Dan Newton
fa739fc467
ENT-6588 Restrict database operations platform flag (#7053)
* ENT-6588 Restrict database operations platform flag

Put the restricting of database operations in `RestrictedConnection` and
`RestrictedEntityManager` behind a platform version flag.

`RESTRICTED_DATABASE_OPERATIONS = 7` was added to signify this.

If the version is less than 7, then the database operations will not be
restricted. A warning is logged to indicate that they are using
potentially dangerous methods.

If the version is 7 or greater, then the database operations are
restricted and throw an error if called.
2022-02-08 10:06:44 +00:00
Viktor Kolomeyko
50c226c29d
ENT-6025: Change the error code after Artemis upgrade (#7041)
Artemis Upgrade (2.17.1) changed the error code for the message we are using to for message acknowledgement in `onLinkRemoteClose`.
More details can be found here: https://issues.apache.org/jira/browse/ARTEMIS-1018
2022-01-21 12:46:09 +00:00
Viktor Kolomeyko
835321bb70
ENT-6286: OS: Upgrade version of Artemis to 2.19.1 (#6975)
Fixes DDoS attack mentioned on the Jira ticket.

PR upgrades Artemis library to version 2.19.1.
This is our own release of Apache Artemis library which has vulnerability fix for v2.20 applied.

**_Breaking changes discovered during Artemis upgrade:_**
1. When the queue is created as temporary, it needs to explicitly be specified as non-durable.
2. By default, Artemis Client performs Host DNS name check against the certificate presented by the server. Our TLS certificates fail this check and this verification has to be explicitly disabled, see use of: `TransportConstants.VERIFY_HOST_PROP_NAME`.
3. Artemis Server now caches login attempts, even unsuccessful ones. When we add RPC users dynamically via DB insert this may have an unexpected outcome if the user with the same `userName` and `password` was not available previously.
To workaround permissions changing dynamically, authorization and authentication caches had to be disabled.
4. When computing `maxMessageSize`, the size of the headers content is now taken into account as well.
5. Artemis handling of start-up errors has changed. E.g. when the port is already bound.
6. A number of deprecated APIs like: `createTemporaryQueue`, `failoverOnInitialAttempt`, `NullOutputStream`, `CoreQueueConfiguration`.
7. Log warning message is produced like: `AMQ212080: Using legacy SSL store provider value: JKS. Please use either 'keyStoreType' or 'trustStoreType' instead as appropriate.`
8. As reported by QA, Artemis now produces more audit logging more details [here](https://r3-cev.atlassian.net/browse/ENT-6540). Log configuration been adjusted to reduce such output.
2022-01-21 09:18:14 +00:00
Lorcan Wogan
c2140f727c
ENT-6025 remote artemis channel does not exist resulting in infinite retry loop (#7020)
* ENT-6025 remote artemis channel does not exist resulting in infinite retry loop

* ENT-6025 rename test

* ENT-6025 fix detekt and add description

* ENT-6025 add check on count of connected stack
2022-01-18 10:13:29 +00:00