Commit Graph

3312 Commits

Author SHA1 Message Date
Shams Asari
de67ab7377 ENT-9876: Encrypting the ledger recovery participant distribution list 2023-07-25 14:59:43 +01:00
Jose Coll
7d1d2297e7
ENT-10289 Ensure Sender and Receiver Distribution records share the same timestamp (#7437) 2023-07-20 09:55:44 +01:00
Adel El-Beik
aa9e41c7c2 ENT-10122: Updated tests to include consuming transaction id in the Vault.Update check. 2023-07-19 16:36:39 +01:00
Adel El-Beik
60bb4c58f2 ENT-10122: Made the consuming tx id field nullable, added missing changelog file. 2023-07-18 17:45:53 +01:00
Adel El-Beik
669d6590af ENT-10122: Add consuming transaction id to vault states table. 2023-07-17 17:58:31 +01:00
Shams Asari
03a9a95216 Merge branch 'release/os/4.10' into shams-4.11-merge-6b655c0f
# Conflicts:
#	.github/workflows/check-pr-title.yml
#	node/src/main/kotlin/net/corda/node/internal/ServicesForResolutionImpl.kt
#	node/src/main/kotlin/net/corda/node/services/vault/NodeVaultService.kt
2023-07-17 10:17:38 +01:00
Tom Stark
58ecce1713
ENT-9875: New network parameters (#7398)
* ENT-9875: Added new network parameters

- Added `transactionRecoveryPeriod`
- Added `confidentialIdentityPreGenerationPeriod`

These new parameters are currently set to be nullable meaning they can be ignored and the duration if not specified will be null rather than, e.g., 0. This currently allows for nothing changing/breaking in the node-api

_Note: if these params can stay as nullable then the deprecated constructor might not even be needed (since the existing one will still work), needs to be discussed._
2023-07-14 12:02:21 +01:00
Shams Asari
5ecb36c302 Merge fix 2023-07-13 11:31:35 +01:00
Shams Asari
3a6deeefa7 Merge branch 'release/os/4.9' into shams-4.10-merge-e6a80822
# Conflicts:
#	.github/workflows/check-pr-title.yml
#	.snyk
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/AMQPClient.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/AMQPClientSslErrorsTest.kt
#	node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt
2023-07-13 10:53:30 +01:00
Shams Asari
1e0f7cd690 Merge fix 2023-07-07 15:29:44 +01:00
Shams Asari
671ce9ef97 Merge branch 'os-4.8-merge-point' into shams-4.9-merge-0b8536b9
# Conflicts:
#	.github/workflows/check-pr-title.yml
#	node/src/main/kotlin/net/corda/node/services/messaging/NodeNettyAcceptorFactory.kt
2023-07-07 15:20:43 +01:00
Chris Cochrane
0f2312a201
ENT-8983 - Postgres migration failure (#7408)
* Set DB transaction isolation level only if its going to change; upgraded dependencies

* Removed duplicate changeset
2023-07-07 13:29:35 +01:00
Jose Coll
0e877958fe
ENT-10009 Enhance SendTransactionFlow to allow sending a txn to multiple sessions. (#7393) 2023-06-09 17:19:43 +01:00
Adel El-Beik
0b8536b97c
Merge pull request #7395 from corda/shams-thread-names
ENT-10016: Give all node threads descriptive names
2023-06-09 16:05:48 +01:00
Shams Asari
5b3180bf7b ENT-10016: Give all node threads descriptive names 2023-06-09 11:17:26 +01:00
Jose Coll
c56ee1cc73
Fix failing slow test. (#7387) 2023-06-07 09:30:24 +01:00
Shams Asari
0bfce451ea ENT-10013: Vault service refactoring backport 2023-06-06 16:17:58 +01:00
Shams Asari
ff0693a598 ENT-9793: Use streams when loading vault query pages 2023-06-06 13:13:22 +01:00
nargas-ritu
5510cb151b
Merge pull request #7375 from corda/shams-vault-page-overlap
ENT-9793: Added Page.previousPageAnchor to allow detection of vault changes whilst pages are loaded
2023-06-06 11:07:45 +01:00
Jose Coll
f791adf442
ENT-9924 (Follow-up) Update recording of transaction flow recovery metadata into Send/Receive transaction flows. (#7382) 2023-06-05 16:59:06 +01:00
Jose Coll
2c775bcc41
ENT-9924 Update recording of transaction flow recovery metadata into Send/Receive transaction flows. (#7374) 2023-06-02 16:05:28 +01:00
Shams Asari
4dcd9245d3 ENT-9806: Using Artemis SSLContextFactory service to pass in custom TrustManagerFactory. This removes the need to copy code from NettyAcceptor. 2023-06-02 15:10:58 +01:00
Shams Asari
5706f89639 Merge branch 'release/os/4.8' into shams-4.9-frwd-merge-a817218b
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/ArtemisTcpTransport.kt
#	node-api/src/test/kotlin/net/corda/nodeapi/internal/revocation/RevocationTest.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/CertificateRevocationListNodeTests.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/ProtonWrapperTests.kt
#	node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt
#	node/src/main/kotlin/net/corda/node/internal/artemis/BrokerJaasLoginModule.kt
#	node/src/main/kotlin/net/corda/node/internal/artemis/CertificateChainCheckPolicy.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/ArtemisMessagingServer.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/NodeNettyAcceptorFactory.kt
2023-06-01 16:37:57 +01:00
Shams Asari
a817218b08
ENT-9806: Added peer info to SSL handshake logging, and other changes for ENT merge (#7380) 2023-06-01 15:51:58 +01:00
Adel El-Beik
e15f92b526
Merge pull request #7355 from corda/shams-crl-netty-threads-blocked
ENT-9806: Netty threads no longer blocked if CRL endpoints are unresponsive
2023-05-27 10:43:41 +01:00
Shams Asari
ed08b2c5de ENT-9793: Added Page.previousPageAnchor to allow detection of vault changes whilst pages are loaded 2023-05-26 12:52:06 +01:00
Jose Coll
2e29e36e01
ENT-9923 Ledger Recovery: split out recovery metadata into own database schema. (#7364) 2023-05-24 09:42:09 +01:00
Shams Asari
e6336666c2 Merge branch 'release/os/4.10' into shams-4.11-fwrd-merge-046ed0ac
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/crypto/X509Utilities.kt
2023-05-22 09:55:19 +01:00
Shams Asari
c065021328 ENT-8827: The ordering of vault query results is clobbered by ServiceHub.loadStates 2023-05-18 15:59:25 +01:00
Shams Asari
0cc3ffe1d6
ENT-9941: Moved new connector factory to node-api (#7369) 2023-05-16 08:45:01 +01:00
Shams Asari
0a617097be ENT-9806: Prevent Netty threads being blocked due to unresponsive CRL endpoints 2023-05-15 12:59:24 +01:00
Shams Asari
a37d09486f Merge branch 'release/os/4.9' into shams-4.10-fwrd-merge-7b7e3847 2023-05-15 09:55:31 +01:00
Shams Asari
13ca00deea ENT-9941: Moved new connector factory to node-api 2023-05-12 17:42:27 +01:00
Shams Asari
151dc16a0e Merge branch 'release/os/4.8' into shams-4.9-frwd-merge-31a34e5a
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/ArtemisTcpTransport.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/NodeNettyAcceptorFactory.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/P2PMessagingClient.kt
2023-05-12 14:34:33 +01:00
Shams Asari
31a34e5a5c
ENT-9941: Improved Netty logging, especially of the embedded broker (#7365) 2023-05-12 10:11:09 +01:00
Shams Asari
0951853207
ENT-6515: Cherry-pick of ENT-6315 - Allow dumping of paused flows (#7008) (#7363)
This is a cherry-pick of https://github.com/corda/corda/pull/7008, which also resolves ENT-6515

Co-authored-by: Dan Newton <dan.newton@r3.com>
2023-05-11 09:49:40 +01:00
Chris Cochrane
cca09fef58
Merge branch 'release/os/4.10' into cc/merge/os-4.10-os-4.11/03-may-2023
Fwd-merge from OS 4.11.
2023-05-03 17:47:44 +01:00
Jose Coll
c3e39a7052
ENT-9842 Re-factor 2PF to support issuance transactions (no notarisation) with observers. (#7349)
Re-factor 2PF to support issuance transactions (no notarisation) with observers.
2023-04-27 16:58:17 +01:00
Chris Cochrane
2f53e8115f
Fwd-merge from OS 4.9 2023-04-27 16:45:16 +01:00
Chris Cochrane
34d4a5f667
Fwd-merge from OS 4.8 2023-04-26 22:54:20 +01:00
Rick Parker
9ba3919980
ENT-9583 Public key caching of encoded form (OS) (#7332) 2023-04-26 17:49:52 +01:00
Jose Coll
1d4feedc62
ENT-9147 Propagate and handle Notary Error (Part 2) (#7346) 2023-04-26 09:06:32 +01:00
Chris Cochrane
e12ab8194f
Merge branch 'release/os/4.7' into cc/merge/os-4.7-os-4.8/25-apr-2023
Fwd-merge from OS 4.7.
2023-04-25 14:54:21 +01:00
Chris Cochrane
91c38fa84b
Merge branch 'release/os/4.6' into cc/merge/os-4.6-os-4.7/24-apr-2023
Fwd-merge from OS 4.6.
2023-04-24 20:13:50 +01:00
Chris Cochrane
e5a6cac9e8
Flakey test fix 2023-04-21 16:00:19 +01:00
Jose Coll
190acdc87c
NOTICK Fix failing StateMachineFinalityErrorHandlingTest's. (#7339) 2023-04-20 15:45:10 +01:00
Chris Cochrane
c58c49d27a
Upgrade corda/crash, snakeyaml 2023-04-20 14:01:20 +01:00
Jose Coll
7bd3f5dd33
ENT-9147 Remove un-notarised transactions upon Double Spend. (#7324) 2023-04-19 15:31:47 +01:00
Chris Cochrane
29dfda3730
Resolved build-check issues 2023-04-18 20:34:42 +01:00
Shams Asari
2556fde971 Merge branch 'release/os/4.10' into shams-4.11-frwd-merge-fc531761 2023-04-13 10:55:03 +01:00
Shams Asari
7b6fc51710 Merge branch 'release/os/4.9' into shams-4.10-fwrd-merge-ceb76c56
# Conflicts:
#	common/logging/src/main/kotlin/net/corda/common/logging/Constants.kt
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/AMQPClient.kt
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/SSLHelper.kt
2023-04-12 16:23:59 +01:00
Shams Asari
c9fd22855e Resolve compile issues 2023-04-11 12:16:02 +01:00
Shams Asari
724a4e5a36 Merge branch 'release/os/4.8' into shams-4.9-fwrd-merge-d140cb59
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/ArtemisTcpTransport.kt
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/AMQPChannelHandler.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/CertificateRevocationListNodeTests.kt
#	node/src/main/kotlin/net/corda/node/internal/artemis/CertificateChainCheckPolicy.kt
2023-04-11 11:54:26 +01:00
Shams Asari
ddb979d693 ENT-8898: Revert change made to externalCrlSource checking
It's now moved back to createPKIXRevocationChecker(). RevocationConfig.enrichExternalCrlSource has also been removed, and moved to where it's called.
2023-04-06 14:42:38 +01:00
Shams Asari
1e6ccfdb60
ENT-8898: Replaced JDK cert revocation with custom plugable implementation (#7322) 2023-04-03 10:26:01 +01:00
Jose Coll
4beeb470df Additional signature verification and validation: recordTransactions() 2023-03-28 12:48:33 +01:00
Jose Coll
b4983597e2
ENT-6875 Two Phase Finality - CLEAN-UP (#7321)
* Remove completed TODOs

* Prevent mis-leading progress tracker message.
2023-03-24 08:55:37 +00:00
Shams Asari
0213861d22
ENT-9569: Apply the 60s SSL handshake timeout to the embedded Artemis server (#7315) 2023-03-22 13:22:12 +00:00
Jose Coll
d2900d54ab
ENT-6875 Two Phase Finality Flow - improve ledger consistency & recoverability (#7290) 2023-03-22 10:47:51 +00:00
Mahmoud Almahroum
021c70143b
ENT-8826 Upgrade Liquibase to latest version - more review feedback (#7279) 2023-03-10 14:28:40 +00:00
Mahmoud Almahroum
1a0d354903
ENT-8983 Upgrade H2 and liquibase to latest version (#7298) 2023-03-03 15:10:25 +00:00
Adel El-Beik
044b83508e ENT-8823: Added copyBaggageToTags configuration option to control if baggage is copied to tags.
Switched SimplLogTelemetryComponent to info logs.
2022-12-08 16:54:38 +00:00
Chris Cochrane
563a7fb6c1
Merge branch 'release/os/4.9' into cc/merge/os-4.9/os-4.10/28-Nov-2022
Fwd-merge from OS-4.9.
2022-11-28 13:14:07 +00:00
rick.parker
3e25f7fd1a ENT-8817 OS port of compute deadlock changes 2022-11-25 09:31:41 +00:00
Chris Cochrane
31d9f0df09
Merge branch 'release/os/4.9' into cc/merge/os-4.9/os-4.10/24-Nov-2022
Fwd-merge from OS 4.9.
2022-11-24 18:27:54 +00:00
Chris Cochrane
0131163db0
ENT-8814 - back-fit changes from Enterprise to OS (#7272) 2022-11-24 18:13:35 +00:00
Adel El-Beik
e46b7bdd5a
ENT-6893: Added interface for clients to grab openetelemetry handle. (#7264)
* ENT-6893: Added interface for clients to grab openetelemetry handle.

* ENT-6893: Make detekt happy.

* ENT-6893: Fix warnings.

* ENT-6893: Make detekt happy.

* ENT-6893: Now shutdown opentelemetry when node stops or client is closed.

* ENT-6893: OpenTelemetryDriver is not not a singleton.
2022-11-24 13:34:08 +00:00
Rick Parker
188027c1f5
ENT-8811 OS port of flow draining fixes (#7269) 2022-11-24 10:27:00 +00:00
Adel El-Beik
7a133f687c
ENT-6893: First cut of telemetry integration. (#7247)
First cut of telemetry integration.

Open telemetry can be enabled in two ways, first is via an opentelemetry java agent specified on the command line. With this way you get the advantage of spans created from other libraries, like hibernate. The java agent does byte code rewriting to insert spans.
The second way is with the open telemetry driver (that links with the opentelemetry sdk). This is a fat jar provided with this project and needs to go into the node drivers directory.
2022-10-28 14:41:39 +01:00
Rick Parker
bdcd25477d
ENT-7039: Notary signature checking requires non-interned SecureHashes. (#7254)
There was a mistake made when we first introduced notary request signature checking, in that we didn't wrap it in SerializedBytes so it always got deserialized as part of the flow message payload. So to check the signature, it has to be re-serialized. This means for cross-version compatibility we can never change the serialized format of NotarisationRequest. In this case we need make sure that every SecureHash mentioned in that data structure is a distinct instance, even if the values are repeated / identical, as that is how it was in Corda 1.

With the introduction of interning of SecureHash, this ceased to be true once again, including undoing the attempts to force it on the sending side that had been introduced in previous versions of Corda. So here we introduce a way to force it, and consolidate the forcing to distinct SecureHash instances in the NotarisationRequest itself, rather than leaving to the caller of the constructor to remember to do it, so that serialized form will always be as per Corda 1.
2022-10-28 14:03:47 +01:00
Rick Parker
b29713d7b9
ENT-6947 Intern common types to reduce heap footprint (#7239)
ENT-6947: Implement interning for SecureHash, CordaX500Name, PublicKey, AsbtractParty and SignatureAttachmentConstraint, including automatic detection of internable types off companion objects in AMQP & Kyro deserialization.  In some cases, add new factory methods to companion objects, and make main code base use them.

Performance tested in performance cluster with no negative impact visible (so default concurrency setting seems okay).

Testing suggests 5-6x memory saving for tokens in TokensSDK in memory selector.  Should see approx. 1 million tokens per GB or better (1.5 million for the tokens we tested with).
2022-10-18 09:28:41 +01:00
Chris Cochrane
5ca5b8d096
Do not black-list AMQP targets that suffer a handshake failure 2022-09-13 11:41:19 +01:00
Chris Cochrane
54fdd12a2d
ENT-6866 support quick RPCs that are not processed via the RPC thread pool (#7213)
* ENT-6866 support quick RPCs that are not processed via the RPC thread pool
2022-07-07 12:33:51 +01:00
HJ Kim
0a59d578ff Merge branch 'release/os/4.9' into hjkim/os-4.9-4.10-merge-24062022
This merge includes a bugfix ENT-6714 which Corda logs database
password.
2022-06-24 16:08:16 +01:00
HJ Kim
8d34be5129 ENT-6714: Fix Corda logging database password
This commit ports the previously implemented fix from Corda ENT. Due to
the unrelated changes and merge conflict, the fix has been manually
copied rather than cherry-picked.
2022-06-23 23:03:10 +01:00
Chris Cochrane
199f635d69
Attempt to fix service priority tests when they are run on Jenkins 2022-06-15 11:56:03 +01:00
Chris Cochrane
c57bbd1c39
Added integration tests for service event priorities 2022-06-14 17:15:31 +01:00
Chris Cochrane
03fa529292
ENT-6791 New service lifecycle event published just before starting the state machine (#7164) 2022-06-10 19:56:02 +01:00
Chris Cochrane
f0d530ab8b
Post review updates; renamed tests/files, neater Kotlin test code 2022-06-08 16:30:06 +01:00
Chris Cochrane
4b1aabb164
Update node/src/integration-test/kotlin/net/corda/node/jmx/NodeStatus.kt
Co-authored-by: Viktor Kolomeyko <viktor.kolomeyko@r3.com>
2022-06-08 16:01:43 +01:00
Chris Cochrane
0232146515
Update node/src/integration-test/kotlin/net/corda/node/jmx/Publish.kt
Co-authored-by: Viktor Kolomeyko <viktor.kolomeyko@r3.com>
2022-06-08 16:01:25 +01:00
Chris Cochrane
3c0c1a01a2
Update node/src/integration-test/kotlin/net/corda/node/jmx/Publish.kt
Co-authored-by: Viktor Kolomeyko <viktor.kolomeyko@r3.com>
2022-06-08 15:50:48 +01:00
Chris Cochrane
fce95dd276
Update node/src/integration-test/kotlin/net/corda/node/jmx/NodeStatus.kt
Co-authored-by: Viktor Kolomeyko <viktor.kolomeyko@r3.com>
2022-06-08 15:50:34 +01:00
Chris Cochrane
518befabeb
Resolved detekt error 2022-06-08 13:54:16 +01:00
Chris Cochrane
500be9cbcd
Post-review updates; added integration tests 2022-06-08 13:36:50 +01:00
Chris Cochrane
b76fc7de59
Introduced node status; published via JMX 2022-06-07 16:20:05 +01:00
Chris Rankin
47ec57186e Merge commit '91ef44dc12b8cc921b9ec0a37b44de75c256d92d' into chrisr3-os-4.8-merge 2022-05-23 12:27:49 +01:00
Chris Rankin
57ddafa148 ENT-6532: Close any deserialization classloader that is evicted from the cache.
Also revert disabling caches when reading URLs.
2022-05-20 11:51:01 +01:00
Adel El-Beik
ad5193f135 ENT-6636: Removing commons-codec - not used, at least not as a compile time dep. 2022-05-10 13:46:48 +01:00
Adel El-Beik
bc6a6ee2ae
Merge pull request #7155 from corda/adel/ENT-6743
ENT-6743: SuspensionMeta in FlowInfo is null when a runnable flow has previously been hospitalized
2022-04-26 17:21:30 +01:00
Adel El-Beik
c1002697c7 ENT-6743: Reverted RetryFlowMockTest as the retry op only available on ENT. 2022-04-25 11:30:24 +01:00
Adel El-Beik
fa607024c2 ENT-6743: Contains method of flow hospital now correctly returns if flow is in hospital. Historic patient records not used. 2022-04-25 11:24:21 +01:00
Adel El-Beik
54a131313d ENT-6728: Archiving of a legal identity now done in its own txn, to cope with clustered notary nodes that have common legal identities between nodes (the service name one is common) 2022-03-15 12:25:39 +00:00
Rick Parker
1e227fdfc4
ENT-6711 - Fix for upgrade to Jackson 2.12 and beyond (actual upgrade to 2.13.1) (#7096)
* NOTICK - Don't know what the JIRA is but wanted to share.

* Updates to resolve bukld issues

* NOTICK: Fixed JDK11 version to prevent capsule version error

* ENT-6711: Added comment for use of jackson_kotlin_version.

* ENT-6711: Avoid deprecation warning, switched to the default method.

Co-authored-by: Chris Cochrane <chris.cochrane@r3.com>
Co-authored-by: Adel El-Beik <adel.el-beik@r3.com>
2022-03-07 11:08:06 +00:00
Adel El-Beik
4594c76259 ENT-6637: Targeted exclude of the jgroups dependency. 2022-03-04 11:52:14 +00:00
Adel El-Beik
4203a56c3a ENT-6637 & ENT-6618: Removed jgroups and dbcp from artemis dependencies. 2022-03-03 16:40:18 +00:00
Conal Smith
5f1e86c17c
ENT-6508 - Prevent directory traversal from zip file names (#7085) 2022-02-25 15:51:49 +00:00
Dan Newton
4b29ad337b
ENT-6710 Corda shell ssh logging again (#7093)
Correct SSH logging again, by pulling the logging of SSH ports to a
singular location.
2022-02-25 15:47:31 +00:00
Viktor Kolomeyko
162f76f710
ENT-6693: Change the way how message sizes are determined (#7090)
Especially relevant for `AMQPLargeMessage`.
2022-02-24 13:54:27 +00:00
Adel El-Beik
ba8cd1b80d Merge remote-tracking branch 'origin/release/os/4.8' into adel/merge-from-4.8-21Feb 2022-02-21 12:52:29 +00:00
Adel El-Beik
20d2e2b3eb
ENT-6677: When you clear the cache via the command line it now also updates node_named_identities table. (#7075) 2022-02-18 15:04:35 +00:00
Adel El-Beik
c11eb382ce
ENT-6588 Restrict database operations platform flag (#7053) (#7073) (#7074) (#7076)
* ENT-6588 Restrict database operations platform flag

Put the restricting of database operations in `RestrictedConnection` and
`RestrictedEntityManager` behind a platform version flag.

`RESTRICTED_DATABASE_OPERATIONS = 7` was added to signify this.

If the version is less than 7, then the database operations will not be
restricted. A warning is logged to indicate that they are using
potentially dangerous methods.

If the version is 7 or greater, then the database operations are
restricted and throw an error if called.

Co-authored-by: Dan Newton <dan.newton@r3.com>
2022-02-18 09:42:59 +00:00
Adel El-Beik
470bce8a6b
ENT-6588 Restrict database operations platform flag (#7053) (#7073) (#7074)
* ENT-6588 Restrict database operations platform flag

Put the restricting of database operations in `RestrictedConnection` and
`RestrictedEntityManager` behind a platform version flag.

`RESTRICTED_DATABASE_OPERATIONS = 7` was added to signify this.

If the version is less than 7, then the database operations will not be
restricted. A warning is logged to indicate that they are using
potentially dangerous methods.

If the version is 7 or greater, then the database operations are
restricted and throw an error if called.

Co-authored-by: Dan Newton <dan.newton@r3.com>

Co-authored-by: Dan Newton <dan.newton@r3.com>
2022-02-17 16:59:34 +00:00
Adel El-Beik
12fd451919
ENT-6588 Restrict database operations platform flag (#7053) (#7073)
* ENT-6588 Restrict database operations platform flag

Put the restricting of database operations in `RestrictedConnection` and
`RestrictedEntityManager` behind a platform version flag.

`RESTRICTED_DATABASE_OPERATIONS = 7` was added to signify this.

If the version is less than 7, then the database operations will not be
restricted. A warning is logged to indicate that they are using
potentially dangerous methods.

If the version is 7 or greater, then the database operations are
restricted and throw an error if called.

Co-authored-by: Dan Newton <dan.newton@r3.com>
2022-02-17 14:07:11 +00:00
Dan Newton
7afb585ae2
ENT-6676 Don't log SSH port if shell not installed (#7059) 2022-02-11 14:05:05 +00:00
Dan Newton
fa739fc467
ENT-6588 Restrict database operations platform flag (#7053)
* ENT-6588 Restrict database operations platform flag

Put the restricting of database operations in `RestrictedConnection` and
`RestrictedEntityManager` behind a platform version flag.

`RESTRICTED_DATABASE_OPERATIONS = 7` was added to signify this.

If the version is less than 7, then the database operations will not be
restricted. A warning is logged to indicate that they are using
potentially dangerous methods.

If the version is 7 or greater, then the database operations are
restricted and throw an error if called.
2022-02-08 10:06:44 +00:00
Adel El-Beik
5c052e9273
ENT-6586: Validate when signing that output states can be deserialized. (#7055)
* ENT-6586: Log warning instead of debug if state cannot be deserialized.
2022-02-07 18:05:59 +00:00
Chris Rankin
758a69f904
Validate LedgerTransaction deserialised from AttachmentsClassLoader. (#7049) (#7052) 2022-01-31 09:40:01 +00:00
Chris Rankin
c955093ce8
Validate LedgerTransaction deserialised from AttachmentsClassLoader. (#7049) 2022-01-28 17:05:59 +00:00
Adel El-Beik
7752fc8c9d
CORDA-4189: Removed existing vendor test (#7048) 2022-01-27 13:49:08 +00:00
Adel El-Beik
fb624a2b06
CORDA-4189: Now compare against Corda Community Edition as well. (#7046)
* CORDA-4189: Now compare against Corda Community Edition as well.

* CORDA-4189: Removed unused imports.
2022-01-26 11:33:23 +00:00
Ramzi El-Yafi
831f6d60cc Merge remote-tracking branch 'origin/release/os/4.8' into ramzi/os-48-49-merge 2022-01-21 10:55:29 +00:00
Viktor Kolomeyko
835321bb70
ENT-6286: OS: Upgrade version of Artemis to 2.19.1 (#6975)
Fixes DDoS attack mentioned on the Jira ticket.

PR upgrades Artemis library to version 2.19.1.
This is our own release of Apache Artemis library which has vulnerability fix for v2.20 applied.

**_Breaking changes discovered during Artemis upgrade:_**
1. When the queue is created as temporary, it needs to explicitly be specified as non-durable.
2. By default, Artemis Client performs Host DNS name check against the certificate presented by the server. Our TLS certificates fail this check and this verification has to be explicitly disabled, see use of: `TransportConstants.VERIFY_HOST_PROP_NAME`.
3. Artemis Server now caches login attempts, even unsuccessful ones. When we add RPC users dynamically via DB insert this may have an unexpected outcome if the user with the same `userName` and `password` was not available previously.
To workaround permissions changing dynamically, authorization and authentication caches had to be disabled.
4. When computing `maxMessageSize`, the size of the headers content is now taken into account as well.
5. Artemis handling of start-up errors has changed. E.g. when the port is already bound.
6. A number of deprecated APIs like: `createTemporaryQueue`, `failoverOnInitialAttempt`, `NullOutputStream`, `CoreQueueConfiguration`.
7. Log warning message is produced like: `AMQ212080: Using legacy SSL store provider value: JKS. Please use either 'keyStoreType' or 'trustStoreType' instead as appropriate.`
8. As reported by QA, Artemis now produces more audit logging more details [here](https://r3-cev.atlassian.net/browse/ENT-6540). Log configuration been adjusted to reduce such output.
2022-01-21 09:18:14 +00:00
Ramzi El-Yafi
ba4d275887 Merge remote-tracking branch 'origin/release/os/4.6' into ramzi/os-46-47-merge 2022-01-20 17:51:36 +00:00
Ramzi El-Yafi
c9d758f84a Merge remote-tracking branch 'origin/release/os/4.5' into ramzi/os-45-46-merge 2022-01-20 14:55:56 +00:00
Adel El-Beik
b17e4571bf
ENT-6386: Revert change of behaviour in rpcOps.wellKnownPartyFromX500Name for revoked identity (#7032)
* ENT-6386: Reverting wellKnownPartyFromX500Name functionality to work … (#4347)
2022-01-20 11:08:13 +00:00
Ramzi El-Yafi
4dfd4c613a Merge remote-tracking branch 'origin/release/os/4.4' into ramzi/os-44-45-merge 2022-01-20 10:19:16 +00:00
Ramzi El-Yafi
a45be650c7
ENT-6533 Update validating notary flow to be non-idempotent (#7031) 2022-01-20 09:52:45 +00:00
Dan Newton
c05c1934cf
ENT-6547 Remove url classloader in shell loading (#7034)
The `corda-shell` jar will already be installed if it exists in the the
node's `/drivers` directory. There is no need to include a
`URLClassLoader` to load its classes.

Rely on the process's main classloader.
2022-01-19 18:26:26 +00:00
Ramzi El-Yafi
caa4de47d3
ENT-6533 Update validating notary flow to be non-idempotent (#7030) 2022-01-19 13:15:51 +00:00
Lorcan Wogan
c2140f727c
ENT-6025 remote artemis channel does not exist resulting in infinite retry loop (#7020)
* ENT-6025 remote artemis channel does not exist resulting in infinite retry loop

* ENT-6025 rename test

* ENT-6025 fix detekt and add description

* ENT-6025 add check on count of connected stack
2022-01-18 10:13:29 +00:00
Chris Rankin
552317f2b3 Merge branch 'release/os/4.8' into chrisr3-os48-merge 2022-01-17 11:02:11 +00:00
Chris Rankin
4f1a07cbcc
ENT-6548: Ensure LazyMappedList is realised with correct SerializationContext. (#7028) 2022-01-17 09:57:33 +00:00
Dan Newton
bd9b50bf7c ENT-6376 Don't hospitalize session end in ReceiveFinalityFlow (#4325)
Do not keep a flow in for observation if it receives an unexpected
session end message while in `ReceiveFinalityFlow` and
`ReceiveTransactionFlow` (due to being called by the former).

This is done by checking the message of the `UnexpectedFlowEndException`
that is thrown when a session end message instead of a data message and
if the stacktrace has `ReceiveTransactionFlow` at the top, after
removing statemachine stack frames.

Checking the stacktrace for `ReceiveTransactionFlow` is important
because the unexpected session end session message is only ok if a
transaction has not already been received. For example, if
`ResolveTransactionsFlow` is in the stack, then this indicates failure
when receiving transaction dependencies on a transaction that should be
recorded.

Also added a test that highlights that the `UnexpectedFlowEndException`
caused by the session end message can be caught, therefore users can
determine their own behaviour if desired.
2022-01-13 11:02:46 +00:00
Dan Newton
56c9d6404f
ENT-6378 Migrate corda-shell to external repo (#7005)
Remove the shell code from the OS code base, this includes the modules:

- `:tools:shell`
- `:tools:shell-cli`

The shell will be run within a node if it exists within the node's `drivers` directory. 

This is done by using a `URLClassloader` to load the `InteractiveShell` class into Corda's JVM process and running `startShell` and `runLocalShell`.

Running the shell within the `:samples` will require adding:

```
cordaDriver "net.corda:corda-shell:<corda_shell_version>"
```

To the module's `build.gradle` containing `deployNodes`. The script will then include the shell in the created nodes.
2022-01-12 11:54:18 +00:00
Dan Newton
f2b3db9c7b
ENT-6315 Allow dumping of paused flows (#7008)
Checkpoint dumping of paused flows was not working because the dumper
expects a flow to have a `FlowState` of `Unstarted` or `Started`,
however due to a memory optimisation paused flows have their `FlowState`
set to `Paused`. This was causing causing an exception as well as a loss
of potentially useful information.

A flag `alwaysDeserializeCheckpoint` has been added to
`Checkpoint.Serialized.deserialize` which skips the memory optimisation
and forces the deserialization of the flow's `FlowState`.

Paused flows are now included in the dumped output along with their real
`FlowState` which is useful to users even if the flow is paused rather
than waiting for something to complete.

The status of the flow has also been added to the JSON output to assist
users in debugging their flows.
2022-01-11 10:22:49 +00:00
Dan Newton
044202550d
ENT-6421 Public version of FlowManagerRPCOps (#7006)
A public version of `FlowManagerRPCOps` which does not live in an
internal package has been added. This new interface shares the same name
as the internal one.

Because of the name sharing, the internal version has been
`@Deprecated`.

`FlowManagerRPCOpsImpl` implements both the new and old interfaces. This
 allows for backwards compatibility, allowing old shells or clients to
 call the old interface on newer nodes without breaking.
2022-01-07 16:40:29 +00:00
Viktor Kolomeyko
505ade9c86
ENT-6388: Eliminate usage of Mina's ConcurrentHashSet (#6977)
* ENT-6388: Eliminate usage of Mina's `ConcurrentHashSet`

* ENT-6388: Restrict to `MutableSet<ScheduledStateRef>`
2021-12-02 13:24:11 +00:00
Kyriakos Tharrouniatis
3cbfb0e024
ENT-6355 Empty flow arguments on error transition (#4307)
A user passed in a `FlowLogic` as an argument into another `FlowLogic`
called `subFlow` on it and had it throw an exception.

This all occurred before the first checkpoint, causing the state machine
to try and persist a FAILED checkpoint containing the flow's arguments.
Because the arguments contained a `FlowLogic` that had been started via
`subFlow` it held a reference to `FlowLogic._stateMachine` which cannot
be serialized.

This caused the flow to fail when trying to persist the fact that it
failed.

The flow arguments are now emptied during `ErrorFlowTransition` to
resolve this issue which mimics the behaviour of the first suspend.
Note, this only takes the arguments out of the serialized checkpoint, it
does not affect the flow metadata and therefore a flow's arguments can
still be viewed.

Co-authored-by: Dan Newton <dan.newton@r3.com>
2021-12-01 14:57:30 +00:00
Chris Rankin
e93e7c2846
ENT-6412: Upgrade ClassGraph 4.8.90 -> 4.8.135. (#6973) 2021-11-23 09:43:41 +00:00
Chris Rankin
caeafb8201
ENT-6357: Deserialize LedgerTransaction elements for each Contract.verify(). (#6962)
* ENT-6357: Deserialize LedgerTransaction elements for each Contract.verify().

* Lock the LedgerTransaction and NetworkParameters objects down for contract verification.

* Refactor BasicVerifier to be package private instead of public.

* Simplify verifyConstraints() operation.

* Review fixes: replace HashSet with LinkedHashSet, and add signing parties to commands via mapIndexed.

* Ensure tests also run notary nodes "out of process".

* Streamline SerializationContext switching.

* Cache deserialised cryptographic instances during contract verification.

* Invoke Class.forName() instead of ClassLoader.loadClass() to reduce contention on the system classloader's lock.

* Deserialization cache key now pre-computes its hash code.

* Allow AttachmentsClassLoader to be used concurrently.

* Cache all Envelope objects for reuse during contract verification.

* Generate CertPathProxy hash code using conventional algorithm.

* Adjust CustomSerializer.Proxy to allow better access to SerializationContext.
2021-11-10 16:38:40 +00:00
Walter Oggioni
883e794853
ENT-6330 Fixed reading jar entries in memory (#6960)
* ENT-6330 Fixed reading jar entries in memory

This is a trivial fix that is however enough to allow to send zip bombs as attachments without the node crashing, a size limit could be added for increased reliability

* added attachment cumulative size check

* added compression ratio check

* added unit test and moved the code to a standalone verifier object

* removed attachment check from AttachmentClassLoader to minimize performance impact
2021-10-13 09:46:20 +01:00
Agnieszka Szczepanska
0b9513ffa3 Merge branch 'release/os/4.7' into aga/make-recordDependencies-suspendable-4.8 2021-04-27 15:08:13 +01:00
Agnieszka Szczepanska
9aa4389902 resolve merge conflicts 2021-04-26 11:35:17 +01:00
Agnieszka Szczepanska
00c43e501c delete duplicated imports 2021-04-21 12:31:03 +02:00
Agnieszka Szczepanska
88f584cac5 fast forward merge 2021-04-21 12:16:22 +02:00
Agnieszka Szczepanska
c48f32215c Merge branch 'release/os/4.4' into aga/make-recordDependencies-suspendable-4.5 2021-04-20 16:25:47 +02:00
Agnieszka Szczepanska
8dd9dee9c4
NOTICK - make recordDependencies suspendable (#6896)
* add suspendable annotation to recordDependencies

* add suspendable annotation to recordDependencies implementation
2021-04-14 08:30:53 +01:00
Stefan Iliev
7b9b2569a9
NAAS-107 StartupQueueTime metric fix (#6886)
* Fixed StartupQueueTime metric producing negative numbers.
2021-03-09 18:00:04 +00:00
Ramzi El-Yafi
fe8baf4b17
NAAS-49 Flag notary/extraConfig block as sensitive (#6880) 2021-02-19 17:05:23 +00:00
Adel El-Beik
9cd02dc62d Merge remote-tracking branch 'origin/release/os/4.4' into adel/merge-from-4.4-19-Feb-21 2021-02-19 13:55:04 +00:00
Denis Rekalov
23fab7ae0c
ENT-5891 Disable flaky test (#6879) 2021-02-19 13:54:34 +00:00
Adel El-Beik
360b3f8d0c
CORDA-4125: Backport AttachmentClassLoader updates to 4.4 to enable ENT-6152 backport (#6878)
* CORDA-3755: Switched attachments map to a WeakHashMap (#6214)

* Bump OS release version 4.6

* CORDA-3755: Switched attachments map to a WeakHashMap

* CORDA-3755: Added explicit strong references to map key.

* CORDA-3755: Keeping detekt happy.

* CORDA-3755: Test a gc in verify.

* CORDA-3755: Making detekt happy.

* CORDA-3755: Suppress warnings for weak reference test.

* CORDA-3755: Fixing build failure with attachments.

* CORDA-3755: Rewrite based on Ricks input - now handles attachment already existing in map!

* CORDA-3755: Refactor WeakReference behaviour into AttachmentsHolderImpl and provide alternate version of this class for core-deterministic.

* CORDA-3755: Added more tests for WeakHashMap.

* CORDA-3755: Ignore the tests using System.gc keep for local testing only

* CORDA-3755: Adding comment to explain the ignored tests.

* Make AttachmentsHolderImpl package-private inside core-deterministic, just like it is inside core.

* CORDA-3755: Update assertions following review comments.

* CORDA-3755: Removing import

* CORDA-3755: Removed unused var.

* CORDA-3755: Reverting files that somehow got changed in rebase.

Co-authored-by: nargas-ritu <ritu.gupta@r3.com>
Co-authored-by: Chris Rankin <chris.rankin@r3.com>

* CORDA-3769: Switched attachments class loader cache to use caffeine (#6326)

* CORDA-3769: Switched attachments class loader cache to use caffeine with original implementation used by determinstic core.

* CORDA-3769: Removed default ctor arguments.

* CORDA-3769: Switched mapping function to Function type to avoid synthetic method being generated.

* CORDA-3769: Now using a cache created from NamedCacheFactory for the attachments class loader cache.

* CORDA-3769: Making detekt happy.

* CORDA-3769: The finality tests now check for UntrustedAttachmentsException which will actually happen in reality.

* CORDA-3769: Refactored after review comments.

* CORDA-3769: Removed the AttachmentsClassLoaderSimpleCacheImpl as DJVM does not need it. Also updated due to review comments.

* CORDA-3769: Removed the generic parameters from AttachmentsClassLoader.

* CORDA-3769: Removed unused imports.

* CORDA-3769: Updates from review comments.

* CORDA-3769: Updated following review comments. MigrationServicesForResolution now uses cache factory. Ctor updated for AttachmentsClassLoaderSimpleCacheImpl.

* CORDA-3769: Reduced max class loader cache size

* CORDA-3769: Fixed the attachments class loader cache size to a fixed default

* CORDA-3769: Switched attachments class loader size to be reduced by fixed value.

* CORDA-4125: Parameter has been added to a private ctor.

Co-authored-by: nargas-ritu <ritu.gupta@r3.com>
Co-authored-by: Chris Rankin <chris.rankin@r3.com>
2021-02-19 13:38:22 +00:00
Dan Newton
41b4f9b3aa
CORDA-4110 startFlowDynamicWithClientId restrict permissions (#6875)
Restrict permissions for `startFlowDynamicWithClientId` to not work with
 `InvokeRpc.startFlow` and instead require `InvokeRpc
 .startFlowWithClientId`.
2021-02-18 14:46:09 +00:00
William Vigor
7261442c98
CORDA-4123 Change MockNetworkParameterStorage to not rehash NetworkParameters (#6872)
* CORDA-4123 Change Mock Network Parameter Storage to not rehash the Network Parameters

* Fix Detekt
2021-02-16 16:25:38 +00:00
Adel El-Beik
1a7401472f
CORDA-4120: Disable the URLConnection cache to prevent file handle leaks in ServiceLoader. (#6869) 2021-02-12 17:24:00 +00:00
William Vigor
20dbbf008d
CORDA-4103 Feature Branch: Serialization injection for transaction building (#6867)
* CORDA-4105 Add public API to allow custom serialization schemes (#6848)

* CORDA-4105 Add public API to allow custom serialization schemes

* Fix Detekt

* Suppress warning

* Fix usused import

* Improve API to use generics

This does not break Java support (only Intelij gets confused).

* Add more detailed documentation to public interfaces

* Change internal variable name after rename public API

* Update Public API to use ByteSquence instead of SerializedBytes

* Remove unused import

* Fix whitespace.

* Add added public API to .ci/api-current.txt

* Improve public interfaces

Rename CustomSchemeContext to SerializationSchemeContext to improve
clarity and move to it's own file. Improve kdoc to make things less
confusing.

* Update API current with changed API

* CORDA-4104 Implement custom serialization scheme discovery mechanism (#6854)

* CORDA-4104 Implement CustomSerializationScheme Discovery Mechanism

Discovers a single CustomSerializationScheme implementation inside
the drivers dir using a system property.

* Started MockNetwork test

* Add driver test of Custom Serialization Scheme

* Fix detekt and minor style error

* Respond to review comments

Allow non-single arg constructors (there must be one no args
constructor), move code from SerializationEnviroment into its
own file, improve exceptions to be more user friendly.

* Fix minor bug in Scheme finding code  + improve error messages

* CORDA-4104 Improve test coverage of custom serialization scheme discovery (#6855)

* CORDA-4104 Add test of classloader scanning for CustomSerializationSchemes

* Fix Detekt

* NOTICK Clarify KDOC on SerializationSchemeContext (#6865)

* CORDA-4111 Change Component Group Serialization to use contex when the lazy map is constructed (#6856)

Currently the component group will recheck the thread local (global)
serialization context when component groups are serialized lazily.
Instead store the serialization context when the lazy map is constructed
and use that latter when doing serialization lazily.

* CORDA-4106 Test wire transaction can still be written to the ledger (#6860)

* Add test that writes transaction to the Database

* Improve test check serialization scheme in test body

* CORDA-4119 Minor changes to serialisation injection for transaction building (#6868)

* CORDA-4119 Minor changes to serialisation injection for transaction building

Scan the CorDapp classloader instead of the drivers classloader.
Add properties map to CustomSerialiaztionContext (copied from SerializationContext).
Change API to let a user pass in the serialization context in TransactionBuilder.toLedgerTransaction

* Improve KDOC + fix shawdowing issue in CordaUtils

* Pass only the properties map into theTransactionBuilder.toWireTransaction

Not the entire serializationContext

* Revert change to CordaUtils

* Improve KDOC explain pitfalls of setting properties
2021-02-11 15:27:03 +00:00
Dan Newton
cef5ea40ca Merge branch 'release/os/4.7' into dan/merge-os-4.7-to-4.8-2021-02-08 2021-02-08 14:38:17 +00:00
Dan Newton
26d6ccdc25 CORDA-4110 startFlowDynamicWithClientId permissions 2021-02-08 11:34:25 +00:00
Dan Newton
d807082cee Merge branch 'release/os/4.6' into dan/os-4.6-to-4.7-merge-2021-02-05
# Conflicts:
#	node/src/main/kotlin/net/corda/node/internal/rpc/proxies/AuthenticatedRpcOpsProxy.kt
2021-02-05 16:13:28 +00:00