Commit Graph

3339 Commits

Author SHA1 Message Date
Shams Asari
a817218b08
ENT-9806: Added peer info to SSL handshake logging, and other changes for ENT merge () 2023-06-01 15:51:58 +01:00
Adel El-Beik
e15f92b526
Merge pull request from corda/shams-crl-netty-threads-blocked
ENT-9806: Netty threads no longer blocked if CRL endpoints are unresponsive
2023-05-27 10:43:41 +01:00
Shams Asari
ed08b2c5de ENT-9793: Added Page.previousPageAnchor to allow detection of vault changes whilst pages are loaded 2023-05-26 12:52:06 +01:00
Jose Coll
2e29e36e01
ENT-9923 Ledger Recovery: split out recovery metadata into own database schema. () 2023-05-24 09:42:09 +01:00
Shams Asari
e6336666c2 Merge branch 'release/os/4.10' into shams-4.11-fwrd-merge-046ed0ac
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/crypto/X509Utilities.kt
2023-05-22 09:55:19 +01:00
Shams Asari
c065021328 ENT-8827: The ordering of vault query results is clobbered by ServiceHub.loadStates 2023-05-18 15:59:25 +01:00
Shams Asari
0cc3ffe1d6
ENT-9941: Moved new connector factory to node-api () 2023-05-16 08:45:01 +01:00
Shams Asari
0a617097be ENT-9806: Prevent Netty threads being blocked due to unresponsive CRL endpoints 2023-05-15 12:59:24 +01:00
Shams Asari
a37d09486f Merge branch 'release/os/4.9' into shams-4.10-fwrd-merge-7b7e3847 2023-05-15 09:55:31 +01:00
Shams Asari
13ca00deea ENT-9941: Moved new connector factory to node-api 2023-05-12 17:42:27 +01:00
Shams Asari
151dc16a0e Merge branch 'release/os/4.8' into shams-4.9-frwd-merge-31a34e5a
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/ArtemisTcpTransport.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/NodeNettyAcceptorFactory.kt
#	node/src/main/kotlin/net/corda/node/services/messaging/P2PMessagingClient.kt
2023-05-12 14:34:33 +01:00
Shams Asari
31a34e5a5c
ENT-9941: Improved Netty logging, especially of the embedded broker () 2023-05-12 10:11:09 +01:00
Shams Asari
0951853207
ENT-6515: Cherry-pick of ENT-6315 - Allow dumping of paused flows () ()
This is a cherry-pick of https://github.com/corda/corda/pull/7008, which also resolves ENT-6515

Co-authored-by: Dan Newton <dan.newton@r3.com>
2023-05-11 09:49:40 +01:00
Chris Cochrane
cca09fef58
Merge branch 'release/os/4.10' into cc/merge/os-4.10-os-4.11/03-may-2023
Fwd-merge from OS 4.11.
2023-05-03 17:47:44 +01:00
Jose Coll
c3e39a7052
ENT-9842 Re-factor 2PF to support issuance transactions (no notarisation) with observers. ()
Re-factor 2PF to support issuance transactions (no notarisation) with observers.
2023-04-27 16:58:17 +01:00
Chris Cochrane
2f53e8115f
Fwd-merge from OS 4.9 2023-04-27 16:45:16 +01:00
Chris Cochrane
34d4a5f667
Fwd-merge from OS 4.8 2023-04-26 22:54:20 +01:00
Rick Parker
9ba3919980
ENT-9583 Public key caching of encoded form (OS) () 2023-04-26 17:49:52 +01:00
Jose Coll
1d4feedc62
ENT-9147 Propagate and handle Notary Error (Part 2) () 2023-04-26 09:06:32 +01:00
Chris Cochrane
e12ab8194f
Merge branch 'release/os/4.7' into cc/merge/os-4.7-os-4.8/25-apr-2023
Fwd-merge from OS 4.7.
2023-04-25 14:54:21 +01:00
Chris Cochrane
91c38fa84b
Merge branch 'release/os/4.6' into cc/merge/os-4.6-os-4.7/24-apr-2023
Fwd-merge from OS 4.6.
2023-04-24 20:13:50 +01:00
Chris Cochrane
e5a6cac9e8
Flakey test fix 2023-04-21 16:00:19 +01:00
Jose Coll
190acdc87c
NOTICK Fix failing StateMachineFinalityErrorHandlingTest's. () 2023-04-20 15:45:10 +01:00
Chris Cochrane
c58c49d27a
Upgrade corda/crash, snakeyaml 2023-04-20 14:01:20 +01:00
Jose Coll
7bd3f5dd33
ENT-9147 Remove un-notarised transactions upon Double Spend. () 2023-04-19 15:31:47 +01:00
Chris Cochrane
29dfda3730
Resolved build-check issues 2023-04-18 20:34:42 +01:00
Shams Asari
2556fde971 Merge branch 'release/os/4.10' into shams-4.11-frwd-merge-fc531761 2023-04-13 10:55:03 +01:00
Shams Asari
7b6fc51710 Merge branch 'release/os/4.9' into shams-4.10-fwrd-merge-ceb76c56
# Conflicts:
#	common/logging/src/main/kotlin/net/corda/common/logging/Constants.kt
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/AMQPClient.kt
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/SSLHelper.kt
2023-04-12 16:23:59 +01:00
Shams Asari
c9fd22855e Resolve compile issues 2023-04-11 12:16:02 +01:00
Shams Asari
724a4e5a36 Merge branch 'release/os/4.8' into shams-4.9-fwrd-merge-d140cb59
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/ArtemisTcpTransport.kt
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/AMQPChannelHandler.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/CertificateRevocationListNodeTests.kt
#	node/src/main/kotlin/net/corda/node/internal/artemis/CertificateChainCheckPolicy.kt
2023-04-11 11:54:26 +01:00
Shams Asari
ddb979d693 ENT-8898: Revert change made to externalCrlSource checking
It's now moved back to createPKIXRevocationChecker(). RevocationConfig.enrichExternalCrlSource has also been removed, and moved to where it's called.
2023-04-06 14:42:38 +01:00
Shams Asari
1e6ccfdb60
ENT-8898: Replaced JDK cert revocation with custom plugable implementation () 2023-04-03 10:26:01 +01:00
Jose Coll
4beeb470df Additional signature verification and validation: recordTransactions() 2023-03-28 12:48:33 +01:00
Jose Coll
b4983597e2
ENT-6875 Two Phase Finality - CLEAN-UP ()
* Remove completed TODOs

* Prevent mis-leading progress tracker message.
2023-03-24 08:55:37 +00:00
Shams Asari
0213861d22
ENT-9569: Apply the 60s SSL handshake timeout to the embedded Artemis server () 2023-03-22 13:22:12 +00:00
Jose Coll
d2900d54ab
ENT-6875 Two Phase Finality Flow - improve ledger consistency & recoverability () 2023-03-22 10:47:51 +00:00
Mahmoud Almahroum
021c70143b
ENT-8826 Upgrade Liquibase to latest version - more review feedback () 2023-03-10 14:28:40 +00:00
Mahmoud Almahroum
1a0d354903
ENT-8983 Upgrade H2 and liquibase to latest version () 2023-03-03 15:10:25 +00:00
Adel El-Beik
044b83508e ENT-8823: Added copyBaggageToTags configuration option to control if baggage is copied to tags.
Switched SimplLogTelemetryComponent to info logs.
2022-12-08 16:54:38 +00:00
Chris Cochrane
563a7fb6c1
Merge branch 'release/os/4.9' into cc/merge/os-4.9/os-4.10/28-Nov-2022
Fwd-merge from OS-4.9.
2022-11-28 13:14:07 +00:00
rick.parker
3e25f7fd1a ENT-8817 OS port of compute deadlock changes 2022-11-25 09:31:41 +00:00
Chris Cochrane
31d9f0df09
Merge branch 'release/os/4.9' into cc/merge/os-4.9/os-4.10/24-Nov-2022
Fwd-merge from OS 4.9.
2022-11-24 18:27:54 +00:00
Chris Cochrane
0131163db0
ENT-8814 - back-fit changes from Enterprise to OS () 2022-11-24 18:13:35 +00:00
Adel El-Beik
e46b7bdd5a
ENT-6893: Added interface for clients to grab openetelemetry handle. ()
* ENT-6893: Added interface for clients to grab openetelemetry handle.

* ENT-6893: Make detekt happy.

* ENT-6893: Fix warnings.

* ENT-6893: Make detekt happy.

* ENT-6893: Now shutdown opentelemetry when node stops or client is closed.

* ENT-6893: OpenTelemetryDriver is not not a singleton.
2022-11-24 13:34:08 +00:00
Rick Parker
188027c1f5
ENT-8811 OS port of flow draining fixes () 2022-11-24 10:27:00 +00:00
Adel El-Beik
7a133f687c
ENT-6893: First cut of telemetry integration. ()
First cut of telemetry integration.

Open telemetry can be enabled in two ways, first is via an opentelemetry java agent specified on the command line. With this way you get the advantage of spans created from other libraries, like hibernate. The java agent does byte code rewriting to insert spans.
The second way is with the open telemetry driver (that links with the opentelemetry sdk). This is a fat jar provided with this project and needs to go into the node drivers directory.
2022-10-28 14:41:39 +01:00
Rick Parker
bdcd25477d
ENT-7039: Notary signature checking requires non-interned SecureHashes. ()
There was a mistake made when we first introduced notary request signature checking, in that we didn't wrap it in SerializedBytes so it always got deserialized as part of the flow message payload. So to check the signature, it has to be re-serialized. This means for cross-version compatibility we can never change the serialized format of NotarisationRequest. In this case we need make sure that every SecureHash mentioned in that data structure is a distinct instance, even if the values are repeated / identical, as that is how it was in Corda 1.

With the introduction of interning of SecureHash, this ceased to be true once again, including undoing the attempts to force it on the sending side that had been introduced in previous versions of Corda. So here we introduce a way to force it, and consolidate the forcing to distinct SecureHash instances in the NotarisationRequest itself, rather than leaving to the caller of the constructor to remember to do it, so that serialized form will always be as per Corda 1.
2022-10-28 14:03:47 +01:00
Rick Parker
b29713d7b9
ENT-6947 Intern common types to reduce heap footprint ()
ENT-6947: Implement interning for SecureHash, CordaX500Name, PublicKey, AsbtractParty and SignatureAttachmentConstraint, including automatic detection of internable types off companion objects in AMQP & Kyro deserialization.  In some cases, add new factory methods to companion objects, and make main code base use them.

Performance tested in performance cluster with no negative impact visible (so default concurrency setting seems okay).

Testing suggests 5-6x memory saving for tokens in TokensSDK in memory selector.  Should see approx. 1 million tokens per GB or better (1.5 million for the tokens we tested with).
2022-10-18 09:28:41 +01:00
Chris Cochrane
5ca5b8d096
Do not black-list AMQP targets that suffer a handshake failure 2022-09-13 11:41:19 +01:00
Chris Cochrane
54fdd12a2d
ENT-6866 support quick RPCs that are not processed via the RPC thread pool ()
* ENT-6866 support quick RPCs that are not processed via the RPC thread pool
2022-07-07 12:33:51 +01:00
HJ Kim
0a59d578ff Merge branch 'release/os/4.9' into hjkim/os-4.9-4.10-merge-24062022
This merge includes a bugfix ENT-6714 which Corda logs database
password.
2022-06-24 16:08:16 +01:00
HJ Kim
8d34be5129 ENT-6714: Fix Corda logging database password
This commit ports the previously implemented fix from Corda ENT. Due to
the unrelated changes and merge conflict, the fix has been manually
copied rather than cherry-picked.
2022-06-23 23:03:10 +01:00
Chris Cochrane
199f635d69
Attempt to fix service priority tests when they are run on Jenkins 2022-06-15 11:56:03 +01:00
Chris Cochrane
c57bbd1c39
Added integration tests for service event priorities 2022-06-14 17:15:31 +01:00
Chris Cochrane
03fa529292
ENT-6791 New service lifecycle event published just before starting the state machine () 2022-06-10 19:56:02 +01:00
Chris Cochrane
f0d530ab8b
Post review updates; renamed tests/files, neater Kotlin test code 2022-06-08 16:30:06 +01:00
Chris Cochrane
4b1aabb164
Update node/src/integration-test/kotlin/net/corda/node/jmx/NodeStatus.kt
Co-authored-by: Viktor Kolomeyko <viktor.kolomeyko@r3.com>
2022-06-08 16:01:43 +01:00
Chris Cochrane
0232146515
Update node/src/integration-test/kotlin/net/corda/node/jmx/Publish.kt
Co-authored-by: Viktor Kolomeyko <viktor.kolomeyko@r3.com>
2022-06-08 16:01:25 +01:00
Chris Cochrane
3c0c1a01a2
Update node/src/integration-test/kotlin/net/corda/node/jmx/Publish.kt
Co-authored-by: Viktor Kolomeyko <viktor.kolomeyko@r3.com>
2022-06-08 15:50:48 +01:00
Chris Cochrane
fce95dd276
Update node/src/integration-test/kotlin/net/corda/node/jmx/NodeStatus.kt
Co-authored-by: Viktor Kolomeyko <viktor.kolomeyko@r3.com>
2022-06-08 15:50:34 +01:00
Chris Cochrane
518befabeb
Resolved detekt error 2022-06-08 13:54:16 +01:00
Chris Cochrane
500be9cbcd
Post-review updates; added integration tests 2022-06-08 13:36:50 +01:00
Chris Cochrane
b76fc7de59
Introduced node status; published via JMX 2022-06-07 16:20:05 +01:00
Chris Rankin
47ec57186e Merge commit '91ef44dc12b8cc921b9ec0a37b44de75c256d92d' into chrisr3-os-4.8-merge 2022-05-23 12:27:49 +01:00
Chris Rankin
57ddafa148 ENT-6532: Close any deserialization classloader that is evicted from the cache.
Also revert disabling caches when reading URLs.
2022-05-20 11:51:01 +01:00
Adel El-Beik
ad5193f135 ENT-6636: Removing commons-codec - not used, at least not as a compile time dep. 2022-05-10 13:46:48 +01:00
Adel El-Beik
bc6a6ee2ae
Merge pull request from corda/adel/ENT-6743
ENT-6743: SuspensionMeta in FlowInfo is null when a runnable flow has previously been hospitalized
2022-04-26 17:21:30 +01:00
Adel El-Beik
c1002697c7 ENT-6743: Reverted RetryFlowMockTest as the retry op only available on ENT. 2022-04-25 11:30:24 +01:00
Adel El-Beik
fa607024c2 ENT-6743: Contains method of flow hospital now correctly returns if flow is in hospital. Historic patient records not used. 2022-04-25 11:24:21 +01:00
Adel El-Beik
54a131313d ENT-6728: Archiving of a legal identity now done in its own txn, to cope with clustered notary nodes that have common legal identities between nodes (the service name one is common) 2022-03-15 12:25:39 +00:00
Rick Parker
1e227fdfc4
ENT-6711 - Fix for upgrade to Jackson 2.12 and beyond (actual upgrade to 2.13.1) ()
* NOTICK - Don't know what the JIRA is but wanted to share.

* Updates to resolve bukld issues

* NOTICK: Fixed JDK11 version to prevent capsule version error

* ENT-6711: Added comment for use of jackson_kotlin_version.

* ENT-6711: Avoid deprecation warning, switched to the default method.

Co-authored-by: Chris Cochrane <chris.cochrane@r3.com>
Co-authored-by: Adel El-Beik <adel.el-beik@r3.com>
2022-03-07 11:08:06 +00:00
Adel El-Beik
4594c76259 ENT-6637: Targeted exclude of the jgroups dependency. 2022-03-04 11:52:14 +00:00
Adel El-Beik
4203a56c3a ENT-6637 & ENT-6618: Removed jgroups and dbcp from artemis dependencies. 2022-03-03 16:40:18 +00:00
Conal Smith
5f1e86c17c
ENT-6508 - Prevent directory traversal from zip file names () 2022-02-25 15:51:49 +00:00
Dan Newton
4b29ad337b
ENT-6710 Corda shell ssh logging again ()
Correct SSH logging again, by pulling the logging of SSH ports to a
singular location.
2022-02-25 15:47:31 +00:00
Viktor Kolomeyko
162f76f710
ENT-6693: Change the way how message sizes are determined ()
Especially relevant for `AMQPLargeMessage`.
2022-02-24 13:54:27 +00:00
Adel El-Beik
ba8cd1b80d Merge remote-tracking branch 'origin/release/os/4.8' into adel/merge-from-4.8-21Feb 2022-02-21 12:52:29 +00:00
Adel El-Beik
20d2e2b3eb
ENT-6677: When you clear the cache via the command line it now also updates node_named_identities table. () 2022-02-18 15:04:35 +00:00
Adel El-Beik
c11eb382ce
ENT-6588 Restrict database operations platform flag () () () ()
* ENT-6588 Restrict database operations platform flag

Put the restricting of database operations in `RestrictedConnection` and
`RestrictedEntityManager` behind a platform version flag.

`RESTRICTED_DATABASE_OPERATIONS = 7` was added to signify this.

If the version is less than 7, then the database operations will not be
restricted. A warning is logged to indicate that they are using
potentially dangerous methods.

If the version is 7 or greater, then the database operations are
restricted and throw an error if called.

Co-authored-by: Dan Newton <dan.newton@r3.com>
2022-02-18 09:42:59 +00:00
Adel El-Beik
470bce8a6b
ENT-6588 Restrict database operations platform flag () () ()
* ENT-6588 Restrict database operations platform flag

Put the restricting of database operations in `RestrictedConnection` and
`RestrictedEntityManager` behind a platform version flag.

`RESTRICTED_DATABASE_OPERATIONS = 7` was added to signify this.

If the version is less than 7, then the database operations will not be
restricted. A warning is logged to indicate that they are using
potentially dangerous methods.

If the version is 7 or greater, then the database operations are
restricted and throw an error if called.

Co-authored-by: Dan Newton <dan.newton@r3.com>

Co-authored-by: Dan Newton <dan.newton@r3.com>
2022-02-17 16:59:34 +00:00
Adel El-Beik
12fd451919
ENT-6588 Restrict database operations platform flag () ()
* ENT-6588 Restrict database operations platform flag

Put the restricting of database operations in `RestrictedConnection` and
`RestrictedEntityManager` behind a platform version flag.

`RESTRICTED_DATABASE_OPERATIONS = 7` was added to signify this.

If the version is less than 7, then the database operations will not be
restricted. A warning is logged to indicate that they are using
potentially dangerous methods.

If the version is 7 or greater, then the database operations are
restricted and throw an error if called.

Co-authored-by: Dan Newton <dan.newton@r3.com>
2022-02-17 14:07:11 +00:00
Dan Newton
7afb585ae2
ENT-6676 Don't log SSH port if shell not installed () 2022-02-11 14:05:05 +00:00
Dan Newton
fa739fc467
ENT-6588 Restrict database operations platform flag ()
* ENT-6588 Restrict database operations platform flag

Put the restricting of database operations in `RestrictedConnection` and
`RestrictedEntityManager` behind a platform version flag.

`RESTRICTED_DATABASE_OPERATIONS = 7` was added to signify this.

If the version is less than 7, then the database operations will not be
restricted. A warning is logged to indicate that they are using
potentially dangerous methods.

If the version is 7 or greater, then the database operations are
restricted and throw an error if called.
2022-02-08 10:06:44 +00:00
Adel El-Beik
5c052e9273
ENT-6586: Validate when signing that output states can be deserialized. ()
* ENT-6586: Log warning instead of debug if state cannot be deserialized.
2022-02-07 18:05:59 +00:00
Chris Rankin
758a69f904
Validate LedgerTransaction deserialised from AttachmentsClassLoader. () () 2022-01-31 09:40:01 +00:00
Chris Rankin
c955093ce8
Validate LedgerTransaction deserialised from AttachmentsClassLoader. () 2022-01-28 17:05:59 +00:00
Adel El-Beik
7752fc8c9d
CORDA-4189: Removed existing vendor test () 2022-01-27 13:49:08 +00:00
Adel El-Beik
fb624a2b06
CORDA-4189: Now compare against Corda Community Edition as well. ()
* CORDA-4189: Now compare against Corda Community Edition as well.

* CORDA-4189: Removed unused imports.
2022-01-26 11:33:23 +00:00
Ramzi El-Yafi
831f6d60cc Merge remote-tracking branch 'origin/release/os/4.8' into ramzi/os-48-49-merge 2022-01-21 10:55:29 +00:00
Viktor Kolomeyko
835321bb70
ENT-6286: OS: Upgrade version of Artemis to 2.19.1 ()
Fixes DDoS attack mentioned on the Jira ticket.

PR upgrades Artemis library to version 2.19.1.
This is our own release of Apache Artemis library which has vulnerability fix for v2.20 applied.

**_Breaking changes discovered during Artemis upgrade:_**
1. When the queue is created as temporary, it needs to explicitly be specified as non-durable.
2. By default, Artemis Client performs Host DNS name check against the certificate presented by the server. Our TLS certificates fail this check and this verification has to be explicitly disabled, see use of: `TransportConstants.VERIFY_HOST_PROP_NAME`.
3. Artemis Server now caches login attempts, even unsuccessful ones. When we add RPC users dynamically via DB insert this may have an unexpected outcome if the user with the same `userName` and `password` was not available previously.
To workaround permissions changing dynamically, authorization and authentication caches had to be disabled.
4. When computing `maxMessageSize`, the size of the headers content is now taken into account as well.
5. Artemis handling of start-up errors has changed. E.g. when the port is already bound.
6. A number of deprecated APIs like: `createTemporaryQueue`, `failoverOnInitialAttempt`, `NullOutputStream`, `CoreQueueConfiguration`.
7. Log warning message is produced like: `AMQ212080: Using legacy SSL store provider value: JKS. Please use either 'keyStoreType' or 'trustStoreType' instead as appropriate.`
8. As reported by QA, Artemis now produces more audit logging more details [here](https://r3-cev.atlassian.net/browse/ENT-6540). Log configuration been adjusted to reduce such output.
2022-01-21 09:18:14 +00:00
Ramzi El-Yafi
ba4d275887 Merge remote-tracking branch 'origin/release/os/4.6' into ramzi/os-46-47-merge 2022-01-20 17:51:36 +00:00
Ramzi El-Yafi
c9d758f84a Merge remote-tracking branch 'origin/release/os/4.5' into ramzi/os-45-46-merge 2022-01-20 14:55:56 +00:00
Adel El-Beik
b17e4571bf
ENT-6386: Revert change of behaviour in rpcOps.wellKnownPartyFromX500Name for revoked identity ()
* ENT-6386: Reverting wellKnownPartyFromX500Name functionality to work … ()
2022-01-20 11:08:13 +00:00
Ramzi El-Yafi
4dfd4c613a Merge remote-tracking branch 'origin/release/os/4.4' into ramzi/os-44-45-merge 2022-01-20 10:19:16 +00:00
Ramzi El-Yafi
a45be650c7
ENT-6533 Update validating notary flow to be non-idempotent () 2022-01-20 09:52:45 +00:00
Dan Newton
c05c1934cf
ENT-6547 Remove url classloader in shell loading ()
The `corda-shell` jar will already be installed if it exists in the the
node's `/drivers` directory. There is no need to include a
`URLClassLoader` to load its classes.

Rely on the process's main classloader.
2022-01-19 18:26:26 +00:00
Ramzi El-Yafi
caa4de47d3
ENT-6533 Update validating notary flow to be non-idempotent () 2022-01-19 13:15:51 +00:00
Lorcan Wogan
c2140f727c
ENT-6025 remote artemis channel does not exist resulting in infinite retry loop ()
* ENT-6025 remote artemis channel does not exist resulting in infinite retry loop

* ENT-6025 rename test

* ENT-6025 fix detekt and add description

* ENT-6025 add check on count of connected stack
2022-01-18 10:13:29 +00:00
Chris Rankin
552317f2b3 Merge branch 'release/os/4.8' into chrisr3-os48-merge 2022-01-17 11:02:11 +00:00
Chris Rankin
4f1a07cbcc
ENT-6548: Ensure LazyMappedList is realised with correct SerializationContext. () 2022-01-17 09:57:33 +00:00
Dan Newton
bd9b50bf7c ENT-6376 Don't hospitalize session end in ReceiveFinalityFlow ()
Do not keep a flow in for observation if it receives an unexpected
session end message while in `ReceiveFinalityFlow` and
`ReceiveTransactionFlow` (due to being called by the former).

This is done by checking the message of the `UnexpectedFlowEndException`
that is thrown when a session end message instead of a data message and
if the stacktrace has `ReceiveTransactionFlow` at the top, after
removing statemachine stack frames.

Checking the stacktrace for `ReceiveTransactionFlow` is important
because the unexpected session end session message is only ok if a
transaction has not already been received. For example, if
`ResolveTransactionsFlow` is in the stack, then this indicates failure
when receiving transaction dependencies on a transaction that should be
recorded.

Also added a test that highlights that the `UnexpectedFlowEndException`
caused by the session end message can be caught, therefore users can
determine their own behaviour if desired.
2022-01-13 11:02:46 +00:00
Dan Newton
56c9d6404f
ENT-6378 Migrate corda-shell to external repo ()
Remove the shell code from the OS code base, this includes the modules:

- `:tools:shell`
- `:tools:shell-cli`

The shell will be run within a node if it exists within the node's `drivers` directory. 

This is done by using a `URLClassloader` to load the `InteractiveShell` class into Corda's JVM process and running `startShell` and `runLocalShell`.

Running the shell within the `:samples` will require adding:

```
cordaDriver "net.corda:corda-shell:<corda_shell_version>"
```

To the module's `build.gradle` containing `deployNodes`. The script will then include the shell in the created nodes.
2022-01-12 11:54:18 +00:00
Dan Newton
f2b3db9c7b
ENT-6315 Allow dumping of paused flows ()
Checkpoint dumping of paused flows was not working because the dumper
expects a flow to have a `FlowState` of `Unstarted` or `Started`,
however due to a memory optimisation paused flows have their `FlowState`
set to `Paused`. This was causing causing an exception as well as a loss
of potentially useful information.

A flag `alwaysDeserializeCheckpoint` has been added to
`Checkpoint.Serialized.deserialize` which skips the memory optimisation
and forces the deserialization of the flow's `FlowState`.

Paused flows are now included in the dumped output along with their real
`FlowState` which is useful to users even if the flow is paused rather
than waiting for something to complete.

The status of the flow has also been added to the JSON output to assist
users in debugging their flows.
2022-01-11 10:22:49 +00:00
Dan Newton
044202550d
ENT-6421 Public version of FlowManagerRPCOps ()
A public version of `FlowManagerRPCOps` which does not live in an
internal package has been added. This new interface shares the same name
as the internal one.

Because of the name sharing, the internal version has been
`@Deprecated`.

`FlowManagerRPCOpsImpl` implements both the new and old interfaces. This
 allows for backwards compatibility, allowing old shells or clients to
 call the old interface on newer nodes without breaking.
2022-01-07 16:40:29 +00:00
Viktor Kolomeyko
505ade9c86
ENT-6388: Eliminate usage of Mina's ConcurrentHashSet ()
* ENT-6388: Eliminate usage of Mina's `ConcurrentHashSet`

* ENT-6388: Restrict to `MutableSet<ScheduledStateRef>`
2021-12-02 13:24:11 +00:00
Kyriakos Tharrouniatis
3cbfb0e024
ENT-6355 Empty flow arguments on error transition ()
A user passed in a `FlowLogic` as an argument into another `FlowLogic`
called `subFlow` on it and had it throw an exception.

This all occurred before the first checkpoint, causing the state machine
to try and persist a FAILED checkpoint containing the flow's arguments.
Because the arguments contained a `FlowLogic` that had been started via
`subFlow` it held a reference to `FlowLogic._stateMachine` which cannot
be serialized.

This caused the flow to fail when trying to persist the fact that it
failed.

The flow arguments are now emptied during `ErrorFlowTransition` to
resolve this issue which mimics the behaviour of the first suspend.
Note, this only takes the arguments out of the serialized checkpoint, it
does not affect the flow metadata and therefore a flow's arguments can
still be viewed.

Co-authored-by: Dan Newton <dan.newton@r3.com>
2021-12-01 14:57:30 +00:00
Chris Rankin
e93e7c2846
ENT-6412: Upgrade ClassGraph 4.8.90 -> 4.8.135. () 2021-11-23 09:43:41 +00:00
Chris Rankin
caeafb8201
ENT-6357: Deserialize LedgerTransaction elements for each Contract.verify(). ()
* ENT-6357: Deserialize LedgerTransaction elements for each Contract.verify().

* Lock the LedgerTransaction and NetworkParameters objects down for contract verification.

* Refactor BasicVerifier to be package private instead of public.

* Simplify verifyConstraints() operation.

* Review fixes: replace HashSet with LinkedHashSet, and add signing parties to commands via mapIndexed.

* Ensure tests also run notary nodes "out of process".

* Streamline SerializationContext switching.

* Cache deserialised cryptographic instances during contract verification.

* Invoke Class.forName() instead of ClassLoader.loadClass() to reduce contention on the system classloader's lock.

* Deserialization cache key now pre-computes its hash code.

* Allow AttachmentsClassLoader to be used concurrently.

* Cache all Envelope objects for reuse during contract verification.

* Generate CertPathProxy hash code using conventional algorithm.

* Adjust CustomSerializer.Proxy to allow better access to SerializationContext.
2021-11-10 16:38:40 +00:00
Walter Oggioni
883e794853
ENT-6330 Fixed reading jar entries in memory ()
* ENT-6330 Fixed reading jar entries in memory

This is a trivial fix that is however enough to allow to send zip bombs as attachments without the node crashing, a size limit could be added for increased reliability

* added attachment cumulative size check

* added compression ratio check

* added unit test and moved the code to a standalone verifier object

* removed attachment check from AttachmentClassLoader to minimize performance impact
2021-10-13 09:46:20 +01:00
Agnieszka Szczepanska
0b9513ffa3 Merge branch 'release/os/4.7' into aga/make-recordDependencies-suspendable-4.8 2021-04-27 15:08:13 +01:00
Agnieszka Szczepanska
9aa4389902 resolve merge conflicts 2021-04-26 11:35:17 +01:00
Agnieszka Szczepanska
00c43e501c delete duplicated imports 2021-04-21 12:31:03 +02:00
Agnieszka Szczepanska
88f584cac5 fast forward merge 2021-04-21 12:16:22 +02:00
Agnieszka Szczepanska
c48f32215c Merge branch 'release/os/4.4' into aga/make-recordDependencies-suspendable-4.5 2021-04-20 16:25:47 +02:00
Agnieszka Szczepanska
8dd9dee9c4
NOTICK - make recordDependencies suspendable ()
* add suspendable annotation to recordDependencies

* add suspendable annotation to recordDependencies implementation
2021-04-14 08:30:53 +01:00
Stefan Iliev
7b9b2569a9
NAAS-107 StartupQueueTime metric fix ()
* Fixed StartupQueueTime metric producing negative numbers.
2021-03-09 18:00:04 +00:00
Ramzi El-Yafi
fe8baf4b17
NAAS-49 Flag notary/extraConfig block as sensitive () 2021-02-19 17:05:23 +00:00
Adel El-Beik
9cd02dc62d Merge remote-tracking branch 'origin/release/os/4.4' into adel/merge-from-4.4-19-Feb-21 2021-02-19 13:55:04 +00:00
Denis Rekalov
23fab7ae0c
ENT-5891 Disable flaky test () 2021-02-19 13:54:34 +00:00
Adel El-Beik
360b3f8d0c
CORDA-4125: Backport AttachmentClassLoader updates to 4.4 to enable ENT-6152 backport ()
* CORDA-3755: Switched attachments map to a WeakHashMap ()

* Bump OS release version 4.6

* CORDA-3755: Switched attachments map to a WeakHashMap

* CORDA-3755: Added explicit strong references to map key.

* CORDA-3755: Keeping detekt happy.

* CORDA-3755: Test a gc in verify.

* CORDA-3755: Making detekt happy.

* CORDA-3755: Suppress warnings for weak reference test.

* CORDA-3755: Fixing build failure with attachments.

* CORDA-3755: Rewrite based on Ricks input - now handles attachment already existing in map!

* CORDA-3755: Refactor WeakReference behaviour into AttachmentsHolderImpl and provide alternate version of this class for core-deterministic.

* CORDA-3755: Added more tests for WeakHashMap.

* CORDA-3755: Ignore the tests using System.gc keep for local testing only

* CORDA-3755: Adding comment to explain the ignored tests.

* Make AttachmentsHolderImpl package-private inside core-deterministic, just like it is inside core.

* CORDA-3755: Update assertions following review comments.

* CORDA-3755: Removing import

* CORDA-3755: Removed unused var.

* CORDA-3755: Reverting files that somehow got changed in rebase.

Co-authored-by: nargas-ritu <ritu.gupta@r3.com>
Co-authored-by: Chris Rankin <chris.rankin@r3.com>

* CORDA-3769: Switched attachments class loader cache to use caffeine ()

* CORDA-3769: Switched attachments class loader cache to use caffeine with original implementation used by determinstic core.

* CORDA-3769: Removed default ctor arguments.

* CORDA-3769: Switched mapping function to Function type to avoid synthetic method being generated.

* CORDA-3769: Now using a cache created from NamedCacheFactory for the attachments class loader cache.

* CORDA-3769: Making detekt happy.

* CORDA-3769: The finality tests now check for UntrustedAttachmentsException which will actually happen in reality.

* CORDA-3769: Refactored after review comments.

* CORDA-3769: Removed the AttachmentsClassLoaderSimpleCacheImpl as DJVM does not need it. Also updated due to review comments.

* CORDA-3769: Removed the generic parameters from AttachmentsClassLoader.

* CORDA-3769: Removed unused imports.

* CORDA-3769: Updates from review comments.

* CORDA-3769: Updated following review comments. MigrationServicesForResolution now uses cache factory. Ctor updated for AttachmentsClassLoaderSimpleCacheImpl.

* CORDA-3769: Reduced max class loader cache size

* CORDA-3769: Fixed the attachments class loader cache size to a fixed default

* CORDA-3769: Switched attachments class loader size to be reduced by fixed value.

* CORDA-4125: Parameter has been added to a private ctor.

Co-authored-by: nargas-ritu <ritu.gupta@r3.com>
Co-authored-by: Chris Rankin <chris.rankin@r3.com>
2021-02-19 13:38:22 +00:00
Dan Newton
41b4f9b3aa
CORDA-4110 startFlowDynamicWithClientId restrict permissions ()
Restrict permissions for `startFlowDynamicWithClientId` to not work with
 `InvokeRpc.startFlow` and instead require `InvokeRpc
 .startFlowWithClientId`.
2021-02-18 14:46:09 +00:00
William Vigor
7261442c98
CORDA-4123 Change MockNetworkParameterStorage to not rehash NetworkParameters ()
* CORDA-4123 Change Mock Network Parameter Storage to not rehash the Network Parameters

* Fix Detekt
2021-02-16 16:25:38 +00:00
Adel El-Beik
1a7401472f
CORDA-4120: Disable the URLConnection cache to prevent file handle leaks in ServiceLoader. () 2021-02-12 17:24:00 +00:00
William Vigor
20dbbf008d
CORDA-4103 Feature Branch: Serialization injection for transaction building ()
* CORDA-4105 Add public API to allow custom serialization schemes ()

* CORDA-4105 Add public API to allow custom serialization schemes

* Fix Detekt

* Suppress warning

* Fix usused import

* Improve API to use generics

This does not break Java support (only Intelij gets confused).

* Add more detailed documentation to public interfaces

* Change internal variable name after rename public API

* Update Public API to use ByteSquence instead of SerializedBytes

* Remove unused import

* Fix whitespace.

* Add added public API to .ci/api-current.txt

* Improve public interfaces

Rename CustomSchemeContext to SerializationSchemeContext to improve
clarity and move to it's own file. Improve kdoc to make things less
confusing.

* Update API current with changed API

* CORDA-4104 Implement custom serialization scheme discovery mechanism ()

* CORDA-4104 Implement CustomSerializationScheme Discovery Mechanism

Discovers a single CustomSerializationScheme implementation inside
the drivers dir using a system property.

* Started MockNetwork test

* Add driver test of Custom Serialization Scheme

* Fix detekt and minor style error

* Respond to review comments

Allow non-single arg constructors (there must be one no args
constructor), move code from SerializationEnviroment into its
own file, improve exceptions to be more user friendly.

* Fix minor bug in Scheme finding code  + improve error messages

* CORDA-4104 Improve test coverage of custom serialization scheme discovery ()

* CORDA-4104 Add test of classloader scanning for CustomSerializationSchemes

* Fix Detekt

* NOTICK Clarify KDOC on SerializationSchemeContext ()

* CORDA-4111 Change Component Group Serialization to use contex when the lazy map is constructed ()

Currently the component group will recheck the thread local (global)
serialization context when component groups are serialized lazily.
Instead store the serialization context when the lazy map is constructed
and use that latter when doing serialization lazily.

* CORDA-4106 Test wire transaction can still be written to the ledger ()

* Add test that writes transaction to the Database

* Improve test check serialization scheme in test body

* CORDA-4119 Minor changes to serialisation injection for transaction building ()

* CORDA-4119 Minor changes to serialisation injection for transaction building

Scan the CorDapp classloader instead of the drivers classloader.
Add properties map to CustomSerialiaztionContext (copied from SerializationContext).
Change API to let a user pass in the serialization context in TransactionBuilder.toLedgerTransaction

* Improve KDOC + fix shawdowing issue in CordaUtils

* Pass only the properties map into theTransactionBuilder.toWireTransaction

Not the entire serializationContext

* Revert change to CordaUtils

* Improve KDOC explain pitfalls of setting properties
2021-02-11 15:27:03 +00:00
Dan Newton
cef5ea40ca Merge branch 'release/os/4.7' into dan/merge-os-4.7-to-4.8-2021-02-08 2021-02-08 14:38:17 +00:00
Dan Newton
26d6ccdc25 CORDA-4110 startFlowDynamicWithClientId permissions 2021-02-08 11:34:25 +00:00
Dan Newton
d807082cee Merge branch 'release/os/4.6' into dan/os-4.6-to-4.7-merge-2021-02-05
# Conflicts:
#	node/src/main/kotlin/net/corda/node/internal/rpc/proxies/AuthenticatedRpcOpsProxy.kt
2021-02-05 16:13:28 +00:00
Dan Newton
4e437ace2a
CORDA-4110 startFlowDynamicWithClientId permissions () 2021-02-05 16:05:55 +00:00
Dan Newton
88172b630d ENT-6142 Use ArrayList for SessionState structures ()
Prevent some serialization errors that occur due to serialization
and deserialization of `ArrayList$SubList` found inside the
`SessionState` data structures.

To prevent this, an explicit `ArrayList` is used rather than a `List`.

Overload the `List` operator functions so that `+` returns an
`ArrayList` instead of a `List`.

Create `toArrayList` for a few conversions.
2021-01-29 11:15:28 +00:00
Dan Newton
c79ad972d0 ENT-6142 Flows become dead due to uncaught exceptions ()
If a flow fails outside of its normal error processing code path it will
end up in `FlowDefaultUncaughtExceptionHandler`.

This handler will put the flow into overnight observation if possible.
This is done in-memory and the database.

Even with this being done, the fiber itself has blown up and therefore
does not manage to get to `SMM.removeFlow` which is where
`SMM.decrementLiveFibers` is called. For example, a flow that errored
will hit this code eventually. This code is also hit when a flow is
suspended and a shutdown event is sent to it.

The `liveFibers` latch blocks the SMM from shutting down until all flows
have finished or processed shutdown events.

The changes described below resolve this problem.

Any flow that goes to the `FlowDefaultUncaughtExceptionHandler` will be
put marked as dead (`StateMachineState.isDead`). Highlighting that the
flow cannot continue to process events normally as it has broken out
of its event loop

Retrying and shutdown are done manually rather than injecting events
into the flow fiber's queue, because it can't execute its event loop.

Killing a dead flow executes an altered version of
`retryFlowFromSafePoint`. It does this so it can delete the checkpoint
and then continue using the checkpoint it just deleted to run the
kill flow transition on a new fiber.

If a killed flow reaches the `FlowDefaultUncaughtExceptionHandler` it
will be forcibly killed via `killFlowForcibly` which deletes the
checkpoint/or updates it to KILLED and then calls `removeFlow` to bypass
any event processing. This means that a flow that was dead and was killed
will be terminated manually if it reaches the handler again. The same is
true for flows that were not dead before but reached the handler after
being killed.

Also, `FlowCreator.createFlowFromCheckpoint` now retains the `isKilled`
state of the previous fiber's state.
2021-01-29 11:15:28 +00:00
Alexey Koren
56df286410
Add ability to set custom initiating flows for NotaryService () 2021-01-26 13:56:42 +00:00
Adel El-Beik
284fd48918
CORDA-4101: Remove attachmentPresence cache ()
* CORDA-4101: Remove the attachmentCache and just rely on attachmentContentCache

* CORDA-4101: Remove the attachmentCacheBound variable.
2021-01-13 16:28:56 +00:00
nkovacsx
be6587a602 Merge branch 'release/os/4.7' into nandor/4.7-4.8-merge-os 2021-01-08 14:25:33 +00:00
nkovacsx
7a079a166b
CORDA-4097 Return stream instead of List in getAllCertificates () 2021-01-08 09:41:31 +00:00
Denis Rekalov
f5d9d97573 Merge branch 'release/os/4.7' into denis/merge-4.7-to-4.8 2020-12-14 10:56:23 +00:00
Stefan Iliev
2c25a5c768
AG-323 Flow startup queue metric () 2020-11-30 16:49:08 +00:00
Denis Rekalov
503644ca29
CORDA-4083: Fix Postresql migration, synch migration scripts from ENT () 2020-11-24 20:03:36 +00:00
Edoardo Ierina
02018b75e9
ENT-6018: renamed hash agility system properties () 2020-11-17 12:50:31 +00:00
Waldemar Żurowski
700cb51091
INFRA-965: forward merge from 4.6 2020 11 15 ()
* CORDA-4045 Disable flaky test ()

Disable `Restart does not set senderUUID` as it is unstable.

* INFRA-965: forward merge from 4.5 (2020-11-15) ()

* EG-4168 Updating contributors.md list for OS 4.5 release branch ()

* INFRA-965: forward merge from 4.4 (2020-11-14) ()

* EG-4168 Updating contributors.md list for OS 4.4 release branch ()

* INFRA-965: forward merge from 4.3 (2020-11-13) ()

* INFRA-965: Jenkins/NexusIQ integration should target patches ()

as well as major/minor releases

* Updated nightly builds to match changes for release branch builds

* Updated JDK11 builds to match changes for release branch builds

Co-authored-by: ivanterziev <61829352+ivanterziev-r3@users.noreply.github.com>

* Backporting updates for check-pr-title workflow

Co-authored-by: ivanterziev <61829352+ivanterziev-r3@users.noreply.github.com>

Co-authored-by: Ross Nicoll <ross.nicoll@r3.com>
Co-authored-by: ivanterziev <61829352+ivanterziev-r3@users.noreply.github.com>
2020-11-16 07:18:22 +00:00
Edoardo Ierina
494654cea6
CORDA-3823: hash agility updates for rc03 ()
* wip

* wip

* wip (need to review IEE comments)

* wip

* wip

* Small refactoring, fixed network-verifier's TestNotaryFlow

* Added command line option to explicitly enable hash agility support

* wip-do-not-push

* wip

* wip

* wip

* aligned merkletree/transaction hash algorithms

* wip

* Added mixed algorithm support for nodes vs leaves and corrected mixed algorithm tests

* moved global computeNonce and componentHash to DigestService

* added comment for failing test to fix

* wip

* Minor cleanups, added deprecated componentHash/computeNonce

* restored exploratory changes to failing SignedTransaction test

* cleaned up and minor rafactoring

* Fixed some tests with hardcoded hash algorithm

* some changes and cleanups following code review

* WIP commit before large change

* WIP Fixed 3 tests

* WIP removed direct references to randomSHA256() and sha256()

* Updated/added liquibase migrations to support larger hash algorithms

* Reviewed, cleanups, comments, fixes

* removing direct references to sha256()

* WIP verifying obligations test errors

* reviewing obligation/attachment issues with sha3_256

* Full review before PR - intermediate commits

* Reviewed and cleaned up

* Futher cleanup

* Fixed partial tree backward compatible json and cleanups

* all tests passing

* Removed couple of unused imports

* Reworked global componentHash function to avoid deprecated warnings

* replaced SHA3s with some alternate SHA2s

* Removed SHA3-256 and SHA3-512 references

* fixed some tests using non ubiquitous hash algorithms

* Fixed ABI compatibility (not for TransactionBuilder)

* Fixed ABI compatibility to TransactionBuilder

* couple of fixes

* fixed DigestService's randomHash

* Removed constructor with loosely typed args for private constructor of LedgerTransaction class (API removal)

* re-introduced LedgerTransaction deprecated ctor for deserialization

* Add possibility to load CustomMessageDigest bypassing JCA ()

* Change api-current for DigestAlgorithm

* disable flaky tests

* addressed liquibase migration script versions

* Removed TODOs and cleanups

* relaxed privacy salt validation

* Fixed privacy salt test to comply with relaxed validation

* detekt and privacySalt validation

* diff cleanup

* diff cleanup

* removed unused import

* removed PrivacySalt's validateFor method and references

* removed invalid character

Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
2020-11-12 17:03:43 +00:00
Denis Rekalov
14e545826c
CORDA-4076: Fix SecureHash compatibility with previous versions () 2020-11-12 09:39:57 +00:00
Denis Rekalov
d9f905cb81
Disable enhanced validation, rename SQL patches () 2020-11-06 12:32:41 +00:00
Edoardo Ierina
82a114a329
[DRAFT] feat/CORDA-3823-hash-agility-qa-ready ()
* wip

* wip

* wip (need to review IEE comments)

* wip

* wip

* Small refactoring, fixed network-verifier's TestNotaryFlow

* Added command line option to explicitly enable hash agility support

* wip-do-not-push

* wip

* wip

* wip

* aligned merkletree/transaction hash algorithms

* wip

* Added mixed algorithm support for nodes vs leaves and corrected mixed algorithm tests

* moved global computeNonce and componentHash to DigestService

* added comment for failing test to fix

* wip

* Minor cleanups, added deprecated componentHash/computeNonce

* restored exploratory changes to failing SignedTransaction test

* cleaned up and minor rafactoring

* Fixed some tests with hardcoded hash algorithm

* some changes and cleanups following code review

* WIP commit before large change

* WIP Fixed 3 tests

* WIP removed direct references to randomSHA256() and sha256()

* Updated/added liquibase migrations to support larger hash algorithms

* Reviewed, cleanups, comments, fixes

* removing direct references to sha256()

* WIP verifying obligations test errors

* reviewing obligation/attachment issues with sha3_256

* Full review before PR - intermediate commits

* Reviewed and cleaned up

* Futher cleanup

* Fixed partial tree backward compatible json and cleanups

* all tests passing

* Removed couple of unused imports

* Reworked global componentHash function to avoid deprecated warnings

* replaced SHA3s with some alternate SHA2s

* Removed SHA3-256 and SHA3-512 references

* fixed some tests using non ubiquitous hash algorithms

* Fixed ABI compatibility (not for TransactionBuilder)

* Fixed ABI compatibility to TransactionBuilder

* couple of fixes

* fixed DigestService's randomHash

* Removed constructor with loosely typed args for private constructor of LedgerTransaction class (API removal)

* re-introduced LedgerTransaction deprecated ctor for deserialization

* Add possibility to load CustomMessageDigest bypassing JCA ()

* Change api-current for DigestAlgorithm

* disable flaky tests

Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
2020-11-05 22:05:29 +00:00
Ross Nicoll
f2dd3ac652
CORDA-4045 Disable flaky test ()
Disable `Restart does not set senderUUID` as it is unstable.
2020-11-01 21:40:55 +00:00
Dan Newton
6b5e76f87a Merge branch 'release/os/4.6' into dan/os-4.6-to-4.7-merge-2020-10-30 2020-10-30 16:08:15 +00:00
Dan Newton
72c359b165
ENT-5886 Parameterised queries in DBCheckpointStorage ()
Direct SQL was causing issues with the saving of instants not
maintaining their timezones properly.

Changing to parameterised queries fixed this issue.

Also changed other queries to do the same, since it is good practice to
query in this way.
2020-10-22 13:38:39 +01:00
Denis Rekalov
c9056f171b
CORDA-4062: Bump platform version to 9 for safe identity key rotation () 2020-10-20 12:31:47 +03:00
Denis Rekalov
4193adf6fd
CORDA-3979: Support for multiple trust roots () 2020-10-20 12:18:00 +03:00
Denis Rekalov
401d8b8856
ENT-5888: Resurrect node_hash_to_key () 2020-10-19 12:56:12 +03:00
Denis Rekalov
551b3f0811
CORDA-4054: combine different identities of the same notary after its key rotation () 2020-10-16 13:53:04 +01:00
Adel El-Beik
e7df5818e4
INFRA-424 CORDA-4058: Disable flaky tests on openj9 () 2020-10-16 13:03:25 +01:00
Ross Nicoll
1ccd84bbcb Merge remote-tracking branch 'origin/release/os/4.6' into rni/notick/merge-46-47 2020-10-14 15:52:56 +01:00
Ross Nicoll
64dc73c7b1
ENT-5679 Correct instability of linear builds () ()
Disable test which triggers OOM
2020-10-14 13:51:35 +01:00
Ross Nicoll
eb65cdc08a
CORDA-4045 Adjust test to remove scope for race conditions ()
Change assertion in `Restart does not set senderUUID` to verify a single message has a sender UUID set, rather than the last to be recorded as sent has no sender UUID.
2020-10-14 13:44:41 +01:00
Ben McMahon
12f1603822
INFRA-504 Change to use an out of process node ()
* Change to use an out of process node
* Remove exception swallowing and add package for test constants to test process
2020-10-12 17:00:39 +01:00
Denis Rekalov
cdd725e79c
CORDA-3972: Support for node identity rotation in IdentityService () 2020-10-12 18:01:32 +03:00
Ramzi El-Yafi
e2efbaea35
AG-341 Notary backpressure mechanism improvements ()
Notary backpressure mechanism improvements
2020-10-08 09:12:45 +01:00
Ben McMahon
2790e945c6
INFRA-687 Change to use an out of process node ()
* Change to use an out of process node
* Remove exception swallowing and add package for test constants to test process
2020-10-05 18:12:31 +01:00
Denis Rekalov
e837bb0ca8 Merge branch 'release/os/4.5' into denis/merge_os_4.5_to_4.6 2020-10-02 21:43:57 +01:00
Dan Newton
dcc65b1d65
NOTICK Change class byteman references ()
Scripts now reference the class `ProxyConnection` which is an implementation of `Connection`, to address JDK 11 compatibility issue.
2020-09-25 15:42:14 +01:00
Ross Nicoll
5c1ca20dae Detekt 2020-09-25 10:38:46 +01:00
Ross Nicoll
5a93f848b0 Merge remote-tracking branch 'origin/release/os/4.4' into rni/notick-merge-44-45 2020-09-25 09:57:10 +01:00
Ross Nicoll
8048cb4966
CORDA-3845: Update BC, log4j, slf4j () ()
* CORDA-3845: Update BC, log4j, slf4j ()

* CORDA-3845: Update BC to 1.64
* CORDA-3845: Upgraded log4j to 2.12.1
* We can remove the use of Manifests from the logging package so that when _it_ logs it doesn't error on the fact the stream was already closed by the default Java logger.
* Remove the logging package as a plugin
* latest BC version
* Remove old test
* Fix some rebased changes to log file handling
* Update slf4j too

Co-authored-by: Ryan Fowler <fowlerrr@users.noreply.github.com>
Co-authored-by: Adel El-Beik <adel.el-beik@r3.com>
2020-09-25 09:54:59 +01:00
Jerome Gerakis
25ab7667a5
backport of bugfix ENT-5752 into OS 4.3 () () 2020-09-24 14:30:35 +01:00
Kyriakos Tharrouniatis
60309114ec NOTICK Properly merge the two RPCPermissionResolver classes as they diverged 2020-09-22 20:57:49 +01:00
Kyriakos Tharrouniatis
5ff95efbce NOTICK Remove unused imports 2020-09-22 19:14:12 +01:00
Kyriakos Tharrouniatis
a274655e7b NOTICK Delete duplicate class after merge 2020-09-22 18:58:56 +01:00
Kyriakos Tharrouniatis
98bf8a7996 NOTICK Remove eliminated code that was re-introduced from merge 2020-09-22 18:53:25 +01:00
Kyriakos Tharrouniatis
2b0d58c7aa NOTICK Fix callsite after merge 2020-09-22 18:42:31 +01:00
Kyriakos Tharrouniatis
88386f38a9 NOTICK Remove leftover var after merge 2020-09-22 18:38:33 +01:00
Ryan Fowler
ed9f6f0ce1 Merge branch 'release/os/4.6' into rfowler-os-4.6-ent-4.6-20200922 2020-09-22 11:42:00 +01:00
Ross Nicoll
f97bb594b2 Merge remote-tracking branch 'origin/release/os/4.3' into rni/notick/merge-43-44 2020-09-18 16:42:26 +01:00
Ross Nicoll
2064a572ba
CORDA-4034 ENT-5679 CORDA-3845 Correct instability of linear builds ()
* CORDA-4034 Reduce forkEvery to 15 to attempt to mitigate memory leak.
* ENT-5679 Disable test which triggers OOM
* Run tests on two Jenkins agents
* Fixed processing JUnit test results by Allure
* Add timeouts to VaultObserverExceptionTest
* Revert "CORDA-3845: Update BC, log4j, slf4j ()" to eliminate introduced memory leaks

Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
2020-09-18 00:23:16 +01:00
Adel El-Beik
01cd8d6415 Merge remote-tracking branch 'origin/release/os/4.5' into adel/merge-from-4.5-17-Sept 2020-09-17 10:34:01 +01:00
Denis Rekalov
5c6acb0909
CORDA-3968: Prepare keystore handling for certificate rotation [OS] ()
* CORDA-3968: Move keystore logic to KeyStoreHandler.

* CORDA-3967: Stop generating node legal identity in runtime.

* CORDA-3969: Remove Node CA access at node normal runtime.

* CORDA-3968: Fix handling of absent keystore file and wrong password.
2020-09-17 10:15:46 +01:00
Denis Rekalov
acb82f77b4
CORDA-3974: NetworkMapCache should link entries with different public keys by X.500 name () 2020-09-17 09:25:42 +01:00
bpaunescu
2498bb3ae1
[ENT-5752]-Fall back on NM cache if NM service is unreachable during node start-up () 2020-09-16 18:13:44 +01:00
Ross Nicoll
cc14c8e53a
CORDA-4034 Rebuild AuthDBTests so H2 shuts down correctly ()
* CORDA-4034 Rebuild AuthDBTests so H2 shuts down correctly
2020-09-16 16:44:31 +01:00
Ryan Fowler
afd3876faf
CORDA-4036: Put the identity failures on the same log so they aren't missed. () 2020-09-16 16:37:16 +01:00
Dan Newton
bd7b96e816
ENT-5768 startFlowWithClientId permissions ()
Do not let a user reattach to a flow started by another user.

Reattaching to a flow using startFlowWithClientId for a flow not
started by the current user throws a PermissionException

Reattaching to a flow using reattachFlowWithClientId for a flow not
started by the current user returns null.

finishedFlowsWithClientIds does not return flows started by other
users.

Normal rpc permissions around startFlowWithClientId and
startFlowDynamicWithClientId has also been added.

To allow admins to remove client ids as well as be able to see all the
client ids on the node, admin versions have been added that bypass the
user restrictions. These can be permitted via rpc to only provide
their usage to admins.
2020-09-16 16:11:06 +01:00
Ross Nicoll
0226f7ccb9
CORDA-4034 Shut down node in AuthDBTests () 2020-09-16 13:10:54 +01:00
Christian Sailer
b746d5cb22
ENT-5759 Give the failing test CorDapp a different package name so it does not get added automatically to all tests. () 2020-09-15 16:52:08 +01:00
Waldemar Zurowski
ebd36c22e7 Removed unused import 2020-09-15 15:42:17 +02:00
Kyriakos Tharrouniatis
342041c737
NOTICK - Remove unused code () 2020-09-15 13:53:57 +01:00
Waldemar Zurowski
99841dcaeb Merge branch 'release/os/4.4' into INFRA-683-os-45-linear-builds 2020-09-15 14:27:34 +02:00
Ross Nicoll
bcce0e9519
INFRA-683 Move OS 4.4 release branch builds to linear ()
* Do not use Security.addProvider(BouncyCastleProvider()) in tests, to avoid disruptions of other tests.
* Forcibly register security providers before starting Jimfs, to resolve a sequencing problem where Jimfs triggers loading of the SFTP filesystem provider, which in turn registers the standard BouncyCastle provider rather than the patched version Corda needs.
* INFRA-683 Move Corda OS release branch builds to serial ()

Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
2020-09-15 13:18:46 +01:00
Ross Nicoll
c0d354e97a
CORDA-3845: Update BC, log4j, slf4j ()
* CORDA-3845: Update BC to 1.64
* CORDA-3845: Upgraded log4j to 2.12.1
* We can remove the use of Manifests from the logging package so that when _it_ logs it doesn't error on the fact the stream was already closed by the default Java logger.
* Remove the logging package as a plugin
* latest BC version
* Remove old test
* Fix some rebased changes to log file handling
* Update slf4j too

Co-authored-by: Ryan Fowler <fowlerrr@users.noreply.github.com>
Co-authored-by: Adel El-Beik <adel.el-beik@r3.com>
2020-09-15 11:59:46 +01:00
Kyriakos Tharrouniatis
6829be7ae9
ENT-5754 - RPC startFlow cannot reattach to existing client id flows when flows draining mode is enabled ()
* Make existing client id flows re-attachable via rpc 'startFlow' when flows draining mode is enabled

* Fix detekt issue

* Remove unneeded/ unreached waiting on flow's return future
2020-09-15 09:04:17 +01:00
Christian Sailer
879e93ed47
ENT-5760 Log db configuration issues ()
ENT-5760 Return config error instead of throwing an exception so that the config error logging mechanism works.
2020-09-14 15:55:38 +01:00
Ross Nicoll
b3d9d1291f
INFRA-683 Move Corda OS release branch builds to serial ()
Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
2020-09-14 11:29:42 +01:00
Christian Sailer
ea423215c2
ENT-5759 check for app schema migration ()
* Fix behaviour for missing schema outside devMode plus test
2020-09-14 11:25:18 +01:00
Ross Nicoll
ba802c6619
CORDA-4028 Forcibly register security providers ()
Forcibly register security providers before starting Jimfs, to resolve a sequencing problem where Jimfs triggers loading of the SFTP filesystem provider, which in turn registers the standard BouncyCastle provider rather than the patched version Corda needs.
2020-09-12 23:52:52 +01:00
Ross Nicoll
df7b1bccd6
NOTICK Do not use Security.addProvider(BouncyCastleProvider()) in tests () ()
Do not use Security.addProvider(BouncyCastleProvider()) in tests, to avoid disruptions of other tests.

Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
2020-09-11 16:50:12 +01:00
Dan Newton
7ec59da318
ENT-5750 Keep killed flows that were started with client ids ()
If a flow is started with a client id, do not delete it from the
database. Instead, set the status to `KILLED` and store a
`KilledFlowException` in the database.
Keeping it around allows it to be reattached to using the same
mechanisms as active, completed or failed flows. Furthermore, without
this change it could be possible for a flow to be triggered again if
killed while the reconnecting rpc client is being used.
If there is no client id related to the flow, then kill flow keeps its
original behaviour of deleting all traces of the flow.
Flows cannot be killed if they are `COMPLETED`, `FAILED` or `KILLED`
already.
Logs have been added if requests to kill flows in these statuses are
made.
Do not update the status + persist the exception if the flow was killed
during flow initialisation (before persisting its first checkpoint).
Remove the client id mapping if the flow was killed and did not persist
its original checkpoint.
2020-09-10 11:58:55 +01:00
Ryan Fowler
6f2cac146c
ENT-5492: Don't do reconnect logic on illegal argument for attachments ()
* ENT-5492: Don't do reconnect logic on illegal argument for attachments

* Use a dedicated exception for missing attachments.
2020-09-09 12:20:16 +01:00
Rick Parker
fd341c7e48
INFRA-670 Stop shutting down the database before some services that use it ()
* INFRA-670 Stop shutting down the database before some services that use it.

* Test fixes
2020-09-08 08:34:19 +01:00
Dan Newton
45fe3f3aef
CORDA-4026 Make early end session test not flakey () 2020-09-07 16:23:37 +01:00
Yash Nabar
d1735b8c42
EG-3456: Make network parameters path configurable ()
* Update docker image

* EG-3456: Make network parameters path configurable

* EG-3456: Remove filename from network parameters path property

* EG-3456: Add tests for network parameters and node configuration

* Revert docker image update

* EG-3456: Resolve network parameters path against base directory

* EG-3456: Add tests for network parameters path, increase test timeout limits

* EG-3456: Replace baseDirectory argument with networkParametersPath in NetworkParametersReader

* EG-3456: Refactor tests to reuse node configuration object

* EG-3456: Pass networkParametersPath as argument instead of baseDirectory
2020-09-07 09:49:37 +01:00
Dan Newton
64dc94bd18
ENT-5666 Extract shutdown events when retrying a flow ()
* ENT-5666 Extract shutdown events when retrying a flow

When a flow is retrying, only a select set of events are transferred
over to the new fiber. Shutdown events were not included in this set.

This meant that if a flow retries when an `Event.SoftShutdown` is in its
queue, it will never process it. This causes the node to hang, as the
node awaits `liveFibers`, which never reaches 0 because the shutdown
event is never processed.

To resolve this, `Event.SoftShutdown` is added to the set of events to
extract and reschedule.

* ENT-5666 Don't schedule extra shutdown event

When a flow is stopped by an `Event.SoftShutdown` it will eventually
reschedule another shutdown even when it reaches `SMM.removeFlow`. It
won't actually be processed because the flow returns an abort
continuation. But, it does look odd.

Therefore, it now does nothing instead since that is what it was
implicitly doing.
2020-09-04 15:44:14 +01:00
Ross Nicoll
0c16957622 Merge remote-tracking branch 'origin/release/os/4.6' into rni/notick/46-47-merge-2020-09-04 2020-09-04 14:15:47 +01:00