Commit Graph

681 Commits

Author SHA1 Message Date
3a6deeefa7 Merge branch 'release/os/4.9' into shams-4.10-merge-e6a80822
# Conflicts:
#	.github/workflows/check-pr-title.yml
#	.snyk
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/AMQPClient.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/AMQPClientSslErrorsTest.kt
#	node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt
2023-07-13 10:53:30 +01:00
671ce9ef97 Merge branch 'os-4.8-merge-point' into shams-4.9-merge-0b8536b9
# Conflicts:
#	.github/workflows/check-pr-title.yml
#	node/src/main/kotlin/net/corda/node/services/messaging/NodeNettyAcceptorFactory.kt
2023-07-07 15:20:43 +01:00
5b3180bf7b ENT-10016: Give all node threads descriptive names 2023-06-09 11:17:26 +01:00
e12ab8194f Merge branch 'release/os/4.7' into cc/merge/os-4.7-os-4.8/25-apr-2023
Fwd-merge from OS 4.7.
2023-04-25 14:54:21 +01:00
91c38fa84b Merge branch 'release/os/4.6' into cc/merge/os-4.6-os-4.7/24-apr-2023
Fwd-merge from OS 4.6.
2023-04-24 20:13:50 +01:00
c58c49d27a Upgrade corda/crash, snakeyaml 2023-04-20 14:01:20 +01:00
c263ba7563 Updated to fix Jackson tests 2023-04-19 13:06:35 +01:00
7b6fc51710 Merge branch 'release/os/4.9' into shams-4.10-fwrd-merge-ceb76c56
# Conflicts:
#	common/logging/src/main/kotlin/net/corda/common/logging/Constants.kt
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/AMQPClient.kt
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/SSLHelper.kt
2023-04-12 16:23:59 +01:00
724a4e5a36 Merge branch 'release/os/4.8' into shams-4.9-fwrd-merge-d140cb59
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/ArtemisTcpTransport.kt
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/AMQPChannelHandler.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/CertificateRevocationListNodeTests.kt
#	node/src/main/kotlin/net/corda/node/internal/artemis/CertificateChainCheckPolicy.kt
2023-04-11 11:54:26 +01:00
95c4a18352 ENT-6336,ENT-6960 - upgrade snakeyaml to address security vulnerabilities 2023-04-06 19:14:30 +01:00
0213861d22 ENT-9569: Apply the 60s SSL handshake timeout to the embedded Artemis server (#7315) 2023-03-22 13:22:12 +00:00
044b83508e ENT-8823: Added copyBaggageToTags configuration option to control if baggage is copied to tags.
Switched SimplLogTelemetryComponent to info logs.
2022-12-08 16:54:38 +00:00
e46b7bdd5a ENT-6893: Added interface for clients to grab openetelemetry handle. (#7264)
* ENT-6893: Added interface for clients to grab openetelemetry handle.

* ENT-6893: Make detekt happy.

* ENT-6893: Fix warnings.

* ENT-6893: Make detekt happy.

* ENT-6893: Now shutdown opentelemetry when node stops or client is closed.

* ENT-6893: OpenTelemetryDriver is not not a singleton.
2022-11-24 13:34:08 +00:00
7a133f687c ENT-6893: First cut of telemetry integration. (#7247)
First cut of telemetry integration.

Open telemetry can be enabled in two ways, first is via an opentelemetry java agent specified on the command line. With this way you get the advantage of spans created from other libraries, like hibernate. The java agent does byte code rewriting to insert spans.
The second way is with the open telemetry driver (that links with the opentelemetry sdk). This is a fat jar provided with this project and needs to go into the node drivers directory.
2022-10-28 14:41:39 +01:00
b29713d7b9 ENT-6947 Intern common types to reduce heap footprint (#7239)
ENT-6947: Implement interning for SecureHash, CordaX500Name, PublicKey, AsbtractParty and SignatureAttachmentConstraint, including automatic detection of internable types off companion objects in AMQP & Kyro deserialization.  In some cases, add new factory methods to companion objects, and make main code base use them.

Performance tested in performance cluster with no negative impact visible (so default concurrency setting seems okay).

Testing suggests 5-6x memory saving for tokens in TokensSDK in memory selector.  Should see approx. 1 million tokens per GB or better (1.5 million for the tokens we tested with).
2022-10-18 09:28:41 +01:00
54fdd12a2d ENT-6866 support quick RPCs that are not processed via the RPC thread pool (#7213)
* ENT-6866 support quick RPCs that are not processed via the RPC thread pool
2022-07-07 12:33:51 +01:00
a5a0c64e2d CORDA-3918: Port of ENT-5417: Allow exceptions to propagate when shutdown commands are called (#6516) 2022-05-16 10:06:01 +01:00
bfcd1d8791 ENT-5379 Reconnecting RPC fixed to recognize shutdown calls and break reconnect attempts (#6316) 2022-05-16 10:06:00 +01:00
1e227fdfc4 ENT-6711 - Fix for upgrade to Jackson 2.12 and beyond (actual upgrade to 2.13.1) (#7096)
* NOTICK - Don't know what the JIRA is but wanted to share.

* Updates to resolve bukld issues

* NOTICK: Fixed JDK11 version to prevent capsule version error

* ENT-6711: Added comment for use of jackson_kotlin_version.

* ENT-6711: Avoid deprecation warning, switched to the default method.

Co-authored-by: Chris Cochrane <chris.cochrane@r3.com>
Co-authored-by: Adel El-Beik <adel.el-beik@r3.com>
2022-03-07 11:08:06 +00:00
4594c76259 ENT-6637: Targeted exclude of the jgroups dependency. 2022-03-04 11:52:14 +00:00
9dfdd71c22 ENT-6631 - upgrade jackson version to get rid of databind vulnerability, keeping jackson-module-kotlin the same 2022-02-24 13:26:37 +00:00
a02f2e9f6a ENT-6414 reattachFlowWithClientId throws NullPointerException if user specifies non-existent client ID (#7044) 2022-01-25 14:42:45 +00:00
835321bb70 ENT-6286: OS: Upgrade version of Artemis to 2.19.1 (#6975)
Fixes DDoS attack mentioned on the Jira ticket.

PR upgrades Artemis library to version 2.19.1.
This is our own release of Apache Artemis library which has vulnerability fix for v2.20 applied.

**_Breaking changes discovered during Artemis upgrade:_**
1. When the queue is created as temporary, it needs to explicitly be specified as non-durable.
2. By default, Artemis Client performs Host DNS name check against the certificate presented by the server. Our TLS certificates fail this check and this verification has to be explicitly disabled, see use of: `TransportConstants.VERIFY_HOST_PROP_NAME`.
3. Artemis Server now caches login attempts, even unsuccessful ones. When we add RPC users dynamically via DB insert this may have an unexpected outcome if the user with the same `userName` and `password` was not available previously.
To workaround permissions changing dynamically, authorization and authentication caches had to be disabled.
4. When computing `maxMessageSize`, the size of the headers content is now taken into account as well.
5. Artemis handling of start-up errors has changed. E.g. when the port is already bound.
6. A number of deprecated APIs like: `createTemporaryQueue`, `failoverOnInitialAttempt`, `NullOutputStream`, `CoreQueueConfiguration`.
7. Log warning message is produced like: `AMQ212080: Using legacy SSL store provider value: JKS. Please use either 'keyStoreType' or 'trustStoreType' instead as appropriate.`
8. As reported by QA, Artemis now produces more audit logging more details [here](https://r3-cev.atlassian.net/browse/ENT-6540). Log configuration been adjusted to reduce such output.
2022-01-21 09:18:14 +00:00
82a114a329 [DRAFT] feat/CORDA-3823-hash-agility-qa-ready (#6789)
* wip

* wip

* wip (need to review IEE comments)

* wip

* wip

* Small refactoring, fixed network-verifier's TestNotaryFlow

* Added command line option to explicitly enable hash agility support

* wip-do-not-push

* wip

* wip

* wip

* aligned merkletree/transaction hash algorithms

* wip

* Added mixed algorithm support for nodes vs leaves and corrected mixed algorithm tests

* moved global computeNonce and componentHash to DigestService

* added comment for failing test to fix

* wip

* Minor cleanups, added deprecated componentHash/computeNonce

* restored exploratory changes to failing SignedTransaction test

* cleaned up and minor rafactoring

* Fixed some tests with hardcoded hash algorithm

* some changes and cleanups following code review

* WIP commit before large change

* WIP Fixed 3 tests

* WIP removed direct references to randomSHA256() and sha256()

* Updated/added liquibase migrations to support larger hash algorithms

* Reviewed, cleanups, comments, fixes

* removing direct references to sha256()

* WIP verifying obligations test errors

* reviewing obligation/attachment issues with sha3_256

* Full review before PR - intermediate commits

* Reviewed and cleaned up

* Futher cleanup

* Fixed partial tree backward compatible json and cleanups

* all tests passing

* Removed couple of unused imports

* Reworked global componentHash function to avoid deprecated warnings

* replaced SHA3s with some alternate SHA2s

* Removed SHA3-256 and SHA3-512 references

* fixed some tests using non ubiquitous hash algorithms

* Fixed ABI compatibility (not for TransactionBuilder)

* Fixed ABI compatibility to TransactionBuilder

* couple of fixes

* fixed DigestService's randomHash

* Removed constructor with loosely typed args for private constructor of LedgerTransaction class (API removal)

* re-introduced LedgerTransaction deprecated ctor for deserialization

* Add possibility to load CustomMessageDigest bypassing JCA (#6798)

* Change api-current for DigestAlgorithm

* disable flaky tests

Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
2020-11-05 22:05:29 +00:00
551b3f0811 CORDA-4054: combine different identities of the same notary after its key rotation (#6734) 2020-10-16 13:53:04 +01:00
ed9f6f0ce1 Merge branch 'release/os/4.6' into rfowler-os-4.6-ent-4.6-20200922 2020-09-22 11:42:00 +01:00
317b59ab9d NOTICK Fix kill flow standalone rpc client test (#6721)
Test was flaky because the flow can finish below it can be killed,
therefore failing the test. Sleep for 1 minute instead to give plenty of
time for the test.
2020-09-18 15:15:44 +01:00
83ec8289b6 ENT-5492 Let the MissingAttachmentException through the RPC (#6719)
* CORDA-4036: Put the identity failures on the same log so they aren't missed.

* ENT-5492: Let the MissingAttachmentException propagate out
2020-09-16 16:41:11 +01:00
bd7b96e816 ENT-5768 startFlowWithClientId permissions (#6708)
Do not let a user reattach to a flow started by another user.

Reattaching to a flow using startFlowWithClientId for a flow not
started by the current user throws a PermissionException

Reattaching to a flow using reattachFlowWithClientId for a flow not
started by the current user returns null.

finishedFlowsWithClientIds does not return flows started by other
users.

Normal rpc permissions around startFlowWithClientId and
startFlowDynamicWithClientId has also been added.

To allow admins to remove client ids as well as be able to see all the
client ids on the node, admin versions have been added that bypass the
user restrictions. These can be permitted via rpc to only provide
their usage to admins.
2020-09-16 16:11:06 +01:00
6829be7ae9 ENT-5754 - RPC startFlow cannot reattach to existing client id flows when flows draining mode is enabled (#6707)
* Make existing client id flows re-attachable via rpc 'startFlow' when flows draining mode is enabled

* Fix detekt issue

* Remove unneeded/ unreached waiting on flow's return future
2020-09-15 09:04:17 +01:00
6f2cac146c ENT-5492: Don't do reconnect logic on illegal argument for attachments (#6693)
* ENT-5492: Don't do reconnect logic on illegal argument for attachments

* Use a dedicated exception for missing attachments.
2020-09-09 12:20:16 +01:00
eecc294820 Merge OS 4.6 OS 4.7
Conflicts:
- .ci/api-current.txt - Kept existing version on 4.7 as I'm about to regenerate the file
2020-09-01 15:28:37 +01:00
90ab495798 NOTICK - Fix concurrency issue in flaky test (#6674)
* Fix concurrency issue in test; should be fixing flaky test
* Replace 'incrementAndGet' occurences with 'getAndIncrement'
2020-08-28 12:48:20 +01:00
57f4858a29 Merge pull request #6638 from corda/nnagy-os-4.6-os-4.7-20200814
NOTICK - OS 4.6 to OS 4.7 merge 20200814
2020-08-24 15:30:39 +01:00
9fc896beb3 CORDA-3960: Port MultiRPCClient to OS (#6644)
* CORDA-3960: Port MultiRPCClient to OS

* CORDA-3960: Carefully restore serialisation environment in `MultiRpcClientTest` to eliminate side effects on other tests

* CORDA-3960: Move ThreadDumpUtils.kt to `core-utils`
2020-08-18 10:35:19 +01:00
4cce83522e merge import fix 2020-08-14 17:27:47 +01:00
c9b2fa11cd Merge branch 'release/os/4.6' into nnagy-os-4.6-os-4.7-20200814
# Conflicts:
#	client/rpc/src/main/kotlin/net/corda/client/rpc/internal/ReconnectingCordaRPCOps.kt
2020-08-14 17:26:58 +01:00
ac9d21f66c CORDA-3882 - Integrate new start method with reconnecting rpc client (#6607)
* Draft version of integrating start flow with client id with reconnecting rpc client

* Add recursive reconnect

* Execute 'doInvoke' on a different thread, fixes hanging

* Tidy up code and add explanatory comment

* Add test timeout

* Add test asserting 'removeClientId' does retry to connect if the node is down

* Modify test to not create a new node on node restart

* Add test asserting 'reattachFlowWithClientId' tries to reconnect if the node is down

* Add test asserting returned flow exception future continue working on node restart

* Add assertions for reconnecting future callbacks

* Remove unused import

* Remove unused local var

* Remove unneeded line breaks

* Remove unneeded parentheses

* Amend existing test; assert flow result reconnectable future returned from 'reattachFlowWithClientId' works

* Update test names

* Add explanatory comment

* Minor comment update

* Fix accidental methods moved

* Update obsolete comment
2020-08-14 12:00:52 +01:00
d55676b452 CORDA-3959: Fix affected smoke test (#6633) 2020-08-14 09:20:11 +01:00
b81eb1559d CORDA-3959: Eliminate InternalCordaRPCOps (#6600)
* CORDA-3959: Make `ThreadContextAdjustingRpcOpsProxy` flexible for `RPCOps` it uses

* CORDA-3959: More changes towards supporting multiple `RPCOps` implementations

* CORDA-3959: Detekt baseline update

* CORDA-3959: Integration test compilation fix

* CORDA-3959: Introduce `CheckpointRPCOpsImpl` and wire it on

* CORDA-3959: Use multiple RPCOps interfaces in the shell commands

* CORDA-3959: Detekt baseline update

* CORDA-3959: Update RPCPermissionsTests

* CORDA-3959: Update RPCSecurityManagerTest

* CORDA-3959: Remove deprecated marker and rename the property

* CORDA-3959: Detekt baseline

* CORDA-3959: Introduce AttachmentTrustInfoRPCOpsImpl and wire it on

* CORDA-3959: Delete `InternalCordaRPCOps`

* CORDA-3959: Detekt baseline update

* CORDA-3959: Rename `CheckpointRPCOps` to `FlowManagerRPCOps`
2020-08-13 14:41:52 +01:00
e234bd9c96 fix for Could not reconnect to the RPC server“ error on shutdown via Shell-Cli (#6595)
an RPCUtils created to centralize method name related operations
2020-08-10 09:51:24 +01:00
e6af60edda NOTICK Migrate recent RPC related changes to OS, as these were initially imp… (#6532)
* Migrate recent RPC related changes to OS, as these were initially implemented in ENT only

* tests cleanup

* cleanup imports
2020-08-07 09:18:09 +01:00
de5568854c INFRA-438 Handle observers not having error handling (#6434)
* INFRA-438 Handle observers not having error handling

When the RPC client connection is closed, it notifies observers using onError(), which may not be the correct approach (TBD) but changing this is a much more invasive change. Where observers do not subscribe to error notifications, this is reflected to the calling client by an exception thrown.

This change catches that exception and lots it as debug rather an error level.
2020-08-05 00:05:33 +01:00
d4189c4f37 CORDA-3918: Port of ENT-5417: Allow exceptions to propagate when shutdown commands are called (#6516) 2020-07-27 14:09:26 +01:00
a2058490ed CORDA-3880: Streamline re-connection logic in RPCClientProxyHandler (#6426)
* CORDA-3880: Streamline re-connection logic in RPCClientProxyHandler

* CORDA-3880: Address PR review comments from @dimosr

* CORDA-3880: Explicitly mention contract around `maxReconnectAttempts`

* CORDA-3880: Handle `maxReconnectAttempts = 0` and do not re-connect
2020-07-03 09:29:43 +01:00
6bc2c79e23 NOTICK NodeBasedTest take in cordapps (#6424)
In enterprise, `AuthDBTests` picked up a schema from a unit test and
included it in the cordapp it builds. This schema does not have a
migration and therefore fails the integration tests.

`NodeBasedTest` now lets cordapps to be defined and passed in to avoid
this issue. It defaults to making a cordapp from the tests base
directory if none are provided.
2020-07-02 16:14:51 +01:00
d97bc7dd12 ENT-5420 stop reconnecting when RejectedCommandException occurs. (#6394)
* ENT-5420 stop reconnecting when RejectedCommandException occurs.

* change to store last exception and return quietly

* changed to rethrow exception. Test added
2020-07-02 11:35:04 +01:00
785715106b ENT-5379 Reconnecting RPC fixed to recognize shutdown calls and break reconnect attempts (#6316) 2020-06-08 18:03:02 +01:00
b43e781f27 ENT-5285, ENT-5296 Ignore ForkJoinPool.commonPool threads in RPCStabilityTests (#6205)
* NOTICK Ignore JUnit time threads in `RPCStabilityTests`

* NOTICK - Ignore ForkJoinPool.commonPool as it's not related to our test

Co-authored-by: LankyDan <danknewton@hotmail.com>
2020-05-11 15:38:23 +01:00
4639a8f68b Revert "NOTICK Ignore JUnit time threads in RPCStabilityTests (#6201)" (#6204)
This reverts commit ecf74053
2020-04-30 13:17:40 +01:00