Commit Graph

681 Commits

Author SHA1 Message Date
Shams Asari
3a6deeefa7 Merge branch 'release/os/4.9' into shams-4.10-merge-e6a80822
# Conflicts:
#	.github/workflows/check-pr-title.yml
#	.snyk
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/AMQPClient.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/AMQPClientSslErrorsTest.kt
#	node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt
2023-07-13 10:53:30 +01:00
Shams Asari
671ce9ef97 Merge branch 'os-4.8-merge-point' into shams-4.9-merge-0b8536b9
# Conflicts:
#	.github/workflows/check-pr-title.yml
#	node/src/main/kotlin/net/corda/node/services/messaging/NodeNettyAcceptorFactory.kt
2023-07-07 15:20:43 +01:00
Shams Asari
5b3180bf7b ENT-10016: Give all node threads descriptive names 2023-06-09 11:17:26 +01:00
Chris Cochrane
e12ab8194f
Merge branch 'release/os/4.7' into cc/merge/os-4.7-os-4.8/25-apr-2023
Fwd-merge from OS 4.7.
2023-04-25 14:54:21 +01:00
Chris Cochrane
91c38fa84b
Merge branch 'release/os/4.6' into cc/merge/os-4.6-os-4.7/24-apr-2023
Fwd-merge from OS 4.6.
2023-04-24 20:13:50 +01:00
Chris Cochrane
c58c49d27a
Upgrade corda/crash, snakeyaml 2023-04-20 14:01:20 +01:00
Chris Cochrane
c263ba7563
Updated to fix Jackson tests 2023-04-19 13:06:35 +01:00
Shams Asari
7b6fc51710 Merge branch 'release/os/4.9' into shams-4.10-fwrd-merge-ceb76c56
# Conflicts:
#	common/logging/src/main/kotlin/net/corda/common/logging/Constants.kt
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/AMQPClient.kt
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/SSLHelper.kt
2023-04-12 16:23:59 +01:00
Shams Asari
724a4e5a36 Merge branch 'release/os/4.8' into shams-4.9-fwrd-merge-d140cb59
# Conflicts:
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/ArtemisTcpTransport.kt
#	node-api/src/main/kotlin/net/corda/nodeapi/internal/protonwrapper/netty/AMQPChannelHandler.kt
#	node/src/integration-test/kotlin/net/corda/node/amqp/CertificateRevocationListNodeTests.kt
#	node/src/main/kotlin/net/corda/node/internal/artemis/CertificateChainCheckPolicy.kt
2023-04-11 11:54:26 +01:00
Chris Cochrane
95c4a18352
ENT-6336,ENT-6960 - upgrade snakeyaml to address security vulnerabilities 2023-04-06 19:14:30 +01:00
Shams Asari
0213861d22
ENT-9569: Apply the 60s SSL handshake timeout to the embedded Artemis server (#7315) 2023-03-22 13:22:12 +00:00
Adel El-Beik
044b83508e ENT-8823: Added copyBaggageToTags configuration option to control if baggage is copied to tags.
Switched SimplLogTelemetryComponent to info logs.
2022-12-08 16:54:38 +00:00
Adel El-Beik
e46b7bdd5a
ENT-6893: Added interface for clients to grab openetelemetry handle. (#7264)
* ENT-6893: Added interface for clients to grab openetelemetry handle.

* ENT-6893: Make detekt happy.

* ENT-6893: Fix warnings.

* ENT-6893: Make detekt happy.

* ENT-6893: Now shutdown opentelemetry when node stops or client is closed.

* ENT-6893: OpenTelemetryDriver is not not a singleton.
2022-11-24 13:34:08 +00:00
Adel El-Beik
7a133f687c
ENT-6893: First cut of telemetry integration. (#7247)
First cut of telemetry integration.

Open telemetry can be enabled in two ways, first is via an opentelemetry java agent specified on the command line. With this way you get the advantage of spans created from other libraries, like hibernate. The java agent does byte code rewriting to insert spans.
The second way is with the open telemetry driver (that links with the opentelemetry sdk). This is a fat jar provided with this project and needs to go into the node drivers directory.
2022-10-28 14:41:39 +01:00
Rick Parker
b29713d7b9
ENT-6947 Intern common types to reduce heap footprint (#7239)
ENT-6947: Implement interning for SecureHash, CordaX500Name, PublicKey, AsbtractParty and SignatureAttachmentConstraint, including automatic detection of internable types off companion objects in AMQP & Kyro deserialization.  In some cases, add new factory methods to companion objects, and make main code base use them.

Performance tested in performance cluster with no negative impact visible (so default concurrency setting seems okay).

Testing suggests 5-6x memory saving for tokens in TokensSDK in memory selector.  Should see approx. 1 million tokens per GB or better (1.5 million for the tokens we tested with).
2022-10-18 09:28:41 +01:00
Chris Cochrane
54fdd12a2d
ENT-6866 support quick RPCs that are not processed via the RPC thread pool (#7213)
* ENT-6866 support quick RPCs that are not processed via the RPC thread pool
2022-07-07 12:33:51 +01:00
Ryan Fowler
a5a0c64e2d CORDA-3918: Port of ENT-5417: Allow exceptions to propagate when shutdown commands are called (#6516) 2022-05-16 10:06:01 +01:00
Tamas Veingartner
bfcd1d8791 ENT-5379 Reconnecting RPC fixed to recognize shutdown calls and break reconnect attempts (#6316) 2022-05-16 10:06:00 +01:00
Rick Parker
1e227fdfc4
ENT-6711 - Fix for upgrade to Jackson 2.12 and beyond (actual upgrade to 2.13.1) (#7096)
* NOTICK - Don't know what the JIRA is but wanted to share.

* Updates to resolve bukld issues

* NOTICK: Fixed JDK11 version to prevent capsule version error

* ENT-6711: Added comment for use of jackson_kotlin_version.

* ENT-6711: Avoid deprecation warning, switched to the default method.

Co-authored-by: Chris Cochrane <chris.cochrane@r3.com>
Co-authored-by: Adel El-Beik <adel.el-beik@r3.com>
2022-03-07 11:08:06 +00:00
Adel El-Beik
4594c76259 ENT-6637: Targeted exclude of the jgroups dependency. 2022-03-04 11:52:14 +00:00
Chris Cochrane
9dfdd71c22
ENT-6631 - upgrade jackson version to get rid of databind vulnerability, keeping jackson-module-kotlin the same 2022-02-24 13:26:37 +00:00
Kyriakos Tharrouniatis
a02f2e9f6a
ENT-6414 reattachFlowWithClientId throws NullPointerException if user specifies non-existent client ID (#7044) 2022-01-25 14:42:45 +00:00
Viktor Kolomeyko
835321bb70
ENT-6286: OS: Upgrade version of Artemis to 2.19.1 (#6975)
Fixes DDoS attack mentioned on the Jira ticket.

PR upgrades Artemis library to version 2.19.1.
This is our own release of Apache Artemis library which has vulnerability fix for v2.20 applied.

**_Breaking changes discovered during Artemis upgrade:_**
1. When the queue is created as temporary, it needs to explicitly be specified as non-durable.
2. By default, Artemis Client performs Host DNS name check against the certificate presented by the server. Our TLS certificates fail this check and this verification has to be explicitly disabled, see use of: `TransportConstants.VERIFY_HOST_PROP_NAME`.
3. Artemis Server now caches login attempts, even unsuccessful ones. When we add RPC users dynamically via DB insert this may have an unexpected outcome if the user with the same `userName` and `password` was not available previously.
To workaround permissions changing dynamically, authorization and authentication caches had to be disabled.
4. When computing `maxMessageSize`, the size of the headers content is now taken into account as well.
5. Artemis handling of start-up errors has changed. E.g. when the port is already bound.
6. A number of deprecated APIs like: `createTemporaryQueue`, `failoverOnInitialAttempt`, `NullOutputStream`, `CoreQueueConfiguration`.
7. Log warning message is produced like: `AMQ212080: Using legacy SSL store provider value: JKS. Please use either 'keyStoreType' or 'trustStoreType' instead as appropriate.`
8. As reported by QA, Artemis now produces more audit logging more details [here](https://r3-cev.atlassian.net/browse/ENT-6540). Log configuration been adjusted to reduce such output.
2022-01-21 09:18:14 +00:00
Edoardo Ierina
82a114a329
[DRAFT] feat/CORDA-3823-hash-agility-qa-ready (#6789)
* wip

* wip

* wip (need to review IEE comments)

* wip

* wip

* Small refactoring, fixed network-verifier's TestNotaryFlow

* Added command line option to explicitly enable hash agility support

* wip-do-not-push

* wip

* wip

* wip

* aligned merkletree/transaction hash algorithms

* wip

* Added mixed algorithm support for nodes vs leaves and corrected mixed algorithm tests

* moved global computeNonce and componentHash to DigestService

* added comment for failing test to fix

* wip

* Minor cleanups, added deprecated componentHash/computeNonce

* restored exploratory changes to failing SignedTransaction test

* cleaned up and minor rafactoring

* Fixed some tests with hardcoded hash algorithm

* some changes and cleanups following code review

* WIP commit before large change

* WIP Fixed 3 tests

* WIP removed direct references to randomSHA256() and sha256()

* Updated/added liquibase migrations to support larger hash algorithms

* Reviewed, cleanups, comments, fixes

* removing direct references to sha256()

* WIP verifying obligations test errors

* reviewing obligation/attachment issues with sha3_256

* Full review before PR - intermediate commits

* Reviewed and cleaned up

* Futher cleanup

* Fixed partial tree backward compatible json and cleanups

* all tests passing

* Removed couple of unused imports

* Reworked global componentHash function to avoid deprecated warnings

* replaced SHA3s with some alternate SHA2s

* Removed SHA3-256 and SHA3-512 references

* fixed some tests using non ubiquitous hash algorithms

* Fixed ABI compatibility (not for TransactionBuilder)

* Fixed ABI compatibility to TransactionBuilder

* couple of fixes

* fixed DigestService's randomHash

* Removed constructor with loosely typed args for private constructor of LedgerTransaction class (API removal)

* re-introduced LedgerTransaction deprecated ctor for deserialization

* Add possibility to load CustomMessageDigest bypassing JCA (#6798)

* Change api-current for DigestAlgorithm

* disable flaky tests

Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
2020-11-05 22:05:29 +00:00
Denis Rekalov
551b3f0811
CORDA-4054: combine different identities of the same notary after its key rotation (#6734) 2020-10-16 13:53:04 +01:00
Ryan Fowler
ed9f6f0ce1 Merge branch 'release/os/4.6' into rfowler-os-4.6-ent-4.6-20200922 2020-09-22 11:42:00 +01:00
Dan Newton
317b59ab9d
NOTICK Fix kill flow standalone rpc client test (#6721)
Test was flaky because the flow can finish below it can be killed,
therefore failing the test. Sleep for 1 minute instead to give plenty of
time for the test.
2020-09-18 15:15:44 +01:00
Ryan Fowler
83ec8289b6
ENT-5492 Let the MissingAttachmentException through the RPC (#6719)
* CORDA-4036: Put the identity failures on the same log so they aren't missed.

* ENT-5492: Let the MissingAttachmentException propagate out
2020-09-16 16:41:11 +01:00
Dan Newton
bd7b96e816
ENT-5768 startFlowWithClientId permissions (#6708)
Do not let a user reattach to a flow started by another user.

Reattaching to a flow using startFlowWithClientId for a flow not
started by the current user throws a PermissionException

Reattaching to a flow using reattachFlowWithClientId for a flow not
started by the current user returns null.

finishedFlowsWithClientIds does not return flows started by other
users.

Normal rpc permissions around startFlowWithClientId and
startFlowDynamicWithClientId has also been added.

To allow admins to remove client ids as well as be able to see all the
client ids on the node, admin versions have been added that bypass the
user restrictions. These can be permitted via rpc to only provide
their usage to admins.
2020-09-16 16:11:06 +01:00
Kyriakos Tharrouniatis
6829be7ae9
ENT-5754 - RPC startFlow cannot reattach to existing client id flows when flows draining mode is enabled (#6707)
* Make existing client id flows re-attachable via rpc 'startFlow' when flows draining mode is enabled

* Fix detekt issue

* Remove unneeded/ unreached waiting on flow's return future
2020-09-15 09:04:17 +01:00
Ryan Fowler
6f2cac146c
ENT-5492: Don't do reconnect logic on illegal argument for attachments (#6693)
* ENT-5492: Don't do reconnect logic on illegal argument for attachments

* Use a dedicated exception for missing attachments.
2020-09-09 12:20:16 +01:00
Joseph Zuniga-Daly
eecc294820 Merge OS 4.6 OS 4.7
Conflicts:
- .ci/api-current.txt - Kept existing version on 4.7 as I'm about to regenerate the file
2020-09-01 15:28:37 +01:00
Kyriakos Tharrouniatis
90ab495798
NOTICK - Fix concurrency issue in flaky test (#6674)
* Fix concurrency issue in test; should be fixing flaky test
* Replace 'incrementAndGet' occurences with 'getAndIncrement'
2020-08-28 12:48:20 +01:00
Matthew Nesbit
57f4858a29
Merge pull request #6638 from corda/nnagy-os-4.6-os-4.7-20200814
NOTICK - OS 4.6 to OS 4.7 merge 20200814
2020-08-24 15:30:39 +01:00
Viktor Kolomeyko
9fc896beb3
CORDA-3960: Port MultiRPCClient to OS (#6644)
* CORDA-3960: Port MultiRPCClient to OS

* CORDA-3960: Carefully restore serialisation environment in `MultiRpcClientTest` to eliminate side effects on other tests

* CORDA-3960: Move ThreadDumpUtils.kt to `core-utils`
2020-08-18 10:35:19 +01:00
nikinagy
4cce83522e merge import fix 2020-08-14 17:27:47 +01:00
nikinagy
c9b2fa11cd Merge branch 'release/os/4.6' into nnagy-os-4.6-os-4.7-20200814
# Conflicts:
#	client/rpc/src/main/kotlin/net/corda/client/rpc/internal/ReconnectingCordaRPCOps.kt
2020-08-14 17:26:58 +01:00
Kyriakos Tharrouniatis
ac9d21f66c
CORDA-3882 - Integrate new start method with reconnecting rpc client (#6607)
* Draft version of integrating start flow with client id with reconnecting rpc client

* Add recursive reconnect

* Execute 'doInvoke' on a different thread, fixes hanging

* Tidy up code and add explanatory comment

* Add test timeout

* Add test asserting 'removeClientId' does retry to connect if the node is down

* Modify test to not create a new node on node restart

* Add test asserting 'reattachFlowWithClientId' tries to reconnect if the node is down

* Add test asserting returned flow exception future continue working on node restart

* Add assertions for reconnecting future callbacks

* Remove unused import

* Remove unused local var

* Remove unneeded line breaks

* Remove unneeded parentheses

* Amend existing test; assert flow result reconnectable future returned from 'reattachFlowWithClientId' works

* Update test names

* Add explanatory comment

* Minor comment update

* Fix accidental methods moved

* Update obsolete comment
2020-08-14 12:00:52 +01:00
Viktor Kolomeyko
d55676b452
CORDA-3959: Fix affected smoke test (#6633) 2020-08-14 09:20:11 +01:00
Viktor Kolomeyko
b81eb1559d
CORDA-3959: Eliminate InternalCordaRPCOps (#6600)
* CORDA-3959: Make `ThreadContextAdjustingRpcOpsProxy` flexible for `RPCOps` it uses

* CORDA-3959: More changes towards supporting multiple `RPCOps` implementations

* CORDA-3959: Detekt baseline update

* CORDA-3959: Integration test compilation fix

* CORDA-3959: Introduce `CheckpointRPCOpsImpl` and wire it on

* CORDA-3959: Use multiple RPCOps interfaces in the shell commands

* CORDA-3959: Detekt baseline update

* CORDA-3959: Update RPCPermissionsTests

* CORDA-3959: Update RPCSecurityManagerTest

* CORDA-3959: Remove deprecated marker and rename the property

* CORDA-3959: Detekt baseline

* CORDA-3959: Introduce AttachmentTrustInfoRPCOpsImpl and wire it on

* CORDA-3959: Delete `InternalCordaRPCOps`

* CORDA-3959: Detekt baseline update

* CORDA-3959: Rename `CheckpointRPCOps` to `FlowManagerRPCOps`
2020-08-13 14:41:52 +01:00
Tamas Veingartner
e234bd9c96
fix for Could not reconnect to the RPC server“ error on shutdown via Shell-Cli (#6595)
an RPCUtils created to centralize method name related operations
2020-08-10 09:51:24 +01:00
Tamas Veingartner
e6af60edda
NOTICK Migrate recent RPC related changes to OS, as these were initially imp… (#6532)
* Migrate recent RPC related changes to OS, as these were initially implemented in ENT only

* tests cleanup

* cleanup imports
2020-08-07 09:18:09 +01:00
Ross Nicoll
de5568854c
INFRA-438 Handle observers not having error handling (#6434)
* INFRA-438 Handle observers not having error handling

When the RPC client connection is closed, it notifies observers using onError(), which may not be the correct approach (TBD) but changing this is a much more invasive change. Where observers do not subscribe to error notifications, this is reflected to the calling client by an exception thrown.

This change catches that exception and lots it as debug rather an error level.
2020-08-05 00:05:33 +01:00
Ryan Fowler
d4189c4f37
CORDA-3918: Port of ENT-5417: Allow exceptions to propagate when shutdown commands are called (#6516) 2020-07-27 14:09:26 +01:00
Viktor Kolomeyko
a2058490ed
CORDA-3880: Streamline re-connection logic in RPCClientProxyHandler (#6426)
* CORDA-3880: Streamline re-connection logic in RPCClientProxyHandler

* CORDA-3880: Address PR review comments from @dimosr

* CORDA-3880: Explicitly mention contract around `maxReconnectAttempts`

* CORDA-3880: Handle `maxReconnectAttempts = 0` and do not re-connect
2020-07-03 09:29:43 +01:00
Dan Newton
6bc2c79e23
NOTICK NodeBasedTest take in cordapps (#6424)
In enterprise, `AuthDBTests` picked up a schema from a unit test and
included it in the cordapp it builds. This schema does not have a
migration and therefore fails the integration tests.

`NodeBasedTest` now lets cordapps to be defined and passed in to avoid
this issue. It defaults to making a cordapp from the tests base
directory if none are provided.
2020-07-02 16:14:51 +01:00
Tamas Veingartner
d97bc7dd12
ENT-5420 stop reconnecting when RejectedCommandException occurs. (#6394)
* ENT-5420 stop reconnecting when RejectedCommandException occurs.

* change to store last exception and return quietly

* changed to rethrow exception. Test added
2020-07-02 11:35:04 +01:00
Tamas Veingartner
785715106b
ENT-5379 Reconnecting RPC fixed to recognize shutdown calls and break reconnect attempts (#6316) 2020-06-08 18:03:02 +01:00
Ryan Fowler
b43e781f27
ENT-5285, ENT-5296 Ignore ForkJoinPool.commonPool threads in RPCStabilityTests (#6205)
* NOTICK Ignore JUnit time threads in `RPCStabilityTests`

* NOTICK - Ignore ForkJoinPool.commonPool as it's not related to our test

Co-authored-by: LankyDan <danknewton@hotmail.com>
2020-05-11 15:38:23 +01:00
Dan Newton
4639a8f68b
Revert "NOTICK Ignore JUnit time threads in RPCStabilityTests (#6201)" (#6204)
This reverts commit ecf74053
2020-04-30 13:17:40 +01:00