Fixes DDoS attack mentioned on the Jira ticket.
PR upgrades Artemis library to version 2.19.1.
This is our own release of Apache Artemis library which has vulnerability fix for v2.20 applied.
**_Breaking changes discovered during Artemis upgrade:_**
1. When the queue is created as temporary, it needs to explicitly be specified as non-durable.
2. By default, Artemis Client performs Host DNS name check against the certificate presented by the server. Our TLS certificates fail this check and this verification has to be explicitly disabled, see use of: `TransportConstants.VERIFY_HOST_PROP_NAME`.
3. Artemis Server now caches login attempts, even unsuccessful ones. When we add RPC users dynamically via DB insert this may have an unexpected outcome if the user with the same `userName` and `password` was not available previously.
To workaround permissions changing dynamically, authorization and authentication caches had to be disabled.
4. When computing `maxMessageSize`, the size of the headers content is now taken into account as well.
5. Artemis handling of start-up errors has changed. E.g. when the port is already bound.
6. A number of deprecated APIs like: `createTemporaryQueue`, `failoverOnInitialAttempt`, `NullOutputStream`, `CoreQueueConfiguration`.
7. Log warning message is produced like: `AMQ212080: Using legacy SSL store provider value: JKS. Please use either 'keyStoreType' or 'trustStoreType' instead as appropriate.`
8. As reported by QA, Artemis now produces more audit logging more details [here](https://r3-cev.atlassian.net/browse/ENT-6540). Log configuration been adjusted to reduce such output.
Remove the shell code from the OS code base, this includes the modules:
- `:tools:shell`
- `:tools:shell-cli`
The shell will be run within a node if it exists within the node's `drivers` directory.
This is done by using a `URLClassloader` to load the `InteractiveShell` class into Corda's JVM process and running `startShell` and `runLocalShell`.
Running the shell within the `:samples` will require adding:
```
cordaDriver "net.corda:corda-shell:<corda_shell_version>"
```
To the module's `build.gradle` containing `deployNodes`. The script will then include the shell in the created nodes.
A public version of `FlowManagerRPCOps` which does not live in an
internal package has been added. This new interface shares the same name
as the internal one.
Because of the name sharing, the internal version has been
`@Deprecated`.
`FlowManagerRPCOpsImpl` implements both the new and old interfaces. This
allows for backwards compatibility, allowing old shells or clients to
call the old interface on newer nodes without breaking.
* wip
* wip
* wip (need to review IEE comments)
* wip
* wip
* Small refactoring, fixed network-verifier's TestNotaryFlow
* Added command line option to explicitly enable hash agility support
* wip-do-not-push
* wip
* wip
* wip
* aligned merkletree/transaction hash algorithms
* wip
* Added mixed algorithm support for nodes vs leaves and corrected mixed algorithm tests
* moved global computeNonce and componentHash to DigestService
* added comment for failing test to fix
* wip
* Minor cleanups, added deprecated componentHash/computeNonce
* restored exploratory changes to failing SignedTransaction test
* cleaned up and minor rafactoring
* Fixed some tests with hardcoded hash algorithm
* some changes and cleanups following code review
* WIP commit before large change
* WIP Fixed 3 tests
* WIP removed direct references to randomSHA256() and sha256()
* Updated/added liquibase migrations to support larger hash algorithms
* Reviewed, cleanups, comments, fixes
* removing direct references to sha256()
* WIP verifying obligations test errors
* reviewing obligation/attachment issues with sha3_256
* Full review before PR - intermediate commits
* Reviewed and cleaned up
* Futher cleanup
* Fixed partial tree backward compatible json and cleanups
* all tests passing
* Removed couple of unused imports
* Reworked global componentHash function to avoid deprecated warnings
* replaced SHA3s with some alternate SHA2s
* Removed SHA3-256 and SHA3-512 references
* fixed some tests using non ubiquitous hash algorithms
* Fixed ABI compatibility (not for TransactionBuilder)
* Fixed ABI compatibility to TransactionBuilder
* couple of fixes
* fixed DigestService's randomHash
* Removed constructor with loosely typed args for private constructor of LedgerTransaction class (API removal)
* re-introduced LedgerTransaction deprecated ctor for deserialization
* Add possibility to load CustomMessageDigest bypassing JCA (#6798)
* Change api-current for DigestAlgorithm
* disable flaky tests
Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
Created a database snapshot of a clean Corda OS 4.5.1 database, which can now be used for testing by both the node driver and mock network.
The MockNetwork was changed from using an in memory database to using an on disk database, and makes use of the snapshot to speed up setup times.
The Node Driver was changed from defaulting to an in-memory database to defaulting to an on-disk database. Tests that do not specify the type of database to use will thus use an on-disk database. Tests that opt in for an in-memory database will continue to use an in-memory database as before.
The database snapshots are copied to the node directory inside the build folder, therefore, they should be cleaned up after a build.
Co-authored-by: Ross Nicoll <ross.nicoll@r3.com>
* INFRA-424 linux1 jenkinsfile
* INFRA-424 full run
* INFRA-424 bigger heap size
* Upgraded DJVM to handle BC - latest version of BC is a multirelease JAR.
When reading JKS keystore if a BC EdDSAPrivateKey is returned then swap for a net.i2p EdDSA private key.
* Temporary downgrade of BC
* Removed the BC EdDSA conversion
* INFRA-424 bigger heap size
* Upgrading Quasar to handle openJ9 different fields.
* INFRA-424: Handle lack of SUPPRESSED_SENTINEL in openj9.
* INFRA-424: If BCEdDSA public or private key is generated convert to net.i2p EdDSA form.
* INFRA-424 bigger heap size
* INFRA-424: On openJ9 only getting upto milli resolution.
* INFRA-424: Handle keystore returning a BCEdDSAPrivateKey.
* INFRA-424: Disable test on JDK11, as it requires the custom cordapp to generate JDK8 contract code, which we now check for.
* INFRA-424: Truncated time test to resolution of millis for openj9.
* INFRA-424 disabling log intensive tests until a fix is developed
* INFRA-424 one more test disabled
* INFRA-424: Disabled a couple of tests failing on openj9.
* INFRA-424: Disabling failing openj9 tests.
* INFRA-424: Disabling test failing on openj9.
* INFRA-424: Ignoring another flaky sleep test on openj9.
* INFRA-424 run integrationTests
* INFRA-424 set timeout to 4 hours
* INFRA-424: Cope with exception message from openj9.
* INFRA-424: Handle the coloured text characters openj9 adds.
* INFRA-424: Disabling test as it is generating JDK11 contract code under JDK11. Currently on JDK8 contract code allowed.
* INFRA-424: Commenting test out for openj9. Output of the processs thats read by the test is sometimes garbled.
* INFRA-424 switching to smoke tests
* INFRA-424 switching to slow integration tests
* INFRA-424 full run
* INFRA-424 moving jenkinsfile
* INFRA-424 removing references
* INFRA-424: Created common IS_OPENJ9 func for ignoring tests.
Co-authored-by: Schife <razvan.codreanu@r3.com>
* CORDA-4003: Now cope with file: prefix not being in class path element.
* CORDA-4003: Switched to new URL type filter.
* CORDA-4003: Switched to a URL comparison. In the string comparison the scheme was removed in latest version of classgraph.
* CORDA-4003: Moved to latest version of classgraph that has support for + in filenames.
* CORDA-4003: Switched to accept version of the deprecated classgraph methods.
* CORDA-3960: Port MultiRPCClient to OS
* CORDA-3960: Carefully restore serialisation environment in `MultiRpcClientTest` to eliminate side effects on other tests
* CORDA-3960: Move ThreadDumpUtils.kt to `core-utils`
* CORDA-3663 MockServices crashes when two of the provided packages to scan are deemed empty in 4.4 RC05
this happends when a given package is not found on the classpath. Now it is handled and an exception is thrown
* replace dummy package names in tests with valid ones
* allow empty package list for CustomCordapps and exclude those from the created jars
* detekt fix
* always true logic fix
* fix to check for empty packages instead of empty classes
* fix for classes and fixups
* logic refactor because of detekt stupidity
* PR related minor refactors
Enable reloading of a flow after every checkpoint is saved. This
includes reloading the checkpoint from the database and recreating the
fiber.
When a flow and its `StateMachineState` is created it checks the node's
config to see if the `reloadCheckpointAfterSuspend` is set to true. If it is
it initialises `StateMachineState.reloadCheckpointAfterSuspendCount`
with the value 0. Otherwise, it remains `null`.
This count represents how many times the flow has reloaded from its
checkpoint (not the same as retrying). It is incremented every time the
flow is reloaded.
When a flow suspends, it processes the suspend event like usual, but
it will now also check if `reloadCheckpointAfterSuspendCount` is not
`null` (that it is activated) and process a
`ReloadFlowFromCheckpointAfterSuspend`event, if and only if
`reloadCheckpointAfterSuspendCount` is greater than
`CheckpointState.numberOfSuspends`.
This means idempotent flows can reload from the start and not reload
again until reaching a new suspension point.
Flows that skip checkpoints can reload from a previously saved
checkpoint (or from the initial checkpoint) and will continue reloading
on reaching the next new suspension point (not the suspension point that
it skipped saving).
If the flow fails to deserialize the checkpoint from the database upon
reloading a `ReloadFlowFromCheckpointException` is throw. This causes
the flow to be kept for observation.
* CORDA-3844: Add new functions to network map client
* CORDA-3844: Apply new fetch logic to nm updater
* CORDA-3844: Fix base url and warnings
* CORDA-3844: Change response object and response validation
In order to make sure that the returned node infos are not maliciously modified, either a signed list response
or a signed reference object would need to be provided. As providing a signed list requires a lot of effort from NM and Signer services,
the signed network map is provided instead, allowing nodes to validate that the list provided conforms to the entries of the signed network map.
* CORDA-3844: Add clarifications and comments
* CORDA-3844: Add error handling for bulk request
* CORDA-3844: Enhance testing
* CORDA-3844: Fix detekt issues
* EG-3844: Apply pr suggestions
* CORDA-3845: Update BC to 1.64
* CORDA-3845: Upgraded log4j to 2.13.3
* We can remove the use of Manifests from the logging package so that when _it_ logs it doesn't error on the fact the stream was already closed by the default Java logger.
* Some more tidy up
* Remove the logging package as a plugin
* latest BC version
* Remove old test
* fix up
* Fix some rebased changes to log file handling
* Fix some rebased changes to log file handling
* Update slf4j too
Co-authored-by: Adel El-Beik <adel.el-beik@r3.com>
Introducing a new flow start method (`startFlowDynamicWithClientId`) passing in a `clientId`.
Once `startFlowDynamicWithClientId` gets called, the `clientId` gets injected into `InvocationContext` and also pushed to the logging context.
If a new flow starts with this method, then a < `clientId` to flow > pair is kept on node side, even after the flow's lifetime. If `startFlowDynamicWithClientId` is called again with the same `clientId` then the node identifies that this `clientId` refers to an existing < `clientId` to flow > pair and returns back to the rpc client a `FlowStateMachineHandle` future, created out of that pair.
`FlowStateMachineHandle` interface was introduced as a thinner `FlowStateMachine`. All `FlowStateMachine` properties used by call sites are moved into this new interface along with `clientId` and then `FlowStateMachine` extends it.
Introducing an acknowledgement method (`removeClientId`). Calling this method removes the < `clientId` to flow > pair on the node side and frees resources.
Wait for log file to indicate that a driver node process is running, instead of trying to open a socket on the port the node is attempting to binding on. This means the driver is more responsive to the node being ready, as well as removing a race condition where the binding test could block the node from starting.
As sometimes nodes do not log this information, after 20 seconds we presume the node is up.
* Add schema migration to smoke tests
* Fix driver to work correctly for out-of-proc node with persistent database.
Co-authored-by: Ross Nicoll <ross.nicoll@r3.com>
