The overload that takes in a bunch of the node parameters with default values has been fixed to the V3 version. New node parameters since V3 now only exist in NodeParameters.
The reason for this is otherwise each new release of Corda that introduces new node parameters will force a new startNode overload to be added to DriverDSL to preserve backwards compatibility.
NodeParameters has been moved to its own file and logLevel is removed as it doesn't do anything.
* CORDA-2254: JmxPolicy fixed so that it turns on monitoring via the jmxMonitoringHttpPort config
To make the API easier to use, httpPort was introduced and both startJmxHttpServer and jmxHttpServerPortAllocation deprecated.
* CORDA-2089 - network parameters tags - part
Data structures changes, storage and notarisation.
Tag transactions with network parameters hash that was in force when tx
was created. Add component group on all core transactions and resolved
parameters on full transactions. The hash should be always visible on
the filtered versions of transactions. Add
notarisation check that the parameters are current.
Implement network parameters storage on services for resolution.
This is only part of the work, next PR will include changes to
ResolveTransactionsFlow to make sure that parameters in the transaction
graph are ordered (this is to prevent the downgrade attack, when the
malicious notary and participants sign transaction that shouldn't be
notarised otherwise).
Probably on network services side we need the default parameters
endpoint for the
transactions that were created before this change - for now it's default
to the current ones.
* Make parameters storage agnostic to cert hierarchy
Test fixes
* Address most PR comments
* Fixes after rebase
* Fixes. Add epoch column to parameters storage.
* Address part of review comments
* Some more comments
* Hopefully fixing what I broke doing rebse
* Address Kostas comments
* Further fixes
* Save all parameters from updates to storage
* Fix integration test
* Address comments
* Fixes after rebase
* Fix test
* Fixes
* Add wrapper for filtering around parameters hash
* API stability fixes
* Add NetworkParametersStorageInternal
* Rename
* add auto acceptance of certain network parameters
* Remove incorrect nullification of newNetworkParameters object within NetworkMapUpdater
* Automatically update network parameters if update accepted and flag day occured
* Comment cleanup
* Add node configuration for auto accepting network parameter changes
* Remove hot swapping of network parameters
* Add docs for auto accept config flag
* Minor change to log line
* Remove unrelated fix that was corrected on master
* Minor name change within NetworkParameters class
* Minor doc rewording
* Fix typo in docs
* Address PR comments
* Add node config option to turn off network param auto-accept on a per param basis
* Address PR comments
* Fix failing Network Map update integration test
FinalityHandler is insecure in that it is open to receive any transaction from any party.
Any CorDapp targeting platform version 4 or above is required use the new c'tors which take in FlowSession objects to the counterpart flow. This flow must subcall ReceiveFinalityFlow to receive and record the finalised transaction.
Old CorDapps (with target platform version < 4) will continue to work as previously. However if there are no old CorDapps loaded then the node will disable FinalityHandler.
* First pass
* Update test.
* Address review comments.
* Added docs and kdocs.
* Clean-up.
* Add extra test.
* Changes to docsite.
* Added try/catch block as recommended by Andras.
* Removed try catch block. It's not required as the checkpoint serialiser deals with this.
* Re-used existing DB session instead of creating a new session.
* Entity manager auto flushes.
* Added java friendly api.
* Addressed review comments.
* Introduce public subset of config to tweak config via mock net work without exposing internal node config.
* Removal of functions exposing (internal) NodeConfiguration from the public test API
* Code review fixes
* Blank lines removed
* Documented mock network API change in upgrade notes.
* Updated documentation and API doc.
* More documentation/API doc
- Make issuableCurrencies config optional
- Allow additionalCordapps to override cordappsForAllNodes
In driver DSL when passing additionalCordapps to startNode and
one of the CordApps is also in cordappsForAllNodes will lead to confilc.
This commit is resolving the conflict by using the CordApp provided
by additionalCordapps
* Migrated all non-BFT notary implementations to use async commits.
* Mock network: await for async operation completion. When calling runNetwork()
it keeps "pumping" messages between participants until no more messages are generated.
The problem comes in when a flow suspends on an async operation: the mock network
thinks the flow finished the work for the current step, and since no more messages
are generated, completes the runNetwork() function. The message that the flow
generates once it resumes after async operation completion never gets processed.
This change makes runNetwork() wait until all flow async operations finish,
and only then check whether no more messages can be transferred.
* Tidy up
* Add install-shell-extensions command
* Make cli tests use same version of picocli as everything else
* Remove initLogging from NodeStartup, it is ran earlier by CordaCLIWrapper
* Use picocli snapshot for testing
* Use RunLast() parser to invoke correct subcommands
* Deprecate old clear-network-map-cache parameter
* Restructure NodeStartup for commands
* Get rid of -c option since the flag method has been deprecated and that didn't exist in last release
* Update documentation
* Update backwards compatibility test
* Get all subcommands working
* Refactor sub commands into seperate classes
* Update docs and fix some tests
* Docs changes
* Fix merge conflicts with master
* Fix renamed parameters
* Fix test failure
* Fix compatibility tests
* Add missing compatibility test for blob inspector
* Remove blob inspector compatibility test as there are import conflicts
* Assorted doc fixes
* Addressing review comments
* More review comments
* Couple more bits
* Fix broken tests
* Fix compilation error
* More merge conflicts
* Make startup logging function a bit more sensible
* Fix broken shell extensions
* Make shell extensions work with subcommands
* Make sure parameters for deprecated options are carried through
* More review comments
* Adding some s's
* One last go
* Fix compilation error on Windows
* Revert logging changes
* Revert docs back to their original imperatively moody state
* first attempt at a flowManager
fix test breakages
add testing around registering subclasses
make flowManager a param of MockNode
extract interface
rename methods
more work around overriding flows
more test fixes
add sample project showing how to use flowOverrides
rebase
* make smallest possible changes to AttachmentSerializationTest and ReceiveAllFlowTests
* add some comments about how flow manager weights flows
* address review comments
add documentation
* address more review comments
The configuration objects for specific notary implementations have been replaced
by a single untyped "extraConfig" Config object that is left to the notary service
itself to parse.
* Remove the raft bootstrapping command from node, we'll need a different
mechanism for that.
* Remove pre-generated identity config value.
* Split up obtainIdentity() in AbstractNode to make it easier to read.
* A temporary workaround for the bootstrapper tool to support BFT notaries.
* Update docs
* Add upgrade notes
* Fix rebase issue
* Add a config diff for the bft notary as well
* ENT-2610: Separate passwords for store and for private keys in Corda OS.
When it comes to KeyStores there are *2* passwords: 1 for the keyStore as a whole and separately there is one private keys within this keyStore.
Unfortunately, those 2 passwords have to be the same due to Artemis limitation, for more details please see:
`org.apache.activemq.artemis.core.remoting.impl.ssl.SSLSupport.loadKeyManagerFactory`
where it is calling `KeyManagerFactory.init()` with store password.
Before change in this PR, throughout our codebase there are multiple places where we assume that storePassword is the same as keyPassword, even in the classes that have nothing to do with Artemis.
This is of course less than ideal as TLS communication may be used not only for Artemis connectivity (e.g. Bridge/Float interaction in Ent) and it is unfair to impose same passwords constraint on that communication channel.
Therefore this PR is removing this limitation and properly separating storePassword from keyPassword.
Linked Jira(https://r3-cev.atlassian.net/browse/ENT-2610) has for more background info.
Suggest to start review from `net.corda.core.crypto.X509NameConstraintsTest` to get an idea about the nature of the changes made.
* ENT-2610: Address PR input from @kchalkias
* ENT-2610: Address PR input from @kchalkias, s/privateKeyPassword/entryPassword/
* ENT-2610: Address PR input from @kchalkias, s/keyPassword/entryPassword/
In the implementation of `CertificateStoreSupplier`
* CORDA-2113 - Include PNM ID in CSR
If Compatibility Zone operator is using private networks and the node
should be joining one, optionally the ID (a UUID) of that network can be
included as part of the node's CSR to to the Doorman.
* fix broken test
The entry point to the API has been simplified to just requireing a list of packages to scan, with sensible defaults provided for the metadata. Because of the wither methods, having parameters for the metadata (with default values) seems unnecessary. Also the ability to scan just individual classes has been made internal, as it seems unlikely app developers would need that level of control when testing their apps.
TestCordappImpl is a data class and thus acts as a natural key for the Jar caching, where previously the key was the package names. This fixes an issue where it was not possible to create two CorDapp Jars of the same package but different metadata.
* Move Raft and BFT notaries into separate modules
* Move schemas
* Fix tests & demos
* Modified logic for creating notary services:
Added a new field 'className' to the notary configuration. The node now
loads the specified implementation via reflection. The default className
value points to the simple notary implementation for backwards compatibility.
Relevant schemas are loaded in a similar fashion.
For backwards compatibility purposes the default SimpleNotaryService will
remain built-in to node, but its cordapp will be generated on startup – so
the loading of notary services is streamlined.
* Move test namedcache factory to test utils
* CORDA-1726 Make sure test frameworks and nodes refuse to start on anything except the right Java versions
Move isValidJavaVersion(), hasMinimumJavaVersion() to Node as it can be use for check in DriverDSL for starting in-process node,
in-process node doesn't go via NodeStartup class where the correct Java version is checked.
Added safeguard to hasMinimumJavaVersion as some JDKs distributions may not have the update version number (e.g. AdoptOpenJDK) which causes unintended parse exception.
* CORDA-1726 Make sure test frameworks and nodes refuse to start on anything except the right Java versions - remove new test, for proper testing this would use reflection and field setting visibility modifiers which easly can has side effects on the subsequent tests
* Addressing PR comments.
* use constant for default platform version value when tests with out of process nodes are run from Intellij.
* node will use constant for platform version instead of manifest file(RPC already uses the constant, no sense in having 2 sources for it)
* fix issues caused by merge
* - Fixed some problems with error handling for Observables.
- Eliminated incorrect double `stop()` call for RpcBroker.
- Added `Schedulers.shutdown()` call in `stop()` implementation for Node and Driver, to avoid stuck processes when observable pipelines go wrong.
* Fixed a missing import.
* Removed `Schedulers.shutdown()` for now.
* Fixed an issue with `pendingFlowsCount()` function.
Allow configuration in node for additional advertised addresses.
fix logic error
Use empty list as default config not null
Allow multiple addresses in NodeInfo
Describe new additionalP2PAddresses property in docs.
Add integration test of additionalP2PAddress feature
Fixup after rebase
Address PR comment
Address PR comments by removing unused element of NodeAddress
* Create constraint, extract Jar signature collection
* Extract JarSignatureCollector into its own file
* Jar signature collection throws exception if signatures are inconsistent
* Focus testing in Jar signature collection
* Extract some helper functions in test
* Patch tests with mock attachment storage
* Assert that generated constraint is satisfied by signed attachment
* Clarify constraint selection logic
* Explicit return types on extension methods
* Link to docsite Signature Contrainsts documentation
* Fix issue with shared JAR reading buffer
Internal tables (the tables from node and finance modules) are now tracked /created by Liquibase script.
Tables backing MappedSchemma in Cordapps are created by Hibernate (as before).
The PR scope added Liquibase library, setup code SchemaMigration and XML scripts and from Enterprise.
For existing database installation - the node will auto-upgrade to use Liquibase.
Method migrateOlderDatabaseToUseLiquibase checks for any 3.X existing Corda database to upgrade database to use Liquibase. When the existing database without Liquibase integral tables is detected, the node (at startup) will create Liquibase tracking tables and fill them with all migration scripts (marked as done), this ensure the database will look as it would use Liquibase from the beginning.
The database changes gradually introduced by the subsequent 3.X releases (3.1, 3.2) are conditionally run by Liquibase.
* ENT-2378 ENT-2377 minimise database access by making more use of caches for filterMyKeys, and don't make a redundant database check for existence of new random keys.
* ENT-2378 Unit test for new method in PersistentIdentityService.
Otherwise PersistentNetworkMapCache only completed the future if a node-info was added, which may not occur after a node restart and thus prevented the SMM from registering its listener with the messaging layer.
Further, the future is only completed if the node-info is other than the node's own one.
* CORDA-1844: Exposing a problem via Unit test.
* CORDA-1844: Unit test update following input from Andras.
* CORDA-1844: Add optional parameter to reduce the time it takes to shutdown RPCServer.
* CORDA-1844: Add optional parameter to reduce the time it takes to shutdown RPCServer and sensibly default it.
Minor changes.
* Reject nodes that have the same organisation name as a previously registered node rather than the same X500 name
DriverDSLImpl -> NetworkVisibilityController -> register check organisation name rather than X500 name and throw IllegalStateException if already exists
Added test to DriverTests to test multiple organisation names end exceptionally
* Remove redundant test and store the organisation names rather than X500 names for driver tests
* Move common matches to testing, add some missing ones
* Clarify test logic
* Move common matches to testing, add some missing ones
* Clarify test logic
* Rename 'randomise'
* endregion
* Fix broken unit test
* Allow MockNetwork to create StartedMockNode from UnstartedMockNode.
* Reimplement by adding a `started` property to UnstartedMockNode.
* Throw IllegalStateException instead of NoSuchElementException.
* Add an isStarted property to UnstartedMockNode.
"app", "rpc", "p2p" and "unknown" have security implications (see isUploaderTrusted method) and thus they are not allowed to be used in the uploader field when importing attachments via the public API.
* Add registerFlowFactory method to public test API
* Return CordaFuture rather than a plain Future
* Rename method
* Improve KDoc
* Hide internal interface with public wrapper
* Modify API current to include ResponderFlowFactory interface
* Take API definition change from generated file
* Note API change in changelog
* InMemoryMessagingNetwork.InMemoryMessaging renamed to MockNodeMessagingService and moved to internal package
* start method added to MockNodeMessagingService which enables AbstractNode to call makeMessagingService in its c'tor
* Removed TopicStringValidator as it's no longer used
* Clean up of TestStartedNode
* Merged InMemoryMessagingTests into InternalMockNetworkTests as it's testing InternalMockNetwork
Or else tests are going to fail on Windows, e.g.:
https://ci-master.corda.r3cev.com/viewLog.html?buildId=103027&tab=buildResultsDiv&buildTypeId=Corda_BuildWindowsTest
```
java.lang.IllegalArgumentException: Illegal character in path at index 95: file:/F:/buildAgent/work/f0cf2ea65c8f73ab/finance/build/libs/corda-finance-4.0-SNAPSHOT.jar/net\corda\finance\contracts\asset\BilateralNetState.class
at java.net.URI.create(URI.java:852)
at net.corda.testing.node.internal.JarEntryInfo$Companion.classFileURL(TestCordappsUtils.kt:209)
at net.corda.testing.node.internal.JarEntryInfo$Companion.access$classFileURL(TestCordappsUtils.kt:199)
at net.corda.testing.node.internal.JarEntryInfo$ClassJarEntryInfo.<init>(TestCordappsUtils.kt:178)
at net.corda.testing.node.internal.TestCordappsUtilsKt.jarEntryInfo(TestCordappsUtils.kt:56)
at net.corda.testing.node.internal.MutableTestCorDapp$Companion.jarEntriesFromClasses(MutableTestCorDapp.kt:42)
...
```
* * First commit for reference input states feature.
* Added docs.
* Added additional test.
* Fixed whitespace.
* Rebased to master.
* Updated Raft and persistent notary implementations.
* Updated changelog.
* Updated topo sort to handle reference states.
* Stubbed out with referenced states flow.
* Added WithReferencedStatesFlow.
* Added Tests for WithReferencedStatesFlow.
* Added ReferenceState type.
* Rebased to latest version of master.
* Added better comments.
* Updated unit test.
* Added comment to explain a little hack.
* Fixed broken contract upgrade RPC test.
* Added minimum platform version check.
* Updated mock network so that notary nodes inherit the platform version set by the network's minimum platform version.
* References states can now only be used when minimum platform version >= 4.
* Created a new file to hold async operations as "WaitForStatesToUpdate" is broadly reusable.
* Refactored WithReferenceStatesFlow
* Updated javadoc for WaitForStatesToUpdate aysnc operation.
* Added network parameters property to InternalMockNetwork.
* Added min platform version capability to mock services.
* Removed erroneous chars from file.
* Made async operation internal and now call it from FlowLogic.
* Moved some transaction checking code around.
* Removed serializable annotation from referenced state and ref.
* Added reference states design doc.
* Added missing KDocs.
* Updated with reference states flow to handle consecutive update races.
* Made platform version info an inheritable thread local.
* Fixed various typos.
* Updated docs.
* Fixed race.
* Removed min platform version checks as API needs more thought.
* Added deprecation to method and supressed warnings.
* Renamed WaitForStatesToUpdate to WaitForStateConsumption.
* Fixed race in WaitForStateConsumption.
* Addressed PR comments and updated comments / KDocs.
* Reverse vault bugfixes.
* * Vault bug fixes.
* * Fixed broken test.
* Moved WaitForStateConsumption to internal package.
* CORDA-1813 fix Postgres db bloat issue
* CORDA-1813 merge fixes
* CORDA-1813 change column type and size to a standard corda type
* CORDA-1813 docs
* CORDA-1813 create custom hibernate type for the checkpoint blob and align with enterprise
* CORDA-1813 Remove max col size
* CORDA-1813 Remove max col size
* CORDA-1813 Fix merge
* CORDA-1813 Remove buggy :serverNameTablePrefix: configuration
Moved start up logic of the various node components out of their c’tors and into “start” methods, which are called from Node.start(). The components themselves are created in the Node’s c’tor with minimal initialisation logic.
Certain things are not immediately available at construction time, which are instead given to the components at start time in an orderly fashion:
* Certs from the node’s key store and trust store
* The network parameters, and thus access to things like maxMessageSize and the contracts whitelist
* A running database - several components were doing database stuff their c’tors
* The node’s NodeInfo, and thus access to things like identities and addresses
The messaging service couldn’t be created in the Node’s c’tor due to initialisation issues with MockNode. This should be fixed in a later commit.
The loading of all node infos in the start method was unnecessary, both for the changePublisher and _loadDBSuccess, and the setting of _registrationFuture was incorrect.
They're prone to cause flaky tests due to the "allocated" ports already being taken by the system when eventually needed. Replaced usages with PortAllocation.Incremental.
Affected unit tests made into integration tests to avoid any issues in the future when unit tests are made to run in parallel.
* CORDA-1743: Remove `isDebug = true` from Node driver to speed-up integration tests execution.
* CORDA-1743: Undo removal of "isDebug = true" from the test that specifically checks for this condition.
* CORDA-1743: Address input from @shamsasari
* ENT-2014 Deletes of NodeInfo can fail to propagate leading to infinite retries
ENT-1880 Move identity key generation to network registration process
(cherry picked from commit c3ac203)
* [CORDA-1634] Destroy child processes when parent exits.
* Add comment.
* Register Shutdownhook for processes regardless of whether the Driver was initialized with
* Add comment.
* Revert "Add comment."
This reverts commit a5e78c379f.
* Add comment.
* Add shutdown hook in ShutdownManager.registerProcessShutdown.
* Initialize the ShutdownManager with a shutdown hook to ensure that is called.
* Add comment.
As reported in [CORDA-1609](https://r3-cev.atlassian.net/browse/CORDA-1609),
`CordaRPCClientConfiguration.default` is not accessible from Java since
`default` is a reserved keyword.
As part of the refactor made in #2831, `CordaRPCClientConfiguration` went
from being a data class to an interface with a backing implementation of
type `CordaRPCClientConfigurationImpl`.
This resulted in Java users having to rewrite code that was on the form:
```java
final CordaRPCClient client = new CordaRPCClient(
nodeAddress, CordaRPCClientConfiguration.DEFAULT
);
```
to something like this:
```java
final CordaRPCClient client = new CordaRPCClient(
nodeAddress, CordaRPCClientConfiguration.Companion.default()
);
```
However, this does not work. The user would get a compilation error because
`default` is a reserved keyword in Java.
Since `CordaRPCClientConfiguration` has been made an interface, there is no
easy way of introducing a static final field on the interface from Kotlin.
Consequently, I've changed this back to using a `class` with a static field
named `DEFAULT` instead of the static method `default()`.
It should be noted that `default()` / `DEFAULT` is currently only used
internally to pass in default values in `CordaRPCClient.kt` and
`CordaRPCClientUtils.kt`. That said, it is exposed as part of our API
surface and consequently shouldn't be broken.
The latter means that in the above example, the user would actually not
have to provide the parameter at all:
```java
final CordaRPCClient client = new CordaRPCClient(nodeAddress);
```
As can be seen from the definition of `CordaRPCClient`:
```kotlin
class CordaRPCClient private constructor(...) {
@JvmOverloads
constructor(
hostAndPort: NetworkHostAndPort,
configuration: CordaRPCClientConfiguration = CordaRPCClientConfiguration.DEFAULT
) : this(hostAndPort, configuration, null)
```
The mentioned [refactor](7a077e76f0 (diff-0948c125db93a22263eb81eaf3161c17R65))
did not make it into the 3.1 release, so from an API-stability perspective,
this change can be applied without affecting our commitment to a
backwards compatible API..
* CORDA-1494: Remove isTimed check from flow hospital when handling timeout exceptions - the timeout might expire just after the TimeFlow has finished and the staff member would throw an exception.
Increase the default flow timeout value in mock network so retries don't happen – this has cause test flakiness.
It will re-run automatically from last checkpoint on node restart, allowing the opportunity to resolve the issue, something required when dealing with contract constraint failures.
* Remove all notion of message level retry.
* Introduce randomness into de-duplication IDs based on the session rather than the flow, in support of idempotent flows.
* CORDA-1494: Re-enable notarisation retries in the new state machine manager.
The original message-based retry approach does not work well with the new
flow state machine due to the way sessions are handled. We decided to move
the retry logic to flow-level: introduce RetryableFlow that won't have
checkpoints persisted and will be restarted after a configurable timeout
if it does not complete in time.
The RetryableFlow functionality will be internal for now, as it's mainly
tailored for the notary client flow, and there are many subtle ways it can
fail when used with arbitrary flows.
