Note: similar sort of changes already made in Ent codebase that fix execution against external DBs.
it will make sense to backport those change to OS to avoid merge conflicts in the future.
It exposed the internal SerializationEnvironment class. It previously documented that the SerializationEnvironmentRule JUnit rule should be used instead.
TransactionBuilder loads attachment using attachment storage instead of CordappLoader,
contract class version is now Integer (format and stored in db as Integer).
* Hash to signature constraints migration #1
* After rebase from Attachments Classloader commit.
* Simplified implementation without CZ whitelisting and relaxing the no-overlap rule slightly.
* Further simplification.
* Detailed implementation.
* Use fully loaded Cash contract jar for hash to signature constraints migration test.
Additional debug logging.
* Minor cleanup.
* Address PR review feedback.
* Minor fix.
* Fixes following rebase from master.
* Implemented `calculateEntriesHashes` to improve classloader validation performance.
* Address minor PR review comments.
* Added integration tests and some minor fixes.
* Minor fixes following rebase from master.
* Updates and fixes following integration testing.
* Added changelog entry.
* Fix broken unit tests.
* Fix compilation errors in DriverDSL tests after rebase from master.
* Minor fix to test cordapp jar signing using explicit keystore.
* Run hash-to-signature constraints integration test out of process using a non-validating notary.
* Address PR review feedback: contract version from database + other minor changes.
* Address final PR review feedback: remove signed attachment field from attachmentWithContext
* Resolve conflicts following rebase from master.
* Fix failing junit test.
* Fix Kryo serialization error (forgot to write new `version` identifier field)
* Removed redundant query carried over from previous commit.
* Added documentation.
* Fix test case where explicit Hash Constraint input and Signature Constraint output explicitly configured.
* Addressing PR review comments from SA.
* AttachmentQueryCriteria API: added wither methods and Java Unit tests.
* Fixed compilation error caused by Unit tests being in wrong module.
* Added @CordaInternal to canBeTransitionedFrom function.
* Minimized AttachmentClassloader overlap duplicates checking.
* Moved JarSignatureTestUtils and ContractJarTestUtils to internal pending clean-up and documentation before public release.
* Minor fix following rebase from master.
* Removed redundant checkNotNull(networkParameters) checks now that these are always passed into the main (non-deprecated) constructor.
* Remove capitalization.
* CORDA-2115: Notary whitelist verification changes
- For regular and contract upgrade transactions: check that the notary is in the network parameter whitelist
- For notary change transactions: check the the new notary is in the network parameter whitelist. This enabled support for network merging: the old notary doesn't have to be in the current network's notary whitelist for re-pointing old states to another notary.
These checks are done during transaction construction/verification and also by the non-validating notary.
* Address comments
* Remove stale todo
* Use notary whitelist of current network parameters for platform versoin 3
* Cleanup test
* Move `getHistoricNotary` to `HistoricNetworkParameterStorage` in `core.internal`
* Require `newNotary` to be notary on the network map during notary change
The httpPort parameter is fundamentally wrong as it gives the same monitoring port to each node!
Added JmxPolicy.defaultEnabled() to workaround the (incorrect) default false value of startJmxHttpServer.
* Enabled system property / env variable cascade for global test port allocation toggles.
* [CORDA-2282]: Ensure global test port allocation prevents port clashes. (fix)
* Make enforceSingleNodeIsRunning internal for test purposes.
* Test that the file is deleted or the file lock is released to be sure that the node has stopped.
* Additional test for enforceSingleNodeIsRunnin
The overload that takes in a bunch of the node parameters with default values has been fixed to the V3 version. New node parameters since V3 now only exist in NodeParameters.
The reason for this is otherwise each new release of Corda that introduces new node parameters will force a new startNode overload to be added to DriverDSL to preserve backwards compatibility.
NodeParameters has been moved to its own file and logLevel is removed as it doesn't do anything.
* CORDA-2254: JmxPolicy fixed so that it turns on monitoring via the jmxMonitoringHttpPort config
To make the API easier to use, httpPort was introduced and both startJmxHttpServer and jmxHttpServerPortAllocation deprecated.
* CORDA-2089 - network parameters tags - part
Data structures changes, storage and notarisation.
Tag transactions with network parameters hash that was in force when tx
was created. Add component group on all core transactions and resolved
parameters on full transactions. The hash should be always visible on
the filtered versions of transactions. Add
notarisation check that the parameters are current.
Implement network parameters storage on services for resolution.
This is only part of the work, next PR will include changes to
ResolveTransactionsFlow to make sure that parameters in the transaction
graph are ordered (this is to prevent the downgrade attack, when the
malicious notary and participants sign transaction that shouldn't be
notarised otherwise).
Probably on network services side we need the default parameters
endpoint for the
transactions that were created before this change - for now it's default
to the current ones.
* Make parameters storage agnostic to cert hierarchy
Test fixes
* Address most PR comments
* Fixes after rebase
* Fixes. Add epoch column to parameters storage.
* Address part of review comments
* Some more comments
* Hopefully fixing what I broke doing rebse
* Address Kostas comments
* Further fixes
* Save all parameters from updates to storage
* Fix integration test
* Address comments
* Fixes after rebase
* Fix test
* Fixes
* Add wrapper for filtering around parameters hash
* API stability fixes
* Add NetworkParametersStorageInternal
* Rename
* add auto acceptance of certain network parameters
* Remove incorrect nullification of newNetworkParameters object within NetworkMapUpdater
* Automatically update network parameters if update accepted and flag day occured
* Comment cleanup
* Add node configuration for auto accepting network parameter changes
* Remove hot swapping of network parameters
* Add docs for auto accept config flag
* Minor change to log line
* Remove unrelated fix that was corrected on master
* Minor name change within NetworkParameters class
* Minor doc rewording
* Fix typo in docs
* Address PR comments
* Add node config option to turn off network param auto-accept on a per param basis
* Address PR comments
* Fix failing Network Map update integration test
FinalityHandler is insecure in that it is open to receive any transaction from any party.
Any CorDapp targeting platform version 4 or above is required use the new c'tors which take in FlowSession objects to the counterpart flow. This flow must subcall ReceiveFinalityFlow to receive and record the finalised transaction.
Old CorDapps (with target platform version < 4) will continue to work as previously. However if there are no old CorDapps loaded then the node will disable FinalityHandler.
* First pass
* Update test.
* Address review comments.
* Added docs and kdocs.
* Clean-up.
* Add extra test.
* Changes to docsite.
* Added try/catch block as recommended by Andras.
* Removed try catch block. It's not required as the checkpoint serialiser deals with this.
* Re-used existing DB session instead of creating a new session.
* Entity manager auto flushes.
* Added java friendly api.
* Addressed review comments.
* Introduce public subset of config to tweak config via mock net work without exposing internal node config.
* Removal of functions exposing (internal) NodeConfiguration from the public test API
* Code review fixes
* Blank lines removed
* Documented mock network API change in upgrade notes.
* Updated documentation and API doc.
* More documentation/API doc
- Make issuableCurrencies config optional
- Allow additionalCordapps to override cordappsForAllNodes
In driver DSL when passing additionalCordapps to startNode and
one of the CordApps is also in cordappsForAllNodes will lead to confilc.
This commit is resolving the conflict by using the CordApp provided
by additionalCordapps
* Migrated all non-BFT notary implementations to use async commits.
* Mock network: await for async operation completion. When calling runNetwork()
it keeps "pumping" messages between participants until no more messages are generated.
The problem comes in when a flow suspends on an async operation: the mock network
thinks the flow finished the work for the current step, and since no more messages
are generated, completes the runNetwork() function. The message that the flow
generates once it resumes after async operation completion never gets processed.
This change makes runNetwork() wait until all flow async operations finish,
and only then check whether no more messages can be transferred.
* Tidy up
* Add install-shell-extensions command
* Make cli tests use same version of picocli as everything else
* Remove initLogging from NodeStartup, it is ran earlier by CordaCLIWrapper
* Use picocli snapshot for testing
* Use RunLast() parser to invoke correct subcommands
* Deprecate old clear-network-map-cache parameter
* Restructure NodeStartup for commands
* Get rid of -c option since the flag method has been deprecated and that didn't exist in last release
* Update documentation
* Update backwards compatibility test
* Get all subcommands working
* Refactor sub commands into seperate classes
* Update docs and fix some tests
* Docs changes
* Fix merge conflicts with master
* Fix renamed parameters
* Fix test failure
* Fix compatibility tests
* Add missing compatibility test for blob inspector
* Remove blob inspector compatibility test as there are import conflicts
* Assorted doc fixes
* Addressing review comments
* More review comments
* Couple more bits
* Fix broken tests
* Fix compilation error
* More merge conflicts
* Make startup logging function a bit more sensible
* Fix broken shell extensions
* Make shell extensions work with subcommands
* Make sure parameters for deprecated options are carried through
* More review comments
* Adding some s's
* One last go
* Fix compilation error on Windows
* Revert logging changes
* Revert docs back to their original imperatively moody state
* first attempt at a flowManager
fix test breakages
add testing around registering subclasses
make flowManager a param of MockNode
extract interface
rename methods
more work around overriding flows
more test fixes
add sample project showing how to use flowOverrides
rebase
* make smallest possible changes to AttachmentSerializationTest and ReceiveAllFlowTests
* add some comments about how flow manager weights flows
* address review comments
add documentation
* address more review comments
The configuration objects for specific notary implementations have been replaced
by a single untyped "extraConfig" Config object that is left to the notary service
itself to parse.
* Remove the raft bootstrapping command from node, we'll need a different
mechanism for that.
* Remove pre-generated identity config value.
* Split up obtainIdentity() in AbstractNode to make it easier to read.
* A temporary workaround for the bootstrapper tool to support BFT notaries.
* Update docs
* Add upgrade notes
* Fix rebase issue
* Add a config diff for the bft notary as well
* ENT-2610: Separate passwords for store and for private keys in Corda OS.
When it comes to KeyStores there are *2* passwords: 1 for the keyStore as a whole and separately there is one private keys within this keyStore.
Unfortunately, those 2 passwords have to be the same due to Artemis limitation, for more details please see:
`org.apache.activemq.artemis.core.remoting.impl.ssl.SSLSupport.loadKeyManagerFactory`
where it is calling `KeyManagerFactory.init()` with store password.
Before change in this PR, throughout our codebase there are multiple places where we assume that storePassword is the same as keyPassword, even in the classes that have nothing to do with Artemis.
This is of course less than ideal as TLS communication may be used not only for Artemis connectivity (e.g. Bridge/Float interaction in Ent) and it is unfair to impose same passwords constraint on that communication channel.
Therefore this PR is removing this limitation and properly separating storePassword from keyPassword.
Linked Jira(https://r3-cev.atlassian.net/browse/ENT-2610) has for more background info.
Suggest to start review from `net.corda.core.crypto.X509NameConstraintsTest` to get an idea about the nature of the changes made.
* ENT-2610: Address PR input from @kchalkias
* ENT-2610: Address PR input from @kchalkias, s/privateKeyPassword/entryPassword/
* ENT-2610: Address PR input from @kchalkias, s/keyPassword/entryPassword/
In the implementation of `CertificateStoreSupplier`
* CORDA-2113 - Include PNM ID in CSR
If Compatibility Zone operator is using private networks and the node
should be joining one, optionally the ID (a UUID) of that network can be
included as part of the node's CSR to to the Doorman.
* fix broken test
The entry point to the API has been simplified to just requireing a list of packages to scan, with sensible defaults provided for the metadata. Because of the wither methods, having parameters for the metadata (with default values) seems unnecessary. Also the ability to scan just individual classes has been made internal, as it seems unlikely app developers would need that level of control when testing their apps.
TestCordappImpl is a data class and thus acts as a natural key for the Jar caching, where previously the key was the package names. This fixes an issue where it was not possible to create two CorDapp Jars of the same package but different metadata.
* Move Raft and BFT notaries into separate modules
* Move schemas
* Fix tests & demos
* Modified logic for creating notary services:
Added a new field 'className' to the notary configuration. The node now
loads the specified implementation via reflection. The default className
value points to the simple notary implementation for backwards compatibility.
Relevant schemas are loaded in a similar fashion.
For backwards compatibility purposes the default SimpleNotaryService will
remain built-in to node, but its cordapp will be generated on startup – so
the loading of notary services is streamlined.
* Move test namedcache factory to test utils
* CORDA-1726 Make sure test frameworks and nodes refuse to start on anything except the right Java versions
Move isValidJavaVersion(), hasMinimumJavaVersion() to Node as it can be use for check in DriverDSL for starting in-process node,
in-process node doesn't go via NodeStartup class where the correct Java version is checked.
Added safeguard to hasMinimumJavaVersion as some JDKs distributions may not have the update version number (e.g. AdoptOpenJDK) which causes unintended parse exception.
* CORDA-1726 Make sure test frameworks and nodes refuse to start on anything except the right Java versions - remove new test, for proper testing this would use reflection and field setting visibility modifiers which easly can has side effects on the subsequent tests
* Addressing PR comments.
* use constant for default platform version value when tests with out of process nodes are run from Intellij.
* node will use constant for platform version instead of manifest file(RPC already uses the constant, no sense in having 2 sources for it)
* fix issues caused by merge
* - Fixed some problems with error handling for Observables.
- Eliminated incorrect double `stop()` call for RpcBroker.
- Added `Schedulers.shutdown()` call in `stop()` implementation for Node and Driver, to avoid stuck processes when observable pipelines go wrong.
* Fixed a missing import.
* Removed `Schedulers.shutdown()` for now.
* Fixed an issue with `pendingFlowsCount()` function.
Allow configuration in node for additional advertised addresses.
fix logic error
Use empty list as default config not null
Allow multiple addresses in NodeInfo
Describe new additionalP2PAddresses property in docs.
Add integration test of additionalP2PAddress feature
Fixup after rebase
Address PR comment
Address PR comments by removing unused element of NodeAddress
* Create constraint, extract Jar signature collection
* Extract JarSignatureCollector into its own file
* Jar signature collection throws exception if signatures are inconsistent
* Focus testing in Jar signature collection
* Extract some helper functions in test
* Patch tests with mock attachment storage
* Assert that generated constraint is satisfied by signed attachment
* Clarify constraint selection logic
* Explicit return types on extension methods
* Link to docsite Signature Contrainsts documentation
* Fix issue with shared JAR reading buffer
Internal tables (the tables from node and finance modules) are now tracked /created by Liquibase script.
Tables backing MappedSchemma in Cordapps are created by Hibernate (as before).
The PR scope added Liquibase library, setup code SchemaMigration and XML scripts and from Enterprise.
For existing database installation - the node will auto-upgrade to use Liquibase.
Method migrateOlderDatabaseToUseLiquibase checks for any 3.X existing Corda database to upgrade database to use Liquibase. When the existing database without Liquibase integral tables is detected, the node (at startup) will create Liquibase tracking tables and fill them with all migration scripts (marked as done), this ensure the database will look as it would use Liquibase from the beginning.
The database changes gradually introduced by the subsequent 3.X releases (3.1, 3.2) are conditionally run by Liquibase.
* ENT-2378 ENT-2377 minimise database access by making more use of caches for filterMyKeys, and don't make a redundant database check for existence of new random keys.
* ENT-2378 Unit test for new method in PersistentIdentityService.
Otherwise PersistentNetworkMapCache only completed the future if a node-info was added, which may not occur after a node restart and thus prevented the SMM from registering its listener with the messaging layer.
Further, the future is only completed if the node-info is other than the node's own one.
* CORDA-1844: Exposing a problem via Unit test.
* CORDA-1844: Unit test update following input from Andras.
* CORDA-1844: Add optional parameter to reduce the time it takes to shutdown RPCServer.
* CORDA-1844: Add optional parameter to reduce the time it takes to shutdown RPCServer and sensibly default it.
Minor changes.
* Reject nodes that have the same organisation name as a previously registered node rather than the same X500 name
DriverDSLImpl -> NetworkVisibilityController -> register check organisation name rather than X500 name and throw IllegalStateException if already exists
Added test to DriverTests to test multiple organisation names end exceptionally
* Remove redundant test and store the organisation names rather than X500 names for driver tests
* Move common matches to testing, add some missing ones
* Clarify test logic
* Move common matches to testing, add some missing ones
* Clarify test logic
* Rename 'randomise'
* endregion
* Fix broken unit test
* Allow MockNetwork to create StartedMockNode from UnstartedMockNode.
* Reimplement by adding a `started` property to UnstartedMockNode.
* Throw IllegalStateException instead of NoSuchElementException.
* Add an isStarted property to UnstartedMockNode.
"app", "rpc", "p2p" and "unknown" have security implications (see isUploaderTrusted method) and thus they are not allowed to be used in the uploader field when importing attachments via the public API.
* Add registerFlowFactory method to public test API
* Return CordaFuture rather than a plain Future
* Rename method
* Improve KDoc
* Hide internal interface with public wrapper
* Modify API current to include ResponderFlowFactory interface
* Take API definition change from generated file
* Note API change in changelog
* InMemoryMessagingNetwork.InMemoryMessaging renamed to MockNodeMessagingService and moved to internal package
* start method added to MockNodeMessagingService which enables AbstractNode to call makeMessagingService in its c'tor
* Removed TopicStringValidator as it's no longer used
* Clean up of TestStartedNode
* Merged InMemoryMessagingTests into InternalMockNetworkTests as it's testing InternalMockNetwork
Or else tests are going to fail on Windows, e.g.:
https://ci-master.corda.r3cev.com/viewLog.html?buildId=103027&tab=buildResultsDiv&buildTypeId=Corda_BuildWindowsTest
```
java.lang.IllegalArgumentException: Illegal character in path at index 95: file:/F:/buildAgent/work/f0cf2ea65c8f73ab/finance/build/libs/corda-finance-4.0-SNAPSHOT.jar/net\corda\finance\contracts\asset\BilateralNetState.class
at java.net.URI.create(URI.java:852)
at net.corda.testing.node.internal.JarEntryInfo$Companion.classFileURL(TestCordappsUtils.kt:209)
at net.corda.testing.node.internal.JarEntryInfo$Companion.access$classFileURL(TestCordappsUtils.kt:199)
at net.corda.testing.node.internal.JarEntryInfo$ClassJarEntryInfo.<init>(TestCordappsUtils.kt:178)
at net.corda.testing.node.internal.TestCordappsUtilsKt.jarEntryInfo(TestCordappsUtils.kt:56)
at net.corda.testing.node.internal.MutableTestCorDapp$Companion.jarEntriesFromClasses(MutableTestCorDapp.kt:42)
...
```
* * First commit for reference input states feature.
* Added docs.
* Added additional test.
* Fixed whitespace.
* Rebased to master.
* Updated Raft and persistent notary implementations.
* Updated changelog.
* Updated topo sort to handle reference states.
* Stubbed out with referenced states flow.
* Added WithReferencedStatesFlow.
* Added Tests for WithReferencedStatesFlow.
* Added ReferenceState type.
* Rebased to latest version of master.
* Added better comments.
* Updated unit test.
* Added comment to explain a little hack.
* Fixed broken contract upgrade RPC test.
* Added minimum platform version check.
* Updated mock network so that notary nodes inherit the platform version set by the network's minimum platform version.
* References states can now only be used when minimum platform version >= 4.
* Created a new file to hold async operations as "WaitForStatesToUpdate" is broadly reusable.
* Refactored WithReferenceStatesFlow
* Updated javadoc for WaitForStatesToUpdate aysnc operation.
* Added network parameters property to InternalMockNetwork.
* Added min platform version capability to mock services.
* Removed erroneous chars from file.
* Made async operation internal and now call it from FlowLogic.
* Moved some transaction checking code around.
* Removed serializable annotation from referenced state and ref.
* Added reference states design doc.
* Added missing KDocs.
* Updated with reference states flow to handle consecutive update races.
* Made platform version info an inheritable thread local.
* Fixed various typos.
* Updated docs.
* Fixed race.
* Removed min platform version checks as API needs more thought.
* Added deprecation to method and supressed warnings.
* Renamed WaitForStatesToUpdate to WaitForStateConsumption.
* Fixed race in WaitForStateConsumption.
* Addressed PR comments and updated comments / KDocs.
* Reverse vault bugfixes.
* * Vault bug fixes.
* * Fixed broken test.
* Moved WaitForStateConsumption to internal package.
* CORDA-1813 fix Postgres db bloat issue
* CORDA-1813 merge fixes
* CORDA-1813 change column type and size to a standard corda type
* CORDA-1813 docs
* CORDA-1813 create custom hibernate type for the checkpoint blob and align with enterprise
* CORDA-1813 Remove max col size
* CORDA-1813 Remove max col size
* CORDA-1813 Fix merge
* CORDA-1813 Remove buggy :serverNameTablePrefix: configuration
Moved start up logic of the various node components out of their c’tors and into “start” methods, which are called from Node.start(). The components themselves are created in the Node’s c’tor with minimal initialisation logic.
Certain things are not immediately available at construction time, which are instead given to the components at start time in an orderly fashion:
* Certs from the node’s key store and trust store
* The network parameters, and thus access to things like maxMessageSize and the contracts whitelist
* A running database - several components were doing database stuff their c’tors
* The node’s NodeInfo, and thus access to things like identities and addresses
The messaging service couldn’t be created in the Node’s c’tor due to initialisation issues with MockNode. This should be fixed in a later commit.
The loading of all node infos in the start method was unnecessary, both for the changePublisher and _loadDBSuccess, and the setting of _registrationFuture was incorrect.
They're prone to cause flaky tests due to the "allocated" ports already being taken by the system when eventually needed. Replaced usages with PortAllocation.Incremental.
Affected unit tests made into integration tests to avoid any issues in the future when unit tests are made to run in parallel.
* CORDA-1743: Remove `isDebug = true` from Node driver to speed-up integration tests execution.
* CORDA-1743: Undo removal of "isDebug = true" from the test that specifically checks for this condition.
* CORDA-1743: Address input from @shamsasari
* ENT-2014 Deletes of NodeInfo can fail to propagate leading to infinite retries
ENT-1880 Move identity key generation to network registration process
(cherry picked from commit c3ac203)
* [CORDA-1634] Destroy child processes when parent exits.
* Add comment.
* Register Shutdownhook for processes regardless of whether the Driver was initialized with
* Add comment.
* Revert "Add comment."
This reverts commit a5e78c379f.
* Add comment.
* Add shutdown hook in ShutdownManager.registerProcessShutdown.
* Initialize the ShutdownManager with a shutdown hook to ensure that is called.
* Add comment.
As reported in [CORDA-1609](https://r3-cev.atlassian.net/browse/CORDA-1609),
`CordaRPCClientConfiguration.default` is not accessible from Java since
`default` is a reserved keyword.
As part of the refactor made in #2831, `CordaRPCClientConfiguration` went
from being a data class to an interface with a backing implementation of
type `CordaRPCClientConfigurationImpl`.
This resulted in Java users having to rewrite code that was on the form:
```java
final CordaRPCClient client = new CordaRPCClient(
nodeAddress, CordaRPCClientConfiguration.DEFAULT
);
```
to something like this:
```java
final CordaRPCClient client = new CordaRPCClient(
nodeAddress, CordaRPCClientConfiguration.Companion.default()
);
```
However, this does not work. The user would get a compilation error because
`default` is a reserved keyword in Java.
Since `CordaRPCClientConfiguration` has been made an interface, there is no
easy way of introducing a static final field on the interface from Kotlin.
Consequently, I've changed this back to using a `class` with a static field
named `DEFAULT` instead of the static method `default()`.
It should be noted that `default()` / `DEFAULT` is currently only used
internally to pass in default values in `CordaRPCClient.kt` and
`CordaRPCClientUtils.kt`. That said, it is exposed as part of our API
surface and consequently shouldn't be broken.
The latter means that in the above example, the user would actually not
have to provide the parameter at all:
```java
final CordaRPCClient client = new CordaRPCClient(nodeAddress);
```
As can be seen from the definition of `CordaRPCClient`:
```kotlin
class CordaRPCClient private constructor(...) {
@JvmOverloads
constructor(
hostAndPort: NetworkHostAndPort,
configuration: CordaRPCClientConfiguration = CordaRPCClientConfiguration.DEFAULT
) : this(hostAndPort, configuration, null)
```
The mentioned [refactor](7a077e76f0 (diff-0948c125db93a22263eb81eaf3161c17R65))
did not make it into the 3.1 release, so from an API-stability perspective,
this change can be applied without affecting our commitment to a
backwards compatible API..
* CORDA-1494: Remove isTimed check from flow hospital when handling timeout exceptions - the timeout might expire just after the TimeFlow has finished and the staff member would throw an exception.
Increase the default flow timeout value in mock network so retries don't happen – this has cause test flakiness.
It will re-run automatically from last checkpoint on node restart, allowing the opportunity to resolve the issue, something required when dealing with contract constraint failures.
* Remove all notion of message level retry.
* Introduce randomness into de-duplication IDs based on the session rather than the flow, in support of idempotent flows.
* CORDA-1494: Re-enable notarisation retries in the new state machine manager.
The original message-based retry approach does not work well with the new
flow state machine due to the way sessions are handled. We decided to move
the retry logic to flow-level: introduce RetryableFlow that won't have
checkpoints persisted and will be restarted after a configurable timeout
if it does not complete in time.
The RetryableFlow functionality will be internal for now, as it's mainly
tailored for the notary client flow, and there are many subtle ways it can
fail when used with arbitrary flows.
* CORDA-1510 - Allow Doorman and NetworkMap to be configured independently
Currently only one compatabilityZoneURL can be specified, however the
two services can be run on as separate servers. Allow nodes to be
configured in this manner
* Partial review comments
* Review comments
* review comments
node-api now depends on this module and upcoming changes will use this as well rather than having to depend on node-api.
EnumEvolveTests.deserializeWithRename and EnumEvolveTests.multiOperations are temporarily ignored since their test resources can't be regenerated due to bugs.
* Client and server support for amqp
* Observable (and supporting) serialisers
Unit Tests
* Fixing tests
* Test fixes
* CORDA-847 - Update api doc with additon of @CordaSerializable annotation
* TestFixes
* review comments
* TestFixes
* Test Fix
* Test Fix
* Test Fix
* Test Fix
* Test Fix
* Test Fix
* TestFix
* Test Fix
* Review Comments
* Sync public API's (and associated internal helpers) between Enterprise and OS.
* Add previous default constructor explicitly.
* Keep the API checker happy!
* Revert addition of ENT makeTestDataSourceProperties() function.
* CORDA-866: Implement removal of stale nodes from network
Backported
* Implement removal of stale nodes from network
Add eventHorizon to NetworkParameters structure. Add republishing of
node info on 1 day intervals - it is treated by network map as heartbeat from node indicating if it's alive or not. Add removal of old node infos on network map signing.
* Add copy method to NetworkParameters data class
Add JvmOverloads annotation to the constructor, because it's data class
exposed in API
* Fix test
* Feature to exclude CorDapps from specific MockNode's in a MockNetwork.
* Fixed API stability issue.
* * Added docs.
* Changed approach for the new parameter to be additive as opposed to exclusive.
* * Updated incorrect comments.
* fix a bug where registration tool will refuse to register for a service identity if the keystore contain NODE_CA key already.
some refactoring
(cherry picked from commit 5ed60ab)
* CORDA-1355: Introduce a dedicated property which controls what is going to be in scope for classpath scanning
* CORDA-1355: Update change log
* CORDA-1355: Minor change to improve readability.
* CORDA-1355: Custom serializers documentation update to mention new system property.
* Improve logging for NetworkMap requests
* Allow interrupt in polling if the process started successfully
* Put `advertiseNewParameters` back
* Additional log line to indicate when all the nodes are started
* Improve logging and use concurrent map since it is updated from multiple threads
* Change NetworkMap response validity duration and rename parameter accordingly
* Changes following code review from @shamsasari
- Existing embedded Shell connects via RPC including checking RPC user credentials (before was a direct use of CordaRPCOps): in dev mode when console terminal is enabled, node created `shell` user.
- New Standalone Shell app with the same functionalities as Shell: connects to a node via RPC Client, can use SSL and run SSH server.
* CORDA-986 and CORDA-985 CompositeKey and Signature verification performance fixes (#2467)
* CORDA-696: Create separate transaction types for contract upgrade transactions.
Add rationale around upgrade transactions
Move contract upgrade transaction resolution logic into internal until it's stabilised.
Throw a better exception when contract attachment not found
Default legacy contract constraint to always accepting - needs to be changed to whitelist constraint before merging
Introduce a new upgraded contract interface that allows specifying the legacy constraint.
Remove StateLoader, make all tx resolution functions take in ServicesForResolution
Contract upgrade transactions can handle whitelist by zone constraints
When creating a contract upgrade transaction, make sure the attachment of the old cordapp gets attached when using hash constraints.
Attachment lookup for a given contract class name only scans currently loaded cordapps, and we don't load old versions of cordapps.
CORDA-696: Update upgrade docs
* ENT-1443 Add cert role to CSR and doorman issue cert according to the cert role (#431)
* Doorman and HSM create certificate base on requested cert role specified in the certificate signing request.
(cherry picked from commit 94f7392)
* remove R3 corda code
* Jolokia agents are loaded dynamically if configured
* Renamed exportJmxTo (never used) to jmxMonitoringHttpPort and take it from config
* Updated documentation and tests
Further, the look up of the node's own node-info from its database has been tightened to ensure there isn't more than one.
Also fixed some brittle code which was assuming exactly one address rather than at least one.
* Update check api changes to look for internals
* Update several more uses of internal
* Make check-api-changes script filter out internal class usages
* Make CordaClock part of API
* Update api-current.txt
* Remove exclusion of nodeapi.internal
* Remove access to CordaPersistence from public api
* Don't expose DB Connection from StartedMockNode and remove unnecessary transaction from CustomVaultQueryTest
* Make internal tests that use need db access use InternalMockNetwork
* Make test certificates internal
* Address further review comments
* Revert some accidental changes to api-current.txt
* Address Shams' review comments
* Update Api Scanner to filter out CordaInternal attribute
* Update api-current.txt
* Remove superfluous brackets
* Add transaction to StartedMockNode
* More leaky transaction fixes
* Added per-cordapp configuration
* Added new API for Cordformation cordapp declarations to support per-cordapp configuration
* Added a cordapp configuration sample
* Remove sslConfiguration from public constructor of CordaRPCClient
* Address review comments
* Update api-current.txt
* sslConfiguration doesn't need to be a property
* Don't expose StartedNode via Node Driver
* Dont expose StartedNode/Abstract Node via MockNetwork
* Remove internal var from constructor as it doesn't hide from public api and change to internal initialisation method
* Update api
* Rename MockNode to StartedMockNode to avoid confusion
Update documentation
Update api-current.txt
* Fix typo
* Fix test failure
* Modify flow tests to use internal mock network and remove additional internal exposures from StartedMockNode
* Fix api-current
* Change InProcess and OutOfProcess to interfaces
* Explicitly declare MockNetwork parameters
Dont expose StateMachineManager
Move affected tests to use internal mock network
* Fix api-current
* Changes requested via review
* Fix IRS Demo address
* Fix api
* Remove internal attribute from classes in internal package
* Remove accidentally added code
* Move useHttps into NodeHandleInternal
* Remove duplicated code
* Update api-current
* Make webAddress internal on NodeHandle
* Make sure parameters in public api are explicitly specified
* Use correct address in IRS Demo
* Get webaddress from webserver handle
* Update api-current
* wire up and enforce max transaction size
* fixup after rebase
moved network parameter from AbstractNode to NodeProperties
* removed TODO
* fix broken import
* address PR issues
* remove API breaking change
address PR issue
* added max transaction size to driver and mock network.
address PR issues
* fix failing test
* added TODO
* fix verifier test
* fix spring driver build error
* Network parameters updates
Add two RPC methods networkParametersFeed and
acceptNewNetworkParameters. Implementation of client handling of network
parameters update event. Partial implementation of accepting new
parameters and installing them on the node as well as node startup with
updated parameters.
Move reading of network parameters on startup to separate
NetworkParametersReader class. Add tests.
Move NetworkParameters and NotaryInfo classes to core.
* Ignore evolvability test - to be fixed later
* Add documentation on update process
* Create CordaInternal attribute for properties on public classes that are not part of the api and apply to FlowLogic.stateMachine
* Remove startFlow from public test api and replace with startFlowAndReturnFuture
* Update api-current with changed signature
* Change test used in documentation to use public test methods
* Remove the rest of the unneccessary usages of the startFlow test utility
* Remove extra whitespace
* Rename startFlowAndReturnFuture back to startFlow
* Update api
* The annotation doesn't appear unless its marked as on the actual getter and setter
* Updated docs and removed pointless attribute
* Deleted whitespace
Minor tweaks to the invocation context code.
1) Un-deprecate FlowInitiator, move the deprecation to the field. This
eliminates large numbers of warnings and means developers are warned
only once in the place where they obtain one.
2) Add documentation for StateMachineInfo and create a type alias to give
it a better name in an ABI compatible way.
3) Improve markup on InvocationContext
4) Rename field from just "context" to "invocationContext" (Context is vague)
* add cmdline option for network root truststore and password, instead of using node's truststore configuration to avoid confusion.
* revert line auto format
* fix failing integration test
* address PR issue
Introduced DigitalSignatureWithCert and SignedDataWithCert as internal APIs, with the expectation that they will become public; renamed the network parameters end-point to network-parameters; updated the network-map.rst doc; and did some refactoring.
* Raft notaries can share a single key pair for the service identity (in contrast to a shared composite public key, and individual signing key pairs). This allows adjusting the cluster size on the fly.
Using the --just-generate-node-info flag for the notary nodes so that their identities can be submitted to the network map server, which does the network parameters generation.
With network parameters the CN is no longer needed to identify notaries. This frees it up to be used in the node's name alongside the other attributes.
Also, the identity generation logic has been simplified, removing the need to have magic string values for storing distributed identities in the keystore. Now there are just two alias prefixes: "identity" as it was previously, and "distributed-notary".
Copying of the node-info files moved out of Cordform and into NetworkParametersGenerator (which is now called NetworkBootstrapper). This class becomes an external tool to enable deployment of nodes in a test setup on a single filesystem.
* * Document TestIdentity entropy and enforce that it actually works
* Ledger/transaction DSL default notary with fresh key
* MockServices default identity with fresh key
* makeTestIdentityService now takes vararg
* Require cordappPackages for MockServices
* DSL automatic serialization init
* Improve error when two MockNetworks used
* * Make cordappPackages required by MockNetwork
* Default identity service in MockServices
* Make notarySpecs Java-friendly
* Able to send hand coded messages to an Artemis node inbox
Get startup race condition fixed. Start cleanup work.
Fixup after rebase
Remove SASL hack for now
Minor tweaks. Enable AMQP mode manually.
Add configuration control
Slight clean up
Stop timeouts that don't work with AMQP
Rename class
Get TLS constants from :node-api
Primitive integration test
Put back commented line
Session per bridge to alow rollback on remote rejects.
Add more tests and handle multiple IP adddresses
Reduce logging
Fixup after rebase
Add a test to verify the remote end AMQP rejection logic works and does cause message replay.
Allow Artemis to duplicate after session rollback
Reduce number of threads
Move legacy bridge related code over to CoreBridgeManager
Shared threadpool for bridges
Add a test to confirm that no side effects when using a shared thread pool.
Address PR comments and remove dead lines
Rebase and add some comments
Remove a couple of blank lines
Ensure AMQP bridges are used in tests
Fixup after removal of testNodeConfiguration
Add a couple of doc comments
Add a couple of doc comments
Make things internal and use CordaFuture
Address some PR comments
Change comment type
* Use Artemis 2.2 to fix AMQP problems. Add explicit test of legacy core bridges, as marking the factory class private had silently broken them.
* Fix change due to using Artemis 2.2
* Take maximum message size from network parameters
* Add epoch handling
* Add handling of network parameters mismatch
Change NetworkMapClient and updater, add handle in
AbstractNode that results in node shutdown on parameters mismatch. Later
on we should implement proper handling of parameters updates.
Add tests of NetworkParameters wiring.
When node starts with compatibilityZone url configured it takes
networkParameters from the networkMap.
* Permit only one network parameters file
On node startup network parameters are read from node's base directory,
we permit only zero or one files to be there. If network map server is
configured the parameters can be downloaded at startup (if not present
in the directory already).
* Update docs on network map endpoints
* Add support for external data source of access control data (RPC/Shell users credential and permissions), with optional in-memory caching.
* Support password encoded with Apache Shiro fully reversible Modular Crypt Format.
* Introduce 'security' field in Node configuration and related docsite page.
* JMX Jolokia instrumentation WIP (driverDSL, webserver, cordformation, hibernate statistics, access policy config file hardening)
* Cordformation changes to support jolokia agent instrumentation at JVM startup.
* Minor updates to reflect usage of Jolokia 1.3.7 (which uses slightly different .war naming)
* Use relative path reference in -javaagent to prevent problem with long path names with spaces.
* Fixed incorrect regex pattern and added assertion to test.
* Enable JMX monitoring.
* Reporting of Hibernate JMX statistics is configurable (by default, only switched on in devMode)
* Make Artemis JMX enablement configurable.
* Re-instate banning of java serialization.
* Improve JUnit.
* Fixes following rebase from master.
* Re-instated correct regex for picking up Jolokia agent jar.
* Fixed broken integration test.
* Updated documentation
* Updated following PR review feedback.
* Fixed compilation error caused by change in DriverDSL argument type.
* Fixed compilation error caused by change in DriverDSL argument type.
* Fail fast if jolokia-agent-jvm.jar is not located.
* Applied changes in cordformation following review feedback from CA.
* Remove MockServices.stateMachineRecordedTransactionMapping which does nothing
* Inline StateLoaderImpl
* Remove unused MockServices
* MockServices well-known identities not needed in a place
* A few things don't need a full-blown ServiceHub
* Generate networkParameteres for Cordformation.
Fix deployNodes task in Cordformation to generate NetworkParameters before running the nodes.
Add TestNetworkParametersGenerator utility loaded after node infos generation step.
* Get rid of bouncy castle provider dependency
For cordform-common. It caused problems with loading our custom
X509EdDSAEngine for generation of network parameters in deployNodes
task.
* new network map object for network map, and verify signature and root in Signed network map and node info
* fixup after rebase
* * added certificate and key to network map server
* move DigitalSignature.WithCert back to NetworkMap.kt, as its breaking API test, will raise another PR to move it back.
* Make DigitalSignature.WithCert not extend WithKey, as per PR discussion.
* various fixes after rebase.
* move Network map back to core/node, as its breaking API test
* revert unintended changes
* move network map objects to node-api
1. The runRPCCashIssue and runWebCashIssue gradle tasks didn't work because they were using the wrong ports
2. Notary lookup was failing because the lookup name didn't include the correct CN for the notary name (this slipped through when reverting the network parameters)
The ports change occurred in #1922 which was attempting the fix the runIssuer gradle task. This is actually a misleading and redundant task as all it does is start up the nodes, which is what the documented deployNodes already does. The ports runIssuer allocated to the nodes were different to the ones specified in deployNodes.
To make sure we have integration tests which closely match deployNodes, the BoC demo has been updated to make use of CordformDefinition. This keeps the node definitions in one place, removing the need to have disparate files in sync. runIssuer has been removed.
* make node info file copying optional by setting "compatabilityZoneURL" in driver
integration test for node using http network map using driver
some bug fixes
* rebase to feature branch and fixup
* add initialRegistration flag to driver
* remove useFileBaseNetworkMap flag, add network map server to DriverTest
* remove useFileBaseNetworkMap flag, add network map server to DriverTest
* use PortAllocation.Incremental instead of random
* * use PortAllocation.Incremental instead of random
* fix NodeInfoWatcher thread leak issue
* reset scheduler before create notary
* move port allocation out of companion object
* move port allocation out of companion object
* make node info file copier lateinit to avoid observable thread pool get created on init
* Revert "CORDA-296: added rpc that returns an observable for node state (#2004)"
This reverts commit 7d1f7ab
* Revert "CORDA-296: added rpc that returns an observable for node state (#2004)"
This reverts commit 7d1f7ab
Previously when de-anonymising a Party instance, the name of the Party was used rather than
the key, meaning a Party could be constructed with a random nonsense key and any name, and be treated as corresponding to the well known identity. This is not a security hole in itself as
in any real scenario a party shouldn't be trusted without having been registered, it creates
a significant risk of a security hole depending on how trusted the anonymous identity is, and
the returned identity is considered.
* Clean up identities in CashTests so that the mini/mega corp keys and identities are correctly
paired together throughout. Previously `miniCorpServices` presented the same key as the
MegaCorp identity, but with the name Mini Corp attached.
* Correct key/name matches in VaultWithCashTest
* Split services in CashTests to not have multiple identities per service hub
The motivation for this came with the recent change that a default notary is started by the driver, which if ignored will leak the notary process.
Also, waitForAllNodesToFinish() has been replaced by a driver parameter.
* Network map cache using Network map client instead of artemis. -- WIP
* fix up after rebase
* address PR issues, split network map update test, added todos to remove sleeps
* move jimfs and baseDir to field variable
* CORDA-296: added rpc that returns an observable for node state; used to let rpc clients know that the know is about to shut down
* replaced node shut down observation String with enum
* Extracted out ShutdownManager into its own file
* Moved RPCDriver and ProcessUtilities into internal package
* Made n.c.testing.performance package internal
network-parameters file read in by the node at startup, of which only the list of notaries is used. For now, the driver and MockNetwork have been updated to require notaries to be started first. This is so that the same set of network parameters can be defined for all the nodes.
CN in the legal name is not longer disallowed since it's no longer reserved for distributed notary names.
Single-node notaries now only have one identity, their main identity. Nodes part of a cluster continue to have two.
(Based off Kasia's work)
* [CORDA-446] Clean up other mentions of network map node and logic
* Rename AbstractNetworkMapService to NetworkMapService and remove the empty NetworkMapService
* fix build
* fix artemismessaging tests
* pr comments
Please see Jira for background info.
The change should help all the Windows developers.
Closer examination shows that in the command line we include the classpath of size ~20,000 characters twice.
Once with `-cp` option - which is fine and second time with `-Djava.class.path=` - the latter is completely unnecessary and can be eliminated shortening the command length dramatically to around: 24,000 characters which should give us some leg room for further expanding the classpath.
approach which assumes a dedicated node for observers: states that are
reported to the node will appear in the database and update feeds as
normal. Apps that expect all updates to be relevant to themselves may
need adjusting if they run on an observer node too, but this is likely
to be rare.
* [CORDA-442] make MockNetwork not start a networkmap node
Now MockNetwork will put the appropriate NodeInfos inside each running node networkMapCache.
Tests relating to networkmap node starting and interaction have been removed since they where relaying on MockNetwork
* Fix-up: Bank Of Corda sample
Use correct CorDapp packages to scan
(cherry picked from commit 2caa134)
* Set adequate permissions for the nodes such that NodeExplorer can connect
(cherry picked from commit ae88242)
* Set adequate permissions for the nodes such that NodeExplorer can connect
(cherry picked from commit ae88242)
* Correct run configuration
* Fix-up port numbers
* [CORDA-442] let Driver run without network map
- Nodes started by driver run without a networkMapNode.
- Driver does not take a networkMapStartStrategy anymore
- a new parameter in the configuration "noNetworkMapServiceMode" allows for a node not to be a networkMapNode nor to connect to one.
- Driver now waits for each node to write its own NodeInfo file to disk and then copies it into each other node.
- When driver starts a node N, it waits for every node to be have N nodes in their network map.
Note: the code to copy around the NodeInfo files was already in DemoBench, the NodeInfoFilesCopier class was just moved from DemoBench into core (I'm very open to core not being the best place, please advise)
* Attempting to make VerifierTests more stable
Main point: Alice node will always be faster to locate Alice by name than NotaryNode. Especially given that it may take some time
for Alice node to get reflected on the Notary's NetworkMap.
Also additional logging which might be helpful during debug and IntelliJ changed project structure reflection.
* Introduce a way for nodes to reliably learn about each other existence in the NetworkMap
Plus minor refactoring of the Driver code.
Rework identity usage in tests to extract identity from nodes by name, rather than just arbitrarily choosing the first identity. This better models the intended design for production (future work).
