* Network parameters updates
Add two RPC methods networkParametersFeed and
acceptNewNetworkParameters. Implementation of client handling of network
parameters update event. Partial implementation of accepting new
parameters and installing them on the node as well as node startup with
updated parameters.
Move reading of network parameters on startup to separate
NetworkParametersReader class. Add tests.
Move NetworkParameters and NotaryInfo classes to core.
* Ignore evolvability test - to be fixed later
* Add documentation on update process
* CORDA-904 - Make evolver work with classes that use setters
* review comments
* review comments
* small fixs
* don't include systemTest in compiler.xml
* CORDA-946 - Fixes to fingerprinting breaks backward compatibility
Demonstrated using the network map parameters signed form as that's
where the problem was first seen
* Review Comments
* CORDA-915 - Replace BEANS introspector with standard reflection
Removes lib dependency and puts something in place we can better
control
* CORDA-915 - Review comment corrections
* Review Comments
Kotlin's own reflection has a habit of throwing weird errors and does not work for private objects - thus the unit test which tests this feature has actually never worked properly, but somehow works by accident. An attempt to upgrade to the latest Kotlin revealed the issue so it must have always been unstable.
Also fixes an odd bug where the inferred type of a getter wasn't
matching the constructor parameter type because that was still unbounded
and seen as T, looking at the raw type allows us to inspect this
properly
This is a multi issue problem
1. Fingerprinting of generics treats <T> and <?> differently, forcing
the evolver to be used when not needed
2. However, the evolver is required sometimes as generics are not
guaranteed to fingerprinting bi-directionally (thanks to type erasure of
deeply nested generic types). However, with serialization now writing
properties in a specific order, we need to ensure they're read back in
that order before applying them to an evolved constructor so as to
not corrupt the object reference cache
* Make SerializationEnvironment private so as not to expose internals.
* Only expose used parts of api
* Make properties lateinit
* Removing java calls to getEnv
* Initialise properties at declaration
* Tidy up imports
Introduced DigitalSignatureWithCert and SignedDataWithCert as internal APIs, with the expectation that they will become public; renamed the network parameters end-point to network-parameters; updated the network-map.rst doc; and did some refactoring.
Can't actually get something to go through the serializer with a
wild card in place as it seems that's an impossible situation
* Review Changes
* CORDA-855 - Review Comments
* Review Comments
* Review comments
* Raft notaries can share a single key pair for the service identity (in contrast to a shared composite public key, and individual signing key pairs). This allows adjusting the cluster size on the fly.
* Make cordform test use new network bootstrapper logic
* Fixing review comments
* Fix issue with backwards compatibility
* Fix issue with setup not being called from CordformDefinitions
* Make sure node dir is created (as CordformDefinition uses it directly if setup is overridden
Make sure tmp dir is created
* Don't crash if node dir is already created
* Stop overwriting errors
Using the --just-generate-node-info flag for the notary nodes so that their identities can be submitted to the network map server, which does the network parameters generation.
* Generate node directories as part of bootstrapping
* Include latest corda.jar in bootstrapper package
Remove SLF4J warnings on startup
* Changes post review
* More review changes
* Review changes
* Making docs clearer
With network parameters the CN is no longer needed to identify notaries. This frees it up to be used in the node's name alongside the other attributes.
Also, the identity generation logic has been simplified, removing the need to have magic string values for storing distributed identities in the keystore. Now there are just two alias prefixes: "identity" as it was previously, and "distributed-notary".
* Add roles to X509 certificates so that the identity service can always determine which certificate in a hierarchy is the well known identity
* Rename CLIENT_CA certificate type to NODE_CA
* Rename DOORMAN role to INTERMEDIATE_CA
* Correct issue in CashTests where instead of providing a well known identity to generateSpend(), a confidential identity was passed in and a confidential identity generated from it.
* Enforce role hierarchy in PKI
* Enforce that party certificates must be well known or confidential identities
* Add network map certificate role
Copying of the node-info files moved out of Cordform and into NetworkParametersGenerator (which is now called NetworkBootstrapper). This class becomes an external tool to enable deployment of nodes in a test setup on a single filesystem.
* * Document TestIdentity entropy and enforce that it actually works
* Ledger/transaction DSL default notary with fresh key
* MockServices default identity with fresh key
* makeTestIdentityService now takes vararg
* Require cordappPackages for MockServices
* DSL automatic serialization init
* Improve error when two MockNetworks used
* * Make cordappPackages required by MockNetwork
* Default identity service in MockServices
* Make notarySpecs Java-friendly
* Able to send hand coded messages to an Artemis node inbox
Get startup race condition fixed. Start cleanup work.
Fixup after rebase
Remove SASL hack for now
Minor tweaks. Enable AMQP mode manually.
Add configuration control
Slight clean up
Stop timeouts that don't work with AMQP
Rename class
Get TLS constants from :node-api
Primitive integration test
Put back commented line
Session per bridge to alow rollback on remote rejects.
Add more tests and handle multiple IP adddresses
Reduce logging
Fixup after rebase
Add a test to verify the remote end AMQP rejection logic works and does cause message replay.
Allow Artemis to duplicate after session rollback
Reduce number of threads
Move legacy bridge related code over to CoreBridgeManager
Shared threadpool for bridges
Add a test to confirm that no side effects when using a shared thread pool.
Address PR comments and remove dead lines
Rebase and add some comments
Remove a couple of blank lines
Ensure AMQP bridges are used in tests
Fixup after removal of testNodeConfiguration
Add a couple of doc comments
Add a couple of doc comments
Make things internal and use CordaFuture
Address some PR comments
Change comment type
* Use Artemis 2.2 to fix AMQP problems. Add explicit test of legacy core bridges, as marking the factory class private had silently broken them.
* Fix change due to using Artemis 2.2
* Take maximum message size from network parameters
* Add epoch handling
* Add handling of network parameters mismatch
Change NetworkMapClient and updater, add handle in
AbstractNode that results in node shutdown on parameters mismatch. Later
on we should implement proper handling of parameters updates.
Add tests of NetworkParameters wiring.
When node starts with compatibilityZone url configured it takes
networkParameters from the networkMap.
* Permit only one network parameters file
On node startup network parameters are read from node's base directory,
we permit only zero or one files to be there. If network map server is
configured the parameters can be downloaded at startup (if not present
in the directory already).
* Update docs on network map endpoints
* SignedNetworkMap verification fix
SignedNetworkMap verification should also include cert path validation,
which was probably moved away by accident, because docs say about the
exception CertPathValidatorException.
* Add support for external data source of access control data (RPC/Shell users credential and permissions), with optional in-memory caching.
* Support password encoded with Apache Shiro fully reversible Modular Crypt Format.
* Introduce 'security' field in Node configuration and related docsite page.
* JMX Jolokia instrumentation WIP (driverDSL, webserver, cordformation, hibernate statistics, access policy config file hardening)
* Cordformation changes to support jolokia agent instrumentation at JVM startup.
* Minor updates to reflect usage of Jolokia 1.3.7 (which uses slightly different .war naming)
* Use relative path reference in -javaagent to prevent problem with long path names with spaces.
* Fixed incorrect regex pattern and added assertion to test.
* Enable JMX monitoring.
* Reporting of Hibernate JMX statistics is configurable (by default, only switched on in devMode)
* Make Artemis JMX enablement configurable.
* Re-instate banning of java serialization.
* Improve JUnit.
* Fixes following rebase from master.
* Re-instated correct regex for picking up Jolokia agent jar.
* Fixed broken integration test.
* Updated documentation
* Updated following PR review feedback.
* Fixed compilation error caused by change in DriverDSL argument type.
* Fixed compilation error caused by change in DriverDSL argument type.
* Fail fast if jolokia-agent-jvm.jar is not located.
* Applied changes in cordformation following review feedback from CA.
* Rename certificate types
* Create separate certificate type for confidential identities
* Add name constraints to dev node CA
* Move dev node CA into getTestPartyAndCertificate()
This removes any need for the user implement and override types from the
super class
* CORDA-786 - Docs update
* CORDA-786 - Remove unneeded second annotation on the proxy objects
* Fix merge conflicts
* Remove MockServices.stateMachineRecordedTransactionMapping which does nothing
* Inline StateLoaderImpl
* Remove unused MockServices
* MockServices well-known identities not needed in a place
* A few things don't need a full-blown ServiceHub
This change doesn't enable anything, it just changes the code to pass
around both relevant schemas instead of a single one from the AMQP
envelope. The actual evolver will build ontop of this
* Generate networkParameteres for Cordformation.
Fix deployNodes task in Cordformation to generate NetworkParameters before running the nodes.
Add TestNetworkParametersGenerator utility loaded after node infos generation step.
* Get rid of bouncy castle provider dependency
For cordform-common. It caused problems with loading our custom
X509EdDSAEngine for generation of network parameters in deployNodes
task.
* new network map object for network map, and verify signature and root in Signed network map and node info
* fixup after rebase
* * added certificate and key to network map server
* move DigitalSignature.WithCert back to NetworkMap.kt, as its breaking API test, will raise another PR to move it back.
* Make DigitalSignature.WithCert not extend WithKey, as per PR discussion.
* various fixes after rebase.
* move Network map back to core/node, as its breaking API test
* revert unintended changes
* move network map objects to node-api
This change doesn't enable anything, it just changes the code to pass
around both relevant schemas instead of a single one from the AMQP
envelope. The actual evolver will build ontop of this
* PartyAndCertificate serialiser has been converted to deal with just CertPath
* Removed X500Name serialiser as we no longer use the BC implementation and have our own CordaX500Name
* Converted X509 cert serialiser to use Java certs and not BC X509 cert holder
* Added to test to AMQP serialisation to make sure c'tor is invoked on deserialisation
network-parameters file read in by the node at startup, of which only the list of notaries is used. For now, the driver and MockNetwork have been updated to require notaries to be started first. This is so that the same set of network parameters can be defined for all the nodes.
CN in the legal name is not longer disallowed since it's no longer reserved for distributed notary names.
Single-node notaries now only have one identity, their main identity. Nodes part of a cluster continue to have two.
(Based off Kasia's work)
Define the two transforms that will be useful for enum evolvability (see
design document for more details).
Furthermore, define the generic mechanism by which transform annotations
on classes are encoded into the AMQP envelope
With nothing to check for these annotations at either end, this is
mostly a no op, but an important step toward getting evolvability in
place
* CORDA-351: force update dependencies and suppress vulnerabilities not affecting corda
* CORDA-351: force update dependencies and suppress vulnerabilities not affecting corda
* [CORDA-442] let Driver run without network map
- Nodes started by driver run without a networkMapNode.
- Driver does not take a networkMapStartStrategy anymore
- a new parameter in the configuration "noNetworkMapServiceMode" allows for a node not to be a networkMapNode nor to connect to one.
- Driver now waits for each node to write its own NodeInfo file to disk and then copies it into each other node.
- When driver starts a node N, it waits for every node to be have N nodes in their network map.
Note: the code to copy around the NodeInfo files was already in DemoBench, the NodeInfoFilesCopier class was just moved from DemoBench into core (I'm very open to core not being the best place, please advise)
* Remove local function because it is serialised as a lambda.
* Don't automatically whitelist Kotlin lambdas unless checkpointing.
* Add comment to @CordaSerializable, warning not to allow AnnotationTarget.EXPRESSION.
* Don't repackage well known key types when converting keys to a well known type
* Remove custom key serializers
* Remove duplicate serializer registration
* Added schedulable flows to cordapp scanning
Fixed a bug where the core flows are included in every cordapp. Added a test to prove the scheduled flows are loaded correctly. Added scheduled flow support to cordapp.
Renabled broken test.
Fixed test to prove cordapps aren't retreived from network.
Review fixes.
Fixed a test issue caused by gradle having slightly different paths to IntelliJ
* Fixed test for real this time.
* Rename and move CordaPluginRegistry to reflect its real purpose now.
* Docs: docsite improvements
* Remove discussion of webserver from 'writing a cordapp' page.
* Fixup some flow docs.
* Add a couple more package descriptions.
* Review comments - always apply default whitelist and no longer load it via ServiceLoader
* Added wording about renaming services resource file
* CORDA-540: Do not use concrete instance of an ArrayList as wire representation of it may be different
* CORDA-540: Make data structures suitable for AMQP serialization
* CORDA-540: Use "name" instead of "toString()"
Classes like "net.corda.finance.contracts.DayCountBasisDay" override "toString()" which leads to error behaviour
Handle PR comments
Add nicer constructors to CordaException and CordaRuntimeException
(cherry picked from commit 89478c8)
Fix ambiguous defaulted constructor
(cherry picked from commit ec9bafe)
Address PR comment
Update a few more custom exceptions
So...
On reflection, and adding a number of tests for static initialisation
with serialised types it looks like there is no chance that the
serializer factory will ever pass a non white-listed type through to the
carpenter in the first place.
As such leaving the plumbing in as it may be useful to pass a blacklist
at some point into the carpenter and the tests are always useful
(ignoring those that won't work without the white-list checking)
The class carpenter should refuse to carpent classes that are not
whitelisted or marked as CordaSerializable. This prevents any security
issue where a malicious message could indicate a class had a member of
some type that on construction did something bad. By respecting the
whitelist we avoid this.
As the carpeter annotates anythign it constructs as CordaSerializable,
it will always be able to carpent classes that contain memebrs that were
unknown, and thus unannotated, carpented classes
The current use of SSL RPC relies on access to the node's keystore file, and further to that some uses where using the NODE_USER to login on the p2p port.
* Transform Kotlin's EmptyList, EmptySet and EmptyMap into Java classes before serialising them.
* Transform Kotlin's EmptyList, EmptySet and EmptyMap to their unmodifiable Java equivalents.
* Contract constraints and attachment loading
Fix compiler warnings.
Fixed IdentitySyncFlowTests in confidential-identities.
Fixes.
Fix AttachmentClassLoaderTests.
Added a TODO.
Renamed cordapp service.
Fix compilation error in java code.
Fix RaftNotaryServiceTests
Fix AttachmentLoadingTest
Fix DistributedServiceTests and LargeTransactionTests.
Add cordapp packages to Verifier tests.
Refactor DummyContractBackdoor back out of internal package.
Resolve compiler warnings.
Consolidate excluding `isolated` project at top-level.
Fix contract attachment serialisation for remote verifier.
Fix integration tests for client:rpc.
Contract constraints and attachment loading
Fix compiler warnings.
Fixed IdentitySyncFlowTests in confidential-identities.
Fixes.
Fix AttachmentClassLoaderTests.
Added a TODO.
Renamed cordapp service.
Fix compilation error in java code.
Fix example compilation.
Fix RaftNotaryServiceTests
Fix AttachmentLoadingTest
Fix DistributedServiceTests and LargeTransactionTests.
Add cordapp packages to Verifier tests.
Refactor DummyContractBackdoor back out of internal package.
Resolve compiler warnings.
Consolidate excluding `isolated` project at top-level.
Fix integration tests for client:rpc.
Fixed issues with node driver and differing ZIPs.
Review changes.
Refactor GeneratedAttachment into node-api module.
Merge branch 'clint/hash-constraint' of https://github.com/corda/corda into clint/hash-constraint
Fixed compile error following rebase.
wip - test to check that app code isn't loaded from attachments sent over the wire.
Use Kotlin copyTo() rather than Apache's IOUtils.
Fixes
more fixes.
Removing unconstrained output.
More fixes.
Fixed another test.
Added missing plugin definition in net.corda.core.node.CordaPluginRegistry: net.corda.finance.contracts.isolated.IsolatedPlugin
Re-added missing magic string used in unit test.
Remove unused FlowSession variable.
* Review fixes.
* More review fixes.
* Moved Cordapp implementation to an internal package.
* More JVMOverloads.
Set things up so that the carpenter must be constructed with a Corda
white list object. Right now we don't do anything with it but this will
capture all the changes in the tests to make it "jsut work" as is before
getting into the meat of the change
Externally this is a "no op" as the serialiser factory already takes a
white list as a construction parameter and the carpenter should use that
anyway
* Added some missing types to the whitelist and removed `KeyPair`.
* Add emptyList/Map/Set for Java. Also check for duplicates.
* Remove java singleton collections since they are already added via kotlin listOf/mapOf/setOf.
* Remove advertisedServices from NodeInfo.
Introduce notaryIdentities in NetworkMapCache, that will be filled in
later from NetworkParameters. Clean up NetworkMapCache API. Expose
notaryIdentities through RPC. For now we assume as temporary solution
that notaries in NetworkMap have to contain "notary" in name.
* Further clean up of NetworkMapCache API
Remve partyNodes. Introduce getAllNodeInfos function
* Remove notaryIdentity from ServiceHub
* Address Shams review comments
* Address Andrius review comments
* Add comments, cleanup
* Fixes
* Address comments
* Yet another commit with comments addressed
* Move ServiceType and ServiceInfo to node-api
Add changelog entry. Address rest of comments.
* Minor comments
* Remove node's main identitiy from NodeInfo.
Preparation for getting rid of services + supporting multiple identities
on the node.
NodeInfo keeps multiple identities as a list. For now the first one is treated as a special one.
Introduced function chooseIdentity in CoreTestUtils as a preparation for proper handling of multiple identities in the future.
Remove legalIdentityKey from ServiceHub, add extension function - chooseIdentity on ServiceHub.
Add `me` field on FlowStateMachineImplemetation, flows should know what the calling identity is.
Remove SERVICES_PREFIX in artemis messaging layer.
* Address minor comments.
* Fixes after rebase.
Remove chooseIdentity from ServiceHub
* Rename me to ourIdentity on FlowLogic
* Fixes after rebase
* Address Ross comments, fixes
* Fix after rebase
* Fix services certificate paths
Apply Patrick's patch.
* Change how NetworkHostAndPort is parsed
Change from using a global extension function to parse
NetworkHostAndPort strings, into using a function on the companion
object. This is a lot easier for Java interop and matches the common
style used elsewhere both in Corda and in Java libraries.
* Move JAR extraction into new utils file
* Move path verification function to ArtemisUtils
Move path verification function "requireOnDefaultFileSystem()" to new
ArtemisUtils.kt file, as this makes more sense from a Java interop
perspective.
* Add JvmName to AMQPSchemaExtensions
* Add JvmName to AMQPSerializationScheme
* Revert "Move JAR extraction into new utils file"
This reverts commit 1f0f41909b68ff21cc24b5efd6a1a360393a0a14.
* Reformat code
* Run formatter on ArtemisUtils
Refactoring meant that the secondary constructor wasn't being called any
more and thus the check wasn't being made. Originally, we always
created copies of fields when constructing a schema which meant we always
called the secondary constructor, now we don't
If the serializer is going to support enumerated types then the class
carpenter also has to
Refactor the Carpenter schema and fields to add an enum type, add code
in the carpenter to generate enum's and of course add tests
* ContractState's contract type has been moved to TransactionState and is now a string representing the class name of the contract class to allow classloading of arbitrary contracts from custom classloaders.
* Upgraded isolated JAR to new version.
* Change RPCApi comments to inline from Dokka, as the existing comments are not API documentation,
and render simply as "The RPC protocol:" in several places, which is unhelpful.
* Rewrite RPCApi comments to no longer be in semi-Dokka format, but instead better reflect they are functional documentation.
* Add Dokka documentation around RPC API
* Change JvmStatics to const
AMQP doesn't define an enum type itself yet the old implementation
listed our snum type as that in the schema despite what we are actually
doing which is serialising the enum down as a list of a string and an
int accompanied by a list of AMQP choices that represent all of the enum
values
Review Comments
Use fingerprinting of the enum types to catch whenever they're changed,
include the enum constants in the fingerprint to avoid any collisions
* Fix the ToStringSerializer so it interacts with reference counting correctly, and stop ref counting byte arrays since they are invariable wrapped in an object we do reference count.
* Pull out object reference condition into a helper function shared by serialization and deserialization
* CORDA-540: Introduce mandatory reason for "kryoSpecific"
... before we forget why they are ignored in such a way
* CORDA-540: Write a test that exposes list serialization problem in AMQP mode
* Revert "Remove CompositeSignaturesWithKeys"
This reverts commit 9b3cad3
* Rename KeyFactory to CompositeKeyFactory
* Move expandedCompositeKeys into TestDSL as the only place that uses it
* Move NullKeys out of their own package
* Move remaining crypto classes into superpackage
* Move utility classes out of crypto and into utilities package
For future testing it would be nice, post serialisation, to have easy
access to the serialised objects schema so we can check how it was
serialised. Adding a helper function to return a data class that does
this in the same way we can for deserialize
