From 760cc1ec48acaa966b5c7893251576c91f342829 Mon Sep 17 00:00:00 2001 From: Katelyn Baker Date: Wed, 11 Apr 2018 13:05:50 +0100 Subject: [PATCH 1/9] RELEASE - merge v3.1 release notes / upgrade notes / changelog to master (#2937) * RELEASE - 3.1 upgrade and release notes * Update docs for change to vno * address vno change in release notes * Update release-notes.rst * make corda links * Review comments * Review comments * review comments * remove ref to reverted bugfix * Review comments --- docs/source/changelog.rst | 42 ++++++++++++------ docs/source/release-notes.rst | 59 +++++++++++++++++++++++++ docs/source/upgrade-notes.rst | 81 ++++++++++++++++++++++++++++++++++- 3 files changed, 169 insertions(+), 13 deletions(-) diff --git a/docs/source/changelog.rst b/docs/source/changelog.rst index 64cd538e2d..9e239532b1 100644 --- a/docs/source/changelog.rst +++ b/docs/source/changelog.rst @@ -5,7 +5,16 @@ Here's a summary of what's changed in each Corda release. For guidance on how to release, see :doc:`upgrade-notes`. Unreleased ----------- +========== + +* java.security.cert.CRLReason added to the default Whitelist. + +* java.security.cert.X509CRL serialization support added. + +* Upgraded H2 to v1.4.197. + +* Shell (embedded available only in dev mode or via SSH) connects to the node via RPC instead of using the ``CordaRPCOps`` object directly. + To enable RPC connectivity ensure node’s ``rpcSettings.address`` and ``rpcSettings.adminAddress`` settings are present. * Errors thrown by a Corda node will now reported to a calling RPC client with attention to serialization and obfuscation of internal data. @@ -19,16 +28,12 @@ Unreleased only once when it was created. Whilst registering serializers that already exist is essentially a no-op, it's a performance overhead for a very frequent operation that hits a synchronisation point (and is thus flagged as contended by our perfomance suite) -* Update the fast-classpath-scanner dependent library version from 2.0.21 to 2.12.3 - - .. note:: Whilst this is not the latest version of this library, that being 2.18.1 at time of writing, versions later - than 2.12.3 (including 2.12.4) exhibit a different issue. - * Node can be shut down abruptly by ``shutdown`` function in `CordaRPCOps` or gracefully (draining flows first) through ``gracefulShutdown`` command from shell. * Carpenter Exceptions will be caught internally by the Serializer and rethrown as a ``NotSerializableException`` - * Specific details of the error encountered are logged to the node's log file. More information can be enabled by setting the debug level to ``trace`` ; this will cause the full stack trace of the error to be dumped into the log. + * Specific details of the error encountered are logged to the node's log file. More information can be enabled by setting the debug level to + ``trace`` ; this will cause the full stack trace of the error to be dumped into the log. * Parsing of ``NodeConfiguration`` will now fail if unknown configuration keys are found. @@ -40,14 +45,27 @@ Unreleased * java.math.BigInteger serialization support added. -* java.security.cert.CRLReason added to the default Whitelist. +.. _changelog_v3.1: -* java.security.cert.X509CRL serialization support added. +Version 3.1 +----------- -* Upgraded H2 to v1.4.197. +* Update the fast-classpath-scanner dependent library version from 2.0.21 to 2.12.3 -* Shell (embedded available only in dev mode or via SSH) connects to the node via RPC instead of using the ``CordaRPCOps`` object directly. - To enable RPC connectivity ensure node’s ``rpcSettings.address`` and ``rpcSettings.adminAddress`` settings are present. + .. note:: Whilst this is not the latest version of this library, that being 2.18.1 at time of writing, versions later + than 2.12.3 (including 2.12.4) exhibit a different issue. + +* Updated the api scanner gradle plugin to work the same way as the version in master. These changes make the api scanner more + accurate and fix a couple of bugs, and change the format of the api-current.txt file slightly. Backporting these changes + to the v3 branch will make it easier for us to ensure that apis are stable for future versions. These changes are + released in gradle plugins version 3.0.10. For more information on the api scanner see + the `documentation `_. + +* Fixed security vulnerability when using the ``HashAttachmentConstraint``. Added strict check that the contract JARs + referenced in a transaction were deployed on the node. + +* Fixed node's behaviour on startup when there is no connectivity to network map. Node continues to work normally if it has + all the needed network data, waiting in the background for network map to become available. .. _changelog_v3: diff --git a/docs/source/release-notes.rst b/docs/source/release-notes.rst index 40455396f8..fba39537be 100644 --- a/docs/source/release-notes.rst +++ b/docs/source/release-notes.rst @@ -1,6 +1,61 @@ Release notes ============= +.. _release_notes_v3_1: + +Release 3.1 +----------- + +This rapid follow-up to Corda 3.0 corrects an issue discovered by some users of Spring Boot and a number of other +smaller issues discovered post release. All users are recommended to upgrade. + +Special Thanks +~~~~~~~~~~~~~~ + +Without passionate and engaged users Corda would be all the poorer. As such, we are extremely grateful to +`Bret Lichtenwald `_ for helping nail down a reproducible test case for the +Spring Boot issue. + +Major Bug Fixes +~~~~~~~~~~~~~~~ + +* **Corda Serialization fails with "Unknown constant pool tag"** + + This issue is most often seen when running a CorDapp with a Rest API using / provided by ``Spring Boot``. + + The fundamental cause was ``Corda 3.0`` shipping with an out of date dependency for the + `fast-classpath-scanner `_ library, where the manifesting + bug was already fixed in a released version newer than our dependant one. In response, we've updated our dependent + version to one including that bug fix. + +* **Corda Versioning** + + Those eagle eyed amongst you will have noticed for the 3.0 release we altered the versioning scheme from that used by previous Corda + releases (1.0.0, 2.0.0, etc) with the addition of an prepended product name, resulting in ``corda-3.0``. The reason for this was so + that developers could clearly distinguish between the base open source platform and any distributions based on on Corda that may + be shipped in the future (including from R3), However, we have heard the complaints and feel the pain that's caused by various + tools not coping well with this change. As such, from now on the versioning scheme will be inverted, with this release being ``3.1-corda``. + + As to those curious as to why we dropped the patch number from the version string, the reason is very simple: there won't + be any patches applied to a release of Corda. Either a release will be a collection of bug fixes and non API breaking + changes, thus eliciting a minor version bump as with this release, or major functional changes or API additions and warrant + a major version bump. Thus, rather than leave a dangling ``.0`` patch version on every release we've just dropped it. In the + case where a major security flaw needed addressing, for example, then that would generate a release of a new minor version. + +Issues Fixed +~~~~~~~~~~~~ + +* RPC server leaks if a single client submits a lot of requests over time [`CORDA-1295 `_] +* Flaky startup, no db transaction in context, when using postgresql [`CORDA-1276 `_] +* Corda's JPA classes should not be final or have final methods [`CORDA-1267 `_] +* Backport api-scanner changes [`CORDA-1178 `_] +* Misleading error message shown when node is restarted after the flag day +* Hash constraints not working from Corda 3.0 onwards +* Serialisation Error between Corda 3 RC01 and Corda 3 +* Nodes don't start when network-map/doorman is down + +.. _release_notes_v3_0: + Release 3.0 ----------- @@ -294,6 +349,8 @@ Minor Changes * Numerous bug fixes and documentation tweaks. * Removed dependency on Jolokia WAR file. +.. _release_notes_v2_0: + Release 2.0 ----------- Following quickly on the heels of the release of Corda 1.0, Corda version 2.0 consolidates @@ -313,6 +370,8 @@ Adds the facility for transparent forwarding of transactions to some third party that entity simply run an Observer node they can simply recieve a stream of digitally signed, de-duplicated reports that can be used for reporting. +.. _release_notes_v1_0: + Release 1.0 ----------- Corda 1.0 is finally here! diff --git a/docs/source/upgrade-notes.rst b/docs/source/upgrade-notes.rst index 28d303566a..2f112d3f4c 100644 --- a/docs/source/upgrade-notes.rst +++ b/docs/source/upgrade-notes.rst @@ -31,6 +31,85 @@ We also strongly recommend cross referencing with the :doc:`changelog` to confir UNRELEASED ---------- +<<< Fill this in >>> + +v3.0 to v3.1 +------------ + +Gradle Plugin Version +^^^^^^^^^^^^^^^^^^^^^ + +Corda 3.1 uses version 3.1.0 of the gradle plugins and your ``build.gradle`` file should be updated to reflect this. + +.. sourcecode:: shell + + ext.corda_gradle_plugins_version = '3.1.0' + +You will also need to update the ``corda_release_version`` identifier in your project gradle file. + +.. sourcecode:: shell + + ext.corda_release_version = '3.1-corda' + +V2.0 to V3.0 +------------ + +Gradle Plugin Version +^^^^^^^^^^^^^^^^^^^^^ + +Corda 3.0 uses version 3.0.9 of the gradle plugins and your ``build.gradle`` file should be updated to reflect this. + +.. sourcecode:: shell + + ext.corda_gradle_plugins_version = '3.0.9' + +You will also need to update the ``corda_release_version`` identifier in your project gradle file. + +.. sourcecode:: shell + + ext.corda_release_version = 'corda-3.0' + +Network Map Service +^^^^^^^^^^^^^^^^^^^ + +With the re-designed network map service the following changes need to be made: + +* The network map is no longer provided by a node and thus the ``networkMapService`` config is ignored. Instead the + network map is either provided by the compatibility zone (CZ) operator (who operates the doorman) and available + using the ``compatibilityZoneURL`` config, or is provided using signed node info files which are copied locally. + See :doc:`network-map` for more details, and :doc:`setting-up-a-corda-network.rst` on how to use the network + bootstrapper for deploying a local network. + +* Configuration for a notary has been simplified. ``extraAdvertisedServiceIds``, ``notaryNodeAddress``, ``notaryClusterAddresses`` + and ``bftSMaRt`` configs have been replaced by a single ``notary`` config object. See :doc:`corda-configuration-file` + for more details. + +* The advertisement of the notary to the rest of the network, and its validation type, is no longer determined by the + ``extraAdvertisedServiceIds`` config. Instead it has been moved to the control of the network operator via + the introduction of network parameters. The network bootstrapper automatically includes the configured notaries + when generating the network parameters file for a local deployment. + +* Any nodes defined in a ``deployNodes`` gradle task performing the function of the network map can be removed, or the + ``NetworkMap`` parameter can be removed for any "controller" node which is both the network map and a notary. + +* For registering a node with the doorman the ``certificateSigningService`` config has been replaced by ``compatibilityZoneURL``. + +Corda Plugins +^^^^^^^^^^^^^ + +* Corda plugins have been modularised further so the following additional gradle entries are necessary: + For example: + + .. sourcecode:: groovy + dependencies { + classpath "net.corda.plugins:cordapp:$corda_gradle_plugins_version" + } + + apply plugin: 'net.corda.plugins.cordapp' + +The plugin needs to be applied in all gradle build files where there is a dependency on Corda using any of: +cordaCompile, cordaRuntime, cordapp + * For existing contract ORM schemas that extend from ``CommonSchemaV1.LinearState`` or ``CommonSchemaV1.FungibleState``, you will need to explicitly map the ``participants`` collection to a database table. Previously this mapping was done in the superclass, but that makes it impossible to properly configure the table name. The required changes are to: @@ -405,4 +484,4 @@ Finance * Adjust imports of Cash flow references * Adjust the ``StartFlow`` permission in ``gradle.build`` files - * Adjust imports of the associated flows (``Cash*Flow``, ``TwoPartyTradeFlow``, ``TwoPartyDealFlow``) \ No newline at end of file + * Adjust imports of the associated flows (``Cash*Flow``, ``TwoPartyTradeFlow``, ``TwoPartyDealFlow``) From 91fd40c8061e06101aef2dc5bca5d123e00b0e13 Mon Sep 17 00:00:00 2001 From: Katelyn Baker Date: Wed, 11 Apr 2018 15:37:47 +0100 Subject: [PATCH 2/9] RELEASE: Merge V3.0 upgrade notes to master This should've been done at release... I clearly missed back-merging this file --- docs/source/upgrade-notes.rst | 100 +++++++++++++++++++++++++++++++++- 1 file changed, 98 insertions(+), 2 deletions(-) diff --git a/docs/source/upgrade-notes.rst b/docs/source/upgrade-notes.rst index 2f112d3f4c..b9634d252e 100644 --- a/docs/source/upgrade-notes.rst +++ b/docs/source/upgrade-notes.rst @@ -133,10 +133,65 @@ cordaCompile, cordaRuntime, cordapp JoinColumn(name = "transaction_id", referencedColumnName = "transaction_id"))) override var participants: MutableSet? = null, -* Shell - to use Shell ensure ``rpcSettings.address`` and ``rpcSettings.adminAddress`` settings are present. +AMQP +^^^^ + +Whilst the enablement of AMQP is a transparent change, as noted in the :doc:`serialization` documentation +the way classes, and states in particular, should be written to work with this new library may require some +alteration to your current implementation. + + * With AMQP enabled Java classes must be compiled with the -parameter flag. + + * If they aren't, then the error message will complain about ``arg`` being an unknown parameter. + * If recompilation is not viable, a custom serializer can be written as per :doc:`cordapp-custom-serializers` + * It is important to bear in mind that with AMQP there must be an implicit mapping between constructor + parameters and properties you wish included in the serialized form of a class. + + * See :doc:`serialization` for more information + + * Error messages of the form + + ``Constructor parameter - "" - doesn't refer to a property of "class "`` + + indicate that a class, in the above example ``some.class.being.serialized``, has a parameter on its primary constructor that + doesn't correlate to a property of the class. This is a problem because the Corda AMQP serialization library uses a class's + constructor (default, primary, or annotated) as the means by which instances of the serialized form are reconstituted. + + See the section "Mismatched Class Properties / Constructor Parameters" in the :doc:`serialization` documentation + +Database schema changes +^^^^^^^^^^^^^^^^^^^^^^^ + +An H2 database instance (represented on the filesystem as a file called `persistence.mv.db`) used in Corda 1.0 or 2.0 +cannot be directly reused with Corda 3.0 due to minor improvements and additions to stabilise the underlying schemas. + +Configuration +^^^^^^^^^^^^^ + +Nodes that do not require SSL to be enabled for RPC clients now need an additional port to be specified as part of their configuration. +To do this, add a block as follows to the nodes configuraiton: + + .. sourcecode:: script + + rpcSettings { + adminAddress "localhost:10007" + } + +to `node.conf` files. + +Also, the property `rpcPort` is now deprecated, so it would be preferable to substitute properties specified that way e.g., `rpcPort=10006` with a block as follows: + + .. sourcecode:: script + + rpcSettings { + address "localhost:10006" + adminAddress "localhost:10007" + } + +Equivalent changes should be performed on classes extending `CordformDefinition`. Testing -~~~~~~~ +^^^^^^^ * The registration mechanism for CorDapps in ``MockNetwork`` unit tests has changed: @@ -146,6 +201,47 @@ Testing * The ``unsetCordappPackages`` method is now redundant and has been removed +* Many classes have been moved between packages, so you will need to update your imports + + .. tip:: We have provided a several scripts (depending upon your operating system of choice) to smooth the upgrade + process for existing projects. This can be found at ``tools\scripts\update-test-packages.sh`` for the Bash shell and + ``tools/scripts/upgrade-test-packages.ps1`` for Windows Power Shell users in the source tree + +* setCordappPackages and unsetCordappPackages have been removed from the ledger/transaction DSL and the flow test framework, + and are now set via a constructor parameter or automatically when constructing the MockServices or MockNetwork object + +* Key constants e.g. ``ALICE_KEY`` have been removed; you can now use TestIdentity to make your own + +* The ledger/transaction DSL must now be provided with MockServices as it no longer makes its own + * In transaction blocks, input and output take their arguments as ContractStates rather than lambdas + * Also in transaction blocks, command takes its arguments as CommandDatas rather than lambdas + +* The MockServices API has changed; please refer to its API documentation + +* TestDependencyInjectionBase has been retired in favour of a JUnit Rule called SerializationEnvironmentRule + * This replaces the initialiseSerialization parameter of ledger/transaction and verifierDriver + * The withTestSerialization method is obsoleted by SerializationEnvironmentRule and has been retired + +* MockNetwork now takes a MockNetworkParameters builder to make it more Java-friendly, like driver's DriverParameters + * Similarly, the MockNetwork.createNode methods now take a MockNodeParameters builder + +* MockNode constructor parameters are now aggregated in MockNodeArgs for easier subclassing + +* MockNetwork.Factory has been retired as you can simply use a lambda + +* testNodeConfiguration has been retired, please use a mock object framework of your choice instead + +* MockNetwork.createSomeNodes and IntegrationTestCategory have been retired with no replacement + +* Starting a flow can now be done directly from a node object. Change calls of the form ``node.getServices().startFlow(...)`` + to ``node.startFlow(...)`` + +* Similarly a tranaction can be executed directly from a node object. Change calls of the form ``node.getDatabase().transaction({ it -> ... })`` + to ``node.transaction({() -> ... })`` + +* ``startFlow`` now returns a ``CordaFuture``, there is no need to call ``startFlow(...).getResultantFuture()`` + + V1.0 to V2.0 ------------ From f19bcea82f53d056963f56554910a07a27a318a5 Mon Sep 17 00:00:00 2001 From: Katelyn Baker Date: Wed, 11 Apr 2018 18:32:23 +0100 Subject: [PATCH 3/9] CORDA-1229 - Setter serialization fails with lists Looks like the super / sub type inference of setter param vs getter param is the wrong way around. Also, Setter Type should be the generic type, not just the type the property must be a supertype of the setter parameter the getter must be a supertype of the setter parameter --- docs/source/changelog.rst | 3 ++ .../serialization/amqp/SerializationHelper.kt | 7 ++-- .../amqp/SetterConstructorTests.java | 34 +++++++++++++++++++ .../amqp/SerializationOutputTests.kt | 1 - 4 files changed, 41 insertions(+), 4 deletions(-) diff --git a/docs/source/changelog.rst b/docs/source/changelog.rst index 9e239532b1..6887eab922 100644 --- a/docs/source/changelog.rst +++ b/docs/source/changelog.rst @@ -7,6 +7,9 @@ release, see :doc:`upgrade-notes`. Unreleased ========== +* Fix CORDA-1229. Setter based serialization was broken with generic types when the property was stored + as the interface type, List for example. + * java.security.cert.CRLReason added to the default Whitelist. * java.security.cert.X509CRL serialization support added. diff --git a/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt b/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt index f9d9efad63..3b8b6dceff 100644 --- a/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt +++ b/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt @@ -305,7 +305,8 @@ fun propertiesForSerializationFromSetters( "takes too many arguments") } - val setterType = setter.parameterTypes[0]!! + //val setterType = setter.parameterTypes[0]!! + val setterType = setter.genericParameterTypes[0]!! if ((property.value.field != null) && (!(TypeToken.of(property.value.field?.genericType!!).isSupertypeOf(setterType)))) { @@ -315,10 +316,10 @@ fun propertiesForSerializationFromSetters( } // make sure the setter returns the same type (within inheritance bounds) the getter accepts - if (!(TypeToken.of (setterType).isSupertypeOf(getter.returnType))) { + if (!(TypeToken.of (getter.genericReturnType).isSupertypeOf(setterType))) { throw NotSerializableException("Defined setter for parameter ${property.value.field?.name} " + "takes parameter of type $setterType yet the defined getter returns a value of type " + - "${getter.returnType}") + "${getter.returnType} [${getter.genericReturnType}]") } this += PropertyAccessorGetterSetter( idx++, diff --git a/node-api/src/test/java/net/corda/nodeapi/internal/serialization/amqp/SetterConstructorTests.java b/node-api/src/test/java/net/corda/nodeapi/internal/serialization/amqp/SetterConstructorTests.java index a0ab7276dd..2c2b938ab9 100644 --- a/node-api/src/test/java/net/corda/nodeapi/internal/serialization/amqp/SetterConstructorTests.java +++ b/node-api/src/test/java/net/corda/nodeapi/internal/serialization/amqp/SetterConstructorTests.java @@ -7,6 +7,8 @@ import org.junit.Test; import static org.junit.Assert.*; import java.io.NotSerializableException; +import java.util.ArrayList; +import java.util.List; public class SetterConstructorTests { @@ -64,6 +66,13 @@ public class SetterConstructorTests { public void setC(int c) { this.c = c; } } + static class CIntList { + private List l; + + public List getL() { return l; } + public void setL(List l) { this.l = l; } + } + static class Inner1 { private String a; @@ -315,4 +324,29 @@ public class SetterConstructorTests { Assertions.assertThatThrownBy(() -> new SerializationOutput(factory1).serialize(tm)).isInstanceOf( NotSerializableException.class); } + + // This not blowing up means it's working + @Test + public void intList() throws NotSerializableException { + CIntList cil = new CIntList(); + + List l = new ArrayList<>(); + l.add(1); + l.add(2); + l.add(3); + + cil.setL(l); + + EvolutionSerializerGetterBase evolutionSerialiserGetter = new EvolutionSerializerGetter(); + FingerPrinter fingerPrinter = new SerializerFingerPrinter(); + SerializerFactory factory1 = new SerializerFactory( + AllWhitelist.INSTANCE, + ClassLoader.getSystemClassLoader(), + evolutionSerialiserGetter, + fingerPrinter); + + // if we've got super / sub types on the setter vs the unerlaying type the wrong way around this will + // explode + new SerializationOutput(factory1).serialize(cil); + } } diff --git a/node-api/src/test/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationOutputTests.kt b/node-api/src/test/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationOutputTests.kt index a41f5b33c0..5b10cde6a9 100644 --- a/node-api/src/test/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationOutputTests.kt +++ b/node-api/src/test/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationOutputTests.kt @@ -1310,6 +1310,5 @@ class SerializationOutputTests(private val compression: CordaSerializationEncodi C(12).serializeE() }.withMessageContaining("has synthetic fields and is likely a nested inner class") } - } From 84914aa5c8fbf3175ffafc8d0a68b3b91eaae6b7 Mon Sep 17 00:00:00 2001 From: Kat Baker Date: Thu, 12 Apr 2018 11:18:39 +0100 Subject: [PATCH 4/9] Remove commented out code --- .../nodeapi/internal/serialization/amqp/SerializationHelper.kt | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt b/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt index 3b8b6dceff..d5220dde49 100644 --- a/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt +++ b/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt @@ -304,8 +304,7 @@ fun propertiesForSerializationFromSetters( throw NotSerializableException("Defined setter for parameter ${property.value.field?.name} " + "takes too many arguments") } - - //val setterType = setter.parameterTypes[0]!! + val setterType = setter.genericParameterTypes[0]!! if ((property.value.field != null) && From c783c431b1d99c5f8a35e4e1957786518b55c917 Mon Sep 17 00:00:00 2001 From: Kat Baker Date: Thu, 12 Apr 2018 11:23:13 +0100 Subject: [PATCH 5/9] Fix broken comment --- .../internal/serialization/amqp/SerializationHelper.kt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt b/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt index d5220dde49..5f92cb9ed2 100644 --- a/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt +++ b/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt @@ -304,7 +304,7 @@ fun propertiesForSerializationFromSetters( throw NotSerializableException("Defined setter for parameter ${property.value.field?.name} " + "takes too many arguments") } - + val setterType = setter.genericParameterTypes[0]!! if ((property.value.field != null) && @@ -314,7 +314,7 @@ fun propertiesForSerializationFromSetters( "${property.value.field?.genericType!!}") } - // make sure the setter returns the same type (within inheritance bounds) the getter accepts + // make sure the getter returns the same type (within inheritance bounds) the setter accepts if (!(TypeToken.of (getter.genericReturnType).isSupertypeOf(setterType))) { throw NotSerializableException("Defined setter for parameter ${property.value.field?.name} " + "takes parameter of type $setterType yet the defined getter returns a value of type " + From 03850dabc2956cfa7a704eb360e4c044923040c9 Mon Sep 17 00:00:00 2001 From: Kat Baker Date: Thu, 12 Apr 2018 11:58:56 +0100 Subject: [PATCH 6/9] Review comments --- docs/source/changelog.rst | 4 ++-- .../internal/serialization/amqp/SerializationHelper.kt | 2 +- .../internal/serialization/amqp/SetterConstructorTests.java | 4 ++-- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/source/changelog.rst b/docs/source/changelog.rst index 6887eab922..8d0ee6cc3e 100644 --- a/docs/source/changelog.rst +++ b/docs/source/changelog.rst @@ -7,8 +7,8 @@ release, see :doc:`upgrade-notes`. Unreleased ========== -* Fix CORDA-1229. Setter based serialization was broken with generic types when the property was stored - as the interface type, List for example. +* Fix CORDA-1229. Setter-based serialization was broken with generic types when the property was stored + as the raw type, List for example. * java.security.cert.CRLReason added to the default Whitelist. diff --git a/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt b/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt index 5f92cb9ed2..119b1e0211 100644 --- a/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt +++ b/node-api/src/main/kotlin/net/corda/nodeapi/internal/serialization/amqp/SerializationHelper.kt @@ -314,7 +314,7 @@ fun propertiesForSerializationFromSetters( "${property.value.field?.genericType!!}") } - // make sure the getter returns the same type (within inheritance bounds) the setter accepts + // Make sure the getter returns the same type (within inheritance bounds) the setter accepts. if (!(TypeToken.of (getter.genericReturnType).isSupertypeOf(setterType))) { throw NotSerializableException("Defined setter for parameter ${property.value.field?.name} " + "takes parameter of type $setterType yet the defined getter returns a value of type " + diff --git a/node-api/src/test/java/net/corda/nodeapi/internal/serialization/amqp/SetterConstructorTests.java b/node-api/src/test/java/net/corda/nodeapi/internal/serialization/amqp/SetterConstructorTests.java index 2c2b938ab9..2e819dfbe7 100644 --- a/node-api/src/test/java/net/corda/nodeapi/internal/serialization/amqp/SetterConstructorTests.java +++ b/node-api/src/test/java/net/corda/nodeapi/internal/serialization/amqp/SetterConstructorTests.java @@ -345,8 +345,8 @@ public class SetterConstructorTests { evolutionSerialiserGetter, fingerPrinter); - // if we've got super / sub types on the setter vs the unerlaying type the wrong way around this will - // explode + // if we've got super / sub types on the setter vs the underlying type the wrong way around this will + // explode. See CORDA-1229 (https://r3-cev.atlassian.net/browse/CORDA-1229) new SerializationOutput(factory1).serialize(cil); } } From e6d352e44632317d0bc9b773167ffa3790338711 Mon Sep 17 00:00:00 2001 From: Kat Baker Date: Thu, 12 Apr 2018 12:09:17 +0100 Subject: [PATCH 7/9] Review comments --- .../internal/serialization/amqp/SetterConstructorTests.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/node-api/src/test/java/net/corda/nodeapi/internal/serialization/amqp/SetterConstructorTests.java b/node-api/src/test/java/net/corda/nodeapi/internal/serialization/amqp/SetterConstructorTests.java index 2e819dfbe7..e9ac48b063 100644 --- a/node-api/src/test/java/net/corda/nodeapi/internal/serialization/amqp/SetterConstructorTests.java +++ b/node-api/src/test/java/net/corda/nodeapi/internal/serialization/amqp/SetterConstructorTests.java @@ -347,6 +347,6 @@ public class SetterConstructorTests { // if we've got super / sub types on the setter vs the underlying type the wrong way around this will // explode. See CORDA-1229 (https://r3-cev.atlassian.net/browse/CORDA-1229) - new SerializationOutput(factory1).serialize(cil); + new DeserializationInput(factory1).deserialize(new SerializationOutput(factory1).serialize(cil), CIntList.class); } } From b5f304a104faabc3659cfb9474da8d018a6be5bf Mon Sep 17 00:00:00 2001 From: Patrick Kuo Date: Thu, 12 Apr 2018 13:05:15 +0100 Subject: [PATCH 8/9] ENT-1323 Network map service to check all identities in submitted node info (#499) * ENT-1323 Network map service to check all identities in submitted node info * fixup after rebase * address PR issues, refactored createValidNodeInfo * address PR issues (cherry picked from commit f9ed55b) --- .../nodeapi/internal/SignedNodeInfoTest.kt | 18 +++++----- .../services/network/NetworkMapClientTest.kt | 4 +-- .../testing/internal/TestNodeInfoBuilder.kt | 35 +++++++++++++++---- 3 files changed, 40 insertions(+), 17 deletions(-) diff --git a/node-api/src/test/kotlin/net/corda/nodeapi/internal/SignedNodeInfoTest.kt b/node-api/src/test/kotlin/net/corda/nodeapi/internal/SignedNodeInfoTest.kt index 83b4662c49..c213b15c3f 100644 --- a/node-api/src/test/kotlin/net/corda/nodeapi/internal/SignedNodeInfoTest.kt +++ b/node-api/src/test/kotlin/net/corda/nodeapi/internal/SignedNodeInfoTest.kt @@ -31,23 +31,23 @@ class SignedNodeInfoTest { @Test fun `verifying single identity`() { - nodeInfoBuilder.addIdentity(ALICE_NAME) + nodeInfoBuilder.addLegalIdentity(ALICE_NAME) val (nodeInfo, signedNodeInfo) = nodeInfoBuilder.buildWithSigned() assertThat(signedNodeInfo.verified()).isEqualTo(nodeInfo) } @Test fun `verifying multiple identities`() { - nodeInfoBuilder.addIdentity(ALICE_NAME) - nodeInfoBuilder.addIdentity(BOB_NAME) + nodeInfoBuilder.addLegalIdentity(ALICE_NAME) + nodeInfoBuilder.addLegalIdentity(BOB_NAME) val (nodeInfo, signedNodeInfo) = nodeInfoBuilder.buildWithSigned() assertThat(signedNodeInfo.verified()).isEqualTo(nodeInfo) } @Test fun `verifying missing signature`() { - val (_, aliceKey) = nodeInfoBuilder.addIdentity(ALICE_NAME) - nodeInfoBuilder.addIdentity(BOB_NAME) + val (_, aliceKey) = nodeInfoBuilder.addLegalIdentity(ALICE_NAME) + nodeInfoBuilder.addLegalIdentity(BOB_NAME) val nodeInfo = nodeInfoBuilder.build() val signedNodeInfo = nodeInfo.signWith(listOf(aliceKey)) assertThatThrownBy { signedNodeInfo.verified() } @@ -70,7 +70,7 @@ class SignedNodeInfoTest { @Test fun `verifying extra signature`() { - val (_, aliceKey) = nodeInfoBuilder.addIdentity(ALICE_NAME) + val (_, aliceKey) = nodeInfoBuilder.addLegalIdentity(ALICE_NAME) val nodeInfo = nodeInfoBuilder.build() val signedNodeInfo = nodeInfo.signWith(listOf(aliceKey, generateKeyPair().private)) assertThatThrownBy { signedNodeInfo.verified() } @@ -80,7 +80,7 @@ class SignedNodeInfoTest { @Test fun `verifying incorrect signature`() { - nodeInfoBuilder.addIdentity(ALICE_NAME) + nodeInfoBuilder.addLegalIdentity(ALICE_NAME) val nodeInfo = nodeInfoBuilder.build() val signedNodeInfo = nodeInfo.signWith(listOf(generateKeyPair().private)) assertThatThrownBy { signedNodeInfo.verified() } @@ -90,8 +90,8 @@ class SignedNodeInfoTest { @Test fun `verifying with signatures in wrong order`() { - val (_, aliceKey) = nodeInfoBuilder.addIdentity(ALICE_NAME) - val (_, bobKey) = nodeInfoBuilder.addIdentity(BOB_NAME) + val (_, aliceKey) = nodeInfoBuilder.addLegalIdentity(ALICE_NAME) + val (_, bobKey) = nodeInfoBuilder.addLegalIdentity(BOB_NAME) val nodeInfo = nodeInfoBuilder.build() val signedNodeInfo = nodeInfo.signWith(listOf(bobKey, aliceKey)) assertThatThrownBy { signedNodeInfo.verified() } diff --git a/node/src/test/kotlin/net/corda/node/services/network/NetworkMapClientTest.kt b/node/src/test/kotlin/net/corda/node/services/network/NetworkMapClientTest.kt index c888ed6818..255016e70b 100644 --- a/node/src/test/kotlin/net/corda/node/services/network/NetworkMapClientTest.kt +++ b/node/src/test/kotlin/net/corda/node/services/network/NetworkMapClientTest.kt @@ -73,8 +73,8 @@ class NetworkMapClientTest { @Test fun `errors return a meaningful error message`() { val nodeInfoBuilder = TestNodeInfoBuilder() - val (_, aliceKey) = nodeInfoBuilder.addIdentity(ALICE_NAME) - nodeInfoBuilder.addIdentity(BOB_NAME) + val (_, aliceKey) = nodeInfoBuilder.addLegalIdentity(ALICE_NAME) + nodeInfoBuilder.addLegalIdentity(BOB_NAME) val nodeInfo3 = nodeInfoBuilder.build() val signedNodeInfo3 = nodeInfo3.signWith(listOf(aliceKey)) diff --git a/testing/test-utils/src/main/kotlin/net/corda/testing/internal/TestNodeInfoBuilder.kt b/testing/test-utils/src/main/kotlin/net/corda/testing/internal/TestNodeInfoBuilder.kt index c311fbaeea..da7396192c 100644 --- a/testing/test-utils/src/main/kotlin/net/corda/testing/internal/TestNodeInfoBuilder.kt +++ b/testing/test-utils/src/main/kotlin/net/corda/testing/internal/TestNodeInfoBuilder.kt @@ -20,7 +20,7 @@ import java.security.cert.X509Certificate class TestNodeInfoBuilder(private val intermediateAndRoot: Pair = DEV_INTERMEDIATE_CA to DEV_ROOT_CA.certificate) { private val identitiesAndPrivateKeys = ArrayList>() - fun addIdentity(name: CordaX500Name, nodeKeyPair: KeyPair = Crypto.generateKeyPair(X509Utilities.DEFAULT_TLS_SIGNATURE_SCHEME)): Pair { + fun addLegalIdentity(name: CordaX500Name, nodeKeyPair: KeyPair = Crypto.generateKeyPair(X509Utilities.DEFAULT_TLS_SIGNATURE_SCHEME)): Pair { val nodeCertificateAndKeyPair = createDevNodeCa(intermediateAndRoot.first, name, nodeKeyPair) val identityKeyPair = Crypto.generateKeyPair() val identityCert = X509Utilities.createCertificate( @@ -29,12 +29,35 @@ class TestNodeInfoBuilder(private val intermediateAndRoot: Pair { + val serviceCert = X509Utilities.createCertificate( + CertificateType.SERVICE_IDENTITY, + intermediateAndRoot.first.certificate, + intermediateAndRoot.first.keyPair, + name.x500Principal, + nodeKeyPair.public) + + val certs = arrayOf(serviceCert) + val key = nodeKeyPair.private + + val certPath = X509Utilities.buildCertPath(*certs, + intermediateAndRoot.first.certificate, + intermediateAndRoot.second) + + return Pair(PartyAndCertificate(certPath), key).also { identitiesAndPrivateKeys += it } } @@ -62,7 +85,7 @@ class TestNodeInfoBuilder(private val intermediateAndRoot: Pair Date: Thu, 12 Apr 2018 17:03:06 +0100 Subject: [PATCH 9/9] CORDA-1312: Removed the need to have whitelist.txt for updating the contracts whitelist using the bootstrapper. (#2954) Instead the current whitelist is read in from the existing network parameters file. --- .ci/api-current.txt | 2 +- .../net/corda/core/node/NetworkParameters.kt | 14 ++ docs/source/changelog.rst | 3 + docs/source/setting-up-a-corda-network.rst | 35 ++--- .../internal/network/NetworkBootstrapper.kt | 133 ++++++++++-------- 5 files changed, 103 insertions(+), 84 deletions(-) diff --git a/.ci/api-current.txt b/.ci/api-current.txt index 4e12066579..23b5465f15 100644 --- a/.ci/api-current.txt +++ b/.ci/api-current.txt @@ -1884,7 +1884,7 @@ public @interface net.corda.core.messaging.RPCReturnsObservables @org.jetbrains.annotations.NotNull public final List getNotaries() @org.jetbrains.annotations.NotNull public final Map getWhitelistedContractImplementations() public int hashCode() - public String toString() + @org.jetbrains.annotations.NotNull public String toString() ## @net.corda.core.serialization.CordaSerializable public final class net.corda.core.node.NodeInfo extends java.lang.Object public (List, List, int, long) diff --git a/core/src/main/kotlin/net/corda/core/node/NetworkParameters.kt b/core/src/main/kotlin/net/corda/core/node/NetworkParameters.kt index 41091074a0..2b8662d348 100644 --- a/core/src/main/kotlin/net/corda/core/node/NetworkParameters.kt +++ b/core/src/main/kotlin/net/corda/core/node/NetworkParameters.kt @@ -37,6 +37,20 @@ data class NetworkParameters( require(maxMessageSize > 0) { "maxMessageSize must be at least 1" } require(maxTransactionSize > 0) { "maxTransactionSize must be at least 1" } } + + override fun toString(): String { + return """NetworkParameters { + minimumPlatformVersion=$minimumPlatformVersion + notaries=$notaries + maxMessageSize=$maxMessageSize + maxTransactionSize=$maxTransactionSize + whitelistedContractImplementations { + ${whitelistedContractImplementations.entries.joinToString("\n ")} + } + modifiedTime=$modifiedTime + epoch=$epoch +}""" + } } /** diff --git a/docs/source/changelog.rst b/docs/source/changelog.rst index 8d0ee6cc3e..679e4b0c0a 100644 --- a/docs/source/changelog.rst +++ b/docs/source/changelog.rst @@ -19,6 +19,9 @@ Unreleased * Shell (embedded available only in dev mode or via SSH) connects to the node via RPC instead of using the ``CordaRPCOps`` object directly. To enable RPC connectivity ensure node’s ``rpcSettings.address`` and ``rpcSettings.adminAddress`` settings are present. +* The network bootstrapper uses the existing network parameters file to update the current contracts whitelist, and no longer + needs the whitelist.txt file. + * Errors thrown by a Corda node will now reported to a calling RPC client with attention to serialization and obfuscation of internal data. * Serializing an inner class (non-static nested class in Java, inner class in Kotlin) will be rejected explicitly by the serialization diff --git a/docs/source/setting-up-a-corda-network.rst b/docs/source/setting-up-a-corda-network.rst index 2ce2fe2b9c..d82c0fa9c7 100644 --- a/docs/source/setting-up-a-corda-network.rst +++ b/docs/source/setting-up-a-corda-network.rst @@ -93,34 +93,16 @@ If you want to create a *Zone whitelist* (see :doc:`api-contract-constraints`), ``java -jar network-bootstrapper.jar ..`` -The CorDapp jars will be hashed and scanned for ``Contract`` classes. -By default the tool would generate a file named ``whitelist.txt`` containing an entry for each contract with the hash of the jar. +The CorDapp jars will be hashed and scanned for ``Contract`` classes. These contract class implementations will become part +of the whitelisted contracts in the network parameters (see ``NetworkParameters.whitelistedContractImplementations`` :doc:`network-map`). +If the network already has a set of network parameters defined (i.e. the node directories all contain the same network-parameters +file) then the new set of contracts will be appended to the current whitelist. -For example: +.. note:: The whitelist can only ever be appended to. Once added a contract implementation can never be removed. -.. sourcecode:: none - - net.corda.finance.contracts.asset.Obligation:decd098666b9657314870e192ced0c3519c2c9d395507a238338f8d003929de8 - net.corda.finance.contracts.asset.Cash:decd098666b9657314870e192ced0c3519c2c9d395507a238338f8d003929de9 - -These will be added to the ``NetworkParameters.whitelistedContractImplementations``. See :doc:`network-map`. - -This means that by default the Network bootstrapper tool will whitelist all contracts found in all passed CorDapps. - -In case there is a ``whitelist.txt`` file in the root dir already, the tool will append the new jar hashes or contracts to it. - -The zone operator will maintain this whitelist file, and, using the tool, will append new versions of CorDapps to it. - -.. warning:: - - The zone operator must ensure that this file is *append only*. - - If the operator removes hashes from the list, all transactions pointing to that version will suddenly fail the constraint verification, and the entire chain is compromised. - - If a contract is removed from the whitelist, then all states created from that moment on will be constrained by the HashAttachmentConstraint. - - Note: In future releases, we will provider a tamper-proof way of maintaining the contract whitelist. - -For fine-grained control of constraints, in case multiple contracts live in the same jar, the tool reads from another file: -``exclude_whitelist.txt``, which contains a list of contracts that should not be whitelisted, and thus default to the very restrictive: -``HashAttachmentConstraint`` +By default the bootstrapper tool will whitelist all the contracts found in all the CorDapp jars. To prevent certain +contracts from being whitelisted, add their fully qualified class name in the ``exclude_whitelist.txt``. These will instead +use the more restrictive ``HashAttachmentConstraint``. For example: @@ -129,7 +111,6 @@ For example: net.corda.finance.contracts.asset.Cash net.corda.finance.contracts.asset.CommercialPaper - Starting the nodes ~~~~~~~~~~~~~~~~~~ diff --git a/node-api/src/main/kotlin/net/corda/nodeapi/internal/network/NetworkBootstrapper.kt b/node-api/src/main/kotlin/net/corda/nodeapi/internal/network/NetworkBootstrapper.kt index 671a3a086d..9799e55715 100644 --- a/node-api/src/main/kotlin/net/corda/nodeapi/internal/network/NetworkBootstrapper.kt +++ b/node-api/src/main/kotlin/net/corda/nodeapi/internal/network/NetworkBootstrapper.kt @@ -2,7 +2,6 @@ package net.corda.nodeapi.internal.network import com.typesafe.config.ConfigFactory import net.corda.cordform.CordformNode -import net.corda.core.crypto.SecureHash.Companion.parse import net.corda.core.identity.Party import net.corda.core.internal.* import net.corda.core.internal.concurrent.fork @@ -11,10 +10,13 @@ import net.corda.core.node.NodeInfo import net.corda.core.node.NotaryInfo import net.corda.core.node.services.AttachmentId import net.corda.core.serialization.SerializationContext +import net.corda.core.serialization.SerializedBytes +import net.corda.core.serialization.deserialize import net.corda.core.serialization.internal.SerializationEnvironmentImpl import net.corda.core.serialization.internal._contextSerializationEnv import net.corda.core.utilities.getOrThrow import net.corda.core.utilities.seconds +import net.corda.nodeapi.internal.DEV_ROOT_CA import net.corda.nodeapi.internal.SignedNodeInfo import net.corda.nodeapi.internal.scanJarForContracts import net.corda.nodeapi.internal.serialization.AMQP_P2P_CONTEXT @@ -23,11 +25,10 @@ import net.corda.nodeapi.internal.serialization.SerializationFactoryImpl import net.corda.nodeapi.internal.serialization.amqp.AMQPServerSerializationScheme import net.corda.nodeapi.internal.serialization.kryo.AbstractKryoSerializationScheme import net.corda.nodeapi.internal.serialization.kryo.kryoMagic -import java.io.PrintStream import java.nio.file.Files import java.nio.file.Path import java.nio.file.Paths -import java.nio.file.StandardCopyOption +import java.nio.file.StandardCopyOption.REPLACE_EXISTING import java.time.Instant import java.util.concurrent.Executors import java.util.concurrent.TimeoutException @@ -47,12 +48,11 @@ class NetworkBootstrapper { ) private const val LOGS_DIR_NAME = "logs" - private const val WHITELIST_FILE_NAME = "whitelist.txt" private const val EXCLUDE_WHITELIST_FILE_NAME = "exclude_whitelist.txt" @JvmStatic fun main(args: Array) { - val baseNodeDirectory = args.firstOrNull() ?: throw IllegalArgumentException("Expecting first argument which is the nodes' parent directory") + val baseNodeDirectory = requireNotNull(args.firstOrNull()) { "Expecting first argument which is the nodes' parent directory" } val cordapps = if (args.size > 1) args.toList().drop(1) else null NetworkBootstrapper().bootstrap(Paths.get(baseNodeDirectory).toAbsolutePath().normalize(), cordapps) } @@ -70,15 +70,17 @@ class NetworkBootstrapper { try { println("Waiting for all nodes to generate their node-info files...") val nodeInfoFiles = gatherNodeInfoFiles(processes, nodeDirs) - println("Distributing all node info-files to all nodes") + println("Distributing all node-info files to all nodes") distributeNodeInfos(nodeDirs, nodeInfoFiles) + print("Loading existing network parameters... ") + val existingNetParams = loadNetworkParameters(nodeDirs) + println(existingNetParams ?: "none found") println("Gathering notary identities") val notaryInfos = gatherNotaryInfos(nodeInfoFiles) - println("Notary identities to be used in network parameters: ${notaryInfos.joinToString("; ") { it.prettyPrint() }}") - val mergedWhiteList = generateWhitelist(directory / WHITELIST_FILE_NAME, directory / EXCLUDE_WHITELIST_FILE_NAME, cordapps?.distinct()) - println("Updating whitelist") - overwriteWhitelist(directory / WHITELIST_FILE_NAME, mergedWhiteList) - installNetworkParameters(notaryInfos, nodeDirs, mergedWhiteList) + println("Generating contract implementations whitelist") + val newWhitelist = generateWhitelist(existingNetParams, directory / EXCLUDE_WHITELIST_FILE_NAME, cordapps?.distinct()) + val netParams = installNetworkParameters(notaryInfos, newWhitelist, existingNetParams, nodeDirs) + println("${if (existingNetParams == null) "New" else "Updated"} $netParams") println("Bootstrapping complete!") } finally { _contextSerializationEnv.set(null) @@ -96,15 +98,15 @@ class NetworkBootstrapper { val nodeName = confFile.fileName.toString().removeSuffix("_node.conf") println("Generating directory for $nodeName") val nodeDir = (directory / nodeName).createDirectories() - confFile.moveTo(nodeDir / "node.conf", StandardCopyOption.REPLACE_EXISTING) - webServerConfFiles.firstOrNull { directory.relativize(it).toString().removeSuffix("_web-server.conf") == nodeName }?.moveTo(nodeDir / "web-server.conf", StandardCopyOption.REPLACE_EXISTING) - Files.copy(cordaJar, (nodeDir / "corda.jar"), StandardCopyOption.REPLACE_EXISTING) + confFile.moveTo(nodeDir / "node.conf", REPLACE_EXISTING) + webServerConfFiles.firstOrNull { directory.relativize(it).toString().removeSuffix("_web-server.conf") == nodeName }?.moveTo(nodeDir / "web-server.conf", REPLACE_EXISTING) + cordaJar.copyToDirectory(nodeDir, REPLACE_EXISTING) } Files.delete(cordaJar) } private fun extractCordaJarTo(directory: Path): Path { - val cordaJarPath = (directory / "corda.jar") + val cordaJarPath = directory / "corda.jar" if (!cordaJarPath.exists()) { println("No corda jar found in root directory. Extracting from jar") Thread.currentThread().contextClassLoader.getResourceAsStream("corda.jar").copyTo(cordaJarPath) @@ -137,7 +139,7 @@ class NetworkBootstrapper { } return try { - future.getOrThrow(60.seconds) + future.getOrThrow(timeout = 60.seconds) } catch (e: TimeoutException) { println("...still waiting. If this is taking longer than usual, check the node logs.") future.getOrThrow() @@ -148,7 +150,7 @@ class NetworkBootstrapper { for (nodeDir in nodeDirs) { val additionalNodeInfosDir = (nodeDir / CordformNode.NODE_INFO_DIRECTORY).createDirectories() for (nodeInfoFile in nodeInfoFiles) { - nodeInfoFile.copyToDirectory(additionalNodeInfosDir, StandardCopyOption.REPLACE_EXISTING) + nodeInfoFile.copyToDirectory(additionalNodeInfosDir, REPLACE_EXISTING) } } } @@ -168,25 +170,67 @@ class NetworkBootstrapper { }.distinct() // We need distinct as nodes part of a distributed notary share the same notary identity } - private fun installNetworkParameters(notaryInfos: List, nodeDirs: List, whitelist: Map>) { - // TODO Add config for minimumPlatformVersion, maxMessageSize and maxTransactionSize - val copier = NetworkParametersCopier(NetworkParameters( - minimumPlatformVersion = 1, - notaries = notaryInfos, - modifiedTime = Instant.now(), - maxMessageSize = 10485760, - maxTransactionSize = Int.MAX_VALUE, - epoch = 1, - whitelistedContractImplementations = whitelist - ), overwriteFile = true) + private fun loadNetworkParameters(nodeDirs: List): NetworkParameters? { + val netParamsFilesGrouped = nodeDirs.mapNotNull { + val netParamsFile = it / NETWORK_PARAMS_FILE_NAME + if (netParamsFile.exists()) netParamsFile else null + }.groupBy { SerializedBytes(it.readAll()) } - nodeDirs.forEach { copier.install(it) } + when (netParamsFilesGrouped.size) { + 0 -> return null + 1 -> return netParamsFilesGrouped.keys.first().deserialize().verifiedNetworkMapCert(DEV_ROOT_CA.certificate) + } + + val msg = StringBuilder("Differing sets of network parameters were found. Make sure all the nodes have the same " + + "network parameters by copying over the correct $NETWORK_PARAMS_FILE_NAME file.\n\n") + + netParamsFilesGrouped.forEach { bytes, netParamsFiles -> + netParamsFiles.map { it.parent.fileName }.joinTo(msg, ", ") + msg.append(":\n") + val netParamsString = try { + bytes.deserialize().verifiedNetworkMapCert(DEV_ROOT_CA.certificate).toString() + } catch (e: Exception) { + "Invalid network parameters file: $e" + } + msg.append(netParamsString) + msg.append("\n\n") + } + + throw IllegalStateException(msg.toString()) } - private fun generateWhitelist(whitelistFile: Path, excludeWhitelistFile: Path, cordapps: List?): Map> { - val existingWhitelist = if (whitelistFile.exists()) readContractWhitelist(whitelistFile) else emptyMap() + private fun installNetworkParameters(notaryInfos: List, + whitelist: Map>, + existingNetParams: NetworkParameters?, + nodeDirs: List): NetworkParameters { + val networkParameters = if (existingNetParams != null) { + existingNetParams.copy( + notaries = notaryInfos, + modifiedTime = Instant.now(), + whitelistedContractImplementations = whitelist, + epoch = existingNetParams.epoch + 1 + ) + } else { + NetworkParameters( + minimumPlatformVersion = 1, + notaries = notaryInfos, + modifiedTime = Instant.now(), + maxMessageSize = 10485760, + maxTransactionSize = Int.MAX_VALUE, + whitelistedContractImplementations = whitelist, + epoch = 1 + ) + } + // TODO Add config for minimumPlatformVersion, maxMessageSize and maxTransactionSize + val copier = NetworkParametersCopier(networkParameters, overwriteFile = true) + nodeDirs.forEach { copier.install(it) } + return networkParameters + } - println(if (existingWhitelist.isEmpty()) "No existing whitelist file found." else "Found existing whitelist: $whitelistFile") + private fun generateWhitelist(networkParameters: NetworkParameters?, + excludeWhitelistFile: Path, + cordapps: List?): Map> { + val existingWhitelist = networkParameters?.whitelistedContractImplementations ?: emptyMap() val excludeContracts = if (excludeWhitelistFile.exists()) readExcludeWhitelist(excludeWhitelistFile) else emptyList() if (excludeContracts.isNotEmpty()) { @@ -200,38 +244,15 @@ class NetworkBootstrapper { } }?.filter { (contractClassName, _) -> contractClassName !in excludeContracts }?.toMap() ?: emptyMap() - println("Calculating whitelist for current installed CorDapps..") - - val merged = (newWhiteList.keys + existingWhitelist.keys).map { contractClassName -> + return (newWhiteList.keys + existingWhitelist.keys).map { contractClassName -> val existing = existingWhitelist[contractClassName] ?: emptyList() val newHash = newWhiteList[contractClassName] contractClassName to (if (newHash == null || newHash in existing) existing else existing + newHash) }.toMap() - - println("CorDapp whitelist " + (if (existingWhitelist.isEmpty()) "generated" else "updated") + " in $whitelistFile") - return merged - } - - private fun overwriteWhitelist(whitelistFile: Path, mergedWhiteList: Map>) { - PrintStream(whitelistFile.toFile().outputStream()).use { out -> - mergedWhiteList.forEach { (contract, attachments) -> - out.println("$contract:${attachments.joinToString(",")}") - } - } - } - - private fun readContractWhitelist(file: Path): Map> { - return file.readAllLines() - .map { line -> line.split(":") } - .map { (contract, attachmentIds) -> - contract to (attachmentIds.split(",").map(::parse)) - }.toMap() } private fun readExcludeWhitelist(file: Path): List = file.readAllLines().map(String::trim) - private fun NotaryInfo.prettyPrint(): String = "${identity.name} (${if (validating) "" else "non-"}validating)" - private fun NodeInfo.notaryIdentity(): Party { return when (legalIdentities.size) { // Single node notaries have just one identity like all other nodes. This identity is the notary identity