From f0c75448b4f87913e5a90d846c27e099b0bc6d4b Mon Sep 17 00:00:00 2001 From: Tudor Malene Date: Thu, 9 May 2019 17:23:52 +0100 Subject: [PATCH] ENT-3161 - Align docs with ENT (#5113) --- docs/source/corda-nodes-index.rst | 1 + .../node-operations-upgrade-cordapps.rst | 74 ++++++++ docs/source/upgrading-cordapps.rst | 165 +++++++++++++----- 3 files changed, 192 insertions(+), 48 deletions(-) create mode 100644 docs/source/node-operations-upgrade-cordapps.rst diff --git a/docs/source/corda-nodes-index.rst b/docs/source/corda-nodes-index.rst index 17be73e58e..61b845907b 100644 --- a/docs/source/corda-nodes-index.rst +++ b/docs/source/corda-nodes-index.rst @@ -13,6 +13,7 @@ Nodes node-database node-database-access-h2 node-database-tables + node-operations-upgrade-cordapps shell clientrpc generating-a-node diff --git a/docs/source/node-operations-upgrade-cordapps.rst b/docs/source/node-operations-upgrade-cordapps.rst new file mode 100644 index 0000000000..e8ddd77f6f --- /dev/null +++ b/docs/source/node-operations-upgrade-cordapps.rst @@ -0,0 +1,74 @@ +Upgrading CorDapps on a node +============================ + +In order to upgrade a CorDapp on a node to a new version, it needs to be determined whether any backwards compatible +changes have been made. These could range from database changes, to changes in the protocol. + +For developer information on upgrading CorDapps, see :doc:`upgrading-cordapps`. + +CorDapps must ship with database migration scripts or clear documentation about how to update the database to be compatible with the new version. + + +Flow upgrades +~~~~~~~~~~~~~ + +If any backwards-incompatible changes have been made (see :ref:`upgrading-cordapps-backwards-incompatible-flow-changes` +for more information), the upgrade method detailed below will need to be followed. Otherwise the CorDapp JAR can just +be replaced with the new version. + +Contract and State upgrades +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +There are two types of contract/state upgrade: + +1. *Implicit:* By allowing multiple implementations of the contract ahead of time, using constraints. See + :doc:`api-contract-constraints` to learn more. +2. *Explicit:* By creating a special *contract upgrade transaction* and getting all participants of a state to sign it using the + contract upgrade flows. + +This documentation only considers the *explicit* type of upgrade, as implicit contract upgrades are handled by the application. + +In an explicit upgrade contracts and states can be changed in arbitrary ways, if and only if all of the state's participants +agree to the proposed upgrade. The following combinations of upgrades are possible: + +* A contract is upgraded while the state definition remains the same. +* A state is upgraded while the contract stays the same. +* The state and the contract are updated simultaneously. + +Running the upgrade +~~~~~~~~~~~~~~~~~~~ + +If a contract or state requires an explicit upgrade then all states will need updating to the new contract at a time agreed +by all participants. The updated CorDapp JAR needs to be distributed to all relevant parties in advance of the changeover +time. + +In order to perform the upgrade, follow the following steps: + +* If required, do a flow drain to avoid the definition of states or contracts changing whilst a flow is in progress (see :ref:`upgrading-cordapps-flow-drains` for more information) + + * By RPC using the ``setFlowsDrainingModeEnabled`` method with the parameter ``true`` + * Via the shell by issuing the following command ``run setFlowsDrainingModeEnabled enabled: true`` + +* Check that all the flows have completed + + * By RPC using the ``stateMachinesSnapshot`` method and checking that there are no results + * Via the shell by issuing the following command ``run stateMachinesSnapshot`` + +* Once all flows have completed, stop the node +* Replace the existing JAR with the new one +* Make any database changes required to any custom vault tables for the upgraded CorDapp, + following the database upgrade steps in :doc:`node-operations-cordapp-deployment`. + The database update for a CorDapp upgrade follows the same steps as database setup for a new CorDapp. + +* Restart the node +* If you drained the node prior to upgrading, switch off flow draining mode to allow the node to continue to receive requests + + * By RPC using the ``setFlowsDrainingModeEnabled`` method with the parameter ``false`` + * Via the shell by issuing the following command ``run setFlowsDrainingModeEnabled enabled: false`` + +* Run the contract upgrade authorisation flow (``ContractUpgradeFlow$Initiate``) for each state that requires updating on every node. + + * You can do this manually via RPC but for anything more than a couple of states it is assumed that a script will be + provided by the CorDapp developer to query the vault and run this for all states + * The contract upgrade initiate flow only needs to be run on one of the participants for each state. The flow will + automatically upgrade the state on all participants. diff --git a/docs/source/upgrading-cordapps.rst b/docs/source/upgrading-cordapps.rst index 6b28ec1fd9..d28baf34c1 100644 --- a/docs/source/upgrading-cordapps.rst +++ b/docs/source/upgrading-cordapps.rst @@ -17,8 +17,26 @@ CorDapp versioning .. UPDATE - This is no longer accurate! Needs to talk about the different types of artifacts ( kernel, workflows) each versioned independently The Corda platform does not mandate a version number on a per-CorDapp basis. Different elements of a CorDapp are -allowed to evolve separately. Sometimes, however, a change to one element will require changes to other elements. For -example, changing a shared data structure may require flow changes that are not backwards-compatible. +allowed to evolve separately: + +* States +* Contracts +* Services +* Flows +* Utilities and library functions +* All, or a subset, of the above + +Sometimes, however, a change to one element will require changes to other elements. For example, changing a shared data +structure may require flow changes that are not backwards-compatible. + +Areas of consideration +---------------------- +This document will consider the following types of versioning: + +* Flow versioning +* State and contract versioning +* State and state schema versioning +* Serialisation of custom types Flow versioning --------------- @@ -32,8 +50,8 @@ The ``version`` property, which defaults to 1, specifies the flow's version. Thi whenever there is a release of a flow which has changes that are not backwards-compatible. A non-backwards compatible change is one that changes the interface of the flow. -Defining a flow's interface -~~~~~~~~~~~~~~~~~~~~~~~~~~~ +What defines the interface of a flow? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The flow interface is defined by the sequence of ``send`` and ``receive`` calls between an ``InitiatingFlow`` and an ``InitiatedBy`` flow, including the types of the data sent and received. We can picture a flow's interface as follows: @@ -59,8 +77,10 @@ As long as both the ``InitiatingFlow`` and the ``InitiatedBy`` flows conform to be implemented in any way you see fit (including adding proprietary business logic that is not shared with other parties). -Non-backwards compatible flow changes -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. _upgrading-cordapps-backwards-incompatible-flow-changes: + +What constitutes a non-backwards compatible flow change? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ A flow can become backwards-incompatible in two main ways: * The sequence of ``send`` and ``receive`` calls changes: @@ -70,8 +90,8 @@ A flow can become backwards-incompatible in two main ways: * The types of the ``send`` and ``receive`` calls changes -Consequences of running flows with incompatible versions -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +What happens when running flows with incompatible versions? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Pairs of ``InitiatingFlow`` flows and ``InitiatedBy`` flows that have incompatible interfaces are likely to exhibit the following behaviour: @@ -82,8 +102,24 @@ following behaviour: * One of the flows ends with an exception: "Counterparty flow terminated early on the other side", because one flow sends some data to another flow, but the latter flow has already ended -Ensuring flow backwards-compatibility -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +How do I upgrade my flows? +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +1. Update the flow and test the changes. Increment the flow version number in the ``InitiatingFlow`` annotation. +2. Ensure that all versions of the existing flow have finished running and there are no pending ``SchedulableFlows`` on + any of the nodes on the business network. This can be done by *draining the node* (see below). +3. Shut down the node. +4. Replace the existing CorDapp JAR with the CorDapp JAR containing the new flow. +5. Start the node. + +If you shut down all nodes and upgrade them all at the same time, any incompatible change can be made. + +In situations where some nodes may still be using previous versions of a flow and thus new versions of your flow may +talk to old versions, the updated flows need to be backwards-compatible. This will be the case for almost any real +deployment in which you cannot easily coordinate the roll-out of new code across the network. + +How do I ensure flow backwards-compatibility? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The ``InitiatingFlow`` version number is included in the flow session handshake and exposed to both parties via the ``FlowLogic.getFlowContext`` method. This method takes a ``Party`` and returns a ``FlowContext`` object which describes the flow running on the other side. In particular, it has a ``flowVersion`` property which can be used to @@ -127,8 +163,8 @@ This code shows a flow that in its first version expected to receive an Int, but expect a String. This flow is still able to communicate with parties that are running the older CorDapp containing the older flow. -Handling interface changes to inlined subflows -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +How do I deal with interface changes to inlined subflows? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Here is an example of an in-lined subflow: .. container:: codeset @@ -222,26 +258,10 @@ Flows which are not an ``InitiatingFlow`` or ``InitiatedBy`` flow, or inlined su ``InitiatingFlow`` or ``InitiatedBy`` flow, can be updated without consideration of backwards-compatibility. Flows of this type include utility flows for querying the vault and flows for reaching out to external systems. -Performing flow upgrades -~~~~~~~~~~~~~~~~~~~~~~~~ +.. _upgrading-cordapps-flow-drains: -1. Update the flow and test the changes. Increment the flow version number in the ``InitiatingFlow`` annotation -2. Ensure that all versions of the existing flow have finished running and there are no pending ``SchedulableFlows`` on - any of the nodes on the business network. This can be done by :ref:`draining_the_node` -3. Shut down the node -4. Replace the existing CorDapp JAR with the CorDapp JAR containing the new flow -5. Start the node - -If you shut down all nodes and upgrade them all at the same time, any incompatible change can be made. - -In situations where some nodes may still be using previous versions of a flow and thus new versions of your flow may -talk to old versions, the updated flows need to be backwards-compatible. This will be the case for almost any real -deployment in which you cannot easily coordinate the roll-out of new code across the network. - -.. _draining_the_node: - -Draining the node -~~~~~~~~~~~~~~~~~ +Flow drains +~~~~~~~~~~~ A flow *checkpoint* is a serialised snapshot of the flow's stack frames and any objects reachable from the stack. Checkpoints are saved to the database automatically when a flow suspends or resumes, which typically happens when @@ -268,10 +288,9 @@ Contract and state versioning There are two types of contract/state upgrade: -1. *Implicit:* By allowing multiple implementations of the contract ahead of time, using constraints. See - :doc:`api-contract-constraints` to learn more -2. *Explicit:* By creating a special *contract upgrade transaction* and getting all participants of a state to sign it - using the contract upgrade flows +1. *Implicit:* By allowing multiple implementations of the contract ahead of time, using constraints. See :doc:`api-contract-constraints` to learn more. +2. *Explicit:* By creating a special *contract upgrade transaction* and getting all participants of a state to sign it using the + contract upgrade flows. The general recommendation for Corda 4 is to use **implicit** upgrades for the reasons described :ref:`here `. @@ -344,18 +363,18 @@ Produce a new CorDapp JAR file. This JAR file should only contain the new contra 4. Distribute the new CorDapp JAR ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Place the new CorDapp JAR file in the ``cordapps`` folder of all the relevant nodes. You can do this while the nodes are still +Place the new CorDapp JAR file in the ``cordapps`` folder of all the relevant nodes. You can do this while the nodes are still running. 5. Stop the nodes ^^^^^^^^^^^^^^^^^ -Have each node operator stop their node. If you are also changing flow definitions, you should perform a -:ref:`node drain ` first to avoid the definition of states or contracts changing whilst a flow is +Have each node operator stop their node. If you are also changing flow definitions, you should perform a +:ref:`node drain ` first to avoid the definition of states or contracts changing whilst a flow is in progress. 6. Re-run the network bootstrapper (only if you want to whitelist the new contract) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -If you're using the network bootstrapper instead of a network map server and have defined any new contracts, you need to +If you're using the network bootstrapper instead of a network map server and have defined any new contracts, you need to re-run the network bootstrapper to whitelist the new contracts. See :doc:`network-bootstrapper`. 7. Restart the nodes @@ -364,7 +383,7 @@ Have each node operator restart their node. 8. Authorise the upgrade ^^^^^^^^^^^^^^^^^^^^^^^^ -Now that new states and contracts are on the classpath for all the relevant nodes, the nodes must all run the +Now that new states and contracts are on the classpath for all the relevant nodes, the next step is for all node to run the ``ContractUpgradeFlow.Authorise`` flow. This flow takes a ``StateAndRef`` of the state to update as well as a reference to the new contract, which must implement the ``UpgradedContract`` interface. @@ -373,7 +392,7 @@ At any point, a node administrator may de-authorise a contract upgrade by runnin 9. Perform the upgrade ^^^^^^^^^^^^^^^^^^^^^^ -Once all nodes have performed the authorisation process, a **single** node must initiate the upgrade via the +Once all nodes have performed the authorisation process, a participant must be chosen to initiate the upgrade via the ``ContractUpgradeFlow.Initiate`` flow for each state object. This flow has the following signature: .. sourcecode:: kotlin @@ -403,21 +422,71 @@ Capabilities of the contract upgrade flows a ``Dog`` state, provided that all participants in the ``Cat`` state agree to the change * If a node has not yet run the contract upgrade authorisation flow, they will not be able to upgrade the contract and/or state objects +* Upgrade authorisations can subsequently be deauthorised +* Upgrades do not have to happen immediately. For a period, the two parties can use the old states and contracts + side-by-side * State schema changes are handled separately +Writing new states and contracts +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +* If a property is removed from a state, any references to it must be removed from the contract code. Otherwise, you + will not be able to compile your contract code. It is generally not advisable to remove properties from states. Mark + them as deprecated instead +* When adding properties to a state, consider how the new properties will affect transaction validation involving this + state. If the contract is not updated to add constraints over the new properties, they will be able to take on any + value +* Updated state objects can use the old contract code as long as there is no requirement to update it + +Permissioning +^^^^^^^^^^^^^ +* Only node administrators are able to run the contract upgrade authorisation and deauthorisation flows + Logistics ^^^^^^^^^ -* All nodes need to run the contract upgrade authorisation flow to upgrade the contract and/or state objects -* Only node administrators are able to run the contract upgrade authorisation and deauthorisation flows -* Upgrade authorisations can subsequently be deauthorised +* All nodes need to run the contract upgrade authorisation flow * Only one node should run the contract upgrade initiation flow. If multiple nodes run it for the same ``StateRef``, a double-spend will occur for all but the first completed upgrade -* Upgrades do not have to happen immediately. For a period, the two parties can use the old states and contracts - side-by-side * The supplied upgrade flows upgrade one state object at a time -State schema versioning ------------------------ +Serialisation +------------- + +Currently, the serialisation format for everything except flow checkpoints (which uses a Kryo-based format) is based +upon AMQP 1.0, a self-describing and controllable serialisation format. AMQP is desirable because it allows us to have +a schema describing what has been serialized alongside the data itself. This assists with versioning and deserialising +long-ago archived data, among other things. + +Writing classes +~~~~~~~~~~~~~~~ +Although not strictly related to versioning, AMQP serialisation dictates that we must write our classes in a particular way: + +* Your class must have a constructor that takes all the properties that you wish to record in the serialized form. This + is required in order for the serialization framework to reconstruct an instance of your class +* If more than one constructor is provided, the serialization framework needs to know which one to use. The + ``@ConstructorForDeserialization`` annotation can be used to indicate the chosen constructor. For a Kotlin class + without the ``@ConstructorForDeserialization`` annotation, the primary constructor is selected +* The class must be compiled with parameter names in the .class file. This is the default in Kotlin but must be turned + on in Java (using the ``-parameters`` command line option to ``javac``) +* Your class must provide a Java Bean getter for each of the properties in the constructor, with a matching name. For + example, if a class has the constructor parameter ``foo``, there must be a getter called ``getFoo()``. If ``foo`` is + a boolean, the getter may optionally be called ``isFoo()``. This is why the class must be compiled with parameter + names turned on +* The class must be annotated with ``@CordaSerializable`` +* The declared types of constructor arguments/getters must be supported, and where generics are used the generic + parameter must be a supported type, an open wildcard (*), or a bounded wildcard which is currently widened to an open + wildcard +* Any superclass must adhere to the same rules, but can be abstract +* Object graph cycles are not supported, so an object cannot refer to itself, directly or indirectly + +Writing enums +~~~~~~~~~~~~~ +Elements cannot be added to enums in a new version of the code. Hence, enums are only a good fit for genuinely static +data that will never change (e.g. days of the week). A ``Buy`` or ``Sell`` flag is another. However, something like +``Trade Type`` or ``Currency Code`` will likely change. For those, it is preferable to choose another representation, +such as a string. + +State schemas +------------- By default, all state objects are serialised to the database as a string of bytes and referenced by their ``StateRef``. However, it is also possible to define custom schemas for serialising particular properties or combinations of properties, so that they can be queried from a source other than the Corda Vault. This is done by implementing the