CORDA-966 - RELEASE - Upgrade notes for V3.0 (#2550)

* DOC: AMQP upgrade notes in brief

* moving test upgrade notes from release notes to upgrade note

* add cordapp/cordformation notes

* Added section for network map changes

* Updated upgrade notes to reflect changes for SSL (#2569)

* Updated upgrade notes to reflect changes for SSL

* Removed some blank lines.

* AMQP Constructor / property mismatch upgrade note

* wip

* Note about feasible (but not recommended) tables that can be migrated from Corda 2.0 to 3.0, basic recipe how to do it.

* Database schema changes - optional partial vault migration described at the end of the paragraph, focus on the overall changes to tables.

* Review Comments

* Database schema changes - split Vault and finance module changes, reformatting, improve migration note.

* Database schema changes - addressing PR comments.

* Database schema changes - move changes to change log, migration to JIRA ticket.

* Finalise versions

* review comments

* CORDA-1171: Add notes on the notary error change

* set release version

docs/source/changelog.rst

@@ -9,6 +9,15 @@ from the previous milestone release.
 Version 3.0
 -----------
 
+* Due to a security risk, the `conflict` property has been removed from `NotaryError.Conflict` error object. It has been replaced
+  with `consumedStates` instead. The new property no longer specifies the original requesting party and transaction id for
+  a consumed state. Instead, only the hash of the transaction id is revealed. For more details why this change had to be
+  made please refer to the release notes.
+
+* Added ``NetworkMapCache.getNodesByLegalName`` for querying nodes belonging to a distributed service such as a notary cluster
+  where they all share a common identity. ``NetworkMapCache.getNodeByLegalName`` has been tightened to throw if more than
+  one node with the legal name is found.
+
 * Per CorDapp configuration is now exposed. ``CordappContext`` now exposes a ``CordappConfig`` object that is populated
   at CorDapp context creation time from a file source during runtime.
 
@@ -36,6 +45,29 @@ Version 3.0
   `participants` collection need to be moved to the actual class. This allows to properly specify the unique table name per a collection.
   See: DummyDealStateSchemaV1.PersistentDummyDealState
 
+* Database schema changes - an H2 database instance of Corda 1.0 and 2.0 cannot be reused for Corda 3.0, listed changes for Vault and Finance module:
+
+    * ``NODE_TRANSACTIONS``:
+       column ``"TRANSACTION”`` renamed to ``TRANSACTION_VALUE``, serialization format of BLOB stored in the column has changed to AMQP
+    * ``VAULT_STATES``:
+       column ``CONTRACT_STATE`` removed
+    * ``VAULT_FUNGIBLE_STATES``:
+        column ``ISSUER_REFERENCE`` renamed to ``ISSUER_REF`` and the field size increased
+    * ``"VAULTSCHEMAV1$VAULTFUNGIBLESTATES_PARTICIPANTS"``:
+        table renamed to ``VAULT_FUNGIBLE_STATES_PARTS``,
+        column ``"VAULTSCHEMAV1$VAULTFUNGIBLESTATES_OUTPUT_INDEX"`` renamed to ``OUTPUT_INDEX``,
+        column ``"VAULTSCHEMAV1$VAULTFUNGIBLESTATES_TRANSACTION_ID"`` renamed to ``TRANSACTION_ID``
+    * ``VAULT_LINEAR_STATES``:
+        type of column ``"UUID"`` changed from ``VARBINARY`` to ``VARCHAR(255)`` - select varbinary column as ``CAST("UUID" AS UUID)`` to get UUID in varchar format
+    * ``"VAULTSCHEMAV1$VAULTLINEARSTATES_PARTICIPANTS"``:
+        table renamed to ``VAULT_LINEAR_STATES_PARTS``,
+        column ``"VAULTSCHEMAV1$VAULTLINEARSTATES_OUTPUT_INDEX"`` renamed to ``OUTPUT_INDEX``,
+        column ``"VAULTSCHEMAV1$VAULTLINEARSTATES_TRANSACTION_ID"`` renamed to ``TRANSACTION_ID``
+    * ``contract_cash_states``:
+        columns storing Base58 representation of the serialised public key (e.g. ``issuer_key``) were changed to store Base58 representation of SHA-256 of public key prefixed with `DL`
+    * ``contract_cp_states``:
+        table renamed to ``cp_states``, column changes as for ``contract_cash_states``
+
 * X.509 certificates now have an extension that specifies the Corda role the certificate is used for, and the role
   hierarchy is now enforced in the validation code. See ``net.corda.core.internal.CertRole`` for the current implementation
   until final documentation is prepared. Certificates at ``NODE_CA``, ``WELL_KNOWN_SERVICE_IDENTITY`` and above must

docs/source/upgrade-notes.rst

@@ -28,8 +28,64 @@ versions you are currently using are still in force.
 
 We also strongly recommend cross referencing with the :doc:`changelog` to confirm changes.
 
-UNRELEASED
+V2.0 to V3.0
-----------
+------------
+
+Gradle Plugin Version
+^^^^^^^^^^^^^^^^^^^^^
+
+Corda 3.0 uses version 3.0.8 of the gradle plugins and your ``build.gradle`` file should be updated to reflect this.
+
+.. sourcecode:: shell
+
+    ext.corda_gradle_plugins_version = '3.0.9'
+
+You will also need to update the ``corda_release_version`` identifier in your project gradle file.
+
+.. sourcecode:: shell
+
+  ext.corda_release_version = 'corda-3.0'
+
+Network Map Service
+^^^^^^^^^^^^^^^^^^^
+
+With the re-designed network map service the following changes need to be made:
+
+* The network map is no longer provided by a node and thus the ``networkMapService`` config is ignored. Instead the
+  network map is either provided by the compatibility zone (CZ) operator (who operates the doorman) and available
+  using the ``compatibilityZoneURL`` config, or is provided using signed node info files which are copied locally.
+  See :doc:`network-map` for more details, and :doc:`setting-up-a-corda-network.rst` on how to use the network
+  bootstrapper for deploying a local network.
+
+* Configuration for a notary has been simplified. ``extraAdvertisedServiceIds``, ``notaryNodeAddress``, ``notaryClusterAddresses``
+  and ``bftSMaRt`` configs have been replaced by a single ``notary`` config object. See :doc:`corda-configuration-file`
+  for more details.
+
+* The advertisement of the notary to the rest of the network, and its validation type, is no longer determined by the
+  ``extraAdvertisedServiceIds`` config. Instead it has been moved to the control of the network operator via
+  the introduction of network parameters. The network bootstrapper automatically includes the configured notaries
+  when generating the network parameters file for a local deployment.
+
+* Any nodes defined in a ``deployNodes`` gradle task performing the function of the network map can be removed, or the
+  ``NetworkMap`` parameter can be removed for any "controller" node which is both the network map and a notary.
+
+* For registering a node with the doorman the ``certificateSigningService`` config has been replaced by ``compatibilityZoneURL``.
+
+Corda Plugins
+^^^^^^^^^^^^^
+
+* Corda plugins have been modularised further so the following additional gradle entries are necessary:
+  For example:
+
+    .. sourcecode:: groovy
+        dependencies {
+            classpath "net.corda.plugins:cordapp:$corda_gradle_plugins_version"
+        }
+
+        apply plugin: 'net.corda.plugins.cordapp'
+
+The plugin needs to be applied in all gradle build files where there is a dependency on Corda using any of:
+cordaCompile, cordaRuntime, cordapp
 
 * For existing contract ORM schemas that extend from ``CommonSchemaV1.LinearState`` or ``CommonSchemaV1.FungibleState``,
   you will need to explicitly map the ``participants`` collection to a database table. Previously this mapping was done
@@ -54,8 +110,65 @@ UNRELEASED
                         JoinColumn(name = "transaction_id", referencedColumnName = "transaction_id")))
                 override var participants: MutableSet<AbstractParty>? = null,
 
+AMQP
+^^^^
+
+Whilst the enablement of AMQP is a transparent change, as noted in the :doc:`serialization` documentation
+the way classes, and states in particular, should be written to work with this new library may require some
+alteration to your current implementation.
+
+  * With AMQP enabled Java classes must be compiled with the -parameter flag.
+
+    * If they aren't, then the error message will complain about ``arg<N>`` being an unknown parameter.
+    * If recompilation is not viable, a custom serializer can be written as per :doc:`cordapp-custom-serializers`
+    * It is important to bear in mind that with AMQP there must be an implicit mapping between constructor
+      parameters and properties you wish included in the serialized form of a class.
+
+      * See :doc:`serialization` for more information
+
+  * Error messages of the form
+
+    ``Constructor parameter - "<some parameter of a constructor>" - doesn't refer to a property of "class <some.class.being.serialized>"``
+
+    indicate that a class, in the above example ``some.class.being.serialized``, has a parameter on its primary constructor that
+    doesn't correlate to a property of the class. This is a problem because the Corda AMQP serialization library uses a class's
+    constructor (default, primary, or annotated) as the means by which instances of the serialized form are reconstituted.
+
+    See the section "Mismatched Class Properties / Constructor Parameters" in the :doc:`serialization` documentation
+
+Database schema changes
+^^^^^^^^^^^^^^^^^^^^^^^
+
+An H2 database instance (represented on the filesystem as a file called `persistence.mv.db`) used in Corda 1.0 or 2.0
+cannot be directly reused with Corda 3.0 due to minor improvements and additions to stabilise the underlying schemas.
+
+Configuration
+^^^^^^^^^^^^^
+
+Nodes that do not require SSL to be enabled for RPC clients now need an additional port to be specified as part of their configuration.
+To do this, add a block as follows to the nodes configuraiton:
+
+  .. sourcecode:: script
+
+    rpcSettings {
+        adminAddress "localhost:10007"
+    }
+
+to `node.conf` files.
+
+Also, the property `rpcPort` is now deprecated, so it would be preferable to substitute properties specified that way e.g., `rpcPort=10006` with a block as follows:
+
+  .. sourcecode:: script
+
+    rpcSettings {
+        address "localhost:10006"
+        adminAddress "localhost:10007"
+    }
+
+Equivalent changes should be performed on classes extending `CordformDefinition`.
+
 Testing
-~~~~~~~
+^^^^^^^
 
 * The registration mechanism for CorDapps in ``MockNetwork`` unit tests has changed:
 
@@ -65,6 +178,47 @@ Testing
 
   * The ``unsetCordappPackages`` method is now redundant and has been removed
 
+* Many classes have been moved between packages, so you will need to update your imports
+
+  .. tip:: We have provided a several scripts (depending upon your operating system of choice) to smooth the upgrade
+     process for existing projects. This can be found at ``tools\scripts\update-test-packages.sh`` for the Bash shell and
+     ``tools/scripts/upgrade-test-packages.ps1`` for Windows Power Shell users in the source tree
+
+* setCordappPackages and unsetCordappPackages have been removed from the ledger/transaction DSL and the flow test framework,
+  and are now set via a constructor parameter or automatically when constructing the MockServices or MockNetwork object
+
+* Key constants e.g. ``ALICE_KEY`` have been removed; you can now use TestIdentity to make your own
+
+* The ledger/transaction DSL must now be provided with MockServices as it no longer makes its own
+  * In transaction blocks, input and output take their arguments as ContractStates rather than lambdas
+  * Also in transaction blocks, command takes its arguments as CommandDatas rather than lambdas
+
+* The MockServices API has changed; please refer to its API documentation
+
+* TestDependencyInjectionBase has been retired in favour of a JUnit Rule called SerializationEnvironmentRule
+  * This replaces the initialiseSerialization parameter of ledger/transaction and verifierDriver
+  * The withTestSerialization method is obsoleted by SerializationEnvironmentRule and has been retired
+
+* MockNetwork now takes a MockNetworkParameters builder to make it more Java-friendly, like driver's DriverParameters
+    * Similarly, the MockNetwork.createNode methods now take a MockNodeParameters builder
+
+* MockNode constructor parameters are now aggregated in MockNodeArgs for easier subclassing
+
+* MockNetwork.Factory has been retired as you can simply use a lambda
+
+* testNodeConfiguration has been retired, please use a mock object framework of your choice instead
+
+* MockNetwork.createSomeNodes and IntegrationTestCategory have been retired with no replacement
+
+* Starting a flow can now be done directly from a node object. Change calls of the form ``node.getServices().startFlow(...)``
+  to ``node.startFlow(...)``
+
+* Similarly a tranaction can be executed directly from a node object. Change calls of the form ``node.getDatabase().transaction({ it -> ... })``
+  to ``node.transaction({() -> ... })``
+
+* ``startFlow`` now returns a ``CordaFuture``, there is no need to call ``startFlow(...).getResultantFuture()``
+
+
 V1.0 to V2.0
 ------------
 
@@ -403,4 +557,4 @@ Finance
 
   * Adjust imports of Cash flow references
   * Adjust the ``StartFlow`` permission in ``gradle.build`` files
-  * Adjust imports of the associated flows (``Cash*Flow``, ``TwoPartyTradeFlow``, ``TwoPartyDealFlow``)
+  * Adjust imports of the associated flows (``Cash*Flow``, ``TwoPartyTradeFlow``, ``TwoPartyDealFlow``)