Docs: release notes and improvements to upgrade notes for Corda 4.

Move release/app upgrade notes to the top of the toctree.

Authors: Kat Baker, Jose Coll, Mike Hearn, Tommy Lillehagen.

docs/source/app-upgrade-notes.rst

@@ -23,17 +23,26 @@ from the new features in the latest release.
 Step 1. Adjust the version numbers in your Gradle build files
 -------------------------------------------------------------
 
+Alter the versions you depend on in your Gradle file like so:
+
 .. sourcecode:: groovy
 
     ext.corda_release_version = '4.0'
-    ext.corda_gradle_plugins_version = '4.0.36'
+    ext.corda_gradle_plugins_version = '4.0.37'
     ext.kotlin_version = '1.2.71'
     ext.quasar_version = '0.7.10'
 
-.. important:: Apps targeting Corda 4 may not at this time use Kotlin 1.3, as it was released too late in the development cycle
+.. note:: You may wish to update your kotlinOptions to use language level 1.2, to benefit from the new features. Apps targeting Corda 4
+   may not at this time use Kotlin 1.3, as it was released too late in the development cycle
    for us to risk an upgrade. Sorry! Future work on app isolation will make it easier for apps to use newer Kotlin versions than
    the node itself uses.
 
+You should also ensure you're using Gradle 4.10 (but not 5). If you use the Gradle wrapper, run::
+
+    ./gradlew wrapper --gradle-version 4.10.3
+
+Otherwise just upgrade your installed copy in the usual manner for your operating system.
+
 Step 2. Add a "cordapp" section to your Gradle build file
 ---------------------------------------------------------
 
@@ -45,29 +54,39 @@ This is used by the Corda Gradle build plugin to populate your app JAR with usef
         contract {
             name "MegaApp Contracts"
             vendor "MegaCorp"
-            license "A liberal, open source license"
+            licence "A liberal, open source licence"
             versionId 1
         }
         workflow {
             name "MegaApp flows"
             vendor "MegaCorp"
-            license "A really expensive proprietary license"
+            licence "A really expensive proprietary licence"
+            versionId 1
         }
     }
 
-Name and vendor can be set to any string you like, they don't have to be Corda identities. Target versioning is a new concept
+.. important:: Watch out for the UK spelling of the word licence (with a c).
+
+Name, vendor and licence can be set to any string you like, they don't have to be Corda identities. Target versioning is a new concept
 introduced in Corda 4. Learn more by reading :doc:`versioning`. Setting a target version of 4 disables workarounds for various
 bugs that may exist in your app, so by doing this you are promising that you have thoroughly tested your app on the new version.
 Using a high target version is a good idea because some features and improvements are only available to apps that opt in.
 
+The new ``versionId`` number is used to block state downgrades: when a state constraint can be satisfied by multiple attachments,
+the version is tracked in the ledger and cannot decrement. This ensures security fixes in CorDapps stick and can't be reversed by
+downgrading states to an earlier version. See ":ref:`contract_non-downgrade_rule_ref`" for more information. In future, the version
+attached to the workflow JAR will also be used to help implement smoother upgrade and migration features. If the "normal" version number
+of your app as defined in your Gradle file follows the convention of always being a whole number starting from 1, you can just refer
+to that variable directly to set the version id.
+
 The duplication between ``contract`` and ``workflow`` blocks exists because you should split your app into two separate JARs/modules,
 one that contains on-ledger validation code like states and contracts, and one for the rest (called by convention the "workflows"
 module although it can contain a lot more than just flows: services would also go here, for instance). For simplicity, here we
 use one JAR for both, but this is in general an anti-pattern and can result in your flow logic code being sent over the network to
 arbitrary third party peers, even though they don't need it.
 
-Step 3. Upgrade your use of FinalityFlow
-----------------------------------------
+Step 3. Security: Upgrade your use of FinalityFlow
+--------------------------------------------------
 
 The previous ``FinalityFlow`` API is insecure. It doesn't have a receive flow, so requires counterparty nodes to accept any and
 all signed transactions that are sent to it, without checks. It is **highly** recommended that existing CorDapps migrate
@@ -168,8 +187,8 @@ finalised transaction. If the initiator is written in a backwards compatible way
 The responder flow may be waiting for the finalised transaction to appear in the local node's vault using ``waitForLedgerCommit``.
 This is no longer necessary with ``ReceiveFinalityFlow`` and the call to ``waitForLedgerCommit`` can be removed.
 
-Step 4. Upgrade your use of SwapIdentitiesFlow
-----------------------------------------------
+Step 4. Security: Upgrade your use of SwapIdentitiesFlow
+--------------------------------------------------------
 
 The :ref:`confidential_identities_ref` API is experimental in Corda 3 and remains so in Corda 4. In this release, the ``SwapIdentitiesFlow``
 has been adjusted in the same way as ``FinalityFlow`` above, to close problems with confidential identities being injectable into a node
@@ -205,22 +224,7 @@ becomes::
 You may need to use the new ``TestCordapp`` API when testing with the node driver or mock network, especially if you decide to stick with the
 pre-Corda 4 ``FinalityFlow`` API. The previous way of pulling in CorDapps into your tests does not honour CorDapp versioning.
 
-Step 6. Security: refactor to avoid violating sealed packages
--------------------------------------------------------------
-
-Hardly any apps will need to do anything in this step.
-
-App isolation has been improved. Version 4 of the finance CorDapp (*corda-finance.jar*) is now built as a sealed and signed JAR file.
-This means classes in your own CorDapps cannot be placed under the following package namespace:  ``net.corda.finance``
-
-In the unlikely event that you were injecting code into ``net.corda.finance.*`` package namespaces from your own apps, you will need to move them
-into a new package, e.g. ``net/corda/finance/flows/MyClass.java`` can be moved to ``com/company/corda/finance/flows/MyClass.java``.
-As a consequence your classes are no longer able to access non-public members of finance CorDapp classes.
-
-When recompiling your JARs for Corda 4, your own apps will also become sealed, meaning other JARs cannot place classes into your own packages.
-This is a security upgrade that ensures package-private visibility in Java code works correctly.
-
-Step 7. Security: Add BelongsToContract annotations
+Step 6. Security: Add BelongsToContract annotations
 ---------------------------------------------------
 
 In versions of the platform prior to v4, it was the responsibility of contract and flow logic to ensure that ``TransactionState`` objects
@@ -237,20 +241,48 @@ to be governed by a contract that is either:
 Learn more by reading ":ref:`implicit_constraint_types`". If an app targets Corda 3 or lower (i.e. does not specify a target version),
 states that point to contracts outside their package will trigger a log warning but validation will proceed.
 
-Step 8. Consider adopting signature constraints
------------------------------------------------
+Step 7. Learn about signature constraints and JAR signing
+---------------------------------------------------------
 
 :doc:`design/data-model-upgrades/signature-constraints` are a new data model feature introduced in Corda 4. They make it much easier to
-deploy application upgrades smoothly and in a decentralised manner. We strongly recommend all apps move to using signature constraints
-as soon as feasible, as they represent the best tradeoff between the different upgrade control models.
-
-.. important:: You will be able to use this feature if the compatibility zone you plan to deploy on has raised its minimum platform version
-   to 4. Otherwise attempting to use signature constraints will throw an exception, because other nodes would not understand it or be able
-   to check the correctness of the transaction. Please take this into account for your own schedule planning.
+deploy application upgrades smoothly and in a decentralised manner. Signature constraints are the new default mode for CorDapps, and
+the act of upgrading your app to use the version 4 Gradle plugins will result in your app being automatically signed, and new states
+automatically using new signature constraints selected automatically based on these signing keys.
 
 You can read more about signature constraints and what they do in :doc:`api-contract-constraints`. The ``TransactionBuilder`` class will
-automatically use them if your application JAR is signed. We recommend all JARs are signed. To start signing your JAR files, read
-:ref:`cordapp_build_system_signing_cordapp_jar_ref`.
+automatically use them if your application JAR is signed. **We recommend all JARs are signed**. To learn how to sign your JAR files, read
+:ref:`cordapp_build_system_signing_cordapp_jar_ref`. In dev mode, all JARs are signed by developer certificates. If a JAR that was signed
+with developer certificates is deployed to a production node, the node will refuse to start. Therefore to deploy apps built for COrda 4
+to production you will need to generate signing keys and integrate them with the build process.
+
+Step 8. Security: package namespace handling
+--------------------------------------------
+
+Almost no apps will be affected by these changes, but they're important to know about.
+
+There are two improvements to how Java package protection is handled in Corda 4:
+
+1. Package sealing
+2. Package namespace ownership
+
+**Sealing.** App isolation has been improved. Version 4 of the finance CorDapp (*corda-finance.jar*) is now built as a sealed and signed JAR file.
+This means classes in your own CorDapps cannot be placed under the following package namespace:  ``net.corda.finance``
+
+In the unlikely event that you were injecting code into ``net.corda.finance.*`` package namespaces from your own apps, you will need to move them
+into a new package, e.g. ``net/corda/finance/flows/MyClass.java`` can be moved to ``com/company/corda/finance/flows/MyClass.java``.
+As a consequence your classes are no longer able to access non-public members of finance CorDapp classes.
+
+When recompiling your JARs for Corda 4, your own apps will also become sealed, meaning other JARs cannot place classes into your own packages.
+This is a security upgrade that ensures package-private visibility in Java code works correctly. If other apps could define classes in your own
+packages, they could call package-private methods, which may not be expected by the developers.
+
+**Namespace ownership.** This part is only relevant if you are joining a production compatibility zone. You may wish to contact your zone operator
+and request ownership of your root package namespaces (e.g. ``com.megacorp.*``), with the signing keys you will be using to sign your app JARs.
+The zone operator can then add your signing key to the network parameters, and prevent attackers defining types in your own package namespaces.
+Whilst this feature is optional and not strictly required, it may be helpful to block attacks at the boundaries of a Corda based application
+where type names may be taken "as read". You can learn more about this feature and the motivation for it by reading
+":doc:`design/data-model-upgrades/package-namespace-ownership`".
+
 
 Step 9. Consider adding extension points to your flows
 ------------------------------------------------------

docs/source/building-a-cordapp-index.rst

@@ -13,7 +13,6 @@ CorDapps
    building-against-master
    debugging-a-cordapp
    versioning
-   upgrade-notes
    upgrading-cordapps
    secure-coding-guidelines
    flow-overriding

docs/source/changelog.rst

@@ -2,10 +2,13 @@ Changelog
 =========
 
 Here's a summary of what's changed in each Corda release. For guidance on how to upgrade code from the previous
-release, see :doc:`upgrade-notes`.
+release, see :doc:`app-upgrade-notes`.
+
+.. _changelog_v4.0:
+
+Version 4.0
+-----------
 
-Unreleased
-----------
 * Fixed race condition between ``NodeVaultService.trackBy`` and ``NodeVaultService.notifyAll``, where there could be states that were not reflected
   in the data feed returned from ``trackBy`` (either in the query's result snapshot or the observable).
 
@@ -97,7 +100,7 @@ Unreleased
 * ``FinalityFlow`` is now an inlined flow and requires ``FlowSession`` s to each party intended to receive the transaction. This is to fix the
   security problem with the old API that required every node to accept any transaction it received without any checks. Existing CorDapp
   binaries relying on this old behaviour will continue to function as previously. However, it is strongly recommended that CorDapps switch to
-  this new API. See :doc:`upgrade-notes` for further details.
+  this new API. See :doc:`app-upgrade-notes` for further details.
 
 * Introduced new optional network bootstrapper command line option (--minimum-platform-version) to set as a network parameter
 
@@ -296,7 +299,7 @@ Unreleased
   The class is used as database Primary Key columns of other entities and databases already impose those columns as non-nullable
   (even if JPA annotation nullable=false was absent).
   In case your Cordapps use this entity class to persist data in own custom tables as non Primary Key columns refer to
-  :doc:`upgrade-notes` for upgrade instructions.
+  :doc:`app-upgrade-notes` for upgrade instructions.
 
 * Adding a public method to check if a public key satisfies Corda recommended algorithm specs, `Crypto.validatePublicKey(java.security.PublicKey)`.
   For instance, this method will check if an ECC key lies on a valid curve or if an RSA key is >= 2048bits. This might

docs/source/index.rst

@@ -28,6 +28,12 @@ We look forward to seeing what you can do with Corda!
 .. _`technical white paper`: _static/corda-technical-whitepaper.pdf
 .. _`download the PDF`: _static/corda-developer-site.pdf
 
+.. toctree::
+   :maxdepth: 1
+
+   release-notes
+   app-upgrade-notes
+
 .. toctree::
    :caption: Development
    :maxdepth: 1
@@ -93,7 +99,6 @@ We look forward to seeing what you can do with Corda!
    contributing-index.rst
    corda-repo-layout.rst
    deterministic-modules.rst
-   release-notes
    changelog
 
 .. conditional-toctree::

docs/source/key-concepts-djvm.rst

@@ -7,22 +7,6 @@ Deterministic JVM
 Introduction
 ~~~~~~~~~~~~
 
-The code in the DJVM module has not yet been integrated with the rest of the platform.  It will eventually become a
-part of the node and enforce deterministic and secure execution of smart contract code, which is mobile and may
-propagate around the network without human intervention.
-
-Currently, it stands alone as an evaluation version. We want to give developers the ability to start trying it out and
-get used to developing deterministic code under the set of constraints that we envision will be placed on contract code
-in the future.
-
-.. warning::
-    The deterministic sandbox is currently a standalone evaluation version of what we, in the future, want to integrate
-    with the Corda platform to protect execution of contract code and ensure deterministic behaviour.
-
-
-Motivation and Overview
-~~~~~~~~~~~~~~~~~~~~~~~
-
 It is important that all nodes that process a transaction always agree on whether it is valid or not. Because
 transaction types are defined using JVM byte code, this means that the execution of that byte code must be fully
 deterministic. Out of the box a standard JVM is not fully deterministic, thus we must make some modifications in order
@@ -32,9 +16,16 @@ So, what does it mean for a piece of code to be fully deterministic?  Ultimately
 as a function, is pure. In other words, given the same set of inputs, it will always produce the same set of outputs
 without inflicting any side-effects that might later affect the computation.
 
+.. important:: The code in the DJVM module has not yet been integrated with the rest of the platform.  It will eventually become a
+   part of the node and enforce deterministic and secure execution of smart contract code, which is mobile and may
+   propagate around the network without human intervention.
+
+   Currently, it stands alone as an evaluation version. We want to give developers the ability to start trying it out and
+   get used to developing deterministic code under the set of constraints that we envision will be placed on contract code
+   in the future.
 
 Non-Determinism
-...............
+~~~~~~~~~~~~~~~
 
 For a program running on the JVM, non-determinism could be introduced by a range of sources, for instance:
 
@@ -65,7 +56,7 @@ generation for ``java.lang.Object``. Contract code is rewritten the first time i
 for future use.
 
 Abstraction
-...........
+~~~~~~~~~~~
 
 The sandbox is abstracted away as an executor which takes as input an implementation of the interface
 ``SandboxedRunnable<in Input, out Output>``, dereferenced by a ``ClassSource``. This interface has a single method that
@@ -319,11 +310,11 @@ Further work is planned:
 Command-line Tool
 ~~~~~~~~~~~~~~~~~
 
-Open your terminal and navigate to the ``djvm`` folder. Then issue the following command:
+Open your terminal and navigate to the ``djvm`` directory in the Corda source tree. Then issue the following command:
 
 ::
 
-  djvm > ./shell/install
+  $ ./shell/install
 
 
 This will build the DJVM tool and install a shortcut on Bash-enabled systems. It will also generate a Bash completion
@@ -331,18 +322,18 @@ file and store it in the ``shell`` folder. This file can be sourced from your Ba
 
 ::
 
-  djvm > cd ~
-  ~ > djvm
+  $ cd ~
+  $ djvm
 
 Now, you can create a new Java file from a skeleton that ``djvm`` provides, compile the file, and consequently run it
 by issuing the following commands:
 
 ::
 
-  ~ > djvm new Hello
-  ~ > vim tmp/net/corda/sandbox/Hello.java
-  ~ > djvm build Hello
-  ~ > djvm run Hello
+  $ djvm new Hello
+  $ vim tmp/net/corda/sandbox/Hello.java
+  $ djvm build Hello
+  $ djvm run Hello
 
 This run will produce some output similar to this:
 

docs/source/network-builder.rst

@@ -6,12 +6,13 @@ Corda Network Builder
 The Corda Network Builder is a tool for building Corda networks for testing purposes. It leverages Docker and
 containers to abstract the complexity of managing a distributed network away from the user.
 
-Currently, the network you build will either be made up of local ``docker`` nodes *or* of nodes spread across Azure
-containers.
+.. image:: _static/images/network-builder-v4.png
 
-The Corda Network Builder can be downloaded from
-https://ci-artifactory.corda.r3cev.com/artifactory/corda-releases/net/corda/corda-network-builder/X.Y-corda/corda-network-builder-X.Y-corda-executable.jar,
-where ``X`` is the major Corda version and ``Y`` is the minor Corda version.
+The network you build will either be made up of local ``docker`` nodes *or* of nodes spread across Azure
+containers. More backends may be added in future. The tool is open source, so contributions to add more
+destinations for the containers are welcome!
+
+`Download the Corda Network Builder <https://ci-artifactory.corda.r3cev.com/artifactory/corda-releases/net/corda/corda-network-builder/4.0-corda/corda-network-builder-4.0-corda-executable.jar>`_.
 
 .. _pre-requisites: