M11 documentation (#609)

* Describe new source-tree directories "verifier" and "webserver", and update descriptions for "node" and "tools".
* Update changelog and release notes for M11.
* Thank Gary Rowe for his contribution to M11.
* Update Copyright and fix some broken links.
* Fix reference to CordaRPCOps class.
* Include changelog for CompositeKey upgrade.
* Fix warnings in changelog
* Update release notes after review.
* Add help for IntelliJ/Gradle.

docs/source/changelog.rst

@@ -1,22 +1,62 @@
 Changelog
 =========
 
-Here are brief summaries of what's changed between each snapshot release.
+Here are brief summaries of what's changed between each snapshot release. This includes guidance on how to upgrade code from the previous milestone release.
 
 UNRELEASED
 ----------
 
 * API changes:
-    * Added extension function ``Database.transaction`` to replace ``databaseTransaction``, which is now deprecated.
+    * ``PluginServiceHub.registerServiceFlow`` has been deprecated and replaced by ``registerServiceFlow`` with the
+      marker Class restricted to ``FlowLogic``.
 
-    * Added ``CompositeSignature`` and ``CompositeSignatureData`` as part of enabling ``java.security`` classes to work with
+    * ``FlowLogic.getCounterpartyMarker`` is no longer used and been deprecated for removal. If you were using this to
-      composite keys and signatures.
+      manage multiple independent message streams with the same party in the same flow then use sub-flows instead.
+
+Milestone 11.0
+--------------
+
+* API changes:
+    * Added extension function ``Database.transaction`` to replace ``databaseTransaction``, which is now deprecated.
 
     * Starting a flow no longer enables progress tracking by default. To enable it, you must now invoke your flow using one of the new ``CordaRPCOps.startTrackedFlow`` functions. ``FlowHandle`` is now an interface, and its ``progress: Observable`` field has been moved to the ``FlowProgressHandle`` child interface. Hence developers no longer need to invoke ``notUsed`` on their flows' unwanted progress-tracking observables.
 
     * Moved ``generateSpend`` and ``generateExit`` functions into ``OnLedgerAsset`` from the vault and
       ``AbstractConserveAmount`` clauses respectively.
 
+    * Added ``CompositeSignature`` and ``CompositeSignatureData`` as part of enabling ``java.security`` classes to work with
+      composite keys and signatures.
+
+    * ``CompositeKey`` now implements ``java.security.PublicKey`` interface, so that keys can be used on standard classes such as ``Certificate``. 
+
+        * There is no longer a need to transform single keys into composite - ``composite`` extension was removed, it is imposible to create ``CompositeKey`` with only one leaf.
+
+        * Constructor of ``CompositeKey`` class is now private. Use ``CompositeKey.Builder`` to create a composite key. Keys emitted by the builder are normalised so that it's impossible to create a composite key with only one node. (Long chains of single nodes are shortened.)
+
+        * Use extension function ``PublicKeys.keys`` to access all keys belonging to an instance of ``PublicKey``. For a ``CompositeKey``, this is equivalent to ``CompositeKey.leafKeys``.
+
+        * Introduced ``containsAny``, ``isFulfilledBy``, ``keys`` extension functions on ``PublicKey`` - ``CompositeKey`` type checking is done there.
+
+* Corda now requires JDK 8u131 or above in order to run. Our Kotlin now also compiles to JDK8 bytecode, and so you'll need to update your CorDapp projects to do the same. E.g. by adding this to ``build.gradle``:
+
+.. parsed-literal::
+
+    tasks.withType(org.jetbrains.kotlin.gradle.tasks.KotlinCompile).all {
+        kotlinOptions {
+            languageVersion = "1.1"
+            apiVersion = "1.1"
+            jvmTarget = "1.8"
+        }
+    }
+
+..
+
+ or by adjusting ``Settings/Build,Execution,Deployment/Compiler/KotlinCompiler`` in IntelliJ::
+
+ -  Language Version: 1.1
+ -  API Version: 1.1
+ -  Target JVM Version: 1.8
+
 * DemoBench is now installed as ``Corda DemoBench`` instead of ``DemoBench``.
 
 * Rewrote standard test identities to have full X.500 distinguished names. As part of this work we standardised on a
@@ -37,12 +77,19 @@ Special thank you to `Qian Hong <https://github.com/fracting>`_, `Marek Skocovsk
 
 * Corda DemoBench:
     * DemoBench is a new tool to make it easy to configure and launch local Corda nodes. A very useful tool to demonstrate to your colleagues the fundamentals of Corda in real-time. It has the following features:
+<<<<<<< HEAD
         * Clicking "Add node" creates a new tab that lets you edit the most important configuration properties of the node before launch, such as its legal name and which CorDapps will be loaded.
+=======
+
+        * Clicking "Add node" creates a new tab that lets you edit the most important configuration properties of the node
+          before launch, such as its legal name and which CorDapps will be loaded.
+>>>>>>> 31861ae... M11 documentation (#609)
         * Each tab contains a terminal emulator, attached to the pseudoterminal of the node. This lets you see console output.
         * You can launch an Corda Explorer instance for each node via the DemoBench UI. Credentials are handed to the Corda Explorer so it starts out logged in already.
         * Some basic statistics are shown about each node, informed via the RPC connection.
         * Another button launches a database viewer in the system browser.
         * The configurations of all running nodes can be saved into a single ``.profile`` file that can be reloaded later.
+
     * You can download Corda DemoBench from `here <https://www.corda.net/downloads/>`_
 
 * Vault:
@@ -373,7 +420,7 @@ New features in this release:
       are trees of public keys in which interior nodes can have validity thresholds attached, thus allowing
       boolean formulas of keys to be created. This is similar to Bitcoin's multi-sig support and the data model
       is the same as the InterLedger Crypto-Conditions spec, which should aid interop in future. Read more about
-      key trees in the ":doc:`transaction-data-types`" article.
+      key trees in the ":doc:`key-concepts-core-types`" article.
     * A new tutorial has been added showing how to use transaction attachments in more detail.
 
 * Testnet
@@ -548,8 +595,8 @@ Highlights of this release:
 We have new documentation on:
 
 * :doc:`event-scheduling`
-* :doc:`transaction-data-types`
+* :doc:`key-concepts-core-types`
-* :doc:`consensus`
+* :doc:`key-concepts-consensus-notaries`
 
 Summary of API changes (not exhaustive):
 

docs/source/conf.py

@@ -46,7 +46,7 @@ master_doc = 'index'
 
 # General information about the project.
 project = u'R3 Corda'
-copyright = u'2016, R3 Limited'
+copyright = u'2017, R3 Limited'
 author = u'R3 DLG'
 
 # The version info for the project you're documenting, acts as replacement for

docs/source/flow-state-machines.rst

@@ -156,7 +156,7 @@ simply flow messages or exceptions. The other two represent the buyer and seller
 Going through the data needed to become a seller, we have:
 
 - ``otherParty: Party`` - the party with which you are trading.
-- ``notaryNode: NodeInfo`` - the entry in the network map for the chosen notary. See ":doc:`consensus`" for more
+- ``notaryNode: NodeInfo`` - the entry in the network map for the chosen notary. See ":doc:`key-concepts-consensus-notaries`" for more
   information on notaries.
 - ``assetToSell: StateAndRef<OwnableState>`` - a pointer to the ledger entry that represents the thing being sold.
 - ``price: Amount<Currency>`` - the agreed on price that the asset is being sold for (without an issuer constraint).

docs/source/getting-set-up-fault-finding.rst

@@ -60,6 +60,35 @@ If IDEA refuses to open a project because an SDK has not been selected, you may
 
 If you are having trouble selecting the correct JDK, the JetBrains website provides the `following guidelines <https://intellij-support.jetbrains.com/hc/en-us/articles/206544879-Selecting-the-JDK-version-the-IDE-will-run-under>`_.
 
+IDEA fails to compile Corda because it refuses to find some dependencies
+************************************************************************
+
+First check that Corda compiles successfully from the command line using ``gradlew clean build``. If this succeeds then IntelliJ may just have imported the project's Gradle files incorrectly. Try opening the
+
+.. parsed-literal::
+
+    View/Tool Windows/Gradle
+
+..
+
+pane and clicking the "Refresh all Gradle projects" button. Then retry compiling Corda within IntelliJ. If this still fails then try
+
+.. parsed-literal::
+
+    File/Invalidate Caches, Restart...
+
+..
+
+or checking the
+
+.. parsed-literal::
+
+    Settings/Build,Execution,Deployment/Build Tools/Gradle/Runner/Delegate IDE build-run actions to gradle
+
+..
+
+checkbox, and then refreshing Gradle again.
+
 IDEA fails to compile in VaultSchemaTest.kt
 *******************************************
 
@@ -87,4 +116,4 @@ Other common issues
 “xterm: command not found”
 **************************
 
-On some machines, running the samples requires xterm. You can download it `here <http://invisible-island.net/xterm/#download>`_.
+On some machines, running the samples requires xterm. You can download it `here <http://invisible-island.net/xterm/#download>`_.

docs/source/inthebox.rst

@@ -42,10 +42,12 @@ The Corda repository comprises the following folders:
 * **gradle** contains the gradle wrapper which you'll use to execute gradle commands.
 * **gradle-plugins** contains some additional plugins which we use to deploy Corda nodes.
 * **lib** contains some dependencies.
-* **node** contains the core code of the Corda node (eg: node driver, servlets, node services, messaging, persistence).
+* **node** contains the core code of the Corda node (eg: node driver, node services, messaging, persistence).
 * **node-api** contains data structures shared between the node and the client module, e.g. types sent via RPC.
 * **node-schemas** contains entity classes used to represent relational database tables.
 * **samples** contains all our Corda demos and code samples.
 * **test-utils** contains some utilities for unit testing contracts ( the contracts testing DSL) and protocols (the
   mock network) implementation.
-* **tools** contains the explorer which is a GUI front-end for Corda.
+* **tools** contains the explorer which is a GUI front-end for Corda, and also the DemoBench which is a GUI tool that allows you to run Corda nodes locally for demonstrations.
+* **verifier** allows out-of-node transaction verification, allowing verification to scale horizontally.
+* **webserver** is a servlet container for CorDapps that export HTTP endpoints. This server is an RPC client of the node.

docs/source/key-concepts-consensus-notaries.rst

@@ -17,7 +17,7 @@ The fundamental unit of consensus in Corda is the **state**. Consensus can be di
 
 1. Consensus over state **validity** -- parties can reach certainty that a transaction is accepted by the contracts pointed
    to by the input and output states, and has all the required signatures. This is achieved by parties independently running
-   the same contract code and validation logic (as described in :doc:`data model <data-model>`)
+   the same contract code and validation logic (as described in :doc:`data model <key-concepts-data-model>`)
 
 2. Consensus over state **uniqueness** -- parties can reach certainty the output states created in a transaction are the
    unique successors to the input states consumed by that transaction (in other words -- an input state has not been previously

docs/source/release-notes.rst

@@ -1,12 +1,21 @@
 Release notes
 =============
 
-Here are release notes for each snapshot release from M9 onwards. This includes guidance on how to upgrade code from
+Here are release notes for each snapshot release from M9 onwards.
-the previous milestone release.
 
 Unreleased
 ----------
 
+Flows can now be versioned using the ``FlowVersion`` annotation, which assigns an integer version number to it. For now
+this enables a node to restrict usage of a flow to a specific version. Support for multiple verisons of the same flow,
+hence achieving backwards compatibility, will be possible once we start loading CorDapps in separate class loaders. Watch
+this space...
+
+Milestone 11
+------------
+
+Special thank you to `Gary Rowe <https://github.com/gary-rowe>`_ for his contribution to Corda's Contracts DSL in M11.
+
 Work has continued on confidential identities, introducing code to enable the Java standard libraries to work with
 composite key signatures. This will form the underlying basis of future work to standardise the public key and signature
 formats to enable interoperability with other systems, as well as enabling the use of composite signatures on X.509
@@ -25,8 +34,9 @@ enforced, however it will be in a later milestone.
 
 We anticipate enforcing the use of distinguished names in node configurations from M12, and across the network from M13.
 
-We have updated DemoBench so that it is installed as "Corda DemoBench" for both Windows and MacOSX. The original version
+We have increased the maximum message size that we can send to Corda over RPC from 100 KB to 10 MB.
-was installed as just "DemoBench", and so will not be overwritten automatically by the new version.
+
+The Corda node now disables any use of ObjectInputStream to prevent Java deserialisation within flows. This is a security fix, and prevents the node from deserialising arbitrary objects.
 
 We've introduced the concept of platform version which is a single integer value which increments by 1 if a release changes
 any of the public APIs of the entire Corda platform. This includes the node's public APIs, the messaging protocol,
@@ -34,6 +44,9 @@ serialisation, etc. The node exposes the platform version it's on and we envisio
 run on older versions of the platform to the one they were compiled against. Platform version borrows heavily from Android's
 API Level.
 
+We have revamped the DemoBench user interface. DemoBench will now also be installed as "Corda DemoBench" for both Windows and MacOSX. The original version
+was installed as just "DemoBench", and so will not be overwritten automatically by the new version.
+
 Milestone 10
 ------------
 
@@ -50,6 +63,7 @@ We have many enhancements planned over time including SSH access, more commands
 The new "DemoBench" makes it easy to configure and launch local Corda nodes. It is a standalone desktop app that can be
 bundled with its own JRE and packaged as either EXE (Windows), DMG (MacOS) or RPM (Linux-based). It has the following
 features:
+
  #. New nodes can be added at the click of a button. Clicking "Add node" creates a new tab that lets you edit the most
     important configuration properties of the node before launch, such as its legal name and which CorDapps will be loaded.
  #. Each tab contains a terminal emulator, attached to the pseudoterminal of the node. This lets you see console output.

docs/source/tutorial-clientrpc-api.rst

@@ -27,7 +27,7 @@ Now we can connect to the node itself using a valid RPC login. We login using th
 
 We start generating transactions in a different thread (``generateTransactions`` to be defined later) using ``proxy``, which exposes the full RPC interface of the node:
 
-.. literalinclude:: ../../node/src/main/kotlin/net/corda/node/services/messaging/CordaRPCOps.kt
+.. literalinclude:: ../../core/src/main/kotlin/net/corda/core/messaging/CordaRPCOps.kt
     :language: kotlin
     :start-after: interface CordaRPCOps
     :end-before: }

docs/source/tutorial-contract.rst

@@ -214,7 +214,7 @@ We have four fields in our state:
   relationships such as a derivative contract.
 * ``faceValue``, an ``Amount<Issued<Currency>>``, which wraps an integer number of pennies and a currency that is
   specific to some issuer (e.g. a regular bank, a central bank, etc). You can read more about this very common
-  type in :doc:`transaction-data-types`.
+  type in :doc:`key-concepts-core-types`.
 * ``maturityDate``, an `Instant <https://docs.oracle.com/javase/8/docs/api/java/time/Instant.html>`_, which is a type
   from the Java 8 standard time library. It defines a point on the timeline.
 
@@ -641,7 +641,7 @@ any ``StateAndRef`` (input), ``ContractState`` (output) or ``Command`` objects a
 for you.
 
 There's one final thing to be aware of: we ask the caller to select a *notary* that controls this state and
-prevents it from being double spent. You can learn more about this topic in the :doc:`consensus` article.
+prevents it from being double spent. You can learn more about this topic in the :doc:`key-concepts-consensus-notaries` article.
 
 .. note:: For now, don't worry about how to pick a notary. More infrastructure will come later to automate this
    decision for you.