Further docsite changes, including reorganising the index.

docs/source/api-contracts.rst

@@ -136,7 +136,7 @@ Commands
 
 ``TransactionForContract`` contains the commands as a list of ``AuthenticatedObject`` instances.
 ``AuthenticatedObject`` pairs an object with a list of signers. In this case, ``AuthenticatedObject`` pairs a command
- with a list of the entities that are required to sign a transaction where this command is present:
+with a list of the entities that are required to sign a transaction where this command is present:
 
 .. container:: codeset
 

docs/source/api-core-types.rst

@@ -4,21 +4,28 @@ API: Core types
 Corda provides a large standard library of data types used to represent the Corda data model. In addition, there are a
 series of helper libraries which provide date manipulation, maths and cryptography functions.
 
+Cryptography and maths support
+------------------------------
+The ``SecureHash`` class represents a secure hash of unknown algorithm. We currently define only a single subclass,
+``SecureHash.SHA256``. There are utility methods to create them, parse them and so on.
+
+We also provide some mathematical utilities, in particular a set of interpolators and classes for working with
+splines. These can be found in the `maths package <api/kotlin/corda/net.corda.core.math/index.html>`_.
+
 NamedByHash and UniqueIdentifier
 --------------------------------
-
 Things which are identified by their hash, like transactions and attachments, should implement the ``NamedByHash``
 interface which standardises how the ID is extracted. Note that a hash is *not* a globally unique identifier: it
 is always a derivative summary of the contents of the underlying data. Sometimes this isn't what you want:
 two deals that have exactly the same parameters and which are made simultaneously but which are logically different
-can't be identified by hash because their contents would be identical. Instead you would use  ``UniqueIdentifier``.
+can't be identified by hash because their contents would be identical.
-This is a combination of a (Java) ``UUID`` representing a globally unique 128 bit random number, and an arbitrary
+
-string which can be paired with it. For instance the string may represent an existing "weak" (not guaranteed unique)
+Instead you would use  ``UniqueIdentifier``. This is a combination of a (Java) ``UUID`` representing a globally
-identifier for convenience purposes.
+unique 128 bit random number, and an arbitrary string which can be paired with it. For instance the string may
+represent an existing "weak" (not guaranteed unique) identifier for convenience purposes.
 
 Party and CompositeKey
 ----------------------
-
 Entities using the network are called *parties*. Parties can sign structures using keys, and a party may have many
 keys under their control.
 
@@ -30,13 +37,15 @@ without any identifying information about who owns it. In contrast, for internal
 a party are required, the ``Party`` class should be used. The identity service provides functionality for resolving
 anonymous parties to full parties.
 
-An ``AuthenticatedObject`` represents an object (like a command) that has been signed by a set of parties.
+.. note:: These types are provisional and will change significantly in future as the identity framework becomes more
+fleshed out.
 
-.. note:: These types are provisional and will change significantly in future as the identity framework becomes more fleshed out.
+AuthenticatedObject
+-------------------
+An ``AuthenticatedObject`` represents an object (like a command) and the list of associated signers.
 
 Multi-signature support
 -----------------------
-
 Corda supports scenarios where more than one key or party is required to authorise a state object transition, for example:
 "Either the CEO or 3 out of 5 of his assistants need to provide signatures".
 
@@ -44,7 +53,6 @@ Corda supports scenarios where more than one key or party is required to authori
 
 Composite Keys
 ^^^^^^^^^^^^^^
-
 This is achieved by public key composition, using a tree data structure ``CompositeKey``. A ``CompositeKey`` is a tree that
 stores the cryptographic public key primitives in its leaves and the composition logic in the intermediary nodes. Every intermediary
 node specifies a *threshold* of how many child signatures it requires.
@@ -64,7 +72,6 @@ then specifies the minimum total weight of all children required. Our previous e
 
 Verification
 ^^^^^^^^^^^^
-
 Signature verification is performed in two stages:
 
   1. Given a list of signatures, each signature is verified against the expected content.
@@ -74,7 +81,6 @@ Signature verification is performed in two stages:
 
 Date support
 ------------
-
 There are a number of supporting interfaces and classes for use by contracts which deal with dates (especially in the
 context of deadlines). As contract negotiation typically deals with deadlines in terms such as "overnight", "T+3",
 etc., it's desirable to allow conversion of these terms to their equivalent deadline. ``Tenor`` models the interval
@@ -86,12 +92,3 @@ Calculating the rollover of a deadline based on working days requires informatio
 bank holidays). The ``BusinessCalendar`` class models these calendars of business holidays; currently it loads these
 from files on disk, but in future this is likely to involve reference data oracles in order to ensure consensus on the
 dates used.
-
-Cryptography and maths support
-------------------------------
-
-The ``SecureHash`` class represents a secure hash of unknown algorithm. We currently define only a single subclass,
-``SecureHash.SHA256``. There are utility methods to create them, parse them and so on.
-
-We also provide some mathematical utilities, in particular a set of interpolators and classes for working with
-splines. These can be found in the `maths package <api/net.corda.core.math/index.html>`_.

docs/source/api-flows.rst

@@ -123,23 +123,9 @@ annotation.
 
 ServiceHub
 ----------
-Within ``FlowLogic.call()``, the flow developer has access to the node's ``ServiceHub`` that provides access to the
+Within ``FlowLogic.call``, the flow developer has access to the node's ``ServiceHub``, which provides access to the
-various services the node provides.
+various services the node provides. See :doc:`api-service-hub` for information about the services the ``ServiceHub``
-
+offers.
-The key ``ServiceHub`` services are:
-
-* ``ServiceHub.networkMapCache``
-    * Provides info on other nodes on the network (e.g. notaries…)
-* ``ServiceHub.vaultService``
-    * Stores the node’s current and historic states
-* ``ServiceHub.storageService``
-    * Stores additional info such as transactions and attachments
-* ``ServiceHub.keyManagementService``
-    * Manages the node’s digital signing keys
-* ``ServiceHub.myInfo``
-    * Other information about the node
-* ``ServiceHub.clock``
-    * Provides access to the node’s internal time and date
 
 Some common tasks performed using the ``ServiceHub`` are:
 

docs/source/api-persistence.rst

@@ -1,23 +1,22 @@
 Persistence
 ===========
 
-Corda offers developers the option to expose all or some part of a contract state to an *Object Relational Mapping* (ORM) tool
+Corda offers developers the option to expose all or some part of a contract state to an *Object Relational Mapping*
-to be persisted in a RDBMS.  The purpose of this is to assist *vault* development by effectively indexing
+(ORM) tool to be persisted in a RDBMS.  The purpose of this is to assist *vault* development by effectively indexing
 persisted contract states held in the vault for the purpose of running queries over them and to allow relational joins
 between Corda data and private data local to the organisation owning a node.
 
-The ORM mapping is specified using the `Java Persistence API <https://en.wikipedia.org/wiki/Java_Persistence_API>`_ (JPA)
+The ORM mapping is specified using the `Java Persistence API <https://en.wikipedia.org/wiki/Java_Persistence_API>`_
-as annotations and is converted to database table rows by the node automatically every time a state is recorded in the
+(JPA) as annotations and is converted to database table rows by the node automatically every time a state is recorded
-node's local vault as part of a transaction.
+in the node's local vault as part of a transaction.
 
-.. note:: Presently the node includes an instance of the H2 database but any database that supports JDBC is a candidate and
+.. note:: Presently the node includes an instance of the H2 database but any database that supports JDBC is a
-   the node will in the future support a range of database implementations via their JDBC drivers.  Much of the node
+candidate and the node will in the future support a range of database implementations via their JDBC drivers.  Much
-   internal state is also persisted there. You can access the internal H2 database via JDBC, please see the info
+of the node internal state is also persisted there. You can access the internal H2 database via JDBC, please see the
-   in ":doc:`node-administration`" for details.
+info in ":doc:`node-administration`" for details.
 
 Schemas
 -------
-
 Every ``ContractState`` can implement the ``QueryableState`` interface if it wishes to be inserted into the node's local
 database and accessible using SQL.
 
@@ -26,10 +25,10 @@ database and accessible using SQL.
     :start-after: DOCSTART QueryableState
     :end-before: DOCEND QueryableState
 
-The ``QueryableState`` interface requires the state to enumerate the different relational schemas it supports, for instance in
+The ``QueryableState`` interface requires the state to enumerate the different relational schemas it supports, for
-cases where the schema has evolved, with each one being represented by a ``MappedSchema`` object return by the
+instance in cases where the schema has evolved, with each one being represented by a ``MappedSchema`` object return
-``supportedSchemas()`` method.  Once a schema is selected it must generate that representation when requested via the
+by the ``supportedSchemas()`` method.  Once a schema is selected it must generate that representation when requested
-``generateMappedObject()`` method which is then passed to the ORM.
+via the ``generateMappedObject()`` method which is then passed to the ORM.
 
 Nodes have an internal ``SchemaService`` which decides what to persist and what not by selecting the ``MappedSchema``
 to use.
@@ -46,44 +45,44 @@ to use.
 
 The ``SchemaService`` can be configured by a node administrator to select the schemas used by each app. In this way the
 relational view of ledger states can evolve in a controlled fashion in lock-step with internal systems or other
-integration points and not necessarily with every upgrade to the contract code.
+integration points and not necessarily with every upgrade to the contract code. It can select from the
-It can select from the ``MappedSchema`` offered by a ``QueryableState``, automatically upgrade to a
+``MappedSchema`` offered by a ``QueryableState``, automatically upgrade to a later version of a schema or even
-later version of a schema or even provide a ``MappedSchema`` not originally offered by the ``QueryableState``.
+provide a ``MappedSchema`` not originally offered by the ``QueryableState``.
 
 It is expected that multiple different contract state implementations might provide mappings to some common schema.
 For example an Interest Rate Swap contract and an Equity OTC Option contract might both provide a mapping to a common
 Derivative schema. The schemas should typically not be part of the contract itself and should exist independently of it
 to encourage re-use of a common set within a particular business area or Cordapp.
 
-``MappedSchema`` offer a family name that is disambiguated using Java package style name-spacing derived from the class name
+``MappedSchema`` offer a family name that is disambiguated using Java package style name-spacing derived from the
-of a *schema family* class that is constant across versions, allowing the ``SchemaService`` to select a preferred version
+class name of a *schema family* class that is constant across versions, allowing the ``SchemaService`` to select a
-of a schema.
+preferred version of a schema.
 
-The ``SchemaService`` is also responsible for the ``SchemaOptions`` that can be configured for a particular ``MappedSchema``
+The ``SchemaService`` is also responsible for the ``SchemaOptions`` that can be configured for a particular
-which allow the configuration of a database schema or table name prefixes to avoid any clash with other ``MappedSchema``.
+``MappedSchema`` which allow the configuration of a database schema or table name prefixes to avoid any clash with
+other ``MappedSchema``.
 
-.. note:: It is intended that there should be plugin support for the ``SchemaService`` to offer the version upgrading and
+.. note:: It is intended that there should be plugin support for the ``SchemaService`` to offer the version upgrading
-   additional schemas as part of Cordapps, and that the active schemas be configurable.  However the present implementation
+   and additional schemas as part of Cordapps, and that the active schemas be configurable.  However the present
-   offers none of this and simply results in all versions of all schemas supported by a ``QueryableState`` being persisted.
+   implementation offers none of this and simply results in all versions of all schemas supported by a
-   This will change in due course.  Similarly, it does not currently support configuring ``SchemaOptions`` but will do so in
+   ``QueryableState`` being persisted. This will change in due course. Similarly, it does not currently support
-   the future.
+   configuring ``SchemaOptions`` but will do so in the future.
 
 Object relational mapping
 -------------------------
-
+The persisted representation of a ``QueryableState`` should be an instance of a ``PersistentState`` subclass,
-The persisted representation of a ``QueryableState`` should be an instance of a ``PersistentState`` subclass, constructed
+constructed either by the state itself or a plugin to the ``SchemaService``.  This allows the ORM layer to always
-either by the state itself or a plugin to the ``SchemaService``.  This allows the ORM layer to always associate a
+associate a ``StateRef`` with a persisted representation of a ``ContractState`` and allows joining with the set of
-``StateRef`` with a persisted representation of a ``ContractState`` and allows joining with the set of unconsumed states
+unconsumed states in the vault.
-in the vault.
 
 The ``PersistentState`` subclass should be marked up as a JPA 2.1 *Entity* with a defined table name and having
 properties (in Kotlin, getters/setters in Java) annotated to map to the appropriate columns and SQL types.  Additional
 entities can be included to model these properties where they are more complex, for example collections, so the mapping
-does not have to be *flat*. The ``MappedSchema`` must provide a list of all of the JPA entity classes for that schema in order
+does not have to be *flat*. The ``MappedSchema`` must provide a list of all of the JPA entity classes for that schema
-to initialise the ORM layer.
+in order to initialise the ORM layer.
 
 Several examples of entities and mappings are provided in the codebase, including ``Cash.State`` and
-``CommercialPaper.State``.  For example, here's the first version of the cash schema.
+``CommercialPaper.State``. For example, here's the first version of the cash schema.
 
 .. literalinclude:: ../../finance/src/main/kotlin/net/corda/schemas/CashSchemaV1.kt
     :language: kotlin

docs/source/api-rpc.rst

@@ -0,0 +1,29 @@
+API: RPC operations
+===================
+The node's owner interacts with the node solely via remote procedure calls (RPC). The node's owner does not have
+access to the node's ``ServiceHub``.
+
+The key RPC operations exposed by the node are:
+
+* ``CordaRPCOps.vaultQueryBy``
+    * Extract states from the node's vault based on a query criteria
+* ``CordaRPCOps.vaultTrackBy``
+    * As above, but also returns an observable of future states matching the query
+* ``CordaRPCOps.verifiedTransactions``
+    * Extract all transactions from the node's local storage, as well as an observable of all future transactions
+* ``CordaRPCOps.networkMapUpdates``
+    * A list of network nodes, and an observable of changes to the network map
+* ``CordaRPCOps.registeredFlows``
+    * See a list of registered flows on the node
+* ``CordaRPCOps.startFlowDynamic``
+    * Start one of the node's registered flows
+* ``CordaRPCOps.startTrackedFlowDynamic``
+    * As above, but also returns a progress handle for the flow
+* ``CordaRPCOps.nodeIdentity``
+    * Returns the node's identity
+* ``CordaRPCOps.currentNodeTime``
+    * Returns the node's current time
+* ``CordaRPCOps.partyFromKey/CordaRPCOps.partyFromX500Name``
+    * Retrieves a party on the network based on a public key or X500 name
+* ``CordaRPCOps.uploadAttachment``/``CordaRPCOps.openAttachment``/``CordaRPCOps.attachmentExists``
+    * Uploads, opens and checks for the existence of attachments

docs/source/api-service-hub.rst

@@ -0,0 +1,29 @@
+API: ServiceHub
+===============
+Within ``FlowLogic.call``, the flow developer has access to the node's ``ServiceHub``, which provides access to the
+various services the node provides. The services offered by the ``ServiceHub`` are split into the following categories:
+
+* ``ServiceHub.networkMapCache``
+    * Provides information on other nodes on the network (e.g. notaries…)
+* ``ServiceHub.identityService``
+    * Allows you to resolve anonymous identities to well-known identities if you have the required certificates
+* ``ServiceHub.vaultService``
+    * Stores the node’s current and historic states
+* ``ServiceHub.storageService``
+    * Stores additional information such as transactions and attachments
+* ``ServiceHub.keyManagementService``
+    * Manages signing transactions and generating fresh public keys
+* ``ServiceHub.myInfo``
+    * Other information about the node
+* ``ServiceHub.clock``
+    * Provides access to the node’s internal time and date
+
+Additional, ``ServiceHub`` exposes the following properties:
+
+* ``ServiceHub.loadState`` and ``ServiceHub.toStateAndRef`` to resolve a ``StateRef`` into a ``TransactionState`` or
+  a ``StateAndRef``
+* ``ServiceHub.signInitialTransaction`` to sign a ``TransactionBuilder`` and convert it into a ``SignedTransaction``
+* ``ServiceHub.createSignature`` and ``ServiceHub.addSignature`` to create and add signatures to a ``SignedTransaction``
+
+Finally, ``ServiceHub`` exposes the node's legal identity key (via ``ServiceHub.legalIdentityKey``) and its notary
+identity key (via ``ServiceHub.notaryIdentityKey``).

docs/source/api-states.rst

@@ -81,6 +81,12 @@ The vault tracks the head (i.e. the most recent version) of each ``LinearState``
 states all sharing a ``linearId``). To create a transaction updating a ``LinearState``, we retrieve the state from the
 vault using its ``linearId``.
 
+UniqueIdentifier
+~~~~~~~~~~~~~~~~
+``UniqueIdentifier`` is a combination of a (Java) ``UUID`` representing a globally unique 128 bit random number, and
+an arbitrary string which can be paired with it. For instance the string may represent an existing "weak" (not
+guaranteed unique) identifier for convenience purposes.
+
 OwnableState
 ^^^^^^^^^^^^
 ``OwnableState`` models fungible assets. Fungible assets are assets for which it's the total amount held that is
@@ -101,7 +107,7 @@ Where:
 * ``owner`` is the ``PublicKey`` of the asset's owner
 
   * ``OwnableState`` also override the default behavior of the vault's relevancy check. The default vault
-  implementation will track any ``OwnableState`` of which it is the owner.
+    implementation will track any ``OwnableState`` of which it is the owner.
 
 * ``withNewOwner(newOwner: PublicKey)`` creates an identical copy of the state, only with a new owner
 

docs/source/api.rst

@@ -4,9 +4,10 @@ API overview
 This section describes the APIs that are available for the development of CorDapps:
 
 * :doc:`api-states`
+* :doc:`api-persistence`
 * :doc:`api-contracts`
 * :doc:`api-transactions`
 * :doc:`api-flows`
 * :doc:`api-core-types`
 
-Before reading this page, you should be familiar with the key concepts of Corda: :doc:`key-concepts`.
+Before reading this page, you should be familiar with the key concepts of Corda: :doc:`key-concepts`.

docs/source/building-a-cordapp-index.rst

@@ -0,0 +1,16 @@
+Building a CorDapp
+==================
+
+.. toctree::
+   :maxdepth: 1
+
+   cordapp-overview
+   writing-cordapps
+   api
+   api-states
+   api-contracts
+   api-transactions
+   api-flows
+   api-service-hub
+   api-core-types
+   cheat-sheet

docs/source/component-library-index.rst

@@ -0,0 +1,10 @@
+Component library
+=================
+
+.. toctree::
+   :maxdepth: 1
+
+   flow-library
+   contract-catalogue
+   financial-model
+   contract-irs

docs/source/contract-irs.rst

@@ -33,7 +33,7 @@ the view of the floating leg receiver / fixed leg payer. The enumerated document
 it progresses (note that, the first version exists before the value date), the dots on the "y=0" represent an interest
 rate value becoming available and then the curved arrow indicates to which period the fixing applies.
 
-.. image:: contract-irs.png
+.. image:: resources/contract-irs.png
 
 Two days (by convention, although this can be modified) before the value date (i.e. at the start of the swap) in the red
 period, the reference rate is observed from an oracle and fixed - in this instance, at 1.1%. At the end of the accrual period,

docs/source/corda-configuration-file.rst

@@ -100,7 +100,7 @@ path to the node's base directory.
 :notaryNodeAddress: The host and port to which to bind the embedded Raft server. Required only when running a distributed
     notary service. A group of Corda nodes can run a distributed notary service by each running an embedded Raft server and
     joining them to the same cluster to replicate the committed state log. Note that the Raft cluster uses a separate transport
-     layer for communication that does not integrate with ArtemisMQ messaging services.
+    layer for communication that does not integrate with ArtemisMQ messaging services.
 
 :notaryClusterAddresses: List of Raft cluster member addresses used to join the cluster. At least one of the specified
     members must be active and be able to communicate with the cluster leader for joining. If empty, a new cluster will be

docs/source/corda-networks-index.rst

@@ -0,0 +1,9 @@
+Corda networks
+==============
+
+.. toctree::
+   :maxdepth: 1
+
+   setting-up-a-corda-network
+   permissioning
+   versioning

docs/source/corda-nodes-index.rst

@@ -0,0 +1,12 @@
+Corda nodes
+===========
+
+.. toctree::
+   :maxdepth: 1
+
+   running-a-node
+   clientrpc
+   shell
+   node-administration
+   corda-configuration-file
+   out-of-process-verification

docs/source/financial-model.rst

@@ -7,8 +7,8 @@ These provide a common language for states and contracts.
 Amount
 ------
 
-The `Amount <api/net.corda.core.contracts/-amount/index.html>`_ class is used to represent an amount of some
+The `Amount <api/kotlin/corda/net.corda.core.contracts/-amount/index.html>`_ class is used to represent an amount of
-fungible asset. It is a generic class which wraps around a type used to define the underlying product, called
+some fungible asset. It is a generic class which wraps around a type used to define the underlying product, called
 the *token*. For instance it can be the standard JDK type ``Currency``, or an ``Issued`` instance, or this can be
 a more complex type such as an obligation contract issuance definition (which in turn contains a token definition
 for whatever the obligation is to be settled in). Custom token types should implement ``TokenizableAssetInfo`` to allow the

docs/source/index.rst

@@ -38,148 +38,19 @@ Documentation Contents:
 =======================
 
 .. toctree::
-   :maxdepth: 1
+   :maxdepth: 2
-   :caption: Quickstart
 
-   getting-set-up
+   quickstart-index.rst
-   tutorial-cordapp
+   key-concepts.rst
-   running-the-demos
+   building-a-cordapp-index.rst
-   CLI-vs-IDE
+   corda-nodes-index.rst
-
+   corda-networks-index.rst
-.. toctree::
+   tutorials-index.rst
-   :maxdepth: 1
+   tools-index.rst
-   :caption: Key concepts
+   node-internals-index.rst
-
+   component-library-index.rst
-   key-concepts
+   release-process-index.rst
-   key-concepts-ecosystem
+   faq.rst
-   key-concepts-ledger
+   troubleshooting.rst
-   key-concepts-states
+   other-index.rst
-   key-concepts-contracts
+   glossary.rst
-   key-concepts-transactions
-   key-concepts-flows
-   key-concepts-consensus
-   key-concepts-notaries
-   key-concepts-time-windows
-   key-concepts-oracles
-   key-concepts-node
-   key-concepts-tradeoffs
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Building a CorDapp
-
-   cordapp-overview
-   writing-cordapps
-   api
-   api-states
-   api-contracts
-   api-transactions
-   api-flows
-   api-core-types
-   cheat-sheet
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Corda nodes
-
-   running-a-node
-   clientrpc
-   shell
-   node-administration
-   corda-configuration-file
-   out-of-process-verification
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Corda networks
-
-   setting-up-a-corda-network
-   permissioning
-   versioning
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Tutorials
-
-   tutorial-contract
-   tutorial-contract-clauses
-   tutorial-test-dsl
-   contract-upgrade
-   tutorial-integration-testing
-   tutorial-clientrpc-api
-   tutorial-building-transactions
-   flow-state-machines
-   flow-testing
-   running-a-notary
-   using-a-notary
-   oracles
-   tutorial-attachments
-   event-scheduling
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Tools
-
-   network-simulator
-   demobench
-   node-explorer
-   azure-vm
-   loadtesting
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Node internals
-
-   node-services
-   vault
-   serialization
-   messaging
-   persistence
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Component library
-
-   flow-library
-   contract-catalogue
-   financial-model
-   contract-irs
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Release process
-
-   release-process
-   release-notes
-   changelog
-   publishing-corda
-   codestyle
-
-.. toctree::
-   :maxdepth: 1
-   :caption: FAQ
-
-   faq
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Troubleshooting
-
-   troubleshooting
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Other
-
-   clauses
-   merkle-trees
-   json
-   secure-coding-guidelines
-   corda-repo-layout
-   building-the-docs
-
-.. toctree::
-   :maxdepth: 1
-   :caption: Glossary
-
-   glossary

docs/source/key-concepts-consensus.rst

@@ -7,6 +7,13 @@ Consensus
    * *Validity consensus requires contractual validity of the transaction and all its dependencies*
    * *Uniqueness consensus prevents double-spends*
 
+Video
+-----
+.. raw:: html
+
+    <iframe src="https://player.vimeo.com/video/214138438" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+    <p></p>
+
 Two types of consensus
 ----------------------
 Determining whether a proposed transaction is a valid ledger update involves reaching two types of consensus:
@@ -34,6 +41,8 @@ transferring us a treasury bond. We can only be sure that the bond transfer is v
 The only way to be sure of both conditions is to walk the transaction's chain. We can visualize this process as follows:
 
 .. image:: resources/validation-consensus.png
+   :scale: 25%
+   :align: center
 
 When verifying a proposed transaction, a given party may not have every transaction in the transaction chain that they
 need to verify. In this case, they can request the missing transactions from the transaction proposer(s). The
@@ -52,6 +61,8 @@ This is a problem because, although both transactions will achieve validity cons
 "double-spend" his USD to get double the amount of GBP and EUR. We can visualize this as follows:
 
 .. image:: resources/uniqueness-consensus.png
+   :scale: 25%
+   :align: center
 
 To prevent this, a valid transaction proposal must also achieve uniqueness consensus. Uniqueness consensus is the
 requirement that none of the inputs to a proposed transaction have already been consumed in another transaction.

docs/source/key-concepts-contracts.rst

@@ -7,6 +7,13 @@ Contracts
    * *Contracts are written in a JVM programming language (e.g. Java or Kotlin)*
    * *Contract execution is deterministic and its acceptance of a transaction is based on the transaction's contents alone*
 
+Video
+-----
+.. raw:: html
+
+    <iframe src="https://player.vimeo.com/video/214168839" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+    <p></p>
+
 Transaction verification
 ------------------------
 Recall that a transaction is only valid if it is digitally signed by all required signers. However, even if a
@@ -23,6 +30,8 @@ transaction gathers all the required signatures, it is only valid if it is also
 We can picture this situation as follows:
 
 .. image:: resources/tx-validation.png
+   :scale: 25%
+   :align: center
 
 The contract code can be written in any JVM language, and has access to the full capabilities of the language,
 including:
@@ -68,6 +77,12 @@ these cases, an oracle is required. See :doc:`key-concepts-oracles` for further
 
 Legal prose
 -----------
+
+.. raw:: html
+
+    <iframe src="https://player.vimeo.com/video/213879293" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+    <p></p>
+
 Each contract also refers to a legal prose document that states the rules governing the evolution of the state over
 time in a way that is compatible with traditional legal systems. This document can be relied upon in the case of
 legal disputes.

docs/source/key-concepts-ecosystem.rst

@@ -31,6 +31,8 @@ This certificate certifies the node's identity when communicating with other par
 We can visualize a network as follows:
 
 .. image:: resources/network.png
+   :scale: 25%
+   :align: center
 
 Network services
 ----------------
@@ -44,7 +46,8 @@ Nodes can provide several types of services:
 These components are illustrated in the following diagram:
 
 .. image:: resources/cordaNetwork.png
-    :align: center
+   :scale: 25%
+   :align: center
 
 In this diagram, Corda infrastructure services are those upon which all participants depend, such as the network map
 and notary services. Corda services may be deployed by participants, third parties or a central network operator

docs/source/key-concepts-flows.rst

@@ -7,37 +7,68 @@ Flows
    * *Communication between nodes only occurs in the context of these flows, and is point-to-point*
    * *Built-in flows are provided to automate common tasks*
 
+Video
+-----
+.. raw:: html
+
+    <iframe src="https://player.vimeo.com/video/214046145" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+    <p></p>
+
+Motivation
+----------
 Corda networks use point-to-point messaging instead of a global broadcast. This means that coordinating a ledger update
 requires network participants to specify exactly what information needs to be sent, to which counterparties, and in
 what order.
 
-Rather than having to specify these steps manually, Corda automates the process using *flows*. A flow is a sequence
+Here is a visualisation of the process of agreeing a simple ledger update between Alice and Bob:
-of actions that tells a node how to achieve a specific ledger update, such as issuing an asset or settling a trade.
-
-Once a given business process has been encapsulated in a flow and installed on the node as part of a CorDapp, the node's
-owner can instruct the node to kick off this business process at any time with a simple RPC call. All activity on the
-node, and all inter-node communication, occurs in the context of these flows.
-
-A flow's steps may include:
-
-* Identifying counterparties on the network
-* Extracting information from the vault or local storage
-* Building a transaction
-* Sending messages
-* Notarising and recording a transaction
-* Reporting progress to observers
-* Requesting human interaction...
-
-A node can have millions of flows active at once and they may last days, across node restarts and even upgrades.
-
-An example flow
----------------
-Here is a visualisation of an example flow, showing Alice and Bob agreeing on a ledger update:
 
 .. image:: resources/flow.gif
+   :scale: 25%
+   :align: center
+
+The flow framework
+------------------
+Rather than having to specify these steps manually, Corda automates the process using *flows*. A flow is a sequence
+of steps that tells a node how to achieve a specific ledger update, such as issuing an asset or settling a trade.
+
+Here is the sequence of flow steps involved in the simple ledger update above:
+
+.. image:: resources/flow-sequence.png
+   :scale: 25%
+   :align: center
+
+Running flows
+-------------
+Once a given business process has been encapsulated in a flow and installed on the node as part of a CorDapp, the node's
+owner can instruct the node to kick off this business process at any time using an RPC call. The flow abstracts all
+the networking, I/O and concurrency issues away from the node owner.
+
+All activity on the node occurs in the context of these flows. Unlike contracts, flows do not execute in a sandbox,
+meaning that nodes can perform actions such as networking, I/O and use sources of randomness within the execution of a
+flow.
+
+Inter-node communication
+^^^^^^^^^^^^^^^^^^^^^^^^
+Nodes communicate by passing messages between flows. Each node has zero or more flow classes that are registered to
+respond to messages from a single other flow.
+
+Suppose Alice is a node on the network and wishes to agree a ledger update with Bob, another network node. To
+communicate with Bob, Alice must:
+
+* Start a flow that Bob is registered to respond to
+* Send Bob a message within the context of that flow
+* Bob will start its registered counterparty flow
+
+Now that a connection is established, Alice and Bob can communicate to agree a ledger update by passing a series of
+messages back and forth, as prescribed by the flow steps.
+
+Subflows
+^^^^^^^^
+Flows can be composed by starting a flow as a subprocess in the context of another flow. The flow that is started as
+a subprocess is known as a *subflow*. The parent flow will wait until the subflow returns.
 
 The flow library
-----------------
+~~~~~~~~~~~~~~~~
 Corda provides a library of flows to handle common tasks, meaning that developers do not have to redefine the
 logic behind common processes such as:
 
@@ -45,4 +76,12 @@ logic behind common processes such as:
 * Gathering signatures from counterparty nodes
 * Verifying a chain of transactions
 
-Further information on the available built-in flows can be found in :doc:`flow-library`.
+Further information on the available built-in flows can be found in :doc:`flow-library`.
+
+Concurrency
+-----------
+The flow framework allows nodes to have many flows active at once. These flows may last days, across node restarts and even upgrades.
+
+This is achieved by serializing flows to disk whenever they enter a blocking state (e.g. when they're waiting on I/O
+or a networking call). Instead of waiting for the flow to become unblocked, the node immediately starts work on any
+other scheduled flows, only returning to the original flow at a later date.

docs/source/key-concepts-ledger.rst

@@ -6,6 +6,13 @@ The ledger
    * *The ledger is subjective from each peer's perspective*
    * *Two peers are always guaranteed to see the exact same version of any on-ledger facts they share*
 
+Video
+-----
+.. raw:: html
+
+    <iframe src="https://player.vimeo.com/video/213812040" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+    <p></p>
+
 Overview
 --------
 In Corda, there is **no single central store of data**. Instead, each node maintains a separate database of known
@@ -15,6 +22,8 @@ entirety.
 For example, imagine a network with five nodes, where each coloured circle represents a shared fact:
 
 .. image:: resources/ledger-venn.png
+   :scale: 25%
+   :align: center
 
 We can see that although Carl, Demi and Ed are aware of shared fact 3, **Alice and Bob are not**.
 
@@ -22,5 +31,7 @@ Equally importantly, Corda guarantees that whenever one of these facts is shared
 in lockstep in the database of every node that is aware of it:
 
 .. image:: resources/ledger-table.png
+   :scale: 25%
+   :align: center
 
 For example, Alice and Bob will both see the **exact same version** of shared facts 1 and 7.

docs/source/key-concepts-node.rst

@@ -11,6 +11,14 @@ Nodes
 
    * *The node's functionality is extended by installing CorDapps in the plugin registry*
 
+Video
+-----
+.. raw:: html
+
+    <p><a href="https://vimeo.com/214168860">Corda Node, CorDapps and Network</a></p>
+    <iframe src="https://player.vimeo.com/video/214168860" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+    <p></p>
+
 Node architecture
 -----------------
 A Corda node is a JVM run-time environment with a unique identity on the network that hosts Corda services and
@@ -19,6 +27,8 @@ CorDapps.
 We can visualize the node's internal architecture as follows:
 
 .. image:: resources/node-architecture.png
+   :scale: 25%
+   :align: center
 
 The core elements of the architecture are:
 
@@ -44,11 +54,8 @@ node's owner does not interact with other network nodes directly.
 
 RPC interface
 -------------
-The node's owner interacts with the node via remote procedure calls (RPC). RPC operations exposed by the node include:
+The node's owner interacts with the node via remote procedure calls (RPC). The key RPC operations the node exposes
-
+are documented in :doc:``api-rpc``.
-* Starting a flow
-* Reading the contents of the vault or the transaction storage
-* Uploading and downloading attachments
 
 The service hub
 ---------------

docs/source/key-concepts-notaries.rst

@@ -7,8 +7,15 @@ Notaries
    * *Notaries may optionally also validate transactions*
    * *A network can have several notaries, each running a different consensus algorithm*
 
-Notarisation
+Video
-------------
+-----
+.. raw:: html
+
+    <iframe src="https://player.vimeo.com/video/214138458" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+    <p></p>
+
+Overview
+--------
 A *notary* is a network service that provides **uniqueness consensus** by attesting that, for a given transaction, it
 has not already signed other transactions that consumes any of the proposed transaction's input states.
 
@@ -28,7 +35,7 @@ Every state has an appointed notary, and a notary will only notarise a transacti
 of all the transaction's input states.
 
 Consensus algorithms
-^^^^^^^^^^^^^^^^^^^^
+--------------------
 Corda has "pluggable" consensus, allowing notaries to choose a consensus algorithm based on their requirements in
 terms of privacy, scalability, legal-system compatibility and algorithmic agility.
 
@@ -60,7 +67,7 @@ In the case of the validating model, the use of anonymous, freshly-generated pub
 identify parties in a transaction limit the information the notary sees.
 
 Multiple notaries
-^^^^^^^^^^^^^^^^^
+-----------------
 Each Corda network can have multiple notaries, each potentially running a different consensus algorithm. This provides
 several benefits:
 

docs/source/key-concepts-oracles.rst

@@ -6,6 +6,15 @@ Oracles
    * *A fact can be included in a transaction as part of a command*
    * *An oracle is a service that will only sign the transaction if the included fact is true*
 
+Video
+-----
+.. raw:: html
+
+    <iframe src="https://player.vimeo.com/video/214157956" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+    <p></p>
+
+Overview
+--------
 In many cases, a transaction's contractual validity depends on some external piece of data, such as the current
 exchange rate. However, if we were to let each participant evaluate the transaction's validity based on their own
 view of the current exchange rate, the contract's execution would be non-deterministic: some signers would consider the
@@ -22,5 +31,49 @@ their transaction, and the oracle will sign the transaction to assert that the f
 If they wish to monetize their services, oracles can choose to only sign a transaction and attest to the validity of
 the fact it contains for a fee.
 
-Transaction tear-offs are used to prevent the oracle from seeing information about the transaction that is not
+Transaction tear-offs
-relevant to them. See :doc:`merkle-trees` for further details.
+---------------------
+To sign a transaction, the only information the oracle needs to see is their embedded command. Providing any
+additional transaction data to the oracle would constitute a privacy leak. Similarly, a non-validating notary only
+needs to see a transaction's input states.
+
+To combat this, the transaction proposer(s) uses a Merkle tree to "tear off" any parts of the transaction that the
+oracle/notary doesn't need to see before presenting it to them for signing. A Merkle tree is a well-known cryptographic
+scheme that is commonly used to provide proofs of inclusion and data integrity. Merkle trees are widely used in
+peer-to-peer networks, blockchain systems and git.
+
+The advantage of a Merkle tree is that the parts of the transaction that were torn off when presenting the transaction
+to the oracle cannot later be changed without also invalidating the oracle's digital signature.
+
+Transaction Merkle trees
+^^^^^^^^^^^^^^^^^^^^^^^^
+A Merkle tree is constructed from a transaction by splitting the transaction into leaves, where each leaf contains
+either an input, an output, a command, or an attachment. The Merkle tree also contains the other fields of the
+``WireTransaction``, such as the timestamp, the notary, the type and the signers.
+
+Next, the Merkle tree is built in the normal way by hashing the concatenation of nodes’ hashes below the current one
+together. It’s visible on the example image below, where ``H`` denotes sha256 function, "+" - concatenation.
+
+.. image:: resources/merkleTree.png
+
+The transaction has two input states, one output state, one attachment, one command and a timestamp. For brevity
+we didn't include all leaves on the diagram (type, notary and signers are presented as one leaf labelled Rest - in
+reality they are separate leaves). Notice that if a tree is not a full binary tree, leaves are padded to the nearest
+power of 2 with zero hash (since finding a pre-image of sha256(x) == 0 is hard computational task) - marked light
+green above. Finally, the hash of the root is the identifier of the transaction, it's also used for signing and
+verification of data integrity. Every change in transaction on a leaf level will change its identifier.
+
+Hiding data
+^^^^^^^^^^^
+Hiding data and providing the proof that it formed a part of a transaction is done by constructing Partial Merkle Trees
+(or Merkle branches). A Merkle branch is a set of hashes, that given the leaves’ data, is used to calculate the
+root’s hash. Then that hash is compared with the hash of a whole transaction and if they match it means that data we
+obtained belongs to that particular transaction.
+
+.. image:: resources/partialMerkle.png
+
+In the example above, the node ``H(f)`` is the one holding command data for signing by Oracle service. Blue leaf
+``H(g)`` is also included since it's holding timestamp information. Nodes labelled ``Provided`` form the Partial
+Merkle Tree, black ones are omitted. Having timestamp with the command that should be in a violet node place and
+branch we are able to calculate root of this tree and compare it with original transaction identifier - we have a
+proof that this command and timestamp belong to this transaction.

docs/source/key-concepts-states.rst

@@ -7,6 +7,15 @@ States
    * *States are evolved by marking the current state as historic and creating an updated state*
    * *Each node has a vault where it stores any relevant states to itself*
 
+Video
+-----
+.. raw:: html
+
+    <iframe src="https://player.vimeo.com/video/213812054" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+    <p></p>
+
+Overview
+--------
 A *state* is an immutable object representing a fact known by one or more Corda nodes at a specific point in time.
 States can contain arbitrary data, allowing them to represent facts of any kind (e.g. stocks, bonds, loans, KYC data,
 identity information...).
@@ -14,6 +23,8 @@ identity information...).
 For example, the following state represents an IOU - an agreement that Alice owes Bob an amount X:
 
 .. image:: resources/state.png
+   :scale: 25%
+   :align: center
 
 Specifically, this state represents an IOU of £10 from Alice to Bob.
 
@@ -32,6 +43,8 @@ This sequence of state replacements gives us a full view of the evolution of the
 picture this situation as follows:
 
 .. image:: resources/state-sequence.png
+   :scale: 25%
+   :align: center
 
 The vault
 ---------
@@ -39,6 +52,8 @@ Each node on the network maintains a *vault* - a database where it tracks all th
 is aware of, and which it considers to be relevant to itself:
 
 .. image:: resources/vault-simple.png
+   :scale: 25%
+   :align: center
 
 We can think of the ledger from each node's point of view as the set of all the current (i.e. non-historic) states that
 it is aware of.

docs/source/key-concepts-time-windows.rst

@@ -7,6 +7,13 @@ Time-windows
    * *The notary is the timestamping authority, refusing to commit transactions outside of that window*
    * *Time-windows can have a start and end time, or be open at either end*
 
+Video
+-----
+.. raw:: html
+
+    <iframe src="https://player.vimeo.com/video/213879314" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+    <p></p>
+
 Time in a distributed system
 ----------------------------
 A notary also act as the *timestamping authority*, verifying that a transaction occurred during a specific time-window
@@ -32,6 +39,8 @@ there can never be "true time", only an approximation of it. Time windows can be
 "before" and "after") or they can be fully bounded.
 
 .. image:: resources/time-window.gif
+   :scale: 25%
+   :align: center
 
 In this way, we express the idea that the *true value* of the fact "the current time" is actually unknowable. Even when
 both a before and an after time are included, the transaction could have occurred at any point within that time-window.
@@ -47,4 +56,4 @@ If a time window needs to be converted to an absolute time (e.g. for display pur
 calculate the mid point.
 
 .. note:: It is assumed that the time feed for a notary is GPS/NaviStar time as defined by the atomic
-clocks at the US Naval Observatory. This time feed is extremely accurate and available globally for free.
+clocks at the US Naval Observatory. This time feed is extremely accurate and available globally for free.

docs/source/key-concepts-transactions.rst

@@ -10,6 +10,15 @@ Transactions
      * *It is contractually valid*
      * *It is signed by the required parties*
 
+Video
+-----
+.. raw:: html
+
+    <iframe src="https://player.vimeo.com/video/213879807" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+    <p></p>
+
+Overview
+--------
 Corda uses a *UTXO* (unspent transaction output) model where every state on the ledger is immutable. The ledger
 evolves over time by applying *transactions*, which update the ledger by marking zero or more existing ledger states
 as historic (the *inputs*) and producing zero or more new ledger states (the *outputs*). Transactions represent a
@@ -18,6 +27,8 @@ single link in the state sequences seen in :doc:`key-concepts-states`.
 Here is an example of an update transaction, with two inputs and two outputs:
 
 .. image:: resources/basic-tx.png
+   :scale: 25%
+   :align: center
 
 A transaction can contain any number of inputs and outputs of any type:
 
@@ -46,6 +57,8 @@ These input states references are a combination of:
 This situation can be illustrated as follows:
 
 .. image:: resources/tx-chain.png
+   :scale: 25%
+   :align: center
 
 These input state references link together transactions over time, forming what is known as a *transaction chain*.
 
@@ -55,15 +68,21 @@ Initially, a transaction is just a **proposal** to update the ledger. It represe
 that is desired by the transaction builder(s):
 
 .. image:: resources/uncommitted_tx.png
+   :scale: 25%
+   :align: center
 
 To become reality, the transaction must receive signatures from all of the *required signers* (see **Commands**, below). Each
 required signer appends their signature to the transaction to indicate that they approve the proposal:
 
 .. image:: resources/tx_with_sigs.png
+   :scale: 25%
+   :align: center
 
 If all of the required signatures are gathered, the transaction becomes committed:
 
 .. image:: resources/committed_tx.png
+   :scale: 25%
+   :align: center
 
 This means that:
 
@@ -96,11 +115,18 @@ For example, a transaction where Alice pays off £5 of an IOU with Bob using a
 attachments and a timestamp, may look as follows:
 
 .. image:: resources/full-tx.png
+   :scale: 25%
+   :align: center
 
 We explore the role played by the remaining transaction components below.
 
 Commands
 ^^^^^^^^
+.. raw:: html
+
+    <iframe src="https://player.vimeo.com/video/213881538" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+    <p></p>
+
 Suppose we have a transaction with a cash state and a bond state as inputs, and a cash state and a bond state as
 outputs. This transaction could represent two different scenarios:
 
@@ -124,9 +150,16 @@ listed in the commands, we get the list of the transaction's required signers. I
 We can visualize this situation as follows:
 
 .. image:: resources/commands.png
+   :scale: 25%
+   :align: center
 
 Attachments
 ^^^^^^^^^^^
+.. raw:: html
+
+    <iframe src="https://player.vimeo.com/video/213879328" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
+    <p></p>
+
 Sometimes, we have a large piece of data that can be reused across many different transactions. Some examples:
 
 * A calendar of public holidays

docs/source/key-concepts.rst

@@ -1,108 +1,33 @@
-Overview
+Key concepts
-========
+============
 
 This section describes the key concepts and features of the Corda platform. It is intended for readers who are new to
 Corda, and want to understand its architecture. It does not contain any code, and is suitable for non-developers.
 
 This section should be read in order:
 
-    * :doc:`key-concepts-ecosystem`
+.. toctree::
-    * :doc:`key-concepts-ledger`
+   :maxdepth: 1
-    * :doc:`key-concepts-states`
+
-    * :doc:`key-concepts-contracts`
+   key-concepts-ecosystem
-    * :doc:`key-concepts-transactions`
+   key-concepts-ledger
-    * :doc:`key-concepts-flows`
+   key-concepts-states
-    * :doc:`key-concepts-consensus`
+   key-concepts-contracts
-    * :doc:`key-concepts-notaries`
+   key-concepts-transactions
-    * :doc:`key-concepts-time-windows`
+   key-concepts-flows
-    * :doc:`key-concepts-oracles`
+   key-concepts-consensus
-    * :doc:`key-concepts-node`
+   key-concepts-notaries
-    * :doc:`key-concepts-tradeoffs`
+   key-concepts-time-windows
+   key-concepts-oracles
+   key-concepts-node
+   key-concepts-tradeoffs
 
 The detailed thinking and rationale behind these concepts are presented in two white papers:
 
     * `Corda: An Introduction`_
     * `Corda: A Distributed Ledger`_ (A.K.A. the Technical White Paper)
 
-Explanations of the key concepts are also available as `videos <https://vimeo.com/album/4555732/>`_:
+Explanations of the key concepts are also available as `videos <https://vimeo.com/album/4555732/>`_.
-
-.. raw:: html
-
-    <p><a href="https://vimeo.com/213812040">The Corda Ledger</a></p>
-    <iframe src="https://player.vimeo.com/video/213812040" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
-    <p></p>
-
-.. raw:: html
-
-    <p><a href="https://vimeo.com/213812054">States</a></p>
-    <iframe src="https://player.vimeo.com/video/213812054" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
-    <p></p>
-
-.. raw:: html
-
-    <p><a href="https://vimeo.com/213879807">Transactions</a></p>
-    <iframe src="https://player.vimeo.com/video/213879807" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
-    <p></p>
-
-.. raw:: html
-
-    <p><a href="https://vimeo.com/214168839">Contracts</a></p>
-    <iframe src="https://player.vimeo.com/video/214168839" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
-    <p></p>
-
-.. raw:: html
-
-    <p><a href="https://vimeo.com/213879293">Legal Prose</a></p>
-    <iframe src="https://player.vimeo.com/video/213879293" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
-    <p></p>
-
-.. raw:: html
-
-    <p><a href="https://vimeo.com/213881538">Commands</a></p>
-    <iframe src="https://player.vimeo.com/video/213881538" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
-    <p></p>
-
-.. raw:: html
-
-    <p><a href="https://vimeo.com/213879314">Timestamps</a></p>
-    <iframe src="https://player.vimeo.com/video/213879314" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
-    <p></p>
-
-.. raw:: html
-
-    <p><a href="https://vimeo.com/213879328">Attachments</a></p>
-    <iframe src="https://player.vimeo.com/video/213879328" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
-    <p></p>
-
-.. raw:: html
-
-    <p><a href="https://vimeo.com/214046145">Flows</a></p>
-    <iframe src="https://player.vimeo.com/video/214046145" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
-    <p></p>
-
-.. raw:: html
-
-    <p><a href="https://vimeo.com/214138438">Consensus</a></p>
-    <iframe src="https://player.vimeo.com/video/214138438" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
-    <p></p>
-
-.. raw:: html
-
-    <p><a href="https://vimeo.com/214138458">Notaries</a></p>
-    <iframe src="https://player.vimeo.com/video/214138458" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
-    <p></p>
-
-.. raw:: html
-
-    <p><a href="https://vimeo.com/214157956">Oracles</a></p>
-    <iframe src="https://player.vimeo.com/video/214157956" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
-    <p></p>
-
-.. raw:: html
-
-    <p><a href="https://vimeo.com/214168860">Corda Node, CorDapps and Network</a></p>
-    <iframe src="https://player.vimeo.com/video/214168860" width="640" height="360" frameborder="0" webkitallowfullscreen mozallowfullscreen allowfullscreen></iframe>
-    <p></p>
 
 .. _`Corda: An Introduction`: _static/corda-introductory-whitepaper.pdf
 .. _`Corda: A Distributed Ledger`: _static/corda-technical-whitepaper.pdf

docs/source/merkle-trees.rst

@@ -1,125 +0,0 @@
-Transaction tear-offs
-=====================
-
-One of the basic data structures in our platform is a transaction. It can be passed around to be signed and verified,
-also by third parties. The construction of transactions assumes that they form a whole entity with input and output states,
-commands and attachments inside. However all sensitive data shouldn’t be revealed to other nodes that take part in
-the creation of transaction on validation level (a good example of this situation is the Oracle which validates only
-embedded commands). How to achieve it in a way that convinces the other party the data they got for signing really did form
-a part of the transaction?
-
-We decided to use well known and described cryptographic scheme to provide proofs of inclusion and data integrity.
-Merkle trees are widely used in peer-to-peer networks, blockchain systems and git.
-You can read more on the concept `here <https://en.wikipedia.org/wiki/Merkle_tree>`_.
-
-Merkle trees in Corda
----------------------
-
-Transactions are split into leaves, each of them contains either input, output, command or attachment. Additionally, in
-transaction id calculation we use other fields of ``WireTransaction`` like timestamp, notary, type and signers.
-Next, the Merkle tree is built in the normal way by hashing the concatenation of nodes’ hashes below the current one together.
-It’s visible on the example image below, where ``H`` denotes sha256 function, "+" - concatenation.
-
-.. image:: resources/merkleTree.png
-
-The transaction has two input states, one of output, attachment and command each and timestamp. For brevity we didn't
-include all leaves on the diagram (type, notary and signers are presented as one leaf labelled Rest - in reality
-they are separate leaves). Notice that if a tree is not a full binary tree, leaves are padded to the nearest power
-of 2 with zero hash (since finding a pre-image of sha256(x) == 0 is hard computational task) - marked light green above.
-Finally, the hash of the root is the identifier of the transaction, it's also used for signing and verification of data integrity.
-Every change in transaction on a leaf level will change its identifier.
-
-Hiding data
------------
-
-Hiding data and providing the proof that it formed a part of a transaction is done by constructing Partial Merkle Trees
-(or Merkle branches). A Merkle branch is a set of hashes, that given the leaves’ data, is used to calculate the root’s hash.
-Then that hash is compared with the hash of a whole transaction and if they match it means that data we obtained belongs
-to that particular transaction.
-
-.. image:: resources/partialMerkle.png
-
-In the example above, the node ``H(f)`` is the one holding command data for signing by Oracle service. Blue leaf ``H(g)`` is also
-included since it's holding timestamp information. Nodes labelled ``Provided`` form the Partial Merkle Tree, black ones
-are omitted. Having timestamp with the command that should be in a violet node place and branch we are able to calculate
-root of this tree and compare it with original transaction identifier - we have a proof that this command and timestamp
-belong to this transaction.
-
-Example of usage
-----------------
-
-Let’s focus on a code example. We want to construct a transaction with commands containing interest rate fix data as in:
-:doc:`oracles`.
-After construction of a partial transaction, with included ``Fix`` commands in it, we want to send it to the Oracle for checking
-and signing. To do so we need to specify which parts of the transaction are going to be revealed. That can be done by constructing
-filtering function over fields of ``WireTransaction`` of type ``(Any) -> Boolean``.
-
-.. container:: codeset
-
-   .. sourcecode:: kotlin
-
-        val partialTx = ...
-        val oracle: Party = ...
-        fun filtering(elem: Any): Boolean {
-                return when (elem) {
-                    is Command -> oracleParty.owningKey in elem.signers && elem.value is Fix
-                    else -> false
-                }
-        }
-
-Assuming that we already assembled partialTx with some commands and know the identity of Oracle service,
-we construct filtering function over commands - ``filtering``. It performs type checking and filters only ``Fix`` commands
-as in IRSDemo example. Then we can construct ``FilteredTransaction``:
-
-.. container:: codeset
-
-   .. sourcecode:: kotlin
-
-        val wtx: WireTransaction = partialTx.toWireTransaction()
-        val ftx: FilteredTransaction = wtx.buildFilteredTransaction(filtering)
-
-In the Oracle example this step takes place in ``RatesFixFlow`` by overriding ``filtering`` function, see: :ref:`filtering_ref`
-
-
-``FilteredTransaction`` holds ``filteredLeaves`` (data that we wanted to reveal) and Merkle branch for them.
-
-.. container:: codeset
-
-   .. sourcecode:: kotlin
-
-        // Direct accsess to included commands, inputs, outputs, attachments etc.
-        val cmds: List<Command> = ftx.filteredLeaves.commands
-        val ins: List<StateRef> = ftx.filteredLeaves.inputs
-        val timestamp: Timestamp? = ftx.filteredLeaves.timestamp
-        ...
-
-.. literalinclude:: ../../samples/irs-demo/src/main/kotlin/net/corda/irs/api/NodeInterestRates.kt
-    :language: kotlin
-    :start-after: DOCSTART 1
-    :end-before: DOCEND 1
-
-Above code snippet is taken from ``NodeInterestRates.kt`` file and implements a signing part of an Oracle.
-You can check only leaves using ``leaves.checkWithFun { check(it) }`` and then verify obtained ``FilteredTransaction``
-to see if data from ``PartialMerkleTree`` belongs to ``WireTransaction`` with provided ``id``. All you need is the root hash
-of the full transaction:
-
-.. container:: codeset
-
-   .. sourcecode:: kotlin
-
-        if (!ftx.verify(merkleRoot)){
-                throw MerkleTreeException("Rate Fix Oracle: Couldn't verify partial Merkle tree.")
-        }
-
-Or combine the two steps together:
-
-.. container:: codeset
-
-   .. sourcecode:: kotlin
-
-        ftx.verifyWithFunction(merkleRoot, ::check)
-
-.. note:: The way the ``FilteredTransaction`` is constructed ensures that after signing of the root hash it's impossible to add or remove
-    leaves. However, it can happen that having transaction with multiple commands one party reveals only subset of them to the Oracle.
-    As signing is done now over the Merkle root hash, the service signs all commands of given type, even though it didn't see
-    all of them. This issue will be handled after implementing partial signatures.

docs/source/network-simulator.rst

@@ -40,7 +40,7 @@ to single JVM simulations.
 Interface
 ---------
 
-.. image:: network-simulator.png
+.. image:: resources/network-simulator.png
 
 The network simulator can be run automatically, or stepped manually through each step of the interest rate swap. The
 options on the simulator window are:

docs/source/node-internals-index.rst

@@ -0,0 +1,11 @@
+Node internals
+==============
+
+.. toctree::
+   :maxdepth: 1
+
+   node-services
+   vault
+   serialization
+   messaging
+   persistence

docs/source/other-index.rst

@@ -0,0 +1,12 @@
+Other
+=====
+
+.. toctree::
+   :maxdepth: 1
+
+   clauses
+   merkle-trees
+   json
+   secure-coding-guidelines
+   corda-repo-layout
+   building-the-docs

docs/source/quickstart-index.rst

@@ -0,0 +1,10 @@
+Quickstart
+==========
+
+.. toctree::
+   :maxdepth: 1
+
+   getting-set-up
+   tutorial-cordapp
+   running-the-demos
+   CLI-vs-IDE

docs/source/release-process-index.rst

@@ -0,0 +1,10 @@
+Release process
+===============
+
+.. toctree::
+   :maxdepth: 1
+
+   release-notes
+   changelog
+   publishing-corda
+   codestyle

docs/source/running-a-node.rst

@@ -151,7 +151,8 @@ deployment.
 
 To use this gradle plugin you must add a new task that is of the type ``net.corda.plugins.Cordform`` to your
 build.gradle and then configure the nodes you wish to deploy with the Node and nodes configuration DSL.
-This DSL is specified in the `JavaDoc <api/index.html>`_. An example of this is in the CorDapp template and below
+This DSL is specified in the `JavaDoc <api/javadoc/>`_. An example of this is in the CorDapp template and
+below
 is a three node example;
 
 .. code-block:: text

docs/source/running-the-demos.rst

@@ -123,7 +123,8 @@ You will notice that successive transactions get signed by different members of
 To run the Raft version of the demo from the command line in Unix:
 
 1. Run ``./gradlew samples:notary-demo:deployNodes``, which will create all three types of notaries' node directories
-   with configs under ``samples/notary-demo/build/nodes/nodesRaft`` (``nodesBFT`` and ``nodesSingle`` for BFT and Single notaries).
+   with configs under ``samples/notary-demo/build/nodes/nodesRaft`` (``nodesBFT`` and ``nodesSingle`` for BFT and
+   Single notaries).
 2. Run ``./samples/notary-demo/build/nodes/nodesRaft/runnodes``, which will start the nodes in separate terminal windows/tabs.
    Wait until a "Node started up and registered in ..." message appears on each of the terminals
 3. Run ``./gradlew samples:notary-demo:notarise`` to make a call to the "Party" node to initiate notarisation requests
@@ -132,7 +133,8 @@ To run the Raft version of the demo from the command line in Unix:
 To run from the command line in Windows:
 
 1. Run ``gradlew samples:notary-demo:deployNodes``, which will create all three types of notaries' node directories
-   with configs under ``samples/notary-demo/build/nodes/nodesRaft`` (``nodesBFT`` and ``nodesSingle`` for BFT and Single notaries).
+   with configs under ``samples/notary-demo/build/nodes/nodesRaft`` (``nodesBFT`` and ``nodesSingle`` for BFT and
+   Single notaries).
 2. Run ``samples\notary-demo\build\nodes\nodesRaft\runnodes``, which will start the nodes in separate terminal windows/tabs.
    Wait until a "Node started up and registered in ..." message appears on each of the terminals
 3. Run ``gradlew samples:notary-demo:notarise`` to make a call to the "Party" node to initiate notarisation requests

docs/source/tools-index.rst

@@ -0,0 +1,11 @@
+Tools
+=====
+
+.. toctree::
+   :maxdepth: 1
+
+   network-simulator
+   demobench
+   node-explorer
+   azure-vm
+   loadtesting

docs/source/tutorial-contract.rst

@@ -30,7 +30,7 @@ value of the commercial paper.
 
 This lifecycle for commercial paper is illustrated in the diagram below:
 
-.. image:: contract-cp.png
+.. image:: resources/contract-cp.png
 
 Where to put your code
 ----------------------
@@ -97,7 +97,7 @@ States
 
 A state is a class that stores data that is checked by the contract. A commercial paper state is structured as below:
 
-.. image:: contract-cp-state.png
+.. image:: resources/contract-cp-state.png
 
 
 .. container:: codeset

docs/source/tutorial-tear-offs.rst

@@ -0,0 +1,80 @@
+Transaction tear-offs
+=====================
+
+Example of usage
+----------------
+Let’s focus on a code example. We want to construct a transaction with commands containing interest rate fix data as in:
+:doc:`oracles`. After construction of a partial transaction, with included ``Fix`` commands in it, we want to send it
+to the Oracle for checking and signing. To do so we need to specify which parts of the transaction are going to be
+revealed. That can be done by constructing filtering function over fields of ``WireTransaction`` of type ``(Any) ->
+Boolean``.
+
+.. container:: codeset
+
+   .. sourcecode:: kotlin
+
+        val partialTx = ...
+        val oracle: Party = ...
+        fun filtering(elem: Any): Boolean {
+                return when (elem) {
+                    is Command -> oracleParty.owningKey in elem.signers && elem.value is Fix
+                    else -> false
+                }
+        }
+
+Assuming that we already assembled partialTx with some commands and know the identity of Oracle service, we construct
+filtering function over commands - ``filtering``. It performs type checking and filters only ``Fix`` commands as in
+IRSDemo example. Then we can construct ``FilteredTransaction``:
+
+.. container:: codeset
+
+   .. sourcecode:: kotlin
+
+        val wtx: WireTransaction = partialTx.toWireTransaction()
+        val ftx: FilteredTransaction = wtx.buildFilteredTransaction(filtering)
+
+In the Oracle example this step takes place in ``RatesFixFlow`` by overriding ``filtering`` function, see:
+:ref:`filtering_ref`.
+
+``FilteredTransaction`` holds ``filteredLeaves`` (data that we wanted to reveal) and Merkle branch for them.
+
+.. container:: codeset
+
+   .. sourcecode:: kotlin
+
+        // Direct accsess to included commands, inputs, outputs, attachments etc.
+        val cmds: List<Command> = ftx.filteredLeaves.commands
+        val ins: List<StateRef> = ftx.filteredLeaves.inputs
+        val timestamp: Timestamp? = ftx.filteredLeaves.timestamp
+        ...
+
+.. literalinclude:: ../../samples/irs-demo/src/main/kotlin/net/corda/irs/api/NodeInterestRates.kt
+    :language: kotlin
+    :start-after: DOCSTART 1
+    :end-before: DOCEND 1
+
+Above code snippet is taken from ``NodeInterestRates.kt`` file and implements a signing part of an Oracle. You can
+check only leaves using ``leaves.checkWithFun { check(it) }`` and then verify obtained ``FilteredTransaction`` to see
+if data from ``PartialMerkleTree`` belongs to ``WireTransaction`` with provided ``id``. All you need is the root hash
+of the full transaction:
+
+.. container:: codeset
+
+   .. sourcecode:: kotlin
+
+        if (!ftx.verify(merkleRoot)){
+                throw MerkleTreeException("Rate Fix Oracle: Couldn't verify partial Merkle tree.")
+        }
+
+Or combine the two steps together:
+
+.. container:: codeset
+
+   .. sourcecode:: kotlin
+
+        ftx.verifyWithFunction(merkleRoot, ::check)
+
+.. note:: The way the ``FilteredTransaction`` is constructed ensures that after signing of the root hash it's impossible to add or remove
+    leaves. However, it can happen that having transaction with multiple commands one party reveals only subset of them to the Oracle.
+    As signing is done now over the Merkle root hash, the service signs all commands of given type, even though it didn't see
+    all of them. This issue will be handled after implementing partial signatures.

docs/source/tutorials-index.rst

@@ -0,0 +1,21 @@
+Tutorials
+=========
+
+.. toctree::
+   :maxdepth: 1
+
+   tutorial-contract
+   tutorial-contract-clauses
+   tutorial-test-dsl
+   contract-upgrade
+   tutorial-integration-testing
+   tutorial-clientrpc-api
+   tutorial-building-transactions
+   flow-state-machines
+   flow-testing
+   running-a-notary
+   using-a-notary
+   oracles
+   tutorial-tear-offs
+   tutorial-attachments
+   event-scheduling