Identity documentation (#1620)

* Sketch initial identity docs

* Restructure confidential identity docs to better fit structure

* Split confidential identities into API and concepts

* Further expansion on basic identity conceptS

* Merge Party type into api-identity.rst

* Address feedback on written content

* Rework inline code with literalinclude

* Start addressing feedback from Richard

* Clarify use of "counterparty"

* Address comments on key concepts

* Correct back to US english

* Clarify distribution/publishing of identities

* Update changelog around confidential identities

docs/source/api-core-types.rst

@@ -20,20 +20,6 @@ Any object that needs to be identified by its hash should implement the ``NamedB
 ``SecureHash`` is a sealed class that only defines a single subclass, ``SecureHash.SHA256``. There are utility methods
 to create and parse ``SecureHash.SHA256`` objects.
 
-Party
------
-Identities on the network are represented by ``AbstractParty``. There are two types of ``AbstractParty``:
-
-* ``Party``, identified by a ``PublicKey`` and a ``CordaX500Name``
-
-* ``AnonymousParty``, identified by a ``PublicKey``
-
-For example, in a transaction sent to your node as part of a chain of custody it is important you can convince yourself
-of the transaction's validity, but equally important that you don't learn anything about who was involved in that
-transaction. In these cases ``AnonymousParty`` should be used. In contrast, for internal processing where extended
-details of a party are required, the ``Party`` class should be used. The identity service provides functionality for
-resolving anonymous parties to full parties.
-
 CompositeKey
 ------------
 Corda supports scenarios where more than one signature is required to authorise a state object transition. For example:

docs/source/api-identity.rst

@@ -0,0 +1,108 @@
+API: Identity
+=============
+
+.. note:: Before reading this page, you should be familiar with the key concepts of :doc:`key-concepts-identity`.
+
+.. contents::
+
+Party
+-----
+Identities on the network are represented by ``AbstractParty``. There are two types of ``AbstractParty``:
+
+* ``Party``, identified by a ``PublicKey`` and a ``CordaX500Name``
+
+* ``AnonymousParty``, identified by a ``PublicKey``
+
+For example, in a transaction sent to your node as part of a chain of custody it is important you can convince yourself
+of the transaction's validity, but equally important that you don't learn anything about who was involved in that
+transaction. In these cases ``AnonymousParty`` should be used by flows constructing when transaction states and commands.
+In contrast, for internal processing where extended details of a party are required, the ``Party`` class should be used
+instead. The identity service provides functionality for flows to resolve anonymous parties to full parties, dependent
+on the anonymous party's identity having been registered with the node earlier (typically this is handled by
+``SwapIdentitiesFlow`` or ``IdentitySyncFlow``, discussed below).
+
+Party names are held within the ``CordaX500Name`` data class, which enforces the structure of names within Corda, as
+well as ensuring a consistent rendering of the names in plain text.
+
+The support for both Party and AnonymousParty classes in Corda enables sophisticated selective disclosure of identity
+information. For example, it is possible to construct a Transaction using an AnonymousParty, so nobody can learn of your
+involvement by inspection of the transaction, yet prove to specific counterparts that this AnonymousParty actually is
+owned by your well known identity. This disclosure is achieved through the use of the PartyAndCertificate data class
+which can be propagated to those who need to know, and contains the Party's X.509 certificate path to provide proof of
+ownership by a well known identity.
+
+The PartyAndCertificate class is also used in the network map service to represent well known identities, in which
+scenario the certificate path proves its issuance by the Doorman service.
+
+
+Confidential Identities
+-----------------------
+
+Confidential identities are key pairs where the corresponding X.509 certificate is not made public, so that parties who
+are not involved in the transaction cannot identify its participants. They are owned by a well known identity, which
+must sign the X.509 certificate. Before constructing a new transaction the involved parties must generate and send new
+confidential identities to each other, a process which managed using ``SwapIdentitiesFlow`` (discussed below). The
+public keys of these confidential identities are then used when generating output states and commands for the transaction.
+
+Where using outputs from a previous transaction in a new transaction, counterparties may need to know who the involved
+parties are. One example is in ``TwoPartyTradeFlow`` which delegates to ``CollectSignaturesFlow`` to gather certificates
+from both parties. ``CollectSignaturesFlow`` requires that a confidential identity of the initiating node has signed
+the transaction, and verifying this requires the receiving node has a copy of the confidential identity for the input
+state. ``IdentitySyncFlow`` can be used to synchronize the confidential identities we have the certificate paths for, in
+a single transaction, to another node.
+
+.. note:: ``CollectSignaturesFlow`` requires that the initiating node has signed the transaction, and as such all nodes
+   providing signatures must recognise the signing key used by the initiating node as being either its well known identity
+   or a confidential identity they have the certificate for.
+
+Swap identities flow
+~~~~~~~~~~~~~~~~~~~~
+
+``SwapIdentitiesFlow`` takes the party to swap identities with in its constructor (the counterparty), and is typically run as a subflow of
+another flow. It returns a mapping from well known identities of the calling flow and our counterparty to the new
+confidential identities; in future this will be extended to handle swapping identities with multiple parties.
+You can see an example of it being used in ``TwoPartyDealFlow.kt``:
+
+.. container:: codeset
+
+    .. literalinclude:: ../../finance/src/main/kotlin/net/corda/finance/flows/TwoPartyDealFlow.kt
+        :language: kotlin
+        :start-after: DOCSTART 2
+        :end-before: DOCEND 2
+
+The swap identities flow goes through the following key steps:
+
+1. Generate a nonce value.
+2. Send nonce value to all counterparties, and receive their nonce values.
+3. Generate a new confidential identity from our well known identity.
+4. Create a data blob containing the new confidential identity, plus the hash of the nonce values.
+5. Sign the resulting data blob with the confidential identity's public key.
+6. Send the confidential identity, data blob signature to all counterparties, while receiving theirs.
+7. Verify the signatures to ensure that identities were generated by the involved set of parties.
+8. Verify the confidential identities are owned by the expected well known identities.
+9. Store the confidential identities and return them to the calling flow.
+
+This ensures not only that the confidential identity certificates are signed by the correct well known identity, but
+also that the confidential identity private key is held by the counterparty.
+
+Identity synchronization flow
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When constructing a transaction whose input states reference confidential identities, it is common for other signing
+entities (counterparties) to require to know which well known identities those confidential identities map to. The
+``IdentitySyncFlow`` handles this process, and you can see an example of its use in ``TwoPartyTradeFlow.kt``:
+
+.. container:: codeset
+
+    .. literalinclude:: ../../finance/src/main/kotlin/net/corda/finance/flows/TwoPartyTradeFlow.kt
+        :language: kotlin
+        :start-after: DOCSTART 6
+        :end-before: DOCEND 6
+
+The identity synchronization flow goes through the following key steps:
+
+1. Extract participant identities from all input and output states and remove any well known identities.
+2. For each counterparty node, send a list of the public keys of the confidential identities, and receive back a list
+   of those the counterparty needs the certificate path for.
+3. Verify the requested list of identities contains only confidential identities in the offered list, and abort otherwise.
+4. Send the requested confidential identities as ``PartyAndCertificate`` instances to the counterparty.

docs/source/api-index.rst

@@ -12,6 +12,7 @@ This section describes the APIs that are available for the development of CorDap
    api-vault-query
    api-transactions
    api-flows
+   api-identity
    api-core-types
 
 Before reading this page, you should be familiar with the :doc:`key concepts of Corda <key-concepts>`.

docs/source/changelog.rst

@@ -26,8 +26,11 @@ Release 1.0
 * Added X509EdDSAEngine to intercept and rewrite EdDSA public keys wrapped in X509Key instances. This corrects an issue
   with verifying certificate paths loaded from a Java Keystore where they contain EdDSA keys.
 
-* generateSpend() now creates a new confidential identity for the change address rather than using the identity of the
-  input state owner.
+* Confidential identities are now complete:
+   * The identity negotiation flow is now called ``SwapIdentitiesFlow``, renamed from ``TransactionKeyFlow``.
+   * generateSpend() now creates a new confidential identity for the change address rather than using the identity of the
+     input state owner.
+   * Please see the documentation :doc:`key-concepts-identity` and :doc:`api-identity` for more details.
 
 * Remove the legacy web front end from the SIMM demo.
 

docs/source/key-concepts-identity.rst

@@ -0,0 +1,77 @@
+Identity
+========
+
+.. topic:: Summary
+
+   * *Identities in Corda can represent legal identities or service identities*
+   * *Identities are attested to by X.509 certificate signed by the Doorman or a well known identity*
+   * *Well known identities are published in the network map*
+   * *Confidential identities are only shared on a need to know basis*
+
+Identities in Corda can represent:
+
+* Legal identity of an organisation
+* Service identity of a network service
+
+Legal identities are used for parties in a transaction, such as the owner of a cash state. Service identities are used
+for those providing transaction-related services, such as notary, or oracle. Service identities are distinct to legal
+identities so that distributed services can exist on nodes owned by different organisations. Such distributed service
+identities are based on ``CompositeKeys``, which describe the valid sets of signers for a signature from the service.
+See :doc:`api-core-types` for more technical detail on composite keys.
+
+Identities are either well known or confidential, depending on whether their X.509 certificate (and corresponding
+certificate path to a trusted root certificate) is published:
+
+* Well known identities are the generally identifiable public key of a legal entity or service, which makes them
+  ill-suited to transactions where confidentiality of participants is required. This certificate is published in the
+  network map service for anyone to access.
+* Confidential identities are only published to those who are involved in transactions with the identity. The public
+  key may be exposed to third parties (for example to the notary service), but distribution of the name and X.500
+  certificate is limited.
+
+Although there are several elements to the Corda transaction privacy model, including ensuring that transactions are
+only shared with those who need to see them, and planned use of Intel SGX, it is important to provide defense in depth against
+privacy breaches. Confidential identities are used to ensure that even if a third party gets access to an unencrypted
+transaction, they cannot identify the participants without additional information.
+
+Name
+----
+
+Identity names are X.500 distinguished names with Corda-specific constraints applied. In order to be compatible with
+other implementations (particularly TLS implementations), we constrain the allowed X.500 attribute types to a subset of
+the minimum supported set for X.509 certificates (specified in RFC 3280), plus the locality attribute:
+
+* organization (O)
+* state (ST)
+* locality (L)
+* country (C)
+* organizational-unit (OU)
+* common name (CN) - used only for service identities
+
+The organisation, locality and country attributes are required, while state, organisational-unit and common name are
+optional. Attributes cannot be be present more than once in the name. The "country" code is strictly restricted to valid
+ISO 3166-1 two letter codes.
+
+Certificates
+------------
+
+Nodes must be able to verify the identity of the owner of a public key, which is achieved using X.509 certificates.
+When first run a node generates a key pair and submits a certificate signing request to the network Doorman service.
+The Doorman service applies appropriate identity checks then issues a certificate to the node, which is used as the
+node certificate authority (CA). From this initial CA certificate the node automatically creates and signs two further
+certificates, a TLS certificate and a signing certificate for the node's well known identity. Finally the node
+builds a node info record containing its address and well known identity, and registers it with the network map service.
+
+From the signing certificate the organisation can create both well known and confidential identities. Use-cases for
+well known identities include clusters of nodes representing a single identity for redundancy purposes, or creating
+identities for organisational units.
+
+It is up to organisations to decide which identities they wish to publish in the network map service, making them
+well known, and which they wish to keep as confidential identities for privacy reasons (typically to avoid exposing
+business sensitive details of transactions). In some cases nodes may also use private network map services in addition
+to the main network map service, for operational reasons. Identities registered with such network maps must be
+considered well known, and it is never appropriate to store confidential identities in a central directory without
+controls applied at the record level to ensure only those who require access to an identity can retrieve its
+certificate.
+
+.. TODO: Revisit once design & use cases of private maps is further fleshed out

docs/source/key-concepts.rst

@@ -13,6 +13,7 @@ This section should be read in order:
 
    key-concepts-ecosystem
    key-concepts-ledger
+   key-concepts-identity
    key-concepts-states
    key-concepts-contracts
    key-concepts-transactions

finance/src/main/kotlin/net/corda/finance/flows/TwoPartyDealFlow.kt

@@ -50,12 +50,14 @@ object TwoPartyDealFlow {
         abstract val notaryParty: Party
         abstract val otherSideSession: FlowSession
 
+        // DOCSTART 2
         @Suspendable
         override fun call(): SignedTransaction {
             progressTracker.currentStep = GENERATING_ID
             val txIdentities = subFlow(SwapIdentitiesFlow(otherSideSession.counterparty))
             val anonymousMe = txIdentities[ourIdentity] ?: ourIdentity.anonymise()
             val anonymousCounterparty = txIdentities[otherSideSession.counterparty] ?: otherSideSession.counterparty.anonymise()
+        // DOCEND 2
             progressTracker.currentStep = SENDING_PROPOSAL
             // Make the first message we'll send to kick off the flow.
             val hello = Handshake(payload, anonymousMe, anonymousCounterparty)

finance/src/main/kotlin/net/corda/finance/flows/TwoPartyTradeFlow.kt

@@ -169,6 +169,7 @@ object TwoPartyTradeFlow {
             progressTracker.currentStep = SIGNING
             val (ptx, cashSigningPubKeys) = assembleSharedTX(assetForSale, tradeRequest, buyerAnonymousIdentity)
 
+            // DOCSTART 6
             // Now sign the transaction with whatever keys we need to move the cash.
             val partSignedTx = serviceHub.signInitialTransaction(ptx, cashSigningPubKeys)
 
@@ -180,6 +181,7 @@ object TwoPartyTradeFlow {
             progressTracker.currentStep = COLLECTING_SIGNATURES
             val sellerSignature = subFlow(CollectSignatureFlow(partSignedTx, sellerSession, sellerSession.counterparty.owningKey))
             val twiceSignedTx = partSignedTx + sellerSignature
+            // DOCEND 6
 
             // Notarise and record the transaction.
             progressTracker.currentStep = RECORDING