ENT-992: Adding docs for the CRL/CRR (#608)

* Adding docs for the CRL/CRR

* Addressing review comments

* Addressing review comments - round 2

docs/source/certificate-revocation.rst

@@ -0,0 +1,57 @@
+Certificate Revocation List
+===========================
+
+The certificate revocation list consists of certificate serial numbers of issued certificates that are no longer valid.
+It is used by nodes when they establish a TLS connection between each other and need to ensure on certificate validity.
+In order to add entries to the certificate revocation list there is the certificate revocation process that resembles
+the one from the certificate signing request (CSR).
+Note that, once added the entries cannot be removed from the certificate revocation list.
+
+In the similar vein as CSR, it is integrated with the JIRA tool, and the submitted requests follow exactly the same lifecycle.
+To support the above functionality, there are two externally available REST endpoints: one for the certificate revocation request submission and
+one for the certificate revocation list retrieval.
+
+Since the certificate revocation list needs to be signed, the revocation process integrates with the HSM signing service.
+The certificate revocation list signing process requires human interaction and there is a separate tool designed for that purpose.
+Once signed the certificate revocation list replaces the current one.
+
+Note: It is assumed that the signed certificate revocation list is always available - even if it's empty.
+
+HTTP certificate revocation protocol
+------------------------------------
+
+The set of REST end-points for the revocation service are as follows.
+
++----------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+
+| Request method | Path                                    | Description                                                                                                                                  |
++================+=========================================+==============================================================================================================================================+
+| POST           | /certificate-revocation-request         | For the node to upload a certificate revocation request.                                                                                     |
++----------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+
+| GET            | /certificate-revocation-list/doorman    | For the node to obtain the certificate revocation list. Returns an ASN.1 DER-encoded java.security.cert.X509CRL object.                              |
++----------------+-----------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------+
+
+Submission of the certificate revocation requests expects the following fields to be present in the request payload:
+
+:certificateSerialNumber: Serial number of the certificate that is to be revoked.
+
+:csrRequestId: Certificate signing request identifier associated with the certificate that is to be revoked.
+
+:legalName: Legal name associated with the certificate that is to be revoked.
+
+:reason: Revocation reason (as specified in the java.security.cert.CRLReason).
+
+:reporter: Issuer of this certificate revocation request.
+
+Note: At least one of the three: certificateSerialNumber, csrRequestId or legalName needs to be specified.
+      Also, Corda AMQP serialization framework is used as the serialization framework.
+
+Because of the proprietary serialization mechanism, it is assumed that those endpoints are used by dedicated tools that support this kind of data encoding.
+
+
+Internal protocol
+-----------------
+
+There is an internal communication protocol between the revocation service and the HSM signing service for producing the signed CRLs.
+This does not use HTTP to avoid exposing any web vulnerabilities to the signing process.
+
+

docs/source/running-doorman.rst

@@ -35,11 +35,11 @@ Allowed parameters are:
 
 :doorman: Doorman specific configuration
 
-    :approveAll: Whether to approve all request (defaults to false), this is for debug only.
+    :approveAll: Whether to approve all requests (defaults to false), this is for debug only.
 
     :approveInterval: How often to process Jira approved requests in seconds.
 
-    :jira: The Jira configuration
+    :jira: The Jira configuration for certificate signing requests
 
         :address: The URL to use to connect to Jira
 
@@ -49,12 +49,38 @@ Allowed parameters are:
 
         :password: Password for Jira
 
+:revocation: Revocation service specific configuration
 
-:networkMap: Network map specific configuration
+        :localSigning: Configuration for local CRL signing using the file key store. If not defined t
 
-    :cacheTimeout: Network map cache entry expiry time (in milliseconds)
+            :crlUpdateInterval: Validity time of the issued certificate revocation lists (in milliseconds).
 
-    :signInterval: How often to sign the network map in seconds
+            :crlEndpoint: REST endpoint under which the certificate revocation list can be obtained.
+                          It is needed as this URL is encoded in the certificate revocation list itself.
+
+        :crlCacheTimeout: Certificate revocation list cache entry expiry time (in milliseconds).
+                          This value indicates for how long the crl is kept on the server side before querying the DB.
+
+        :approveInterval: How often to process Jira approved requests in seconds.
+                          Processing in this context means: querying the JIRA for approved/rejected request and syncing with the Doorman persistence.
+
+        :approveAll: Whether to approve all requests (defaults to false), this is for debug only.
+
+        :jira: The Jira configuration for certificate revocation requests
+
+            :address: The URL to use to connect to Jira
+
+            :projectCode: The project code on Jira
+
+            :username: Username for Jira
+
+            :password: Password for Jira
+
+:networkMap: Network map specific configuration.
+
+    :cacheTimeout: Network map cache entry expiry time (in milliseconds).
+
+    :signInterval: How often to sign the network map in seconds.
 
 :keystorePath: Path for the keystore. If not set (or null is passed) doorman will NOT perform any signing.
     This is required in case of the HSM integration where signing process is offloaded (from doorman) to an external service
@@ -80,7 +106,6 @@ Note that when reading from file:
 
 Bootstrapping the network map
 -----------------------------
-
 The network map is periodically refreshed, with frequency driven by the 'signInterval' parameter when local signing is in use.
 In case of an external signing service it depends on that service configuration. Due to the design decisions dictated by the security concerns
 around the external signing service, doorman is not allowed to connect directly with the signing sevice. Instead, the external service is
@@ -91,4 +116,18 @@ See the :doc:`signing-service` for a more detailed description of the service.
 
 When dealing with a fresh deployment (i.e. no previous data is present in the doorman database),
 it may take some time until the network map is available. This is caused by the aforementioned decoupling of the signing
-process from doorman itself.
+process from doorman itself.
+
+Bootstrapping the certificate revocation list
+---------------------------------------------
+Upon doorman startup, the revocation service becomes available serving the certificate revocation list and providing endpoints
+for certificate revocation request submission. It is assumed, that an empty signed CRL exists prior to the revocation service startup.
+The revocation service exposes its API in two ways: via REST endpoints and via sockets.
+While the former are meant to be used externally by network nodes (e.g. for the certificate revocation request submission,
+certificate revocation list retrieval...), the latter is designed for internal communication with other proprietary services (e.g. HSM signing service).
+The certificate revocation requests have the same lifecycle as the certificate signing requests.
+For that purpose (and in the same manner) the revocation service is integrated with JIRA which is configured according
+to the parameters specified in the doorman configuration file.
+As mentioned, the revocation service provides the certificate revocation list. The list itself is signed externally (i.e. HSM signing service).
+Therefore some delay, during the initial deployment of the service, is expected as those two services execute independently.
+

docs/source/running-signing-service.rst

@@ -46,6 +46,10 @@ Allowed parameters are:
 
     :crlDistributionPoint: Certificate revocation list location for the node CA certificate.
 
+    :crlServerSocketAddress: Address of the socket connection serving the certificate revocation list.
+
+    :crlUpdatePeriod: Validity time of the issued certificate revocation lists (in milliseconds).
+
     :authParameters: Authentication configuration for the CSR signing process.
 
         :mode: Authentication mode. Allowed values are: PASSWORD, CARD_READER and KEY_FILE

network-management/README.md

@@ -80,8 +80,8 @@ Doorman service can be started with the following options :
 
 ### JIRA
 
-The doorman service can use JIRA to manage the certificate signing request approval workflow. This can be turned on by providing
+The doorman service can use JIRA to manage both the certificate signing request and the certificate revocation request approval work flows.
-JIRA connection configuration in the config file.
+This can be turned on by providing JIRA connection configuration in the config file.
    ```
    doorman {
        jira {
@@ -151,6 +151,21 @@ doorman {
   }
 }
 
+# Comment out this section if running without the revocation service
+revocation {
+  approveInterval = 10000
+  approveAll = false
+  crlUpdateInterval = 86400000
+  crlEndpoint = "http://test.com/crl"
+  crlCacheTimeout = 86400000
+  jira {
+    address = "https://doorman-jira-host.com/"
+    projectCode = "CRR"
+    username = "username"
+    password = "password"
+  }
+}
+
 # Comment out this section if running without network map service
 networkMap {
   cacheTimeout = 600000

network-management/doorman.conf

@@ -20,13 +20,28 @@ database {
 
 h2port = 0
 
-# Comment out this section if running without doorman service
+# Comment out this section if running without the doorman service
 doorman {
   approveInterval = 10000
   approveAll = false
   jira {
     address = "https://doorman-jira-host.com/"
-    projectCode = "TD"
+    projectCode = "CSR"
+    username = "username"
+    password = "password"
+  }
+}
+
+# Comment out this section if running without the revocation service
+revocation {
+  approveInterval = 10000
+  approveAll = false
+  crlUpdateInterval = 86400000
+  crlEndpoint = "http://test.com/crl"
+  crlCacheTimeout = 86400000
+  jira {
+    address = "https://doorman-jira-host.com/"
+    projectCode = "CRR"
     username = "username"
     password = "password"
   }

network-management/src/integration-test/kotlin/com/r3/corda/networkmanage/doorman/NodeRegistrationTest.kt

@@ -16,7 +16,9 @@ import com.r3.corda.networkmanage.doorman.signer.LocalSigner
 import net.corda.cordform.CordformNode
 import net.corda.core.crypto.random63BitValue
 import net.corda.core.identity.CordaX500Name
-import net.corda.core.internal.*
+import net.corda.core.internal.div
+import net.corda.core.internal.exists
+import net.corda.core.internal.list
 import net.corda.core.messaging.startFlow
 import net.corda.core.utilities.OpaqueBytes
 import net.corda.core.utilities.getOrThrow
@@ -26,7 +28,8 @@ import net.corda.finance.flows.CashIssueAndPaymentFlow
 import net.corda.nodeapi.internal.createDevNetworkMapCa
 import net.corda.nodeapi.internal.crypto.CertificateAndKeyPair
 import net.corda.nodeapi.internal.persistence.DatabaseConfig
-import net.corda.testing.core.*
+import net.corda.testing.core.SerializationEnvironmentRule
+import net.corda.testing.core.singleIdentity
 import net.corda.testing.driver.NodeHandle
 import net.corda.testing.driver.PortAllocation
 import net.corda.testing.driver.internal.NodeHandleInternal
@@ -164,8 +167,10 @@ class NodeRegistrationTest : IntegrationTest() {
                         jira = null,
                         approveInterval = timeoutMillis,
                         crlCacheTimeout = timeoutMillis,
-                        crlEndpoint = URL("http://test.com/crl"),
+                        localSigning = CertificateRevocationConfig.LocalSigning(
-                        crlUpdateInterval = timeoutMillis),
+                                crlEndpoint = URL("http://test.com/crl"),
+                                crlUpdateInterval = timeoutMillis)
+                ),
                 if (startNetworkMap) {
                     NetworkMapStartParams(
                             LocalSigner(networkMapCa),

network-management/src/integration-test/kotlin/com/r3/corda/networkmanage/hsm/SigningServiceIntegrationTest.kt

@@ -107,10 +107,12 @@ class SigningServiceIntegrationTest : HsmBaseTest() {
                     revocationConfig = CertificateRevocationConfig(
                             approveAll = true,
                             jira = null,
-                            crlUpdateInterval = 2.hours.toMillis(),
                             crlCacheTimeout = 30.minutes.toMillis(),
-                            crlEndpoint = URL("http://test.com/crl"),
+                            approveInterval = 10.minutes.toMillis(),
-                            approveInterval = 10.minutes.toMillis()
+                            localSigning = CertificateRevocationConfig.LocalSigning(
+                                    crlEndpoint = URL("http://test.com/crl"),
+                                    crlUpdateInterval = 2.hours.toMillis()
+                            )
                     ),
                     startNetworkMap = null)
             val doormanHostAndPort = server.hostAndPort

network-management/src/main/kotlin/com/r3/corda/networkmanage/doorman/DoormanParameters.kt

@@ -57,20 +57,22 @@ data class DoormanConfig(val approveAll: Boolean = false,
 }
 
 data class CertificateRevocationConfig(val approveAll: Boolean = false,
-                         val jira: JiraConfig? = null,
+                                       val jira: JiraConfig? = null,
-                         val crlUpdateInterval: Long,
+                                       val localSigning: LocalSigning?,
-                         val crlEndpoint: URL,
+                                       val crlCacheTimeout: Long,
-                         val crlCacheTimeout: Long,
+                                       val approveInterval: Long = NetworkManagementServerConfig.DEFAULT_APPROVE_INTERVAL.toMillis()) {
-                         val approveInterval: Long = NetworkManagementServerConfig.DEFAULT_APPROVE_INTERVAL.toMillis()) {
     init {
         require(Booleans.countTrue(approveAll, jira != null) == 1) {
             "Either 'approveAll' or 'jira' config settings need to be specified but not both"
         }
     }
+
+    data class LocalSigning(val crlUpdateInterval: Long,
+                            val crlEndpoint: URL)
 }
 
 data class NetworkMapConfig(val cacheTimeout: Long,
-                            // TODO: Move signing to signing server.
+        // TODO: Move signing to signing server.
                             val signInterval: Long = NetworkManagementServerConfig.DEFAULT_SIGN_INTERVAL.toMillis())
 
 enum class Mode {

network-management/src/main/kotlin/com/r3/corda/networkmanage/doorman/NetworkManagementServer.kt

@@ -129,7 +129,11 @@ class NetworkManagementServer(dataSourceProperties: Properties, databaseConfig:
         val crlStorage = PersistentCertificateRevocationListStorage(database)
 
         val crlHandler = csrCertPathAndKeyPair?.let {
-            LocalCrlHandler(crrStorage, crlStorage, CertificateAndKeyPair(it.certPath.first(), it.toKeyPair()), Duration.ofMillis(config.crlUpdateInterval), config.crlEndpoint)
+            LocalCrlHandler(crrStorage,
+                    crlStorage,
+                    CertificateAndKeyPair(it.certPath.first(), it.toKeyPair()),
+                    Duration.ofMillis(config.localSigning!!.crlUpdateInterval),
+                    config.localSigning.crlEndpoint)
         }
 
         val jiraConfig = config.jira