diff --git a/docs/source/changelog.rst b/docs/source/changelog.rst index e67802aa20..ed402eb2c3 100644 --- a/docs/source/changelog.rst +++ b/docs/source/changelog.rst @@ -16,6 +16,29 @@ Unreleased * Vault storage of contract state constraints metadata and associated vault query functions to retrieve and sort by constraint type. +* UPGRADE REQUIRED: changes have been made to how notary implementations are configured and loaded. + No upgrade steps are required for the single-node notary (both validating and non-validating variants). + Other notary implementations have been moved out of the Corda node into individual Cordapps, and require configuration + file updates. + + To run a notary you will now need to include the appropriate notary CorDapp in the ``cordapps/`` directory: + + * ``corda-notary-raft`` for the Raft notary. + * ``corda-notary-bft-smart`` for the BFT-Smart notary. + + It is now required to specify the fully qualified notary service class name, ``className``, and the legal name of + the notary service in case of distributed notaries: ``serviceLegalName``. + + Implementation-specific configuration values have been moved to the ``extraConfig`` configuration block. + + Example configuration changes for the Raft notary: + + .. image:: resources/notary-config-update.png + + Example configuration changes for the BFT-Smart notary: + + .. image:: resources/notary-config-update-bft.png + * New overload for ``CordaRPCClient.start()`` method allowing to specify target legal identity to use for RPC call. * Case insensitive vault queries can be specified via a boolean on applicable SQL criteria builder operators. By default diff --git a/docs/source/corda-configuration-file.rst b/docs/source/corda-configuration-file.rst index fe4e7455b8..db98ef46da 100644 --- a/docs/source/corda-configuration-file.rst +++ b/docs/source/corda-configuration-file.rst @@ -151,34 +151,17 @@ absolute path to the node's base directory. :security: Contains various nested fields controlling user authentication/authorization, in particular for RPC accesses. See :doc:`clientrpc` for details. -:notary: Optional configuration object which if present configures the node to run as a notary. If part of a Raft or BFT SMaRt - cluster then specify ``raft`` or ``bftSMaRt`` respectively as described below. If a single node notary then omit both. +:notary: Optional configuration object which if present configures the node to run as a notary. :validating: Boolean to determine whether the notary is a validating or non-validating one. :serviceLegalName: If the node is part of a distributed cluster, specify the legal name of the cluster. At runtime, Corda checks whether this name matches the name of the certificate of the notary cluster. - :raft: If part of a distributed Raft cluster specify this config object, with the following settings: + :className: The fully qualified class name of the notary service to run. The class is expected to be loaded from + a notary CorDapp. Defaults to run the ``SimpleNotaryService``, which is built in. - :nodeAddress: The host and port to which to bind the embedded Raft server. Note that the Raft cluster uses a - separate transport layer for communication that does not integrate with ArtemisMQ messaging services. - - :clusterAddresses: Must list the addresses of all the members in the cluster. At least one of the members must - be active and be able to communicate with the cluster leader for the node to join the cluster. If empty, a - new cluster will be bootstrapped. - - :bftSMaRt: If part of a distributed BFT-SMaRt cluster specify this config object, with the following settings: - - :replicaId: The zero-based index of the current replica. All replicas must specify a unique replica id. - - :clusterAddresses: Must list the addresses of all the members in the cluster. At least one of the members must - be active and be able to communicate with the cluster leader for the node to join the cluster. If empty, a - new cluster will be bootstrapped. - - :custom: If `true`, will load and install a notary service from a CorDapp. See :doc:`tutorial-custom-notary`. - - Only one of ``raft``, ``bftSMaRt`` or ``custom`` configuration values may be specified. + :extraConfig: an optional configuration block for providing notary implementation-specific values. :rpcUsers: A list of users who are authorised to access the RPC system. Each user in the list is a config object with the following fields: diff --git a/docs/source/resources/notary-config-update-bft.png b/docs/source/resources/notary-config-update-bft.png new file mode 100644 index 0000000000..207d87cdef Binary files /dev/null and b/docs/source/resources/notary-config-update-bft.png differ diff --git a/docs/source/resources/notary-config-update.png b/docs/source/resources/notary-config-update.png new file mode 100644 index 0000000000..345c3854fe Binary files /dev/null and b/docs/source/resources/notary-config-update.png differ diff --git a/docs/source/running-a-node.rst b/docs/source/running-a-node.rst index 2c4b16dc19..b852778543 100644 --- a/docs/source/running-a-node.rst +++ b/docs/source/running-a-node.rst @@ -48,8 +48,6 @@ Command-line options The node can optionally be started with the following command-line options: * ``--base-directory``, ``-b``: The node working directory where all the files are kept (default: ``.``). -* ``--bootstrap-raft-cluster``: Bootstraps Raft cluster. The node forms a single node cluster (ignoring otherwise configured peer - addresses), acting as a seed for other nodes to join the cluster. * ``--clear-network-map-cache``, ``-c``: Clears local copy of network map, on node startup it will be restored from server or file system. * ``--config-file``, ``-f``: The path to the config file. Defaults to ``node.conf``. * ``--dev-mode``, ``-d``: Runs the node in developer mode. Unsafe in production. Defaults to true on MacOS and desktop versions of Windows. False otherwise. diff --git a/experimental/notary-bft-smart/src/main/kotlin/net/corda/notary/bftsmart/BFTSMaRtConfig.kt b/experimental/notary-bft-smart/src/main/kotlin/net/corda/notary/bftsmart/BFTSMaRtConfig.kt index d31e6fb203..a69985bafc 100644 --- a/experimental/notary-bft-smart/src/main/kotlin/net/corda/notary/bftsmart/BFTSMaRtConfig.kt +++ b/experimental/notary-bft-smart/src/main/kotlin/net/corda/notary/bftsmart/BFTSMaRtConfig.kt @@ -13,6 +13,24 @@ import java.net.SocketException import java.nio.file.Files import java.util.concurrent.TimeUnit.MILLISECONDS +data class BFTSMaRtConfiguration( + /** The zero-based index of the current replica. All replicas must specify a unique replica id. */ + val replicaId: Int, + /** + * Must list the addresses of all the members in the cluster. At least one of the members must be active and + * be able to communicate with the cluster leader for the node to join the cluster. If empty, + * a new cluster will be bootstrapped. + */ + val clusterAddresses: List, + val debug: Boolean = false, + /** Used for testing purposes only. */ + val exposeRaces: Boolean = false +) { + init { + require(replicaId >= 0) { "replicaId cannot be negative" } + } +} + /** * BFT SMaRt can only be configured via files in a configHome directory. * Each instance of this class creates such a configHome, accessible via [path]. diff --git a/experimental/notary-bft-smart/src/main/kotlin/net/corda/notary/bftsmart/BftSmartNotaryService.kt b/experimental/notary-bft-smart/src/main/kotlin/net/corda/notary/bftsmart/BftSmartNotaryService.kt index 391b92a630..905a8fe569 100644 --- a/experimental/notary-bft-smart/src/main/kotlin/net/corda/notary/bftsmart/BftSmartNotaryService.kt +++ b/experimental/notary-bft-smart/src/main/kotlin/net/corda/notary/bftsmart/BftSmartNotaryService.kt @@ -20,9 +20,9 @@ import net.corda.core.utilities.debug import net.corda.core.utilities.getOrThrow import net.corda.core.utilities.unwrap import net.corda.node.services.api.ServiceHubInternal -import net.corda.node.services.config.BFTSMaRtConfiguration import net.corda.node.services.transactions.PersistentUniquenessProvider import net.corda.node.utilities.AppendOnlyPersistentMap +import net.corda.nodeapi.internal.config.parseAs import net.corda.nodeapi.internal.persistence.NODE_DATABASE_PREFIX import java.security.PublicKey import javax.persistence.Entity @@ -54,10 +54,13 @@ class BftSmartNotaryService( } private val notaryConfig = services.configuration.notary - ?: throw IllegalArgumentException("Failed to register ${this::class.java}: notary configuration not present") + ?: throw IllegalArgumentException("Failed to register ${BftSmartNotaryService::class.java}: notary configuration not present") - private val bftSMaRtConfig = notaryConfig.bftSMaRt - ?: throw IllegalArgumentException("Failed to register ${this::class.java}: raft configuration not present") + private val bftSMaRtConfig = try { + notaryConfig.extraConfig!!.parseAs() + } catch (e: Exception) { + throw IllegalArgumentException("Failed to register ${BftSmartNotaryService::class.java}: BFT-Smart configuration not present") + } private val cluster: BFTSMaRt.Cluster = makeBFTCluster(notaryIdentityKey, bftSMaRtConfig) diff --git a/experimental/notary-bft-smart/src/test/kotlin/net/corda/notary/bftsmart/BFTNotaryServiceTests.kt b/experimental/notary-bft-smart/src/test/kotlin/net/corda/notary/bftsmart/BFTNotaryServiceTests.kt index 2fc3806206..abe6a359bf 100644 --- a/experimental/notary-bft-smart/src/test/kotlin/net/corda/notary/bftsmart/BFTNotaryServiceTests.kt +++ b/experimental/notary-bft-smart/src/test/kotlin/net/corda/notary/bftsmart/BFTNotaryServiceTests.kt @@ -21,17 +21,17 @@ import net.corda.core.utilities.NetworkHostAndPort import net.corda.core.utilities.Try import net.corda.core.utilities.getOrThrow import net.corda.core.utilities.seconds -import net.corda.node.services.config.BFTSMaRtConfiguration import net.corda.node.services.config.NotaryConfig import net.corda.nodeapi.internal.DevIdentityGenerator +import net.corda.nodeapi.internal.config.toConfig import net.corda.nodeapi.internal.network.NetworkParametersCopier import net.corda.testing.common.internal.testNetworkParameters import net.corda.testing.contracts.DummyContract import net.corda.testing.core.dummyCommand import net.corda.testing.core.singleIdentity -import net.corda.testing.node.TestClock import net.corda.testing.internal.IntegrationTest import net.corda.testing.internal.IntegrationTestSchemas +import net.corda.testing.node.TestClock import net.corda.testing.node.internal.* import org.hamcrest.Matchers.instanceOf import org.junit.AfterClass @@ -64,7 +64,7 @@ class BFTNotaryServiceTests { @JvmStatic fun before() { IntegrationTest.globalSetUp() //Enterprise only - remote db setup - mockNet = InternalMockNetwork(cordappsForAllNodes = cordappsForPackages("net.corda.testing.contracts")) + mockNet = InternalMockNetwork(cordappsForAllNodes = cordappsForPackages("net.corda.testing.contracts", "net.corda.notary.bftsmart")) val clusterSize = minClusterSize(1) val started = startBftClusterAndNode(clusterSize, mockNet) notary = started.first @@ -81,10 +81,10 @@ class BFTNotaryServiceTests { fun startBftClusterAndNode(clusterSize: Int, mockNet: InternalMockNetwork, exposeRaces: Boolean = false): Pair { (Paths.get("config") / "currentView").deleteIfExists() // XXX: Make config object warn if this exists? val replicaIds = (0 until clusterSize) - + val serviceLegalName = CordaX500Name("BFT", "Zurich", "CH") val notaryIdentity = DevIdentityGenerator.generateDistributedNotaryCompositeIdentity( replicaIds.map { mockNet.baseDirectory(mockNet.nextNodeId + it) }, - CordaX500Name("BFT", "Zurich", "CH")) + serviceLegalName) val networkParameters = NetworkParametersCopier(testNetworkParameters(listOf(NotaryInfo(notaryIdentity, false)))) @@ -94,8 +94,9 @@ class BFTNotaryServiceTests { mockNet.createUnstartedNode(InternalMockNodeParameters(configOverrides = { val notary = NotaryConfig( validating = false, - bftSMaRt = BFTSMaRtConfiguration(replicaId, clusterAddresses, exposeRaces = exposeRaces), - className = "net.corda.notary.bftsmart.BftSmartNotaryService" + extraConfig = BFTSMaRtConfiguration(replicaId, clusterAddresses, exposeRaces = exposeRaces).toConfig(), + className = "net.corda.notary.bftsmart.BftSmartNotaryService", + serviceLegalName = serviceLegalName ) doReturn(notary).whenever(it).notary })) diff --git a/experimental/notary-raft/src/main/kotlin/net/corda/notary/raft/RaftConfig.kt b/experimental/notary-raft/src/main/kotlin/net/corda/notary/raft/RaftConfig.kt new file mode 100644 index 0000000000..75996a6ebf --- /dev/null +++ b/experimental/notary-raft/src/main/kotlin/net/corda/notary/raft/RaftConfig.kt @@ -0,0 +1,18 @@ +package net.corda.notary.raft + +import net.corda.core.utilities.NetworkHostAndPort + +/** Configuration properties specific to the RaftNotaryService. */ +data class RaftConfig( + /** + * The host and port to which to bind the embedded Raft server. Note that the Raft cluster uses a + * separate transport layer for communication that does not integrate with ArtemisMQ messaging services. + */ + val nodeAddress: NetworkHostAndPort, + /** + * Must list the addresses of all the members in the cluster. At least one of the members mustbe active and + * be able to communicate with the cluster leader for the node to join the cluster. If empty, a new cluster + * will be bootstrapped. + */ + val clusterAddresses: List +) \ No newline at end of file diff --git a/experimental/notary-raft/src/main/kotlin/net/corda/notary/raft/RaftNotaryService.kt b/experimental/notary-raft/src/main/kotlin/net/corda/notary/raft/RaftNotaryService.kt index 3913551802..97843cb13c 100644 --- a/experimental/notary-raft/src/main/kotlin/net/corda/notary/raft/RaftNotaryService.kt +++ b/experimental/notary-raft/src/main/kotlin/net/corda/notary/raft/RaftNotaryService.kt @@ -6,6 +6,7 @@ import net.corda.core.internal.notary.TrustedAuthorityNotaryService import net.corda.node.services.api.ServiceHubInternal import net.corda.node.services.transactions.NonValidatingNotaryFlow import net.corda.node.services.transactions.ValidatingNotaryFlow +import net.corda.nodeapi.internal.config.parseAs import java.security.PublicKey /** A highly available notary service using the Raft algorithm to achieve consensus. */ @@ -14,11 +15,14 @@ class RaftNotaryService( override val notaryIdentityKey: PublicKey ) : TrustedAuthorityNotaryService() { private val notaryConfig = services.configuration.notary - ?: throw IllegalArgumentException("Failed to register ${this::class.java}: notary configuration not present") + ?: throw IllegalArgumentException("Failed to register ${RaftNotaryService::class.java}: notary configuration not present") override val uniquenessProvider = with(services) { - val raftConfig = notaryConfig.raft - ?: throw IllegalArgumentException("Failed to register ${this::class.java}: raft configuration not present") + val raftConfig = try { + notaryConfig.extraConfig!!.parseAs() + } catch (e: Exception) { + throw IllegalArgumentException("Failed to register ${RaftNotaryService::class.java}: raft configuration not present") + } RaftUniquenessProvider( configuration.baseDirectory, configuration.p2pSslOptions, diff --git a/experimental/notary-raft/src/main/kotlin/net/corda/notary/raft/RaftUniquenessProvider.kt b/experimental/notary-raft/src/main/kotlin/net/corda/notary/raft/RaftUniquenessProvider.kt index e9aac5a99e..f3b7671a3e 100644 --- a/experimental/notary-raft/src/main/kotlin/net/corda/notary/raft/RaftUniquenessProvider.kt +++ b/experimental/notary-raft/src/main/kotlin/net/corda/notary/raft/RaftUniquenessProvider.kt @@ -26,7 +26,6 @@ import net.corda.core.serialization.SingletonSerializeAsToken import net.corda.core.serialization.serialize import net.corda.core.utilities.contextLogger import net.corda.core.utilities.debug -import net.corda.node.services.config.RaftConfig import net.corda.node.utilities.AppendOnlyPersistentMap import net.corda.nodeapi.internal.config.MutualSslConfiguration import net.corda.nodeapi.internal.persistence.CordaPersistence diff --git a/node-api/src/main/kotlin/net/corda/nodeapi/internal/network/NetworkBootstrapper.kt b/node-api/src/main/kotlin/net/corda/nodeapi/internal/network/NetworkBootstrapper.kt index 1747cc2333..c581577c57 100644 --- a/node-api/src/main/kotlin/net/corda/nodeapi/internal/network/NetworkBootstrapper.kt +++ b/node-api/src/main/kotlin/net/corda/nodeapi/internal/network/NetworkBootstrapper.kt @@ -126,8 +126,8 @@ internal constructor(private val initSerEnv: Boolean, } return clusteredNotaries.groupBy { it.first }.map { (k, vs) -> val cs = vs.map { it.second.config } - if (cs.any { it.hasPath("notary.bftSMaRt") }) { - require(cs.all { it.hasPath("notary.bftSMaRt") }) { "Mix of BFT and non-BFT notaries with service name $k" } + if (cs.any { isBFTNotary(it) }) { + require(cs.all { isBFTNotary(it) }) { "Mix of BFT and non-BFT notaries with service name $k" } NotaryCluster.BFT(k) to vs.map { it.second.directory } } else { NotaryCluster.CFT(k) to vs.map { it.second.directory } @@ -135,6 +135,12 @@ internal constructor(private val initSerEnv: Boolean, }.toMap() } + private fun isBFTNotary(config: Config): Boolean { + // TODO: pass a commandline parameter to the bootstrapper instead. Better yet, a notary config map + // specifying the notary identities and the type (single-node, CFT, BFT) of each notary to set up. + return config.getString ("notary.className").contains("BFT", true) + } + private fun generateServiceIdentitiesForNotaryClusters(configs: Map) { notaryClusters(configs).forEach { (cluster, directories) -> when (cluster) { diff --git a/node/src/main/kotlin/net/corda/node/NodeCmdLineOptions.kt b/node/src/main/kotlin/net/corda/node/NodeCmdLineOptions.kt index 6d04567d9d..4f4ba98c1a 100644 --- a/node/src/main/kotlin/net/corda/node/NodeCmdLineOptions.kt +++ b/node/src/main/kotlin/net/corda/node/NodeCmdLineOptions.kt @@ -88,12 +88,6 @@ class NodeCmdLineOptions { ) var justGenerateRpcSslCerts: Boolean = false - @Option( - names = ["--bootstrap-raft-cluster"], - description = ["Bootstraps Raft cluster. The node forms a single node cluster (ignoring otherwise configured peer addresses), acting as a seed for other nodes to join the cluster."] - ) - var bootstrapRaftCluster: Boolean = false - @Option( names = ["-c", "--clear-network-map-cache"], description = ["Clears local copy of network map, on node startup it will be restored from server or file system."] diff --git a/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt b/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt index 66eb3cef1b..fa61074df4 100644 --- a/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt +++ b/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt @@ -270,7 +270,7 @@ abstract class AbstractNode(val configuration: NodeConfiguration, check(started == null) { "Node has already been started" } log.info("Generating nodeInfo ...") val trustRoot = initKeyStores() - val (identity, identityKeyPair) = obtainIdentity(notaryConfig = null) + val (identity, identityKeyPair) = obtainIdentity() startDatabase() val nodeCa = configuration.signingCertificateStore.get()[CORDA_CLIENT_CA] identityService.start(trustRoot, listOf(identity.certificate, nodeCa)) @@ -325,7 +325,7 @@ abstract class AbstractNode(val configuration: NodeConfiguration, networkMapCache.start(netParams.notaries) startDatabase() - val (identity, identityKeyPair) = obtainIdentity(notaryConfig = null) + val (identity, identityKeyPair) = obtainIdentity() identityService.start(trustRoot, listOf(identity.certificate, nodeCa)) val mutualExclusionConfiguration = configuration.enterpriseConfiguration.mutualExclusionConfiguration @@ -417,8 +417,8 @@ abstract class AbstractNode(val configuration: NodeConfiguration, val keyPairs = mutableSetOf(identityKeyPair) val myNotaryIdentity = configuration.notary?.let { - if (it.isClusterConfig) { - val (notaryIdentity, notaryIdentityKeyPair) = obtainIdentity(it) + if (it.serviceLegalName != null) { + val (notaryIdentity, notaryIdentityKeyPair) = loadNotaryClusterIdentity(it.serviceLegalName) keyPairs += notaryIdentityKeyPair notaryIdentity } else { @@ -870,24 +870,14 @@ abstract class AbstractNode(val configuration: NodeConfiguration, myNotaryIdentity: PartyAndCertificate?, networkParameters: NetworkParameters) - private fun obtainIdentity(notaryConfig: NotaryConfig?): Pair { + /** Loads or generates the node's legal identity and key-pair. */ + private fun obtainIdentity(): Pair { val keyStore = configuration.signingCertificateStore.get() + val legalName = configuration.myLegalName - val (id, singleName) = if (notaryConfig == null || !notaryConfig.isClusterConfig) { - // Node's main identity or if it's a single node notary. - Pair(NODE_IDENTITY_ALIAS_PREFIX, configuration.myLegalName) - } else { - // The node is part of a distributed notary whose identity must already be generated beforehand. - Pair(DISTRIBUTED_NOTARY_ALIAS_PREFIX, null) - } // TODO: Integrate with Key management service? - val privateKeyAlias = "$id-private-key" - + val privateKeyAlias = "$NODE_IDENTITY_ALIAS_PREFIX-private-key" if (privateKeyAlias !in keyStore) { - // We shouldn't have a distributed notary at this stage, so singleName should NOT be null. - requireNotNull(singleName) { - "Unable to find in the key store the identity of the distributed notary the node is part of" - } log.info("$privateKeyAlias not found in key store, generating fresh key!") keyStore.storeLegalIdentity(privateKeyAlias, generateKeyPair()) } @@ -895,30 +885,41 @@ abstract class AbstractNode(val configuration: NodeConfiguration, val (x509Cert, keyPair) = keyStore.query { getCertificateAndKeyPair(privateKeyAlias, keyStore.entryPassword) } // TODO: Use configuration to indicate composite key should be used instead of public key for the identity. - val compositeKeyAlias = "$id-composite-key" + val certificates = keyStore.query { getCertificateChain(privateKeyAlias) } + check(certificates.first() == x509Cert) { + "Certificates from key store do not line up!" + } + + val subject = CordaX500Name.build(certificates.first().subjectX500Principal) + if (subject != legalName) { + throw ConfigurationException("The name '$legalName' for $NODE_IDENTITY_ALIAS_PREFIX doesn't match what's in the key store: $subject") + } + + val certPath = X509Utilities.buildCertPath(certificates) + return Pair(PartyAndCertificate(certPath), keyPair) + } + + /** Loads pre-generated notary service cluster identity. */ + private fun loadNotaryClusterIdentity(serviceLegalName: CordaX500Name): Pair { + val keyStore = configuration.signingCertificateStore.get() + + val privateKeyAlias = "$DISTRIBUTED_NOTARY_ALIAS_PREFIX-private-key" + val keyPair = keyStore.query { getCertificateAndKeyPair(privateKeyAlias, keyStore.entryPassword) }.keyPair + + val compositeKeyAlias = "$DISTRIBUTED_NOTARY_ALIAS_PREFIX-composite-key" val certificates = if (compositeKeyAlias in keyStore) { - // Use composite key instead if it exists. val certificate = keyStore[compositeKeyAlias] // We have to create the certificate chain for the composite key manually, this is because we don't have a keystore // provider that understand compositeKey-privateKey combo. The cert chain is created using the composite key certificate + // the tail of the private key certificates, as they are both signed by the same certificate chain. listOf(certificate) + keyStore.query { getCertificateChain(privateKeyAlias) }.drop(1) - } else { - keyStore.query { getCertificateChain(privateKeyAlias) }.let { - check(it[0] == x509Cert) { "Certificates from key store do not line up!" } - it - } - } + } else throw IllegalStateException("The identity public key for the notary service $serviceLegalName was not found in the key store.") - val subject = CordaX500Name.build(certificates[0].subjectX500Principal) - if (singleName != null && subject != singleName) { - throw ConfigurationException("The name '$singleName' for $id doesn't match what's in the key store: $subject") - } else if (notaryConfig != null && notaryConfig.isClusterConfig && notaryConfig.serviceLegalName != null && subject != notaryConfig.serviceLegalName) { - // Note that we're not checking if `notaryConfig.serviceLegalName` is not present for backwards compatibility. - throw ConfigurationException("The name of the notary service '${notaryConfig.serviceLegalName}' for $id doesn't " + + val subject = CordaX500Name.build(certificates.first().subjectX500Principal) + if (subject != serviceLegalName) { + throw ConfigurationException("The name of the notary service '$serviceLegalName' for $DISTRIBUTED_NOTARY_ALIAS_PREFIX doesn't " + "match what's in the key store: $subject. You might need to adjust the configuration of `notary.serviceLegalName`.") } - val certPath = X509Utilities.buildCertPath(certificates) return Pair(PartyAndCertificate(certPath), keyPair) } diff --git a/node/src/main/kotlin/net/corda/node/internal/NodeStartup.kt b/node/src/main/kotlin/net/corda/node/internal/NodeStartup.kt index 42914cbb8e..33444e6f2d 100644 --- a/node/src/main/kotlin/net/corda/node/internal/NodeStartup.kt +++ b/node/src/main/kotlin/net/corda/node/internal/NodeStartup.kt @@ -18,7 +18,6 @@ import net.corda.node.* import net.corda.node.internal.Node.Companion.isValidJavaVersion import net.corda.node.internal.cordapp.MultipleCordappsForFlowException import net.corda.node.services.config.NodeConfiguration -import net.corda.node.services.config.NodeConfigurationImpl import net.corda.node.services.config.shouldStartLocalShell import net.corda.node.services.config.shouldStartSSHDaemon import net.corda.node.utilities.createKeyPairAndSelfSignedTLSCertificate @@ -27,7 +26,6 @@ import net.corda.node.utilities.registration.NodeRegistrationException import net.corda.node.utilities.registration.NodeRegistrationHelper import net.corda.node.utilities.saveToKeyStore import net.corda.node.utilities.saveToTrustStore -import net.corda.core.internal.PLATFORM_VERSION import net.corda.nodeapi.internal.addShutdownHook import net.corda.nodeapi.internal.config.UnknownConfigurationKeysException import net.corda.nodeapi.internal.persistence.CouldNotCreateDataSourceException @@ -191,14 +189,7 @@ open class NodeStartup : CordaCliWrapper("corda", "Runs a Corda Node") { if (cmdLineOptions.devMode == true) { println("Config:\n${rawConfig.root().render(ConfigRenderOptions.defaults())}") } - val configuration = configurationResult.getOrThrow() - return if (cmdLineOptions.bootstrapRaftCluster) { - println("Bootstrapping raft cluster (starting up as seed node).") - // Ignore the configured clusterAddresses to make the node bootstrap a cluster instead of joining. - (configuration as NodeConfigurationImpl).copy(notary = configuration.notary?.copy(raft = configuration.notary?.raft?.copy(clusterAddresses = emptyList()))) - } else { - configuration - } + return configurationResult.getOrThrow() } private fun checkRegistrationMode(): Boolean { @@ -300,12 +291,12 @@ open class NodeStartup : CordaCliWrapper("corda", "Runs a Corda Node") { val console: Console? = System.console() when (console) { - // In this case, the JVM is not connected to the console so we need to exit. + // In this case, the JVM is not connected to the console so we need to exit. null -> { println("Not connected to console. Exiting") exitProcess(1) } - // Otherwise we can proceed normally. + // Otherwise we can proceed normally. else -> { while (true) { val keystorePassword1 = console.readPassword("Enter the RPC keystore password => ") diff --git a/node/src/main/kotlin/net/corda/node/services/config/NodeConfiguration.kt b/node/src/main/kotlin/net/corda/node/services/config/NodeConfiguration.kt index 5bc625c0b5..2965180898 100644 --- a/node/src/main/kotlin/net/corda/node/services/config/NodeConfiguration.kt +++ b/node/src/main/kotlin/net/corda/node/services/config/NodeConfiguration.kt @@ -11,12 +11,7 @@ import net.corda.core.utilities.loggerFor import net.corda.core.utilities.seconds import net.corda.node.services.config.rpc.NodeRpcOptions import net.corda.nodeapi.BrokerRpcSslOptions -import net.corda.nodeapi.internal.config.FileBasedCertificateStoreSupplier -import net.corda.nodeapi.internal.config.SslConfiguration -import net.corda.nodeapi.internal.config.MutualSslConfiguration -import net.corda.nodeapi.internal.config.UnknownConfigKeysPolicy -import net.corda.nodeapi.internal.config.User -import net.corda.nodeapi.internal.config.parseAs +import net.corda.nodeapi.internal.config.* import net.corda.nodeapi.internal.persistence.CordaPersistence.DataSourceConfigTag import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.tools.shell.SSHDConfiguration @@ -79,7 +74,7 @@ interface NodeConfiguration { val flowMonitorPeriodMillis: Duration get() = DEFAULT_FLOW_MONITOR_PERIOD_MILLIS val flowMonitorSuspensionLoggingThresholdMillis: Duration get() = DEFAULT_FLOW_MONITOR_SUSPENSION_LOGGING_THRESHOLD_MILLIS val crlCheckSoftFail: Boolean - val jmxReporterType : JmxReporterType? get() = defaultJmxReporterType + val jmxReporterType: JmxReporterType? get() = defaultJmxReporterType val baseDirectory: Path val certificatesDirectory: Path @@ -132,71 +127,16 @@ fun NodeConfiguration.shouldStartSSHDaemon() = this.sshd != null fun NodeConfiguration.shouldStartLocalShell() = !this.noLocalShell && System.console() != null && this.devMode fun NodeConfiguration.shouldInitCrashShell() = shouldStartLocalShell() || shouldStartSSHDaemon() -data class NotaryConfig(val validating: Boolean, - val raft: RaftConfig? = null, - val bftSMaRt: BFTSMaRtConfiguration? = null, - val mysql: MySQLConfiguration? = null, - val serviceLegalName: CordaX500Name? = null, - val className: String = "net.corda.node.services.transactions.SimpleNotaryService", - val batchSize: Int = 128, - val batchTimeoutMs: Long = 1L, - val maxInputStates: Int = 2000, - val maxDBTransactionRetryCount: Int = 10, - val backOffBaseMs: Long = 20L -) { - init { - require(raft == null || bftSMaRt == null || mysql == null) { - "raft, bftSMaRt, and mysql configs cannot be specified together" - } - } - - val isClusterConfig: Boolean get() = raft != null || bftSMaRt != null || mysql != null -} - -data class MySQLConfiguration( - val dataSource: Properties, - /** - * Number of times to attempt to reconnect to the database. - */ - val connectionRetries: Int = 2, // Default value for a 3 server cluster. - /** - * Time increment between re-connection attempts. - * - * The total back-off duration is calculated as: backOffIncrement * backOffBase ^ currentRetryCount - */ - val backOffIncrement: Int = 500, - /** Exponential back-off multiplier base. */ - val backOffBase: Double = 1.5, - /** The maximum number of transactions processed in a single batch. */ - val maxBatchSize: Int = 500, - /** The maximum combined number of input states processed in a single batch. */ - val maxBatchInputStates: Int = 10_000, - /** A batch will be processed after a specified timeout even if it has not yet reached full capacity. */ - val batchTimeoutMs: Long = 200, - /** - * The maximum number of commit requests in flight. Once the capacity is reached the service will block on - * further commit requests. - */ - val maxQueueSize: Int = 100_000 -) { - init { - require(connectionRetries >= 0) { "connectionRetries cannot be negative" } - } -} - -data class RaftConfig(val nodeAddress: NetworkHostAndPort, val clusterAddresses: List) - -/** @param exposeRaces for testing only, so its default is not in reference.conf but here. */ -data class BFTSMaRtConfiguration( - val replicaId: Int, - val clusterAddresses: List, - val debug: Boolean = false, - val exposeRaces: Boolean = false -) { - init { - require(replicaId >= 0) { "replicaId cannot be negative" } - } -} +data class NotaryConfig( + /** Specifies whether the notary validates transactions or not. */ + val validating: Boolean, + /** The legal name of cluster in case of a distributed notary service. */ + val serviceLegalName: CordaX500Name? = null, + /** The name of the notary service class to load. */ + val className: String = "net.corda.node.services.transactions.SimpleNotaryService", + /** Notary implementation-specific configuration parameters. */ + val extraConfig: Config? = null +) /** * Used as an alternative to the older compatibilityZoneURL to allow the doorman and network map @@ -217,7 +157,7 @@ data class NetworkServicesConfig( val doormanURL: URL, val networkMapURL: URL, val pnm: UUID? = null, - val inferred : Boolean = false + val inferred: Boolean = false ) /** @@ -416,7 +356,7 @@ data class NodeConfigurationImpl( override val effectiveH2Settings: NodeH2Settings? get() = when { - h2port != null -> NodeH2Settings(address = NetworkHostAndPort(host="localhost", port=h2port)) + h2port != null -> NodeH2Settings(address = NetworkHostAndPort(host = "localhost", port = h2port)) else -> h2Settings } @@ -453,7 +393,7 @@ data class NodeConfigurationImpl( // Check for usage of deprecated config @Suppress("DEPRECATION") - if(certificateChainCheckPolicies.isNotEmpty()) { + if (certificateChainCheckPolicies.isNotEmpty()) { logger.warn("""You are configuring certificateChainCheckPolicies. This is a setting that is not used, and will be removed in a future version. |Please contact the R3 team on the public slack to discuss your use case. """.trimMargin()) diff --git a/node/src/main/resources/net.corda.node.internal.NodeStartup.yml b/node/src/main/resources/net.corda.node.internal.NodeStartup.yml index a67e7cee5a..28f0651a78 100644 --- a/node/src/main/resources/net.corda.node.internal.NodeStartup.yml +++ b/node/src/main/resources/net.corda.node.internal.NodeStartup.yml @@ -6,11 +6,6 @@ required: false multiParam: true acceptableValues: [] - - parameterName: "--bootstrap-raft-cluster" - parameterType: "boolean" - required: false - multiParam: false - acceptableValues: [] - parameterName: "--clear-network-map-cache" parameterType: "boolean" required: false diff --git a/node/src/test/kotlin/net/corda/node/internal/NodeStartupTest.kt b/node/src/test/kotlin/net/corda/node/internal/NodeStartupTest.kt index 947c6f0e73..ec1da7a226 100644 --- a/node/src/test/kotlin/net/corda/node/internal/NodeStartupTest.kt +++ b/node/src/test/kotlin/net/corda/node/internal/NodeStartupTest.kt @@ -35,7 +35,6 @@ class NodeStartupTest { assertThat(startup.cmdLineOptions.sshdServer).isEqualTo(false) assertThat(startup.cmdLineOptions.justGenerateNodeInfo).isEqualTo(false) assertThat(startup.cmdLineOptions.justGenerateRpcSslCerts).isEqualTo(false) - assertThat(startup.cmdLineOptions.bootstrapRaftCluster).isEqualTo(false) assertThat(startup.cmdLineOptions.unknownConfigKeysPolicy).isEqualTo(UnknownConfigKeysPolicy.FAIL) assertThat(startup.cmdLineOptions.devMode).isEqualTo(null) assertThat(startup.cmdLineOptions.clearNetworkMapCache).isEqualTo(false) diff --git a/node/src/test/kotlin/net/corda/node/services/TimedFlowTests.kt b/node/src/test/kotlin/net/corda/node/services/TimedFlowTests.kt index 0983cb064c..97fb55379f 100644 --- a/node/src/test/kotlin/net/corda/node/services/TimedFlowTests.kt +++ b/node/src/test/kotlin/net/corda/node/services/TimedFlowTests.kt @@ -52,15 +52,16 @@ class TimedFlowTestRule(val clusterSize: Int) : ExternalResource() { lateinit var notary: Party lateinit var node: TestStartedNode - private fun startClusterAndNode(mockNet: InternalMockNetwork): Pair { - val replicaIds = (0 until clusterSize) - val notaryIdentity = DevIdentityGenerator.generateDistributedNotaryCompositeIdentity( - replicaIds.map { mockNet.baseDirectory(mockNet.nextNodeId + it) }, - CordaX500Name("Custom Notary", "Zurich", "CH")) + private fun startClusterAndNode(mockNet: InternalMockNetwork): Pair { + val replicaIds = (0 until clusterSize) + val serviceLegalName = CordaX500Name("Custom Notary", "Zurich", "CH") + val notaryIdentity = DevIdentityGenerator.generateDistributedNotaryCompositeIdentity( + replicaIds.map { mockNet.baseDirectory(mockNet.nextNodeId + it) }, + serviceLegalName) val networkParameters = NetworkParametersCopier(testNetworkParameters(listOf(NotaryInfo(notaryIdentity, true)))) val notaryConfig = mock { - whenever(it.isClusterConfig).thenReturn(true) + whenever(it.serviceLegalName).thenReturn(serviceLegalName) whenever(it.validating).thenReturn(true) whenever(it.className).thenReturn(TimedFlowTests.TestNotaryService::class.java.name) } diff --git a/samples/notary-demo/build.gradle b/samples/notary-demo/build.gradle index 7b8b206c71..357aa041d3 100644 --- a/samples/notary-demo/build.gradle +++ b/samples/notary-demo/build.gradle @@ -112,7 +112,7 @@ task deployNodesRaft(type: Cordform, dependsOn: ['jar', nodeTask, webTask]) { notary = [ validating: true, serviceLegalName: "O=Raft,L=Zurich,C=CH", - raft: [ + extraConfig: [ nodeAddress: "localhost:10008" ], className: className @@ -128,7 +128,7 @@ task deployNodesRaft(type: Cordform, dependsOn: ['jar', nodeTask, webTask]) { notary = [ validating: true, serviceLegalName: "O=Raft,L=Zurich,C=CH", - raft: [ + extraConfig: [ nodeAddress: "localhost:10012", clusterAddresses: ["localhost:10008"] ], @@ -145,7 +145,7 @@ task deployNodesRaft(type: Cordform, dependsOn: ['jar', nodeTask, webTask]) { notary = [ validating: true, serviceLegalName: "O=Raft,L=Zurich,C=CH", - raft: [ + extraConfig: [ nodeAddress: "localhost:10016", clusterAddresses: ["localhost:10008"] ], @@ -181,7 +181,7 @@ task deployNodesBFT(type: Cordform, dependsOn: ['jar', nodeTask, webTask]) { notary = [ validating: false, serviceLegalName: "O=BFT,L=Zurich,C=CH", - bftSMaRt: [ + extraConfig: [ replicaId: 0, clusterAddresses: clusterAddresses ], @@ -198,7 +198,7 @@ task deployNodesBFT(type: Cordform, dependsOn: ['jar', nodeTask, webTask]) { notary = [ validating: false, serviceLegalName: "O=BFT,L=Zurich,C=CH", - bftSMaRt: [ + extraConfig: [ replicaId: 1, clusterAddresses: clusterAddresses ], @@ -215,7 +215,7 @@ task deployNodesBFT(type: Cordform, dependsOn: ['jar', nodeTask, webTask]) { notary = [ validating: false, serviceLegalName: "O=BFT,L=Zurich,C=CH", - bftSMaRt: [ + extraConfig: [ replicaId: 2, clusterAddresses: clusterAddresses ], @@ -232,7 +232,7 @@ task deployNodesBFT(type: Cordform, dependsOn: ['jar', nodeTask, webTask]) { notary = [ validating: false, serviceLegalName: "O=BFT,L=Zurich,C=CH", - bftSMaRt: [ + extraConfig: [ replicaId: 3, clusterAddresses: clusterAddresses ], diff --git a/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/DriverDSLImpl.kt b/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/DriverDSLImpl.kt index 0a8a28c024..060372f8ba 100644 --- a/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/DriverDSLImpl.kt +++ b/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/DriverDSLImpl.kt @@ -172,7 +172,7 @@ class DriverDSLImpl( private fun establishRpc(config: NodeConfig, processDeathFuture: CordaFuture): CordaFuture { val rpcAddress = config.corda.rpcOptions.address - val clientRpcSslOptions = clientSslOptionsCompatibleWith(config.corda.rpcOptions) + val clientRpcSslOptions = clientSslOptionsCompatibleWith(config.corda.rpcOptions) val client = createCordaRPCClientWithSslAndClassLoader(rpcAddress, sslConfiguration = clientRpcSslOptions) val connectionFuture = poll(executorService, "RPC connection") { try { @@ -506,29 +506,30 @@ class DriverDSLImpl( } } - // TODO This mapping is done is several places including the gradle plugin. In general we need a better way of - // generating the configs for the nodes, probably making use of Any.toConfig() - private fun NotaryConfig.toConfigMap(): Map = mapOf("notary" to toConfig().root().unwrapped()) - private fun startSingleNotary(spec: NotarySpec, localNetworkMap: LocalNetworkMap?, customOverrides: Map): CordaFuture> { + val notaryConfig = mapOf("notary" to mapOf("validating" to spec.validating)) return startRegisteredNode( spec.name, localNetworkMap, spec.rpcUsers, spec.verifierType, - customOverrides = NotaryConfig(spec.validating).toConfigMap() + customOverrides + customOverrides = notaryConfig + customOverrides ).map { listOf(it) } } private fun startRaftNotaryCluster(spec: NotarySpec, localNetworkMap: LocalNetworkMap?): CordaFuture> { fun notaryConfig(nodeAddress: NetworkHostAndPort, clusterAddress: NetworkHostAndPort? = null): Map { val clusterAddresses = if (clusterAddress != null) listOf(clusterAddress) else emptyList() - val config = NotaryConfig( - validating = spec.validating, - serviceLegalName = spec.name, - className = "net.corda.notary.raft.RaftNotaryService", - raft = RaftConfig(nodeAddress = nodeAddress, clusterAddresses = clusterAddresses)) - return config.toConfigMap() + val config = configOf("notary" to mapOf( + "validating" to spec.validating, + "serviceLegalName" to spec.name.toString(), + "className" to "net.corda.notary.raft.RaftNotaryService", + "extraConfig" to mapOf( + "nodeAddress" to nodeAddress.toString(), + "clusterAddresses" to clusterAddresses.map { it.toString() } + )) + ) + return config.root().unwrapped() } val nodeNames = generateNodeNames(spec)