CORDA-1510 - Allow Doorman and NetworkMap to be configured independently (#3485)

* CORDA-1510 - Allow Doorman and NetworkMap to be configured independently (#3220)

* CORDA-1510 - Allow Doorman and NetworkMap to be configured independently

Currently only one compatabilityZoneURL can be specified, however the
two services can be run on as separate servers. Allow nodes to be
configured in this manner

* Partial review comments

* Review comments

* review comments

* Test fix

* spell fix
This commit is contained in:
Katelyn Baker 2018-07-02 15:32:51 +01:00 committed by GitHub
parent d2bb19e095
commit f7aa2f8294
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
13 changed files with 222 additions and 36 deletions

View File

@ -1,8 +1,14 @@
Changelog
=========
Here are brief summaries of what's changed between each snapshot release. This includes guidance on how to upgrade code
from the previous milestone release.
.. _changelog_v3.2:
Version 3.2
-----------
* Doorman and NetworkMap URLs can now be configured individually rather than being assumed to be
the same server. Current ``compatibilityZoneURL`` configurations remain valid. See both :doc:`corda-configuration-file`
and :doc:`permissioning` for details.
.. _changelog_v3.1:

View File

@ -169,7 +169,16 @@ path to the node's base directory.
interfaces, and then by sending an IP discovery request to the network map service. Set to ``false`` to disable.
:compatibilityZoneURL: The root address of Corda compatibility zone network management services, it is used by the Corda node to register with the network and
obtain Corda node certificate, (See :doc:`permissioning` for more information.) and also used by the node to obtain network map information.
obtain Corda node certificate, (See :doc:`permissioning` for more information.) and also used by the node to obtain network map information. Cannot be
set at the same time as the ``networkServices`` option.
:networkServices: If the Corda compatibility zone services, both network map and registration (doorman), are not running on the same endpoint
and thus have different URLs then this option should be used in place of the ``compatibilityZoneURL`` setting.
:doormanURL: Root address of the network registration service.
:networkMapURL: Root address of the network map service.
.. note:: Only one of ``compatibilityZoneURL`` or ``networkServices`` should be used.
:jvmArgs: An optional list of JVM args, as strings, which replace those inherited from the command line when launching via ``corda.jar``
only. e.g. ``jvmArgs = [ "-Xmx220m", "-Xms220m", "-XX:+UseG1GC" ]``
@ -195,4 +204,4 @@ path to the node's base directory.
Otherwise defaults to 10MB
:attachmentCacheBound: Optionally specify how many attachments should be cached locally. Note that this includes only the key and
metadata, the content is cached separately and can be loaded lazily. Defaults to 1024.
metadata, the content is cached separately and can be loaded lazily. Defaults to 1024.

View File

@ -0,0 +1,25 @@
myLegalName : "O=Bank A,L=London,C=GB"
keyStorePassword : "cordacadevpass"
trustStorePassword : "trustpass"
crlCheckSoftFail: true
dataSourceProperties : {
dataSourceClassName : org.h2.jdbcx.JdbcDataSource
dataSource.url : "jdbc:h2:file:"${baseDirectory}"/persistence"
dataSource.user : sa
dataSource.password : ""
}
p2pAddress : "my-corda-node:10002"
rpcSettings = {
useSsl = false
standAloneBroker = false
address : "my-corda-node:10003"
adminAddress : "my-corda-node:10004"
}
rpcUsers : [
{ username=user1, password=letmein, permissions=[ StartFlow.net.corda.protocols.CashProtocol ] }
]
devMode : false
networkServices : {
doormanURL = "https://registration.corda.net"
networkMapURL = "https://cz.corda.net"
}

View File

@ -192,21 +192,25 @@ This can be overridden with the additional ``--network-root-truststore`` flag.
The certificate signing request will be created based on node information obtained from the node configuration.
The following information from the node configuration file is needed to generate the request.
:myLegalName: Your company's legal name as an X.500 string. X.500 allows differentiation between entities with the same
name as the legal name needs to be unique on the network. If another node has already been permissioned with this
name then the permissioning server will automatically reject the request. The request will also be rejected if it
violates legal name rules, see :ref:`node_naming` for more information.
* **myLegalName** Your company's legal name as an X.500 string. X.500 allows differentiation between entities with the same
name as the legal name needs to be unique on the network. If another node has already been permissioned with this
name then the permissioning server will automatically reject the request. The request will also be rejected if it
violates legal name rules, see :ref:`node_naming` for more information.
:emailAddress: e.g. "admin@company.com"
* **emailAddress** e.g. "admin@company.com"
:devMode: must be set to false
* **devMode** must be set to false
:compatibilityZoneURL: Corda compatibility zone network management service root URL.
* **networkServices or compatibilityZoneURL** The Corda compatibility zone services must be configured. This must be either:
A new pair of private and public keys generated by the Corda node will be used to create the request.
* **compatibilityZoneURL** The Corda compatibility zone network management service root URL.
* **networkServices** Replaces the ``compatibilityZoneURL`` when the Doorman and Network Map services
are configured to operate on different URL endpoints. The ``doorman`` entry is used for registration.
The utility will submit the request to the doorman server and poll for a result periodically to retrieve the certificates.
Once the request has been approved and the certificates downloaded from the server, the node will create the keystore and trust store using the certificates and the generated private key.
A new pair of private and public keys generated by the Corda node will be used to create the request.
The utility will submit the request to the doorman server and poll for a result periodically to retrieve the certificates.
Once the request has been approved and the certificates downloaded from the server, the node will create the keystore and trust store using the certificates and the generated private key.
.. note:: You can exit the utility at any time if the approval process is taking longer than expected. The request process will resume on restart.

View File

@ -13,26 +13,25 @@ import net.corda.core.utilities.seconds
import net.corda.nodeapi.internal.network.NETWORK_PARAMS_FILE_NAME
import net.corda.nodeapi.internal.network.SignedNetworkParameters
import net.corda.testing.common.internal.testNetworkParameters
import net.corda.testing.core.ALICE_NAME
import net.corda.testing.core.BOB_NAME
import net.corda.testing.core.SerializationEnvironmentRule
import net.corda.testing.core.*
import net.corda.testing.driver.NodeHandle
import net.corda.testing.driver.PortAllocation
import net.corda.testing.driver.internal.RandomFree
import net.corda.testing.node.internal.CompatibilityZoneParams
import net.corda.testing.node.internal.internalDriver
import net.corda.testing.node.internal.*
import net.corda.testing.node.internal.network.NetworkMapServer
import org.assertj.core.api.Assertions.assertThat
import org.junit.After
import org.junit.Before
import org.junit.Rule
import org.junit.Test
import org.junit.runner.RunWith
import org.junit.runners.Parameterized
import java.net.URL
import java.time.Instant
import kotlin.streams.toList
import kotlin.test.assertEquals
class NetworkMapTest {
@RunWith(Parameterized::class)
class NetworkMapTest(var initFunc: (URL, NetworkMapServer) -> CompatibilityZoneParams) {
@Rule
@JvmField
val testSerialization = SerializationEnvironmentRule(true)
@ -43,13 +42,37 @@ class NetworkMapTest {
private lateinit var networkMapServer: NetworkMapServer
private lateinit var compatibilityZone: CompatibilityZoneParams
companion object {
@JvmStatic
@Parameterized.Parameters(name = "{0}")
fun runParams() = listOf(
{ addr: URL, nms: NetworkMapServer ->
SharedCompatibilityZoneParams(
addr,
publishNotaries = {
nms.networkParameters = testNetworkParameters(it, modifiedTime = Instant.ofEpochMilli(random63BitValue()), epoch = 2)
}
)
},
{ addr: URL, nms: NetworkMapServer ->
SplitCompatibilityZoneParams(
doormanURL = URL("http://I/Don't/Exist"),
networkMapURL = addr,
publishNotaries = {
nms.networkParameters = testNetworkParameters(it, modifiedTime = Instant.ofEpochMilli(random63BitValue()), epoch = 2)
}
)
}
)
}
@Before
fun start() {
networkMapServer = NetworkMapServer(cacheTimeout, portAllocation.nextHostAndPort())
val address = networkMapServer.start()
compatibilityZone = CompatibilityZoneParams(URL("http://$address"), publishNotaries = {
networkMapServer.networkParameters = testNetworkParameters(it, modifiedTime = Instant.ofEpochMilli(random63BitValue()), epoch = 2)
})
compatibilityZone = initFunc(URL("http://$address"), networkMapServer)
}
@After

View File

@ -22,6 +22,7 @@ import net.corda.testing.driver.PortAllocation
import net.corda.testing.internal.DEV_ROOT_CA
import net.corda.testing.node.NotarySpec
import net.corda.testing.node.internal.CompatibilityZoneParams
import net.corda.testing.node.internal.SharedCompatibilityZoneParams
import net.corda.testing.node.internal.internalDriver
import net.corda.testing.node.internal.network.NetworkMapServer
import org.assertj.core.api.Assertions.assertThat
@ -77,7 +78,7 @@ class NodeRegistrationTest {
@Test
fun `node registration correct root cert`() {
val compatibilityZone = CompatibilityZoneParams(
val compatibilityZone = SharedCompatibilityZoneParams(
URL("http://$serverHostAndPort"),
publishNotaries = { server.networkParameters = testNetworkParameters(it) },
rootCert = DEV_ROOT_CA.certificate)

View File

@ -91,11 +91,14 @@ data class CmdLineOptions(val baseDirectory: Path,
val noLocalShell: Boolean,
val sshdServer: Boolean,
val justGenerateNodeInfo: Boolean,
val bootstrapRaftCluster: Boolean) {
val bootstrapRaftCluster: Boolean
) {
fun loadConfig(): NodeConfiguration {
val config = ConfigHelper.loadConfig(baseDirectory, configFile).parseAsNodeConfiguration()
if (isRegistration) {
requireNotNull(config.compatibilityZoneURL) { "Compatibility Zone Url must be provided in registration mode." }
require(config.compatibilityZoneURL != null || config.networkServices != null) {
"compatibilityZoneURL or networkServices must be present in the node configuration file in registration mode."
}
requireNotNull(networkRootTruststorePath) { "Network root trust store path must be provided in registration mode." }
requireNotNull(networkRootTruststorePassword) { "Network root trust store password must be provided in registration mode." }
}

View File

@ -197,7 +197,7 @@ abstract class AbstractNode(val configuration: NodeConfiguration,
val (identity, identityKeyPair) = obtainIdentity(notaryConfig = null)
val identityService = makeIdentityService(identity.certificate)
networkMapClient = configuration.compatibilityZoneURL?.let { NetworkMapClient(it, identityService.trustRoot) }
networkMapClient = configuration.networkServices?.let { NetworkMapClient(it.networkMapURL, identityService.trustRoot) }
val networkParameters = NetworkParametersReader(identityService.trustRoot, networkMapClient, configuration.baseDirectory).networkParameters
check(networkParameters.minimumPlatformVersion <= versionInfo.platformVersion) {

View File

@ -188,8 +188,10 @@ open class NodeStartup(val args: Array<String>) {
return !(!cmdlineOptions.isRegistration || compatibilityZoneURL == null)
}
open protected fun registerWithNetwork(conf: NodeConfiguration, networkRootTruststorePath: Path, networkRootTruststorePassword: String) {
val compatibilityZoneURL = conf.compatibilityZoneURL!!
protected open fun registerWithNetwork(conf: NodeConfiguration, networkRootTruststorePath: Path, networkRootTruststorePassword: String) {
val compatibilityZoneURL = conf.networkServices?.doormanURL ?: throw RuntimeException(
"compatibilityZoneURL or networkServices must be configured!")
println()
println("******************************************************************")
println("* *")

View File

@ -32,6 +32,7 @@ interface NodeConfiguration : NodeSSLConfiguration {
val devMode: Boolean
val devModeOptions: DevModeOptions?
val compatibilityZoneURL: URL?
val networkServices: NetworkServicesConfig?
val certificateChainCheckPolicies: List<CertChainPolicyConfig>
val verifierType: VerifierType
val messageRedeliveryDelaySeconds: Int
@ -107,6 +108,25 @@ data class BridgeConfiguration(val retryIntervalMs: Long,
data class ActiveMqServerConfiguration(val bridge: BridgeConfiguration)
/**
* Used as an alternative to the older compatibilityZoneURL to allow the doorman and network map
* services for a node to be configured as different URLs. Cannot be set at the same time as the
* compatibilityZoneURL, and will be defaulted (if not set) to both point at the configured
* compatibilityZoneURL.
*
* @property doormanURL The URL of the tls certificate signing service.
* @property networkMapURL The URL of the Network Map service.
* @property inferred Non user setting that indicates weather the Network Services configuration was
* set explicitly ([inferred] == false) or weather they have been inferred via the compatibilityZoneURL parameter
* ([inferred] == true) where both the network map and doorman are running on the same endpoint. Only one,
* compatibilityZoneURL or networkServices, can be set at any one time.
*/
data class NetworkServicesConfig(
val doormanURL: URL,
val networkMapURL: URL,
val inferred : Boolean = false
)
fun Config.parseAsNodeConfiguration(): NodeConfiguration = parseAs<NodeConfigurationImpl>()
data class NodeConfigurationImpl(
@ -118,6 +138,7 @@ data class NodeConfigurationImpl(
override val trustStorePassword: String,
override val dataSourceProperties: Properties,
override val compatibilityZoneURL: URL? = null,
override var networkServices: NetworkServicesConfig? = null,
override val rpcUsers: List<User>,
override val security : SecurityConfiguration? = null,
override val verifierType: VerifierType,
@ -156,6 +177,7 @@ data class NodeConfigurationImpl(
explicitAddress != null -> {
require(settings.address == null) { "Can't provide top-level rpcAddress and rpcSettings.address (they control the same property)." }
logger.warn("Top-level declaration of property 'rpcAddress' is deprecated. Please use 'rpcSettings.address' instead.")
settings.copy(address = explicitAddress)
}
else -> settings
@ -165,6 +187,7 @@ data class NodeConfigurationImpl(
override fun validate(): List<String> {
val errors = mutableListOf<String>()
errors += validateRpcOptions(rpcOptions)
errors += validateNetworkServices()
return errors
}
@ -179,6 +202,17 @@ data class NodeConfigurationImpl(
}
override val exportJMXto: String get() = "http"
private fun validateNetworkServices(): List<String> {
val errors = mutableListOf<String>()
if (compatibilityZoneURL != null && networkServices != null && !(networkServices!!.inferred)) {
errors += "Cannot configure both compatibilityZoneUrl and networkServices simultaneously"
}
return errors
}
override val transactionCacheSizeBytes: Long
get() = transactionCacheSizeMegaBytes?.MB ?: super.transactionCacheSizeBytes
override val attachmentContentCacheSizeBytes: Long
@ -192,6 +226,10 @@ data class NodeConfigurationImpl(
require(security == null || rpcUsers.isEmpty()) {
"Cannot specify both 'rpcUsers' and 'security' in configuration"
}
if (compatibilityZoneURL != null && networkServices == null) {
networkServices = NetworkServicesConfig(compatibilityZoneURL, compatibilityZoneURL, true)
}
}
}

View File

@ -4,8 +4,12 @@ import net.corda.core.internal.div
import net.corda.core.utilities.NetworkHostAndPort
import net.corda.testing.core.ALICE_NAME
import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties
import org.assertj.core.api.Assertions.assertThatThrownBy
import org.assertj.core.api.Assertions.*
import org.junit.Assert.assertEquals
import org.junit.Assert.assertNotNull
import org.junit.Test
import java.net.URI
import java.net.URL
import java.nio.file.Paths
import java.util.*
import kotlin.test.assertFalse
@ -29,6 +33,34 @@ class NodeConfigurationImplTest {
assertFalse { configDebugOptions(true, DevModeOptions(true)).shouldCheckCheckpoints() }
}
@Test
fun `validation has error when both compatibilityZoneURL and networkServices are configured`() {
val configuration = testConfiguration.copy(
devMode = false,
compatibilityZoneURL = URL("https://r3.com"),
networkServices = NetworkServicesConfig(
URL("https://r3.com.doorman"),
URL("https://r3.com/nm")))
val errors = configuration.validate()
assertThat(errors).hasOnlyOneElementSatisfying {
error -> error.contains("Cannot configure both compatibilityZoneUrl and networkServices simultaneously")
}
}
@Test
fun `compatiilityZoneURL populates NetworkServices`() {
val compatibilityZoneURL = URI.create("https://r3.com").toURL()
val configuration = testConfiguration.copy(
devMode = false,
compatibilityZoneURL = compatibilityZoneURL)
assertNotNull(configuration.networkServices)
assertEquals(compatibilityZoneURL, configuration.networkServices!!.doormanURL)
assertEquals(compatibilityZoneURL, configuration.networkServices!!.networkMapURL)
}
private fun configDebugOptions(devMode: Boolean, devModeOptions: DevModeOptions?): NodeConfiguration {
return testConfiguration.copy(devMode = devMode, devModeOptions = devModeOptions)
}

View File

@ -200,7 +200,7 @@ class DriverDSLImpl(
}
val registrationFuture = if (compatibilityZone?.rootCert != null) {
// We don't need the network map to be available to be able to register the node
startNodeRegistration(name, compatibilityZone.rootCert, compatibilityZone.url)
startNodeRegistration(name, compatibilityZone.rootCert, compatibilityZone.doormanURL())
} else {
doneFuture(Unit)
}
@ -225,7 +225,15 @@ class DriverDSLImpl(
val rpcAdminAddress = portAllocation.nextHostAndPort()
val webAddress = portAllocation.nextHostAndPort()
val users = rpcUsers.map { it.copy(permissions = it.permissions + DRIVER_REQUIRED_PERMISSIONS) }
val czUrlConfig = if (compatibilityZone != null) mapOf("compatibilityZoneURL" to compatibilityZone.url.toString()) else emptyMap()
val czUrlConfig = when (compatibilityZone) {
null -> emptyMap()
is SharedCompatibilityZoneParams ->
mapOf("compatibilityZoneURL" to compatibilityZone.doormanURL().toString())
is SplitCompatibilityZoneParams ->
mapOf("networkServices.doormanURL" to compatibilityZone.doormanURL().toString(),
"networkServices.networkMapURL" to compatibilityZone.networkMapURL().toString())
}
val config = NodeConfig(ConfigHelper.loadConfig(
baseDirectory = baseDirectory(name),
allowMissingConfig = true,
@ -413,7 +421,7 @@ class DriverDSLImpl(
startNotaryIdentityGeneration()
} else {
// With a root cert specified we delegate generation of the notary identities to the CZ.
startAllNotaryRegistrations(compatibilityZone.rootCert, compatibilityZone.url)
startAllNotaryRegistrations(compatibilityZone.rootCert, compatibilityZone.doormanURL())
}
notaryInfosFuture.map { notaryInfos ->
compatibilityZone.publishNotaries(notaryInfos)
@ -1013,15 +1021,49 @@ fun <DI : DriverDSL, D : InternalDriverDSL, A> genericDriver(
/**
* Internal API to enable testing of the network map service and node registration process using the internal driver.
* @property url The base CZ URL for registration and network map updates
*
* @property publishNotaries Hook for a network map server to capture the generated [NotaryInfo] objects needed for
* creating the network parameters. This is needed as the network map server is expected to distribute it. The callback
* will occur on a different thread to the driver-calling thread.
* @property rootCert If specified then the nodes will register themselves with the doorman service using [url] and expect
* the registration response to be rooted at this cert. If not specified then no registration is performed and the dev
* root cert is used as normal.
*
* @see SharedCompatibilityZoneParams
* @see SplitCompatibilityZoneParams
*/
data class CompatibilityZoneParams(val url: URL, val publishNotaries: (List<NotaryInfo>) -> Unit, val rootCert: X509Certificate? = null)
sealed class CompatibilityZoneParams(
val publishNotaries: (List<NotaryInfo>) -> Unit,
val rootCert: X509Certificate? = null
) {
abstract fun networkMapURL(): URL
abstract fun doormanURL(): URL
}
/**
* Represent network management services, network map and doorman, running on the same URL
*/
class SharedCompatibilityZoneParams(
private val url: URL,
publishNotaries: (List<NotaryInfo>) -> Unit,
rootCert: X509Certificate? = null
) : CompatibilityZoneParams(publishNotaries, rootCert) {
override fun doormanURL() = url
override fun networkMapURL() = url
}
/**
* Represent network management services, network map and doorman, running on different URLs
*/
class SplitCompatibilityZoneParams(
private val doormanURL: URL,
private val networkMapURL: URL,
publishNotaries: (List<NotaryInfo>) -> Unit,
rootCert: X509Certificate? = null
) : CompatibilityZoneParams(publishNotaries, rootCert) {
override fun doormanURL() = doormanURL
override fun networkMapURL() = networkMapURL
}
fun <A> internalDriver(
isDebug: Boolean = DriverParameters().isDebug,

View File

@ -464,6 +464,7 @@ private fun mockNodeConfiguration(): NodeConfiguration {
doReturn(true).whenever(it).devMode
doReturn(null).whenever(it).compatibilityZoneURL
doReturn(emptyList<CertChainPolicyConfig>()).whenever(it).certificateChainCheckPolicies
doReturn(null).whenever(it).networkServices
doReturn(VerifierType.InMemory).whenever(it).verifierType
doReturn(5).whenever(it).messageRedeliveryDelaySeconds
doReturn(5.seconds.toMillis()).whenever(it).additionalNodeInfoPollingFrequencyMsec