From 857ed1897a8dc0be1c9ae9bd166aaa533262a800 Mon Sep 17 00:00:00 2001 From: Anthony Keenan Date: Mon, 9 Jul 2018 17:38:31 +0100 Subject: [PATCH 1/4] CORDA-1712 - Make documentation clearer for Network Bootstrapper (#3504) --- .idea/compiler.xml | 3 + docs/source/corda-configuration-file.rst | 2 +- docs/source/network-bootstrapper.rst | 185 ++++++++++++++++++++--- 3 files changed, 170 insertions(+), 20 deletions(-) diff --git a/.idea/compiler.xml b/.idea/compiler.xml index e1cb19e8be..fbd48220f1 100644 --- a/.idea/compiler.xml +++ b/.idea/compiler.xml @@ -79,6 +79,9 @@ + + + diff --git a/docs/source/corda-configuration-file.rst b/docs/source/corda-configuration-file.rst index 0e3efe6629..3a2c462a42 100644 --- a/docs/source/corda-configuration-file.rst +++ b/docs/source/corda-configuration-file.rst @@ -134,7 +134,7 @@ absolute path to the node's base directory. :validating: Boolean to determine whether the notary is a validating or non-validating one. :serviceLegalName: If the node is part of a distributed cluster, specify the legal name of the cluster. At runtime, Corda - checks whether this name matches the name of the certificate of the notary cluster. + checks whether this name matches the name of the certificate of the notary cluster. :raft: If part of a distributed Raft cluster specify this config object, with the following settings: diff --git a/docs/source/network-bootstrapper.rst b/docs/source/network-bootstrapper.rst index e0c3900927..5d28f87d1d 100644 --- a/docs/source/network-bootstrapper.rst +++ b/docs/source/network-bootstrapper.rst @@ -45,8 +45,6 @@ will generate directories containing three nodes: ``notary``, ``partya`` and ``p that comes with the bootstrapper. If a different version of Corda is required then simply place that ``corda.jar`` file alongside the configuration files in the directory. -The directory can also contain CorDapp JARs which will be copied to each node's ``cordapps`` directory. - You can also have the node directories containing their "node.conf" files already laid out. The previous example would be: .. sourcecode:: none @@ -61,33 +59,34 @@ You can also have the node directories containing their "node.conf" files alread Similarly, each node directory may contain its own ``corda.jar``, which the bootstrapper will use instead. -Synchronisation -~~~~~~~~~~~~~~~ +Providing CorDapps to the Network Bootstrapper +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -This tool only bootstraps a network. It cannot dynamically update if a new node needs to join the network or if an existing -one has changed something in their node-info, e.g. their P2P address. For this the new node-info file will need to be placed -in the other nodes' ``additional-node-infos`` directory. A simple way to do this is to use `rsync `_. -However, if it's known beforehand the set of nodes that will eventually form part of the network then all the node directories -can be pregenerated in the bootstrap and only started when needed. +If you would like the Network Bootstrapper to include your CorDapps in each generated node, just place them in the directory +alongside the config files. For example, if your directory has this structure: -Running the bootstrapper again on the same network will allow a new node to be added or an existing one to have its updated -node-info re-distributed. However this comes at the expense of having to temporarily collect the node directories back -together again under a common parent directory. +.. sourcecode:: none + + . + ├── notary_node.conf // The notary's node.conf file + ├── partya_node.conf // Party A's node.conf file + ├── partyb_node.conf // Party B's node.conf file + ├── cordapp-a.jar // A cordapp to be installed on all nodes + └── cordapp-b.jar // Another cordapp to be installed on all nodes + +The ``cordapp-a.jar`` and ``cordapp-b.jar`` will be installed in each node directory, and any contracts within them will be +added to the Contract Whitelist (see below). Whitelisting contracts -~~~~~~~~~~~~~~~~~~~~~~ +---------------------- -The CorDapp JARs are also automatically used to create the *Zone whitelist* (see :doc:`api-contract-constraints`) for -the network. +Any CorDapps provided when bootstrapping a network will be scanned for contracts which will be used to create the +*Zone whitelist* (see :doc:`api-contract-constraints`) for the network. .. note:: If you only wish to whitelist the CorDapps but not copy them to each node then run with the ``--no-copy`` flag. The CorDapp JARs will be hashed and scanned for ``Contract`` classes. These contract class implementations will become part of the whitelisted contracts in the network parameters (see ``NetworkParameters.whitelistedContractImplementations`` :doc:`network-map`). -If the network already has a set of network parameters defined (i.e. the node directories all contain the same network-parameters -file) then the new set of contracts will be appended to the current whitelist. - -.. note:: The whitelist can only ever be appended to. Once added a contract implementation can never be removed. By default the bootstrapper will whitelist all the contracts found in all the CorDapp JARs. To prevent certain contracts from being whitelisted, add their fully qualified class name in the ``exclude_whitelist.txt``. These will instead @@ -99,3 +98,151 @@ For example: net.corda.finance.contracts.asset.Cash net.corda.finance.contracts.asset.CommercialPaper + +Modifying a bootstrapped network +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The network bootstrapper is provided as a development tool for setting up Corda networks for development and testing. +There is some limited functionality which can be used to make changes to a network, but for anything more complicated consider +using a :doc:`network-map` server. + +When running the Network Bootstrapper, each ``node-info`` file needs to be gathered together in one directory. If +the nodes are being run on different machines you need to do the following: + +* Copy the node directories from each machine into one directory, on one machine +* Depending on the modification being made (see below for more information), add any new files required to the root directory +* Run the Network Bootstrapper from the root directory +* Copy each individual node's directory back to the original machine + +The network bootstrapper cannot dynamically update the network if an existing node has changed something in their node-info, +e.g. their P2P address. For this the new node-info file will need to be placed in the other nodes' ``additional-node-infos`` directory. +If the nodes are located on different machines, then a utility such as `rsync `_ can be used +so that the nodes can share node-infos. + +Adding a new node to the network +-------------------------------- + +Running the bootstrapper again on the same network will allow a new node to be added and its +node-info distributed to the existing nodes. + +As an example, if we have an existing bootstrapped network, with a Notary and PartyA and we want to add a PartyB, we +can use the network bootstrapper on the following network structure: + +.. sourcecode:: none + + . + ├── notary // existing node directories + │   ├── node.conf + │ ├── network-parameters + │ ├── node-info-notary + │ └── additional-node-infos + │ ├── node-info-notary + │ └── node-info-partya + ├── partya + │   ├── node.conf + │ ├── network-parameters + │ ├── node-info-partya + │ └── additional-node-infos + │ ├── node-info-notary + │ └── node-info-partya + └── partyb_node.conf // the node.conf for the node to be added + +Then run the network bootstrapper again from the root dir: + +``java -jar network-bootstrapper-VERSION.jar --dir `` + +Which will give the following: + +.. sourcecode:: none + + . + ├── notary // the contents of the existing nodes (keys, db's etc...) are unchanged + │   ├── node.conf + │ ├── network-parameters + │ ├── node-info-notary + │ └── additional-node-infos + │ ├── node-info-notary + │ ├── node-info-partya + │ └── node-info-partyb + ├── partya + │   ├── node.conf + │ ├── network-parameters + │ ├── node-info-partya + │ └── additional-node-infos + │ ├── node-info-notary + │ ├── node-info-partya + │ └── node-info-partyb + └── partyb // a new node directory is created for PartyB +    ├── node.conf + ├── network-parameters + ├── node-info-partyb + └── additional-node-infos + ├── node-info-notary + ├── node-info-partya + └── node-info-partyb + +The bootstrapper will generate a directory and the ``node-info`` file for PartyB, and will also make sure a copy of each +nodes' ``node-info`` file is in the ``additional-node-info`` directory of every node. Any other files in the existing nodes, +such a generated keys, will be unaffected. + +.. note:: The bootstrapper is provided for test deployments and can only generate information for nodes collected on + the same machine. If a network needs to be updated using the bootstrapper once deployed, the nodes will need + collecting back together. + +Updating the contract whitelist for bootstrapped networks +--------------------------------------------------------- + +If the network already has a set of network parameters defined (i.e. the node directories all contain the same network-parameters +file) then the bootstrapper can be used to append contracts from new CorDapps to the current whitelist. +For example, with the following pre-generated network: + +.. sourcecode:: none + + . + ├── notary + │   ├── node.conf + │ ├── network-parameters + │ └── cordapps + │ └── cordapp-a.jar + ├── partya + │   ├── node.conf + │ ├── network-parameters + │ └── cordapps + │ └── cordapp-a.jar + ├── partyb + │   ├── node.conf + │ ├── network-parameters + │ └── cordapps + │ └── cordapp-a.jar + └── cordapp-b.jar // The new cordapp to add to the existing nodes + +Then run the network bootstrapper again from the root dir: + +``java -jar network-bootstrapper-VERSION.jar --dir `` + +To give the following: + +.. sourcecode:: none + + . + ├── notary + │   ├── node.conf + │ ├── network-parameters // The contracts from cordapp-b are appended to the whitelist in network-parameters + │ └── cordapps + │ ├── cordapp-a.jar + │ └── cordapp-b.jar // The updated cordapp is placed in the nodes cordapp directory + ├── partya + │   ├── node.conf + │ ├── network-parameters // The contracts from cordapp-b are appended to the whitelist in network-parameters + │ └── cordapps + │ ├── cordapp-a.jar + │ └── cordapp-b.jar // The updated cordapp is placed in the nodes cordapp directory + └── partyb +    ├── node.conf + ├── network-parameters // The contracts from cordapp-b are appended to the whitelist in network-parameters + └── cordapps + ├── cordapp-a.jar + └── cordapp-b.jar // The updated cordapp is placed in the nodes cordapp directory + +.. note:: The whitelist can only ever be appended to. Once added a contract implementation can never be removed. + From c4d3522ddc3052218776f4ab02cbbe1caab1f5aa Mon Sep 17 00:00:00 2001 From: Gavin Thomas Date: Mon, 9 Jul 2018 17:40:18 +0100 Subject: [PATCH 2/4] ENT-2100 quick start page to include user journeys (#3523) * ENT-2100 quick start page to include user journeys * Restructures table. * add anchors for easy navigation * fix anchors for easy navigation * another fix anchors for easy navigation * Reformats table. * another fix anchors for easy navigation * and another fix anchors for easy navigation * and yet another fix anchors for easy navigation * table tweak * table header fix * add missing table headers --- docs/source/quickstart-index.rst | 108 +++++++++++++++++++++++++++++-- 1 file changed, 104 insertions(+), 4 deletions(-) diff --git a/docs/source/quickstart-index.rst b/docs/source/quickstart-index.rst index 58523d090c..40a2798807 100644 --- a/docs/source/quickstart-index.rst +++ b/docs/source/quickstart-index.rst @@ -10,7 +10,107 @@ Quickstart getting-set-up.rst tutorial-cordapp.rst -* :doc:`Set up your machine for CorDapp development ` -* :doc:`Run the Example CorDapp ` -* `View CorDapps in Corda Explore `_ -* `Download sample CorDapps `_ \ No newline at end of file +Welcome to the Corda Quickstart Guide. Follow the links below to help get going quickly with Corda. + +I want to: + +* :ref:`Learn ` about Corda for the first time +* :ref:`Develop ` a CorDapp +* :ref:`Run ` and test a CorDapp on a local Corda network +* :ref:`Add ` a node to an existing test Corda network +* :ref:`Add ` a node to an existing production network + +.. _quickstart-learn: + +Learn about Corda for the first time +------------------------------------ + ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| Useful links | Description | ++============================================+=========================================================================================================+ +| :doc:`key-concepts` | The key concepts and features of the Corda Platform | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`getting-set-up` | Set up your machine for running and developing CorDapps | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`tutorial-cordapp` | A guide to running a simple CorDapp | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ + +.. _quickstart-develop: + +Develop a CorDapp +----------------- + ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| Useful links | Description | ++============================================+=========================================================================================================+ +| :doc:`hello-world-introduction` | A coding walk-through of a basic CorDapp | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`cordapp-overview` | An introduction to CordApps | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`writing-a-cordapp` | How to structure a CorDapp project | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`cordapp-build-systems` | How to build a CorDapp | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`corda-api` | A guide to the CorDapp API | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ + +.. _quickstart-run: + +Run and test a CorDapp on local Corda network +--------------------------------------------- + ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| Useful links | Description | ++============================================+=========================================================================================================+ +| :doc:`generating-a-node` | Guidance on creating Corda nodes for development and testing locally and on Docker | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`node-structure` | The Corda node folder structure and how to name your node | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`corda-configuration-file` | A detailed description of the Corda node configuration file with examples | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`running-a-node` | Guidance on running Corda nodes locally and on Docker | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`setting-up-a-corda-network` | Considerations for setting up a Corda network | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`shell` | Guidance on using an embedded command line to control and monitor a node | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`node-administration` | How to monitor a Corda node using an RPC interface | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`node-explorer` | A GUI-based tool to view transactional data and transactional history for a node | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ + +.. _quickstart-add: + +Add a node to an existing test Corda network +-------------------------------------------- + ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| Useful links | Description | ++============================================+=========================================================================================================+ +| :doc:`node-structure` | The Corda node folder structure and how to name your node | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`corda-configuration-file` | A detailed description of the Corda node configuration file with examples | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`deploying-a-node` | A step-by-step guide on deploying a Corda node to your own server | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`azure-vm` | A step-by-step guide on creating a Corda Network on Azure | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`aws-vm` | A step-by-step guide on creating a Corda Network on AWS | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`shell` | Guidance on using an embedded command line to control and monitor a node | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`node-administration` | How to monitor a Corda node using an RPC interface | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`node-explorer` | A GUI-based tool to view transactional data and transactional history for a node | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ +| :doc:`blob-inspector` | A troubleshooting tool allowing you to read the contents of a binary blob file | ++--------------------------------------------+---------------------------------------------------------------------------------------------------------+ + +.. _quickstart-production: + +Add a node to an existing production network +-------------------------------------------- + ++---------------------------------------------------------------------------------------------------------+ +| Contact R3 Solutions Engineering at support@r3.com | ++---------------------------------------------------------------------------------------------------------+ From 54161a630a8241682231e441bc918cf683bf0fde Mon Sep 17 00:00:00 2001 From: Michele Sollecito Date: Mon, 9 Jul 2018 18:45:20 +0100 Subject: [PATCH 3/4] [CORDA-1659]: Improve handling/logging of failed address binding. (#3498) --- .../errors/AddressBindingException.kt | 20 ++++ .../corda/node/AddressBindingFailureTests.kt | 48 ++++++++++ .../kotlin/net/corda/node/BootTests.kt | 2 +- .../net/corda/node/internal/AbstractNode.kt | 94 ++++++++++++++++--- .../kotlin/net/corda/node/internal/Node.kt | 16 +++- .../net/corda/node/internal/NodeStartup.kt | 11 ++- .../node/internal/artemis/ArtemisBroker.kt | 6 +- .../messaging/ArtemisMessagingServer.kt | 14 ++- .../node/services/rpc/ArtemisRpcBroker.kt | 12 ++- .../kotlin/net/corda/webserver/WebServer.kt | 4 + .../corda/webserver/internal/NodeWebServer.kt | 27 +++--- 11 files changed, 220 insertions(+), 34 deletions(-) create mode 100644 core/src/main/kotlin/net/corda/core/internal/errors/AddressBindingException.kt create mode 100644 node/src/integration-test/kotlin/net/corda/node/AddressBindingFailureTests.kt diff --git a/core/src/main/kotlin/net/corda/core/internal/errors/AddressBindingException.kt b/core/src/main/kotlin/net/corda/core/internal/errors/AddressBindingException.kt new file mode 100644 index 0000000000..8c597f65e4 --- /dev/null +++ b/core/src/main/kotlin/net/corda/core/internal/errors/AddressBindingException.kt @@ -0,0 +1,20 @@ +package net.corda.core.internal.errors + +import net.corda.core.CordaRuntimeException +import net.corda.core.utilities.NetworkHostAndPort + +class AddressBindingException(val addresses: Set) : CordaRuntimeException(message(addresses)) { + + constructor(address: NetworkHostAndPort) : this(setOf(address)) + + private companion object { + private fun message(addresses: Set): String { + require(addresses.isNotEmpty()) + return if (addresses.size > 1) { + "Failed to bind on an address in ${addresses.joinToString(", ", "[", "]")}." + } else { + "Failed to bind on address ${addresses.single()}." + } + } + } +} \ No newline at end of file diff --git a/node/src/integration-test/kotlin/net/corda/node/AddressBindingFailureTests.kt b/node/src/integration-test/kotlin/net/corda/node/AddressBindingFailureTests.kt new file mode 100644 index 0000000000..425e7a4e53 --- /dev/null +++ b/node/src/integration-test/kotlin/net/corda/node/AddressBindingFailureTests.kt @@ -0,0 +1,48 @@ +package net.corda.node + +import net.corda.core.utilities.NetworkHostAndPort +import net.corda.core.utilities.getOrThrow +import net.corda.core.internal.errors.AddressBindingException +import net.corda.testing.driver.DriverParameters +import net.corda.testing.driver.PortAllocation +import net.corda.testing.driver.driver +import org.assertj.core.api.Assertions.assertThat +import org.assertj.core.api.Assertions.assertThatThrownBy +import org.junit.Test +import java.net.InetSocketAddress +import java.net.ServerSocket + +class AddressBindingFailureTests { + + companion object { + private val portAllocation = PortAllocation.Incremental(20_000) + } + + @Test + fun `p2p address`() = assertBindExceptionForOverrides { address -> mapOf("p2pAddress" to address.toString()) } + + @Test + fun `rpc address`() = assertBindExceptionForOverrides { address -> mapOf("rpcSettings" to mapOf("address" to address.toString())) } + + @Test + fun `rpc admin address`() = assertBindExceptionForOverrides { address -> mapOf("rpcSettings" to mapOf("adminAddress" to address.toString())) } + + @Test + fun `H2 address`() = assertBindExceptionForOverrides { address -> mapOf("h2Settings" to mapOf("address" to address.toString())) } + + private fun assertBindExceptionForOverrides(overrides: (NetworkHostAndPort) -> Map) { + + ServerSocket(0).use { socket -> + + val address = InetSocketAddress(socket.localPort).toNetworkHostAndPort() + driver(DriverParameters(startNodesInProcess = true, notarySpecs = emptyList(), inMemoryDB = false, portAllocation = portAllocation)) { + + assertThatThrownBy { startNode(customOverrides = overrides(address)).getOrThrow() }.isInstanceOfSatisfying(AddressBindingException::class.java) { exception -> + assertThat(exception.addresses).contains(address).withFailMessage("Expected addresses to contain $address but was ${exception.addresses}.") + } + } + } + } + + private fun InetSocketAddress.toNetworkHostAndPort() = NetworkHostAndPort(hostName, port) +} \ No newline at end of file diff --git a/node/src/integration-test/kotlin/net/corda/node/BootTests.kt b/node/src/integration-test/kotlin/net/corda/node/BootTests.kt index 53dcd1f666..fcb56b7c00 100644 --- a/node/src/integration-test/kotlin/net/corda/node/BootTests.kt +++ b/node/src/integration-test/kotlin/net/corda/node/BootTests.kt @@ -36,7 +36,7 @@ class BootTests { @Test fun `double node start doesn't write into log file`() { - driver { + driver(DriverParameters(notarySpecs = emptyList())) { val alice = startNode(providedName = ALICE_NAME).get() val logFolder = alice.baseDirectory / NodeStartup.LOGS_DIRECTORY_NAME val logFile = logFolder.list { it.filter { it.fileName.toString().endsWith(".log") }.findAny().get() } diff --git a/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt b/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt index abb110c31c..3ac42322d1 100644 --- a/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt +++ b/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt @@ -11,7 +11,16 @@ import net.corda.core.concurrent.CordaFuture import net.corda.core.context.InvocationContext import net.corda.core.crypto.newSecureRandom import net.corda.core.crypto.sign -import net.corda.core.flows.* +import net.corda.core.flows.ContractUpgradeFlow +import net.corda.core.flows.FinalityFlow +import net.corda.core.flows.FlowLogic +import net.corda.core.flows.FlowLogicRefFactory +import net.corda.core.flows.FlowSession +import net.corda.core.flows.InitiatedBy +import net.corda.core.flows.InitiatingFlow +import net.corda.core.flows.NotaryChangeFlow +import net.corda.core.flows.NotaryFlow +import net.corda.core.flows.StartableByService import net.corda.core.identity.AbstractParty import net.corda.core.identity.CordaX500Name import net.corda.core.identity.Party @@ -23,14 +32,31 @@ import net.corda.core.internal.concurrent.map import net.corda.core.internal.concurrent.openFuture import net.corda.core.internal.notary.NotaryService import net.corda.core.internal.uncheckedCast -import net.corda.core.messaging.* -import net.corda.core.node.* -import net.corda.core.node.services.* +import net.corda.core.messaging.CordaRPCOps +import net.corda.core.messaging.FlowHandle +import net.corda.core.messaging.FlowHandleImpl +import net.corda.core.messaging.FlowProgressHandle +import net.corda.core.messaging.FlowProgressHandleImpl +import net.corda.core.messaging.RPCOps +import net.corda.core.node.AppServiceHub +import net.corda.core.node.NetworkParameters +import net.corda.core.node.NodeInfo +import net.corda.core.node.ServiceHub +import net.corda.core.node.ServicesForResolution +import net.corda.core.node.services.AttachmentStorage +import net.corda.core.node.services.CordaService +import net.corda.core.node.services.IdentityService +import net.corda.core.node.services.KeyManagementService +import net.corda.core.node.services.TransactionVerifierService import net.corda.core.serialization.SerializationWhitelist import net.corda.core.serialization.SerializeAsToken import net.corda.core.serialization.SingletonSerializeAsToken import net.corda.core.serialization.serialize -import net.corda.core.utilities.* +import net.corda.core.utilities.NetworkHostAndPort +import net.corda.core.utilities.days +import net.corda.core.utilities.debug +import net.corda.core.utilities.getOrThrow +import net.corda.core.utilities.minutes import net.corda.node.CordaClock import net.corda.node.VersionInfo import net.corda.node.internal.CheckpointVerifier.verifyCheckpointsCompatible @@ -45,21 +71,61 @@ import net.corda.node.internal.security.RPCSecurityManager import net.corda.node.services.ContractUpgradeHandler import net.corda.node.services.FinalityHandler import net.corda.node.services.NotaryChangeHandler -import net.corda.node.services.api.* -import net.corda.node.services.config.* +import net.corda.node.services.api.CheckpointStorage +import net.corda.node.services.api.DummyAuditService +import net.corda.node.services.api.FlowStarter +import net.corda.node.services.api.IdentityServiceInternal +import net.corda.node.services.api.MonitoringService +import net.corda.node.services.api.NetworkMapCacheBaseInternal +import net.corda.node.services.api.NetworkMapCacheInternal +import net.corda.node.services.api.NodePropertiesStore +import net.corda.node.services.api.SchedulerService +import net.corda.node.services.api.SchemaService +import net.corda.node.services.api.ServiceHubInternal +import net.corda.node.services.api.StartedNodeServices +import net.corda.node.services.api.VaultServiceInternal +import net.corda.node.services.api.WritableTransactionStorage +import net.corda.node.services.config.BFTSMaRtConfiguration +import net.corda.node.services.config.NodeConfiguration +import net.corda.node.services.config.NotaryConfig +import net.corda.node.services.config.configureWithDevSSLCertificate import net.corda.node.services.config.shell.toShellConfig +import net.corda.node.services.config.shouldInitCrashShell import net.corda.node.services.events.NodeSchedulerService import net.corda.node.services.events.ScheduledActivityObserver import net.corda.node.services.identity.PersistentIdentityService import net.corda.node.services.keys.PersistentKeyManagementService import net.corda.node.services.messaging.DeduplicationHandler import net.corda.node.services.messaging.MessagingService -import net.corda.node.services.network.* -import net.corda.node.services.persistence.* +import net.corda.node.services.network.NetworkMapCacheImpl +import net.corda.node.services.network.NetworkMapClient +import net.corda.node.services.network.NetworkMapUpdater +import net.corda.node.services.network.NodeInfoWatcher +import net.corda.node.services.network.PersistentNetworkMapCache +import net.corda.node.services.persistence.AbstractPartyDescriptor +import net.corda.node.services.persistence.AbstractPartyToX500NameAsStringConverter +import net.corda.node.services.persistence.DBCheckpointStorage +import net.corda.node.services.persistence.DBTransactionMappingStorage +import net.corda.node.services.persistence.DBTransactionStorage +import net.corda.node.services.persistence.NodeAttachmentService +import net.corda.node.services.persistence.NodePropertiesPersistentStore import net.corda.node.services.schema.HibernateObserver import net.corda.node.services.schema.NodeSchemaService -import net.corda.node.services.statemachine.* -import net.corda.node.services.transactions.* +import net.corda.node.services.statemachine.ExternalEvent +import net.corda.node.services.statemachine.FlowLogicRefFactoryImpl +import net.corda.node.services.statemachine.FlowMonitor +import net.corda.node.services.statemachine.SingleThreadedStateMachineManager +import net.corda.node.services.statemachine.StateMachineManager +import net.corda.node.services.statemachine.StateMachineManagerInternal +import net.corda.node.services.statemachine.appName +import net.corda.node.services.statemachine.flowVersionAndInitiatingClass +import net.corda.node.services.transactions.BFTNonValidatingNotaryService +import net.corda.node.services.transactions.BFTSMaRt +import net.corda.node.services.transactions.RaftNonValidatingNotaryService +import net.corda.node.services.transactions.RaftUniquenessProvider +import net.corda.node.services.transactions.RaftValidatingNotaryService +import net.corda.node.services.transactions.SimpleNotaryService +import net.corda.node.services.transactions.ValidatingNotaryService import net.corda.node.services.upgrade.ContractUpgradeServiceImpl import net.corda.node.services.vault.NodeVaultService import net.corda.node.utilities.AffinityExecutor @@ -368,7 +434,11 @@ abstract class AbstractNode(val configuration: NodeConfiguration, open fun startShell() { if (configuration.shouldInitCrashShell()) { - InteractiveShell.startShellInternal(configuration.toShellConfig(), cordappLoader.appClassLoader) + val shellConfiguration = configuration.toShellConfig() + shellConfiguration.sshHostKeyDirectory?.let { + log.info("Binding Shell SSHD server on port $it.") + } + InteractiveShell.startShellInternal(shellConfiguration, cordappLoader.appClassLoader) } } diff --git a/node/src/main/kotlin/net/corda/node/internal/Node.kt b/node/src/main/kotlin/net/corda/node/internal/Node.kt index 00285e5c14..08aabad98c 100644 --- a/node/src/main/kotlin/net/corda/node/internal/Node.kt +++ b/node/src/main/kotlin/net/corda/node/internal/Node.kt @@ -26,6 +26,7 @@ import net.corda.node.VersionInfo import net.corda.node.internal.artemis.ArtemisBroker import net.corda.node.internal.artemis.BrokerAddresses import net.corda.node.internal.cordapp.CordappLoader +import net.corda.core.internal.errors.AddressBindingException import net.corda.node.internal.security.RPCSecurityManagerImpl import net.corda.node.internal.security.RPCSecurityManagerWithAdditionalUser import net.corda.node.serialization.amqp.AMQPServerSerializationScheme @@ -36,7 +37,6 @@ import net.corda.node.services.api.NodePropertiesStore import net.corda.node.services.api.SchemaService import net.corda.node.services.config.NodeConfiguration import net.corda.node.services.config.SecurityConfiguration -import net.corda.node.services.config.VerifierType import net.corda.node.services.config.shouldInitCrashShell import net.corda.node.services.config.shouldStartLocalShell import net.corda.node.services.messaging.ArtemisMessagingServer @@ -61,10 +61,12 @@ import net.corda.serialization.internal.AMQP_RPC_CLIENT_CONTEXT import net.corda.serialization.internal.AMQP_RPC_SERVER_CONTEXT import net.corda.serialization.internal.AMQP_STORAGE_CONTEXT import net.corda.serialization.internal.SerializationFactoryImpl +import org.h2.jdbc.JdbcSQLException import org.slf4j.Logger import org.slf4j.LoggerFactory import rx.Scheduler import rx.schedulers.Schedulers +import java.net.BindException import java.nio.file.Path import java.security.PublicKey import java.time.Clock @@ -328,7 +330,7 @@ open class Node(configuration: NodeConfiguration, if (databaseUrl != null && databaseUrl.startsWith(h2Prefix)) { val effectiveH2Settings = configuration.effectiveH2Settings - if(effectiveH2Settings != null && effectiveH2Settings.address != null) { + if (effectiveH2Settings?.address != null) { val databaseName = databaseUrl.removePrefix(h2Prefix).substringBefore(';') val server = org.h2.tools.Server.createTcpServer( "-tcpPort", effectiveH2Settings.address.port.toString(), @@ -338,7 +340,15 @@ open class Node(configuration: NodeConfiguration, // override interface that createTcpServer listens on (which is always 0.0.0.0) System.setProperty("h2.bindAddress", effectiveH2Settings.address.host) runOnStop += server::stop - val url = server.start().url + val url = try { + server.start().url + } catch (e: JdbcSQLException) { + if (e.cause is BindException) { + throw AddressBindingException(effectiveH2Settings.address) + } else { + throw e + } + } printBasicNodeInfo("Database connection url is", "jdbc:h2:$url/node") } } diff --git a/node/src/main/kotlin/net/corda/node/internal/NodeStartup.kt b/node/src/main/kotlin/net/corda/node/internal/NodeStartup.kt index 4424236b05..a4e87cedc7 100644 --- a/node/src/main/kotlin/net/corda/node/internal/NodeStartup.kt +++ b/node/src/main/kotlin/net/corda/node/internal/NodeStartup.kt @@ -6,8 +6,14 @@ import com.typesafe.config.ConfigException import com.typesafe.config.ConfigRenderOptions import io.netty.channel.unix.Errors import net.corda.core.crypto.Crypto -import net.corda.core.internal.* +import net.corda.core.internal.Emoji import net.corda.core.internal.concurrent.thenMatch +import net.corda.core.internal.createDirectories +import net.corda.core.internal.div +import net.corda.core.internal.errors.AddressBindingException +import net.corda.core.internal.exists +import net.corda.core.internal.location +import net.corda.core.internal.randomOrNull import net.corda.core.utilities.Try import net.corda.core.utilities.loggerFor import net.corda.node.* @@ -149,6 +155,9 @@ open class NodeStartup(val args: Array) { } catch (e: CheckpointIncompatibleException) { logger.error(e.message) return false + } catch (e: AddressBindingException) { + logger.error(e.message) + return false } catch (e: NetworkParametersReader.Error) { logger.error(e.message) return false diff --git a/node/src/main/kotlin/net/corda/node/internal/artemis/ArtemisBroker.kt b/node/src/main/kotlin/net/corda/node/internal/artemis/ArtemisBroker.kt index 2803f52039..5dc7e2c4ea 100644 --- a/node/src/main/kotlin/net/corda/node/internal/artemis/ArtemisBroker.kt +++ b/node/src/main/kotlin/net/corda/node/internal/artemis/ArtemisBroker.kt @@ -1,8 +1,10 @@ package net.corda.node.internal.artemis +import io.netty.channel.unix.Errors import net.corda.core.utilities.NetworkHostAndPort import net.corda.node.internal.LifecycleSupport import org.apache.activemq.artemis.api.core.management.ActiveMQServerControl +import java.net.BindException interface ArtemisBroker : LifecycleSupport, AutoCloseable { val addresses: BrokerAddresses @@ -14,4 +16,6 @@ interface ArtemisBroker : LifecycleSupport, AutoCloseable { data class BrokerAddresses(val primary: NetworkHostAndPort, private val adminArg: NetworkHostAndPort?) { val admin = adminArg ?: primary -} \ No newline at end of file +} + +fun java.io.IOException.isBindingError() = this is BindException || this is Errors.NativeIoException && message?.contains("Address already in use") == true \ No newline at end of file diff --git a/node/src/main/kotlin/net/corda/node/services/messaging/ArtemisMessagingServer.kt b/node/src/main/kotlin/net/corda/node/services/messaging/ArtemisMessagingServer.kt index 88c4ff0962..7b4d697a2c 100644 --- a/node/src/main/kotlin/net/corda/node/services/messaging/ArtemisMessagingServer.kt +++ b/node/src/main/kotlin/net/corda/node/services/messaging/ArtemisMessagingServer.kt @@ -1,5 +1,6 @@ package net.corda.node.services.messaging +import io.netty.channel.unix.Errors import net.corda.core.internal.ThreadBox import net.corda.core.internal.div import net.corda.core.serialization.SingletonSerializeAsToken @@ -9,6 +10,7 @@ import net.corda.core.utilities.debug import net.corda.node.internal.artemis.* import net.corda.node.internal.artemis.BrokerJaasLoginModule.Companion.NODE_P2P_ROLE import net.corda.node.internal.artemis.BrokerJaasLoginModule.Companion.PEER_ROLE +import net.corda.core.internal.errors.AddressBindingException import net.corda.node.services.config.NodeConfiguration import net.corda.nodeapi.ArtemisTcpTransport.Companion.p2pAcceptorTcpTransport import net.corda.nodeapi.internal.AmqpMessageSizeChecksInterceptor @@ -91,7 +93,7 @@ class ArtemisMessagingServer(private val config: NodeConfiguration, // TODO: Maybe wrap [IOException] on a key store load error so that it's clearly splitting key store loading from // Artemis IO errors - @Throws(IOException::class, KeyStoreException::class) + @Throws(IOException::class, AddressBindingException::class, KeyStoreException::class) private fun configureAndStartServer() { val artemisConfig = createArtemisConfig() val securityManager = createArtemisSecurityManager() @@ -104,7 +106,15 @@ class ArtemisMessagingServer(private val config: NodeConfiguration, registerPostQueueDeletionCallback { address, qName -> log.debug { "Queue deleted: $qName for $address" } } } - activeMQServer.start() + try { + activeMQServer.start() + } catch (e: java.io.IOException) { + if (e.isBindingError()) { + throw AddressBindingException(config.p2pAddress) + } else { + throw e + } + } activeMQServer.remotingService.addIncomingInterceptor(ArtemisMessageSizeChecksInterceptor(maxMessageSize)) activeMQServer.remotingService.addIncomingInterceptor(AmqpMessageSizeChecksInterceptor(maxMessageSize)) // Config driven switch between legacy CORE bridges and the newer AMQP protocol bridges. diff --git a/node/src/main/kotlin/net/corda/node/services/rpc/ArtemisRpcBroker.kt b/node/src/main/kotlin/net/corda/node/services/rpc/ArtemisRpcBroker.kt index f7ed434ea0..d8b02869da 100644 --- a/node/src/main/kotlin/net/corda/node/services/rpc/ArtemisRpcBroker.kt +++ b/node/src/main/kotlin/net/corda/node/services/rpc/ArtemisRpcBroker.kt @@ -1,10 +1,12 @@ package net.corda.node.services.rpc +import io.netty.channel.unix.Errors import net.corda.core.utilities.NetworkHostAndPort import net.corda.core.utilities.loggerFor import net.corda.node.internal.artemis.* import net.corda.node.internal.artemis.BrokerJaasLoginModule.Companion.NODE_SECURITY_CONFIG import net.corda.node.internal.artemis.BrokerJaasLoginModule.Companion.RPC_SECURITY_CONFIG +import net.corda.core.internal.errors.AddressBindingException import net.corda.node.internal.security.RPCSecurityManager import net.corda.nodeapi.BrokerRpcSslOptions import net.corda.nodeapi.internal.config.SSLConfiguration @@ -44,7 +46,15 @@ internal class ArtemisRpcBroker internal constructor( override fun start() { logger.debug("Artemis RPC broker is starting.") - server.start() + try { + server.start() + } catch (e: java.io.IOException) { + if (e.isBindingError()) { + throw AddressBindingException(adminAddressOptional?.let { setOf(it, addresses.primary) } ?: setOf(addresses.primary)) + } else { + throw e + } + } logger.debug("Artemis RPC broker is started.") } diff --git a/webserver/src/main/kotlin/net/corda/webserver/WebServer.kt b/webserver/src/main/kotlin/net/corda/webserver/WebServer.kt index 3c0f2ec00a..e33adc70ff 100644 --- a/webserver/src/main/kotlin/net/corda/webserver/WebServer.kt +++ b/webserver/src/main/kotlin/net/corda/webserver/WebServer.kt @@ -4,6 +4,7 @@ package net.corda.webserver import com.typesafe.config.ConfigException import net.corda.core.internal.div +import net.corda.core.internal.errors.AddressBindingException import net.corda.core.internal.location import net.corda.core.internal.rootCause import net.corda.webserver.internal.NodeWebServer @@ -66,6 +67,9 @@ fun main(args: Array) { val elapsed = (System.currentTimeMillis() - startTime) / 10 / 100.0 println("Webserver started up in $elapsed sec") server.run() + } catch (e: AddressBindingException) { + log.error(e.message) + exitProcess(1) } catch (e: Exception) { log.error("Exception during node startup", e) exitProcess(1) diff --git a/webserver/src/main/kotlin/net/corda/webserver/internal/NodeWebServer.kt b/webserver/src/main/kotlin/net/corda/webserver/internal/NodeWebServer.kt index d94764dab4..5bf16e4a7a 100644 --- a/webserver/src/main/kotlin/net/corda/webserver/internal/NodeWebServer.kt +++ b/webserver/src/main/kotlin/net/corda/webserver/internal/NodeWebServer.kt @@ -1,26 +1,18 @@ package net.corda.webserver.internal import com.google.common.html.HtmlEscapers.htmlEscaper +import io.netty.channel.unix.Errors import net.corda.client.jackson.JacksonSupport import net.corda.client.rpc.CordaRPCClient import net.corda.client.rpc.RPCException +import net.corda.core.internal.errors.AddressBindingException import net.corda.core.messaging.CordaRPCOps import net.corda.core.utilities.contextLogger import net.corda.webserver.WebServerConfig import net.corda.webserver.converters.CordaConverterProvider import net.corda.webserver.services.WebServerPluginRegistry -import net.corda.webserver.servlets.AttachmentDownloadServlet -import net.corda.webserver.servlets.CorDappInfoServlet -import net.corda.webserver.servlets.DataUploadServlet -import net.corda.webserver.servlets.ObjectMapperConfig -import net.corda.webserver.servlets.ResponseFilter -import org.eclipse.jetty.server.Connector -import org.eclipse.jetty.server.HttpConfiguration -import org.eclipse.jetty.server.HttpConnectionFactory -import org.eclipse.jetty.server.SecureRequestCustomizer -import org.eclipse.jetty.server.Server -import org.eclipse.jetty.server.ServerConnector -import org.eclipse.jetty.server.SslConnectionFactory +import net.corda.webserver.servlets.* +import org.eclipse.jetty.server.* import org.eclipse.jetty.server.handler.ErrorHandler import org.eclipse.jetty.server.handler.HandlerCollection import org.eclipse.jetty.servlet.DefaultServlet @@ -34,6 +26,7 @@ import org.slf4j.LoggerFactory import java.io.IOException import java.io.Writer import java.lang.reflect.InvocationTargetException +import java.net.BindException import java.nio.file.NoSuchFileException import java.util.* import javax.servlet.http.HttpServletRequest @@ -95,7 +88,15 @@ class NodeWebServer(val config: WebServerConfig) { server.connectors = arrayOf(connector) server.handler = handlerCollection - server.start() + try { + server.start() + } catch (e: IOException) { + if (e is BindException || e is Errors.NativeIoException && e.message?.contains("Address already in use") == true) { + throw AddressBindingException(address) + } else { + throw e + } + } log.info("Starting webserver on address $address") return server } From 7b4ace16e18c73a23c4ac05d60536c0d1c6df588 Mon Sep 17 00:00:00 2001 From: Dan Newton Date: Tue, 10 Jul 2018 08:35:14 +0100 Subject: [PATCH 4/4] Remove duplicated code between PersistentIdentityService and InMemoryIdentityService (#3428) --- .../core/node/services/IdentityService.kt | 39 ++++++++-- docs/source/changelog.rst | 3 + .../services/api/IdentityServiceInternal.kt | 67 ++++++++++++++++ .../services/identity/IdentityServiceUtil.kt | 24 ------ .../identity/InMemoryIdentityService.kt | 68 ++-------------- .../identity/PersistentIdentityService.kt | 77 ++++--------------- 6 files changed, 125 insertions(+), 153 deletions(-) delete mode 100644 node/src/main/kotlin/net/corda/node/services/identity/IdentityServiceUtil.kt diff --git a/core/src/main/kotlin/net/corda/core/node/services/IdentityService.kt b/core/src/main/kotlin/net/corda/core/node/services/IdentityService.kt index 1aa8e35f24..91c714b51f 100644 --- a/core/src/main/kotlin/net/corda/core/node/services/IdentityService.kt +++ b/core/src/main/kotlin/net/corda/core/node/services/IdentityService.kt @@ -3,6 +3,7 @@ package net.corda.core.node.services import net.corda.core.CordaException import net.corda.core.DoNotImplement import net.corda.core.contracts.PartyAndReference +import net.corda.core.crypto.toStringShort import net.corda.core.identity.* import java.security.InvalidAlgorithmParameterException import java.security.PublicKey @@ -37,10 +38,17 @@ interface IdentityService { * Asserts that an anonymous party maps to the given full party, by looking up the certificate chain associated with * the anonymous party and resolving it back to the given full party. * - * @throws IllegalStateException if the anonymous party is not owned by the full party. + * @throws UnknownAnonymousPartyException if the anonymous party is not owned by the full party. */ - @Throws(IllegalStateException::class) - fun assertOwnership(party: Party, anonymousParty: AnonymousParty) + @Throws(UnknownAnonymousPartyException::class) + fun assertOwnership(party: Party, anonymousParty: AnonymousParty) { + val anonymousIdentity = certificateFromKey(anonymousParty.owningKey) + ?: throw UnknownAnonymousPartyException("Unknown $anonymousParty") + val issuingCert = anonymousIdentity.certPath.certificates[1] + require(issuingCert.publicKey == party.owningKey) { + "Issuing certificate's public key must match the party key ${party.owningKey.toStringShort()}." + } + } /** * Get all identities known to the service. This is expensive, and [partyFromKey] or [partyFromX500Name] should be @@ -63,7 +71,7 @@ interface IdentityService { * @param key The owning [PublicKey] of the [Party]. * @return Returns a [Party] with a matching owningKey if known, else returns null. */ - fun partyFromKey(key: PublicKey): Party? + fun partyFromKey(key: PublicKey): Party? = certificateFromKey(key)?.party /** * Resolves a party name to the well known identity [Party] instance for this name. Where possible well known identity @@ -82,7 +90,21 @@ interface IdentityService { * @param party identity to determine well known identity for. * @return well known identity, if found. */ - fun wellKnownPartyFromAnonymous(party: AbstractParty): Party? + fun wellKnownPartyFromAnonymous(party: AbstractParty): Party? { + // The original version of this would return the party as-is if it was a Party (rather than AnonymousParty), + // however that means that we don't verify that we know who owns the key. As such as now enforce turning the key + // into a party, and from there figure out the well known party. + val candidate = partyFromKey(party.owningKey) + // TODO: This should be done via the network map cache, which is the authoritative source of well known identities + return if (candidate != null) { + require(party.nameOrNull() == null || party.nameOrNull() == candidate.name) { + "Candidate party $candidate does not match expected $party" + } + wellKnownPartyFromX500Name(candidate.name) + } else { + null + } + } /** * Resolves a (optionally) confidential identity to the corresponding well known identity [Party]. @@ -93,7 +115,7 @@ interface IdentityService { * @param partyRef identity (and reference, which is unused) to determine well known identity for. * @return the well known identity, or null if unknown. */ - fun wellKnownPartyFromAnonymous(partyRef: PartyAndReference) = wellKnownPartyFromAnonymous(partyRef.party) + fun wellKnownPartyFromAnonymous(partyRef: PartyAndReference): Party? = wellKnownPartyFromAnonymous(partyRef.party) /** * Resolve the well known identity of a party. Throws an exception if the party cannot be identified. @@ -102,7 +124,10 @@ interface IdentityService { * @return the well known identity. * @throws IllegalArgumentException */ - fun requireWellKnownPartyFromAnonymous(party: AbstractParty): Party + fun requireWellKnownPartyFromAnonymous(party: AbstractParty): Party { + return wellKnownPartyFromAnonymous(party) + ?: throw IllegalStateException("Could not deanonymise party ${party.owningKey.toStringShort()}") + } /** * Returns a list of candidate matches for a given string, with optional fuzzy(ish) matching. Fuzzy matching may diff --git a/docs/source/changelog.rst b/docs/source/changelog.rst index 95ed758c45..1048335bc6 100644 --- a/docs/source/changelog.rst +++ b/docs/source/changelog.rst @@ -6,6 +6,9 @@ release, see :doc:`upgrade-notes`. Unreleased ---------- +* Docs for IdentityService. assertOwnership updated to correctly state that an UnknownAnonymousPartyException is thrown + rather than IllegalStateException. + * The Corda JPA entities no longer implement java.io.Serializable, as this was causing persistence errors in obscure cases. Java serialization is disabled globally in the node, but in the unlikely event you were relying on these types being Java serializable please contact us. diff --git a/node/src/main/kotlin/net/corda/node/services/api/IdentityServiceInternal.kt b/node/src/main/kotlin/net/corda/node/services/api/IdentityServiceInternal.kt index 5694a68940..1f5c1f0a6f 100644 --- a/node/src/main/kotlin/net/corda/node/services/api/IdentityServiceInternal.kt +++ b/node/src/main/kotlin/net/corda/node/services/api/IdentityServiceInternal.kt @@ -1,11 +1,78 @@ package net.corda.node.services.api +import net.corda.core.identity.CordaX500Name +import net.corda.core.identity.Party import net.corda.core.identity.PartyAndCertificate +import net.corda.core.internal.CertRole import net.corda.core.node.services.IdentityService +import net.corda.core.utilities.contextLogger +import net.corda.nodeapi.internal.crypto.X509Utilities +import net.corda.nodeapi.internal.crypto.x509Certificates +import java.security.InvalidAlgorithmParameterException +import java.security.cert.CertPathValidatorException +import java.security.cert.CertificateExpiredException +import java.security.cert.CertificateNotYetValidException +import java.security.cert.TrustAnchor interface IdentityServiceInternal : IdentityService { + + private companion object { + val log = contextLogger() + } + /** This method exists so it can be mocked with doNothing, rather than having to make up a possibly invalid return value. */ fun justVerifyAndRegisterIdentity(identity: PartyAndCertificate) { verifyAndRegisterIdentity(identity) } + + fun partiesFromName(query: String, exactMatch: Boolean, x500name: CordaX500Name, results: LinkedHashSet, party: Party) { + val components = listOfNotNull(x500name.commonName, x500name.organisationUnit, x500name.organisation, x500name.locality, x500name.state, x500name.country) + components.forEach { component -> + if (exactMatch && component == query) { + results += party + } else if (!exactMatch) { + // We can imagine this being a query over a lucene index in future. + // + // Kostas says: We can easily use the Jaro-Winkler distance metric as it is best suited for short + // strings such as entity/company names, and to detect small typos. We can also apply it for city + // or any keyword related search in lists of records (not raw text - for raw text we need indexing) + // and we can return results in hierarchical order (based on normalised String similarity 0.0-1.0). + if (component.contains(query, ignoreCase = true)) + results += party + } + } + } + + /** + * Verifies that an identity is valid. + * + * @param trustAnchor The trust anchor that will verify the identity's validity + * @param identity The identity to verify + */ + @Throws(CertificateExpiredException::class, CertificateNotYetValidException::class, InvalidAlgorithmParameterException::class) + fun verifyAndRegisterIdentity(trustAnchor: TrustAnchor, identity: PartyAndCertificate): PartyAndCertificate? { + // Validate the chain first, before we do anything clever with it + val identityCertChain = identity.certPath.x509Certificates + try { + identity.verify(trustAnchor) + } catch (e: CertPathValidatorException) { + log.warn("Certificate validation failed for ${identity.name} against trusted root ${trustAnchor.trustedCert.subjectX500Principal}.") + log.warn("Certificate path :") + identityCertChain.reversed().forEachIndexed { index, certificate -> + val space = (0 until index).joinToString("") { " " } + log.warn("$space${certificate.subjectX500Principal}") + } + throw e + } + // Ensure we record the first identity of the same name, first + val wellKnownCert = identityCertChain.single { CertRole.extract(it)?.isWellKnown ?: false } + if (wellKnownCert != identity.certificate) { + val idx = identityCertChain.lastIndexOf(wellKnownCert) + val firstPath = X509Utilities.buildCertPath(identityCertChain.slice(idx until identityCertChain.size)) + verifyAndRegisterIdentity(trustAnchor, PartyAndCertificate(firstPath)) + } + return registerIdentity(identity) + } + + fun registerIdentity(identity: PartyAndCertificate): PartyAndCertificate? } diff --git a/node/src/main/kotlin/net/corda/node/services/identity/IdentityServiceUtil.kt b/node/src/main/kotlin/net/corda/node/services/identity/IdentityServiceUtil.kt deleted file mode 100644 index b5f065cc85..0000000000 --- a/node/src/main/kotlin/net/corda/node/services/identity/IdentityServiceUtil.kt +++ /dev/null @@ -1,24 +0,0 @@ -package net.corda.node.services.identity - -import net.corda.core.identity.CordaX500Name -import net.corda.core.identity.Party - - -fun partiesFromName(query: String, exactMatch: Boolean, x500name: CordaX500Name, results: LinkedHashSet, party: Party) { - - val components = listOfNotNull(x500name.commonName, x500name.organisationUnit, x500name.organisation, x500name.locality, x500name.state, x500name.country) - components.forEach { component -> - if (exactMatch && component == query) { - results += party - } else if (!exactMatch) { - // We can imagine this being a query over a lucene index in future. - // - // Kostas says: We can easily use the Jaro-Winkler distance metric as it is best suited for short - // strings such as entity/company names, and to detect small typos. We can also apply it for city - // or any keyword related search in lists of records (not raw text - for raw text we need indexing) - // and we can return results in hierarchical order (based on normalised String similarity 0.0-1.0). - if (component.contains(query, ignoreCase = true)) - results += party - } - } -} \ No newline at end of file diff --git a/node/src/main/kotlin/net/corda/node/services/identity/InMemoryIdentityService.kt b/node/src/main/kotlin/net/corda/node/services/identity/InMemoryIdentityService.kt index adcc86892a..35b1c51fe6 100644 --- a/node/src/main/kotlin/net/corda/node/services/identity/InMemoryIdentityService.kt +++ b/node/src/main/kotlin/net/corda/node/services/identity/InMemoryIdentityService.kt @@ -1,15 +1,12 @@ package net.corda.node.services.identity -import net.corda.core.contracts.PartyAndReference -import net.corda.core.crypto.toStringShort -import net.corda.core.identity.* -import net.corda.core.internal.CertRole -import net.corda.core.node.services.IdentityService -import net.corda.core.node.services.UnknownAnonymousPartyException +import net.corda.core.identity.CordaX500Name +import net.corda.core.identity.Party +import net.corda.core.identity.PartyAndCertificate import net.corda.core.serialization.SingletonSerializeAsToken import net.corda.core.utilities.contextLogger import net.corda.core.utilities.trace -import net.corda.nodeapi.internal.crypto.X509Utilities +import net.corda.node.services.api.IdentityServiceInternal import net.corda.nodeapi.internal.crypto.x509Certificates import java.security.InvalidAlgorithmParameterException import java.security.PublicKey @@ -22,10 +19,9 @@ import javax.annotation.concurrent.ThreadSafe * * @param identities initial set of identities for the service, typically only used for unit tests. */ -// TODO There is duplicated logic between this and PersistentIdentityService @ThreadSafe class InMemoryIdentityService(identities: List = emptyList(), - override val trustRoot: X509Certificate) : SingletonSerializeAsToken(), IdentityService { + override val trustRoot: X509Certificate) : SingletonSerializeAsToken(), IdentityServiceInternal { companion object { private val log = contextLogger() } @@ -44,29 +40,10 @@ class InMemoryIdentityService(identities: List = emptyList( } @Throws(CertificateExpiredException::class, CertificateNotYetValidException::class, InvalidAlgorithmParameterException::class) - override fun verifyAndRegisterIdentity(identity: PartyAndCertificate): PartyAndCertificate? { - // Validate the chain first, before we do anything clever with it + override fun verifyAndRegisterIdentity(identity: PartyAndCertificate): PartyAndCertificate? = verifyAndRegisterIdentity(trustAnchor, identity) + + override fun registerIdentity(identity: PartyAndCertificate): PartyAndCertificate? { val identityCertChain = identity.certPath.x509Certificates - try { - identity.verify(trustAnchor) - } catch (e: CertPathValidatorException) { - log.warn("Certificate validation failed for ${identity.name} against trusted root ${trustAnchor.trustedCert.subjectX500Principal}.") - log.warn("Certificate path :") - identityCertChain.reversed().forEachIndexed { index, certificate -> - val space = (0 until index).joinToString("") { " " } - log.warn("$space${certificate.subjectX500Principal}") - } - throw e - } - - // Ensure we record the first identity of the same name, first - val wellKnownCert = identityCertChain.single { CertRole.extract(it)?.isWellKnown ?: false } - if (wellKnownCert != identity.certificate) { - val idx = identityCertChain.lastIndexOf(wellKnownCert) - val firstPath = X509Utilities.buildCertPath(identityCertChain.slice(idx until identityCertChain.size)) - verifyAndRegisterIdentity(PartyAndCertificate(firstPath)) - } - log.trace { "Registering identity $identity" } keyToParties[identity.owningKey] = identity // Always keep the first party we registered, as that's the well known identity @@ -79,26 +56,7 @@ class InMemoryIdentityService(identities: List = emptyList( // We give the caller a copy of the data set to avoid any locking problems override fun getAllIdentities(): Iterable = ArrayList(keyToParties.values) - override fun partyFromKey(key: PublicKey): Party? = keyToParties[key]?.party override fun wellKnownPartyFromX500Name(name: CordaX500Name): Party? = principalToParties[name]?.party - override fun wellKnownPartyFromAnonymous(party: AbstractParty): Party? { - // The original version of this would return the party as-is if it was a Party (rather than AnonymousParty), - // however that means that we don't verify that we know who owns the key. As such as now enforce turning the key - // into a party, and from there figure out the well known party. - val candidate = partyFromKey(party.owningKey) - // TODO: This should be done via the network map cache, which is the authoritative source of well known identities - return if (candidate != null) { - require(party.nameOrNull() == null || party.nameOrNull() == candidate.name) { "Candidate party $candidate does not match expected $party" } - wellKnownPartyFromX500Name(candidate.name) - } else { - null - } - } - - override fun wellKnownPartyFromAnonymous(partyRef: PartyAndReference) = wellKnownPartyFromAnonymous(partyRef.party) - override fun requireWellKnownPartyFromAnonymous(party: AbstractParty): Party { - return wellKnownPartyFromAnonymous(party) ?: throw IllegalStateException("Could not deanonymise party ${party.owningKey.toStringShort()}") - } override fun partiesFromName(query: String, exactMatch: Boolean): Set { val results = LinkedHashSet() @@ -107,14 +65,4 @@ class InMemoryIdentityService(identities: List = emptyList( } return results } - - @Throws(UnknownAnonymousPartyException::class) - override fun assertOwnership(party: Party, anonymousParty: AnonymousParty) { - val anonymousIdentity = keyToParties[anonymousParty.owningKey] ?: - throw UnknownAnonymousPartyException("Unknown $anonymousParty") - val issuingCert = anonymousIdentity.certPath.certificates[1] - require(issuingCert.publicKey == party.owningKey) { - "Issuing certificate's public key must match the party key ${party.owningKey.toStringShort()}." - } - } } diff --git a/node/src/main/kotlin/net/corda/node/services/identity/PersistentIdentityService.kt b/node/src/main/kotlin/net/corda/node/services/identity/PersistentIdentityService.kt index 59a37b21fc..baa1899cba 100644 --- a/node/src/main/kotlin/net/corda/node/services/identity/PersistentIdentityService.kt +++ b/node/src/main/kotlin/net/corda/node/services/identity/PersistentIdentityService.kt @@ -1,10 +1,7 @@ package net.corda.node.services.identity -import net.corda.core.contracts.PartyAndReference import net.corda.core.crypto.SecureHash -import net.corda.core.crypto.toStringShort import net.corda.core.identity.* -import net.corda.core.internal.CertRole import net.corda.core.internal.hash import net.corda.core.node.services.UnknownAnonymousPartyException import net.corda.core.serialization.SingletonSerializeAsToken @@ -14,7 +11,6 @@ import net.corda.core.utilities.debug import net.corda.node.services.api.IdentityServiceInternal import net.corda.node.utilities.AppendOnlyPersistentMap import net.corda.nodeapi.internal.crypto.X509CertificateFactory -import net.corda.nodeapi.internal.crypto.X509Utilities import net.corda.nodeapi.internal.crypto.x509Certificates import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.NODE_DATABASE_PREFIX @@ -35,7 +31,6 @@ import javax.persistence.Lob * @param trustRoot certificate from the zone operator for identity on the network. * @param caCertificates list of additional certificates. */ -// TODO There is duplicated logic between this and InMemoryIdentityService @ThreadSafe class PersistentIdentityService(override val trustRoot: X509Certificate, private val database: CordaPersistence, @@ -127,38 +122,21 @@ class PersistentIdentityService(override val trustRoot: X509Certificate, @Throws(CertificateExpiredException::class, CertificateNotYetValidException::class, InvalidAlgorithmParameterException::class) override fun verifyAndRegisterIdentity(identity: PartyAndCertificate): PartyAndCertificate? { return database.transaction { - - // Validate the chain first, before we do anything clever with it - val identityCertChain = identity.certPath.x509Certificates - try { - identity.verify(trustAnchor) - } catch (e: CertPathValidatorException) { - log.warn(e.localizedMessage) - log.warn("Path = ") - identityCertChain.reversed().forEach { - log.warn(it.subjectX500Principal.toString()) - } - throw e - } - - // Ensure we record the first identity of the same name, first - val wellKnownCert = identityCertChain.single { CertRole.extract(it)?.isWellKnown ?: false } - if (wellKnownCert != identity.certificate) { - val idx = identityCertChain.lastIndexOf(wellKnownCert) - val firstPath = X509Utilities.buildCertPath(identityCertChain.slice(idx until identityCertChain.size)) - verifyAndRegisterIdentity(PartyAndCertificate(firstPath)) - } - - log.debug { "Registering identity $identity" } - val key = mapToKey(identity) - keyToParties.addWithDuplicatesAllowed(key, identity) - // Always keep the first party we registered, as that's the well known identity - principalToParties.addWithDuplicatesAllowed(identity.name, key, false) - val parentId = mapToKey(identityCertChain[1].publicKey) - keyToParties[parentId] + verifyAndRegisterIdentity(trustAnchor, identity) } } + override fun registerIdentity(identity: PartyAndCertificate): PartyAndCertificate? { + val identityCertChain = identity.certPath.x509Certificates + log.debug { "Registering identity $identity" } + val key = mapToKey(identity) + keyToParties.addWithDuplicatesAllowed(key, identity) + // Always keep the first party we registered, as that's the well known identity + principalToParties.addWithDuplicatesAllowed(identity.name, key, false) + val parentId = mapToKey(identityCertChain[1].publicKey) + return keyToParties[parentId] + } + override fun certificateFromKey(owningKey: PublicKey): PartyAndCertificate? = database.transaction { keyToParties[mapToKey(owningKey)] } private fun certificateFromCordaX500Name(name: CordaX500Name): PartyAndCertificate? { @@ -173,27 +151,9 @@ class PersistentIdentityService(override val trustRoot: X509Certificate, // We give the caller a copy of the data set to avoid any locking problems override fun getAllIdentities(): Iterable = database.transaction { keyToParties.allPersisted().map { it.second }.asIterable() } - override fun partyFromKey(key: PublicKey): Party? = certificateFromKey(key)?.party override fun wellKnownPartyFromX500Name(name: CordaX500Name): Party? = certificateFromCordaX500Name(name)?.party - override fun wellKnownPartyFromAnonymous(party: AbstractParty): Party? { - return database.transaction { - // The original version of this would return the party as-is if it was a Party (rather than AnonymousParty), - // however that means that we don't verify that we know who owns the key. As such as now enforce turning the key - // into a party, and from there figure out the well known party. - val candidate = partyFromKey(party.owningKey) - // TODO: This should be done via the network map cache, which is the authoritative source of well known identities - if (candidate != null) { - wellKnownPartyFromX500Name(candidate.name) - } else { - null - } - } - } - override fun wellKnownPartyFromAnonymous(partyRef: PartyAndReference) = wellKnownPartyFromAnonymous(partyRef.party) - override fun requireWellKnownPartyFromAnonymous(party: AbstractParty): Party { - return wellKnownPartyFromAnonymous(party) ?: throw IllegalStateException("Could not deanonymise party ${party.owningKey.toStringShort()}") - } + override fun wellKnownPartyFromAnonymous(party: AbstractParty): Party? = database.transaction { super.wellKnownPartyFromAnonymous(party) } override fun partiesFromName(query: String, exactMatch: Boolean): Set { return database.transaction { @@ -206,13 +166,6 @@ class PersistentIdentityService(override val trustRoot: X509Certificate, } @Throws(UnknownAnonymousPartyException::class) - override fun assertOwnership(party: Party, anonymousParty: AnonymousParty) { - database.transaction { - val anonymousIdentity = certificateFromKey(anonymousParty.owningKey) ?: throw UnknownAnonymousPartyException("Unknown $anonymousParty") - val issuingCert = anonymousIdentity.certPath.certificates[1] - require(issuingCert.publicKey == party.owningKey) { - "Issuing certificate's public key must match the party key ${party.owningKey.toStringShort()}." - } - } - } + override fun assertOwnership(party: Party, anonymousParty: AnonymousParty) = database.transaction { super.assertOwnership(party, anonymousParty) } + }