From 3cf1450fc1839a7fbf134f75b3b95004367cdce1 Mon Sep 17 00:00:00 2001 From: Michele Sollecito Date: Wed, 3 Oct 2018 18:16:07 +0200 Subject: [PATCH] [CORDA-2066]: `setting-up-a-corda-network` docs file is misleading (fixed). (#4025) --- docs/source/corda-networks-index.rst | 2 +- docs/source/corda-test-networks.rst | 77 +++++++++ docs/source/setting-up-a-corda-network.rst | 190 --------------------- 3 files changed, 78 insertions(+), 191 deletions(-) create mode 100644 docs/source/corda-test-networks.rst delete mode 100644 docs/source/setting-up-a-corda-network.rst diff --git a/docs/source/corda-networks-index.rst b/docs/source/corda-networks-index.rst index 67f544d69c..d1bf86f03e 100644 --- a/docs/source/corda-networks-index.rst +++ b/docs/source/corda-networks-index.rst @@ -5,7 +5,7 @@ Networks :maxdepth: 1 joining-a-network - setting-up-a-corda-network + corda-test-networks running-a-notary permissioning network-map diff --git a/docs/source/corda-test-networks.rst b/docs/source/corda-test-networks.rst new file mode 100644 index 0000000000..f5c2991fcf --- /dev/null +++ b/docs/source/corda-test-networks.rst @@ -0,0 +1,77 @@ +.. _log4j2: http://logging.apache.org/log4j/2.x/ + +Corda networks +============== + +A Corda network consists of a number of machines running nodes. These nodes communicate using persistent protocols in +order to create and validate transactions. + +There are three broader categories of functionality one such node may have. These pieces of functionality are provided +as services, and one node may run several of them. + +* Notary: Nodes running a notary service witness state spends and have the final say in whether a transaction is a + double-spend or not +* Oracle: Network services that link the ledger to the outside world by providing facts that affect the validity of + transactions +* Regular node: All nodes have a vault and may start protocols communicating with other nodes, notaries and oracles and + evolve their private ledger + +Bootstrap your own test network +------------------------------- + +Certificates +~~~~~~~~~~~~ + +Every node in a given Corda network must have an identity certificate signed by the network's root CA. See +:doc:`permissioning` for more information. + +Configuration +~~~~~~~~~~~~~ + +A node can be configured by adding/editing ``node.conf`` in the node's directory. For details see :doc:`corda-configuration-file`. + +An example configuration: + +.. literalinclude:: example-code/src/main/resources/example-node.conf +:language: cfg + + The most important fields regarding network configuration are: + + * ``p2pAddress``: This specifies a host and port to which Artemis will bind for messaging with other nodes. Note that the + address bound will **NOT** be ``my-corda-node``, but rather ``::`` (all addresses on all network interfaces). The hostname specified + is the hostname *that must be externally resolvable by other nodes in the network*. In the above configuration this is the + resolvable name of a machine in a VPN. +* ``rpcAddress``: The address to which Artemis will bind for RPC calls. +* ``webAddress``: The address the webserver should bind. Note that the port must be distinct from that of ``p2pAddress`` and ``rpcAddress`` if they are on the same machine. + +Starting the nodes +~~~~~~~~~~~~~~~~~~ + +You will first need to create the local network by bootstrapping it with the bootstrapper. Details of how to do that +can be found in :doc:`network-bootstrapper`. + +Once that's done you may now start the nodes in any order. You should see a banner, some log lines and eventually +``Node started up and registered``, indicating that the node is fully started. + +.. TODO: Add a better way of polling for startup. A programmatic way of determining whether a node is up is to check whether it's ``webAddress`` is bound. + +In terms of process management there is no prescribed method. You may start the jars by hand or perhaps use systemd and friends. + +Logging +~~~~~~~ + +Only a handful of important lines are printed to the console. For +details/diagnosing problems check the logs. + +Logging is standard log4j2_ and may be configured accordingly. Logs +are by default redirected to files in ``NODE_DIRECTORY/logs/``. + +Connecting to the nodes +~~~~~~~~~~~~~~~~~~~~~~~ + +Once a node has started up successfully you may connect to it as a client to initiate protocols/query state etc. +Depending on your network setup you may need to tunnel to do this remotely. + +See the :doc:`tutorial-clientrpc-api` on how to establish an RPC link. + +Sidenote: A client is always associated with a single node with a single identity, which only sees their part of the ledger. diff --git a/docs/source/setting-up-a-corda-network.rst b/docs/source/setting-up-a-corda-network.rst deleted file mode 100644 index 320cca5f0d..0000000000 --- a/docs/source/setting-up-a-corda-network.rst +++ /dev/null @@ -1,190 +0,0 @@ -.. _log4j2: http://logging.apache.org/log4j/2.x/ - -Setting up a Corda network -========================== - -.. contents:: - -Bootstrapping a development network ------------------------------------ - -When testing CorDapps during development, you should use the :doc:`bootstrapper tool ` to create -a local test network. - -Creating your own compatibility zone ------------------------------------- - -This section documents how to implement your own doorman and network map servers, which is the basic process required to -create a dedicated zone. At this time we do not provide tooling to do this, because the needs of each zone are different -and no generic, configurable doorman codebase has been written. - -Do you need a zone? -^^^^^^^^^^^^^^^^^^^ - -Think twice before going down this route: - -1. It isn't necessary for testing. -2. It isn't necessary for adding another layer of permissioning or 'know your customer' requirements onto your app. - -**Testing.** Creating a production-ready zone isn't necessary for testing as you can use the :doc:`network bootstrapper ` -tool to create all the certificates, keys, and distribute the needed map files to run many nodes. The bootstrapper can -create a network locally on your desktop/laptop but it also knows how to automate cloud providers via their APIs and -using Docker. In this way you can bring up a simulation of a real Corda network with different nodes on different -machines in the cloud for your own testing. Testing this way has several advantages, most obviously that you avoid -race conditions in your tests caused by nodes/tests starting before all map data has propagated to all nodes. -You can read more about the reasons for the creation of the bootstrapper tool -`in a blog post on the design thinking behind Corda's network map infrastructure `__. - -**Permissioning.** And creating a zone is also unnecessary for imposing permissioning requirements beyond that of the -base Corda network. You can control who can use your app by creating a *business network*. A business network is what we -call a coalition of nodes that have chosen to run a particular app within a given commercial context. Business networks -aren't represented in the Corda API at this time, partly because the technical side is so simple. You can create one -via a simple three step process: - -1. Distribute a list of X.500 names that are members of your business network, e.g. a simple way to do this is by - hosting a text file with one name per line on your website at a fixed HTTPS URL. You could also write a simple - request/response flow that serves the list over the Corda protocol itself, although this requires the business - network to have a node for itself. -2. Write a bit of code that downloads and caches the contents of this file on disk, and which loads it into memory in - the node. A good place to do this is in a class annotated with ``@CordaService``, because this class can expose - a ``Set`` field representing the membership of your service. -3. In your flows use ``serviceHub.findService`` to get a reference to your ``@CordaService`` class, read the list of - members and at the start of each flow, throw a FlowException if the counterparty isn't in the membership list. - -In this way you can impose a centrally controlled ACL that all members will collectively enforce. - -.. note:: A production-ready Corda network and a new iteration of the testnet will be available soon. - -Why create your own zone? -^^^^^^^^^^^^^^^^^^^^^^^^^ - -The primary reason to create a zone and provide the associated infrastructure is control over *network parameters*. These -are settings that control Corda's operation, and on which all users in a network must agree. Failure to agree would create -the Corda equivalent of a blockchain "hard fork". Parameters control things like the root of identity, -how quickly users should upgrade, how long nodes can be offline before they are evicted from the system and so on. - -Creating a zone involves the following steps: - -1. Create the zone private keys and certificates. This procedure is conventional and no special knowledge is required: - any self-signed set of certificates can be used. A professional quality zone will probably keep the keys inside a - hardware security module (as the main Corda network and test networks do). -2. Write a network map server. -3. Optionally, create a doorman server. -4. Finally, you would select and generate your network parameter file. - -Writing a network map server -^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -This server implements a simple HTTP based protocol described in the ":doc:`network-map`" page. -The map server is responsible for gathering NodeInfo files from nodes, storing them, and distributing them back to the -nodes in the zone. By doing this it is also responsible for choosing who is in and who is out: having a signed -identity certificate is not enough to be a part of a Corda zone, you also need to be listed in the network map. -It can be thought of as a DNS equivalent. If you want to de-list a user, you would do it here. - -It is very likely that your map server won't be entirely standalone, but rather, integrated with whatever your master -user database is. - -The network map server also distributes signed network parameter files and controls the roll-out schedule for when they -become available for download and opt-in, and when they become enforced. This is again a policy decision you will -probably choose to place some simple UI or workflow tooling around, in particular to enforce restrictions on who can -edit the map or the parameters. - -Writing a doorman server -^^^^^^^^^^^^^^^^^^^^^^^^ - -This step is optional because your users can obtain a signed certificate in many different ways. The doorman protocol -is again a very simple HTTP based approach in which a node creates keys and requests a certificate, polling until it -gets back what it expects. However, you could also integrate this process with the rest of your signup process. For example, -by building a tool that's integrated with your payment flow (if payment is required to take part in your zone at all). -Alternatively you may wish to distribute USB smartcard tokens that generate the private key on first use, as is typically -seen in national PKIs. There are many options. - -If you do choose to make a doorman server, the bulk of the code you write will be workflow related. For instance, -related to keeping track of an applicant as they proceed through approval. You should also impose any naming policies -you have in the doorman process. If names are meant to match identities registered in government databases then that -should be enforced here, alternatively, if names can be self-selected or anonymous, you would only bother with a -deduplication check. Again it will likely be integrated with a master user database. - -Corda does not currently provide a doorman or network map service out of the box, partly because when stripped of the -zone specific policy there isn't much to them: just a basic HTTP server that most programmers will have favourite -frameworks for anyway. - -The protocol is: - -* If $URL = ``https://some.server.com/some/path`` -* Node submits a PKCS#10 certificate signing request using HTTP POST to ``$URL/certificate``. It will have a MIME - type of ``application/octet-stream``. The ``Platform-Version`` header is set to be "1.0" and the ``Client-Version`` header to reflect the node software version -* The server returns an opaque string that references this request (let's call it ``$requestid``, or an HTTP error if something went wrong -* The returned request ID should be persisted to disk, to handle zones where approval may take a long time due to manual - intervention being required -* The node starts polling ``$URL/$requestid`` using HTTP GET. The poll interval can be controlled by the server returning - a response with a ``Cache-Control`` header -* If the request is answered with a ``200 OK`` response, the body is expected to be a zip file. Each file is expected to - be a binary X.509 certificate, and the certs are expected to be in order -* If the request is answered with a ``204 No Content`` response, the node will try again later -* If the request is answered with a ``403 Not Authorized`` response, the node will treat that as request rejection and give up -* Other response codes will cause the node to abort with an exception - -You can use any standard key tools to create the required key pairs and certificates. The ``X509Utilities`` class in the -`Corda repository -`__ -shows how to generate the required key pairs and certificates using Bouncy Castle. - -Setting zone parameters -^^^^^^^^^^^^^^^^^^^^^^^ - -Zone parameters are stored in a file containing a Corda AMQP serialised ``SignedDataWithCert`` -object. It is easy to create such a file with a small Java or Kotlin program. The ``NetworkParameters`` object is a -simple data holder that could be read from e.g. a config file, or settings from a database. Signing and saving the -resulting file is just a few lines of code. A full example can be found in `NetworkParametersCopier.kt -`__, -but a flavour of it looks like this: - -.. container:: codeset - - .. sourcecode:: java - - NetworkParameters networkParameters = new NetworkParameters( - 4, // minPlatformVersion - Collections.emptyList(), // notaries - 1024 * 1024 * 20, // maxMessageSize - 1024 * 1024 * 15, // maxTransactionSize - Instant.now(), // modifiedTime - 2, // epoch - Collections.emptyMap() // whitelist - ); - CertificateAndKeyPair signingCertAndKeyPair = loadNetworkMapCA(); - SerializedBytes> bytes = SerializedBytes.from(netMapCA.sign(networkParameters)); - Files.copy(bytes.open(), Paths.get("params-file")); - - .. sourcecode:: kotlin - - val networkParameters = NetworkParameters( - minimumPlatformVersion = 4, - notaries = listOf(...), - maxMessageSize = 1024 * 1024 * 20 // 20mb, for example. - maxTransactionSize = 1024 * 1024 * 15, - modifiedTime = Instant.now(), - epoch = 2, - ... etc ... - ) - val signingCertAndKeyPair: CertificateAndKeyPair = loadNetworkMapCA() - val signedParams: SerializedBytes = signingCertAndKeyPair.sign(networkParameters).serialize() - signedParams.open().copyTo(Paths.get("/some/path")) - -Each individual parameter is documented in `the JavaDocs/KDocs for the NetworkParameters class -`__. The network map -certificate is usually chained off the root certificate, and can be created according to the instructions above. Each -time the zone parameters are changed, the epoch should be incremented. Epochs are essentially version numbers for the -parameters, and they therefore cannot go backwards. Once saved, the new parameters can be served by the network map server. - -Selecting parameter values -^^^^^^^^^^^^^^^^^^^^^^^^^^ - -How to choose the parameters? This is the most complex question facing you as a new zone operator. Some settings may seem -straightforward and others may involve cost/benefit trade-offs specific to your business. For example, you could choose -to run a validating notary yourself, in which case you would (in the absence of SGX) see all the users' data. Or you could -run a non-validating notary, with BFT fault tolerance, which implies recruiting others to take part in the cluster. - -New network parameters will be added over time as Corda evolves. You will need to ensure that when your users upgrade, -all the new network parameters are being served. You can ask for advice on the `corda-dev mailing list `__. \ No newline at end of file