corda/docs/source/network-bootstrapper.rst
Anthony Keenan b7d04b1c6e
[CORDA-2235]: Add overrides for network parameters via command line and file (#4279)
* Temp commit

* Print the error message first by default, makes error output more natural.

* Polishing

* Further modifications after testing

* Documentation updates

* Couple of fixes after review

* Removing unnecessary tests

* Fix broken test

* Add interface to bootstrapper for testign

* Added unit tests

* Remove unused class

* Fix up bootstrapper unit tests and add a couple more

* Refactor the tests slightly

* Review comments

* Couple of minor tweaks
2018-11-26 17:11:05 +00:00

20 KiB

Network Bootstrapper

Test deployments

Nodes within a network see each other using the network map. This is a collection of statically signed node-info files, one for each node. Most production deployments will use a highly available, secure distribution of the network map via HTTP.

For test deployments where the nodes (at least initially) reside on the same filesystem, these node-info files can be placed directly in the node's additional-node-infos directory from where the node will pick them up and store them in its local network map cache. The node generates its own node-info file on startup.

In addition to the network map, all the nodes must also use the same set of network parameters. These are a set of constants which guarantee interoperability between the nodes. The HTTP network map distributes the network parameters which are downloaded automatically by the nodes. In the absence of this the network parameters must be generated locally.

For these reasons, test deployments can avail themselves of the Network Bootstrapper. This is a tool that scans all the node configurations from a common directory to generate the network parameters file, which is then copied to all the nodes' directories. It also copies each node's node-info file to every other node so that they can all be visible to each other.

You can find out more about network maps and network parameters from network-map.

Bootstrapping a test network

The Corda Network Bootstrapper can be downloaded from here.

Create a directory containing a node config file, ending in "_node.conf", for each node you want to create. Then run the following command:

java -jar network-bootstrapper-VERSION.jar --dir <nodes-root-dir>

For example running the command on a directory containing these files:

.
├── notary_node.conf             // The notary's node.conf file
├── partya_node.conf             // Party A's node.conf file
└── partyb_node.conf             // Party B's node.conf file

will generate directories containing three nodes: notary, partya and partyb. They will each use the corda.jar that comes with the Network Bootstrapper. If a different version of Corda is required then simply place that corda.jar file alongside the configuration files in the directory.

You can also have the node directories containing their "node.conf" files already laid out. The previous example would be:

.
├── notary
│   └── node.conf
├── partya
│   └── node.conf
└── partyb
    └── node.conf

Similarly, each node directory may contain its own corda.jar, which the Bootstrapper will use instead.

Providing CorDapps to the Network Bootstrapper

If you would like the Network Bootstrapper to include your CorDapps in each generated node, just place them in the directory alongside the config files. For example, if your directory has this structure:

.
├── notary_node.conf            // The notary's node.conf file
├── partya_node.conf            // Party A's node.conf file
├── partyb_node.conf            // Party B's node.conf file
├── cordapp-a.jar               // A cordapp to be installed on all nodes
└── cordapp-b.jar               // Another cordapp to be installed on all nodes

The cordapp-a.jar and cordapp-b.jar will be installed in each node directory, and any contracts within them will be added to the Contract Whitelist (see below).

Whitelisting contracts

Any CorDapps provided when bootstrapping a network will be scanned for contracts which will be used to create the Zone whitelist (see api-contract-constraints) for the network.

Note

If you only wish to whitelist the CorDapps but not copy them to each node then run with the --no-copy flag.

The CorDapp JARs will be hashed and scanned for Contract classes. These contract class implementations will become part of the whitelisted contracts in the network parameters (see NetworkParameters.whitelistedContractImplementations network-map).

By default the Bootstrapper will whitelist all the contracts found in the unsigned CorDapp JARs (a JAR file not signed by jarSigner tool). Whitelisted contracts are checked by Zone constraints, while contract classes from signed JARs will be checked by Signature constraints. To prevent certain contracts from unsigned JARs from being whitelisted, add their fully qualified class name in the exclude_whitelist.txt. These will instead use the more restrictive HashAttachmentConstraint. Refer to api-contract-constraints to understand the implication of different constraint types before adding exclude_whitelist.txt files.

For example:

net.corda.finance.contracts.asset.Cash
net.corda.finance.contracts.asset.CommercialPaper

Modifying a bootstrapped network

The Network Bootstrapper is provided as a development tool for setting up Corda networks for development and testing. There is some limited functionality which can be used to make changes to a network, but for anything more complicated consider using a network-map server.

When running the Network Bootstrapper, each node-info file needs to be gathered together in one directory. If the nodes are being run on different machines you need to do the following:

  • Copy the node directories from each machine into one directory, on one machine
  • Depending on the modification being made (see below for more information), add any new files required to the root directory
  • Run the Network Bootstrapper from the root directory
  • Copy each individual node's directory back to the original machine

The Network Bootstrapper cannot dynamically update the network if an existing node has changed something in their node-info, e.g. their P2P address. For this the new node-info file will need to be placed in the other nodes' additional-node-infos directory. If the nodes are located on different machines, then a utility such as rsync can be used so that the nodes can share node-infos.

Adding a new node to the network

Running the Bootstrapper again on the same network will allow a new node to be added and its node-info distributed to the existing nodes.

As an example, if we have an existing bootstrapped network, with a Notary and PartyA and we want to add a PartyB, we can use the Network Bootstrapper on the following network structure:

.
├── notary                      // existing node directories
│   ├── node.conf
│   ├── network-parameters
│   ├── node-info-notary
│   └── additional-node-infos
│       ├── node-info-notary
│       └── node-info-partya
├── partya
│   ├── node.conf
│   ├── network-parameters
│   ├── node-info-partya
│   └── additional-node-infos
│       ├── node-info-notary
│       └── node-info-partya
└── partyb_node.conf            // the node.conf for the node to be added

Then run the Network Bootstrapper again from the root dir:

java -jar network-bootstrapper-VERSION.jar --dir <nodes-root-dir>

Which will give the following:

.
├── notary                      // the contents of the existing nodes (keys, db's etc...) are unchanged
│   ├── node.conf
│   ├── network-parameters
│   ├── node-info-notary
│   └── additional-node-infos
│       ├── node-info-notary
│       ├── node-info-partya
│       └── node-info-partyb
├── partya
│   ├── node.conf
│   ├── network-parameters
│   ├── node-info-partya
│   └── additional-node-infos
│       ├── node-info-notary
│       ├── node-info-partya
│       └── node-info-partyb
└── partyb                      // a new node directory is created for PartyB
    ├── node.conf
    ├── network-parameters
    ├── node-info-partyb
    └── additional-node-infos
        ├── node-info-notary
        ├── node-info-partya
        └── node-info-partyb

The Bootstrapper will generate a directory and the node-info file for PartyB, and will also make sure a copy of each nodes' node-info file is in the additional-node-info directory of every node. Any other files in the existing nodes, such a generated keys, will be unaffected.

Note

The Network Bootstrapper is provided for test deployments and can only generate information for nodes collected on the same machine. If a network needs to be updated using the Bootstrapper once deployed, the nodes will need collecting back together.

Updating the contract whitelist for bootstrapped networks

If the network already has a set of network parameters defined (i.e. the node directories all contain the same network-parameters file) then the Network Bootstrapper can be used to append contracts from new CorDapps to the current whitelist. For example, with the following pre-generated network:

.
├── notary
│   ├── node.conf
│   ├── network-parameters
│   └── cordapps
│       └── cordapp-a.jar
├── partya
│   ├── node.conf
│   ├── network-parameters
│   └── cordapps
│       └── cordapp-a.jar
├── partyb
│   ├── node.conf
│   ├── network-parameters
│   └── cordapps
│       └── cordapp-a.jar
└── cordapp-b.jar               // The new cordapp to add to the existing nodes

Then run the Network Bootstrapper again from the root dir:

java -jar network-bootstrapper-VERSION.jar --dir <nodes-root-dir>

To give the following:

.
├── notary
│   ├── node.conf
│   ├── network-parameters      // The contracts from cordapp-b are appended to the whitelist in network-parameters
│   └── cordapps
│       ├── cordapp-a.jar
│       └── cordapp-b.jar       // The updated cordapp is placed in the nodes cordapp directory
├── partya
│   ├── node.conf
│   ├── network-parameters      // The contracts from cordapp-b are appended to the whitelist in network-parameters
│   └── cordapps
│       ├── cordapp-a.jar
│       └── cordapp-b.jar       // The updated cordapp is placed in the nodes cordapp directory
└── partyb
    ├── node.conf
    ├── network-parameters      // The contracts from cordapp-b are appended to the whitelist in network-parameters
    └── cordapps
        ├── cordapp-a.jar
        └── cordapp-b.jar       // The updated cordapp is placed in the nodes cordapp directory

Note

The whitelist can only ever be appended to. Once added a contract implementation can never be removed.

Modifying the network parameters

The Network Bootstrapper creates a network parameters file when bootstrapping a network, using a set of sensible defaults. However, if you would like to override these defaults when testing, there are two ways of doing this. Options can be overridden via the command line or by supplying a configuration file. If the same parameter is overridden both by a command line argument and in the configuration file, the command line value will take precedence.

Overriding network parameters via command line

The --minimum-platform-version, --max-message-size, --max-transaction-size and --event-horizon command line parameters can be used to override the default network parameters. See Command line options for more information.

Overriding network parameters via a file

You can provide a network parameters overrides file using the following syntax:

java -jar network-bootstrapper-VERSION.jar --network-parameters-overrides=<path_to_file>

Or alternatively, by using the short form version:

java -jar network-bootstrapper-VERSION.jar -n=<path_to_file>

The network parameter overrides file is a HOCON file with the following fields, all of which are optional. Any field that is not provided will be ignored. If a field is not provided and you are bootstrapping a new network, a sensible default value will be used. If a field is not provided and you are updating an existing network, the value in the existing network parameters file will be used.

Note

All fields can be used with placeholders for environment variables. For example: ${KEY_STORE_PASSWORD} would be replaced by the contents of environment

variable KEY_STORE_PASSWORD. See: corda-configuration-hiding-sensitive-data .

The available configuration fields are listed below:

minimumPlatformVersion

The minimum supported version of the Corda platform that is required for nodes in the network.

maxMessageSize

The maximum permitted message size, in bytes. This is currently ignored but will be used in a future release.

maxTransactionSize

The maximum permitted transaction size, in bytes.

eventHorizon

The time after which nodes will be removed from the network map if they have not been seen during this period. This parameter uses the parse function on the java.time.Duration class to interpret the data. See here for information on valid inputs.

packageOwnership

A list of package owners. See Package namespace ownership for more information. For each package owner, the following fields are required:

packageName

Java package name (e.g com.my_company ).

keystore

The path of the keystore file containing the signed certificate.

keystorePassword

The password for the given keystore (not to be confused with the key password).

keystoreAlias

The alias for the name associated with the certificate to be associated with the package namespace.

An example configuration file:

minimumPlatformVersion=4 maxMessageSize=10485760 maxTransactionSize=524288000 eventHorizon="30 days" packageOwnership=[ { packageName="com.example" keystore="myteststore" keystorePassword="MyStorePassword" keystoreAlias="MyKeyAlias" } ]

Package namespace ownership

Package namespace ownership is a Corda security feature that allows a compatibility zone to give ownership of parts of the Java package namespace to registered users (e.g. a CorDapp development organisation). The exact mechanism used to claim a namespace is up to the zone operator. A typical approach would be to accept an SSL certificate with the domain in it as proof of domain ownership, or to accept an email from that domain.

Note

Read more about Package ownership here<design/data-model-upgrades/package-namespace-ownership>.

A Java package namespace is case insensitive and cannot be a sub-package of an existing registered namespace. See Naming a Package and Naming Conventions for guidelines on naming conventions.

The registration of a Java package namespace requires the creation of a signed certificate as generated by the Java keytool.

The packages can be registered by supplying a network parameters override config file via the command line, using the --network-parameters-overrides command.

For each package to be registered, the following are required:

packageName

Java package name (e.g com.my_company ).

keystore

The path of the keystore file containing the signed certificate. If a relative path is provided, it is assumed to be relative to the location of the configuration file.

keystorePassword

The password for the given keystore (not to be confused with the key password).

keystoreAlias

The alias for the name associated with the certificate to be associated with the package namespace.

Using the Example CorDapp as an example, we will initialise a simple network and then register and unregister a package namespace. Checkout the Example CorDapp and follow the instructions to build it here.

Note

You can point to any existing bootstrapped corda network (this will have the effect of updating the associated network parameters file).

  1. Create a new public key to use for signing the Java package namespace we wish to register:

    $JAVA_HOME/bin/keytool -genkeypair -keystore _teststore -storepass MyStorePassword -keyalg RSA -alias MyKeyAlias -keypass MyKeyPassword -dname "O=Alice Corp, L=Madrid, C=ES"

    This will generate a key store file called _teststore in the current directory.

  2. Create a network-parameters.conf file in the same directory, with the following information:

    packageOwnership=[
    {

    packageName="com.example" keystore="_teststore" keystorePassword="MyStorePassword" keystoreAlias="MyKeyAlias"

    }

    ]

  3. Register the package namespace to be claimed by the public key generated above:

    # Register the Java package namespace using the Network Bootstrapper
    java -jar network-bootstrapper.jar --dir build/nodes --network-parameter-overrides=network-parameters.conf
  4. To unregister the package namespace, edit the network-parameters.conf file to remove the package:

    packageOwnership=[]

  5. Unregister the package namespace:

    # Unregister the Java package namespace using the Network Bootstrapper
    java -jar network-bootstrapper.jar --dir build/nodes --network-parameter-overrides=network-parameters.conf

Command line options

The Network Bootstrapper can be started with the following command line options:

bootstrapper [-hvV] [--no-copy] [--dir=<dir>] [--event-horizon=<eventHorizon>]
         [--logging-level=<loggingLevel>]
         [--max-message-size=<maxMessageSize>]
         [--max-transaction-size=<maxTransactionSize>]
         [--minimum-platform-version=<minimumPlatformVersion>]
         [-n=<networkParametersFile>] [COMMAND]
  • --dir=<dir>: Root directory containing the node configuration files and CorDapp JARs that will form the test network.

    It may also contain existing node directories. Defaults to the current directory.

  • --no-copy: Don't copy the CorDapp JARs into the nodes' "cordapps" directories.
  • --verbose, --log-to-console, -v: If set, prints logging to the console as well as to a file.
  • --logging-level=<loggingLevel>: Enable logging at this level and higher. Possible values: ERROR, WARN, INFO, DEBUG, TRACE. Default: INFO.
  • --help, -h: Show this help message and exit.
  • --version, -V: Print version information and exit.
  • --minimum-platform-version: The minimum platform version to use in the network-parameters.
  • --max-message-size: The maximum message size to use in the network-parameters, in bytes.
  • --max-transaction-size: The maximum transaction size to use in the network-parameters, in bytes.
  • --event-horizon: The event horizon to use in the network-parameters.
  • --network-parameter-overrides=<networkParametersFile>, ``-n=<networkParametersFile>: Overrides the default network parameters with those in the given file. See `Overriding network parameters via a file_ for more information.

Sub-commands

install-shell-extensions: Install bootstrapper alias and auto completion for bash and zsh. See cli-application-shell-extensions for more info.