Merge remote-tracking branch 'open/master' into anthony-os-merge-20190914-2

# Conflicts:
#	docs/source/index.rst
#	docs/source/running-a-node.rst
#	node/build.gradle

docs/source/blob-inspector.rst

@@ -97,3 +97,25 @@ Here's what a node-info file from the node's data directory may look like:
 
 Notice the file is actually a serialised ``SignedNodeInfo`` object, which has a ``raw`` property of type ``SerializedBytes<NodeInfo>``.
 This property is materialised into a ``NodeInfo`` and is output under the ``deserialized`` field.
+
+Command-line options
+~~~~~~~~~~~~~~~~~~~~
+
+The blob inspector can be started with the following command-line options:
+
+.. code-block:: shell
+
+    blob-inspector [-hvV] [--full-parties] [--install-shell-extensions] [--schema]
+                   [--format=type] [--input-format=type]
+                   [--logging-level=<loggingLevel>] [SOURCE]
+
+* ``--format=type``: Output format. Possible values: [YAML, JSON]. Default: YAML.
+* ``--input-format=type``: Input format. If the file can't be decoded with the given value it's auto-detected, so you should
+  never normally need to specify this. Possible values [BINARY, HEX, BASE64]. Default: BINARY.
+* ``--full-parties``: Display the owningKey and certPath properties of Party and PartyAndReference objects respectively.
+* ``--schema``: Print the blob's schema first.
+* ``--verbose``, ``--log-to-console``, ``-v``: If set, prints logging to the console as well as to a file.
+* ``--logging-level=<loggingLevel>``: Enable logging at this level and higher. Possible values: ERROR, WARN, INFO, DEBUG, TRACE. Default: INFO.
+* ``--install-shell-extensions``: Install ``blob-inspector`` alias and auto completion for bash and zsh. See :doc:`cli-application-shell-extensions` for more info.
+* ``--help``, ``-h``: Show this help message and exit.
+* ``--version``, ``-V``: Print version information and exit.

docs/source/cli-application-shell-extensions.rst

@@ -0,0 +1,76 @@
+Shell extensions for CLI Applications
+=====================================
+
+.. _installing-shell-extensions:
+
+Installing shell extensions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Users of ``bash`` or ``zsh`` can install an alias and auto-completion for Corda applications that contain a command line interface. Run:
+
+.. code-block:: shell
+
+   java -jar <name-of-JAR>.jar --install-shell-extensions
+
+Then, either restart your shell, or for ``bash`` users run:
+
+.. code-block:: shell
+
+   . ~/.bashrc
+
+Or, for ``zsh`` run:
+
+.. code-block:: shell
+
+   . ~/.zshrc
+
+You will now be able to run the command line application from anywhere by running the following:
+
+.. code-block:: shell
+
+   <alias> --<option>
+
+For example, for the Corda node, install the shell extensions using
+
+.. code-block:: shell
+
+   java -jar corda-<version>.jar --install-shell-extensions
+
+And then run the node by running:
+
+.. code-block:: shell
+
+   corda --<option>
+
+Upgrading shell extensions
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Once the shell extensions have been installed, you can upgrade them in one of two ways.
+
+1) Overwrite the existing JAR with the newer version. The next time you run the application, it will automatically update
+   the completion file. Either restart the shell or see :ref:`above<installing-shell-extensions>` for instructions
+   on making the changes take effect immediately.
+2) If you wish to use a new JAR from a different directory, navigate to that directory and run:
+
+   .. code-block:: shell
+
+      java -jar <name-of-JAR>
+
+   Which will update the alias to point to the new location, and update command line completion functionality. Either
+   restart the shell or see :ref:`above<installing-shell-extensions>` for instructions on making the changes take effect immediately.
+
+List of existing CLI applications
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
++----------------------------------------------------------------+--------------------------------------------------------------+--------------------------------+
+| Description                                                    | JAR name                                                     | Alias                          |
++----------------------------------------------------------------+--------------------------------------------------------------+--------------------------------+
+| :ref:`Corda node<starting-an-individual-corda-node>`           | ``corda-<version>.jar``                                      | ``corda --<option>``           |
++----------------------------------------------------------------+--------------------------------------------------------------+--------------------------------+
+| :doc:`Network bootstrapper<network-bootstrapper>`              | ``corda-tools-network-bootstrapper-<version>.jar``           | ``bootstrapper --<option>``    |
++----------------------------------------------------------------+--------------------------------------------------------------+--------------------------------+
+| :ref:`Standalone shell<standalone-shell>`                      | ``corda-tools-shell-cli-<version>.jar``                      | ``corda-shell --<option>``     |
++----------------------------------------------------------------+--------------------------------------------------------------+--------------------------------+
+| :doc:`Blob inspector<blob-inspector>`                          | ``corda-tools-blob-inspector-<version>.jar``                 | ``blob-inspector --<option>``  |
++----------------------------------------------------------------+--------------------------------------------------------------+--------------------------------+
+

docs/source/network-bootstrapper.rst

@@ -245,3 +245,22 @@ To give the following:
 
 .. note:: The whitelist can only ever be appended to. Once added a contract implementation can never be removed.
 
+
+Command-line options
+--------------------
+
+The network bootstrapper can be started with the following command-line options:
+
+.. code-block:: shell
+
+    bootstrapper [-hvV] [--install-shell-extensions] [--no-copy] [--dir=<dir>]
+                 [--logging-level=<loggingLevel>]
+
+* ``--dir=<dir>``: Root directory containing the node configuration files and CorDapp JARs that will form the test network.
+  It may also contain existing node directories. Defaults to the current directory.
+* ``--no-copy``: Don't copy the CorDapp JARs into the nodes' "cordapps" directories.
+* ``--verbose``, ``--log-to-console``, ``-v``: If set, prints logging to the console as well as to a file.
+* ``--logging-level=<loggingLevel>``: Enable logging at this level and higher. Possible values: ERROR, WARN, INFO, DEBUG, TRACE. Default: INFO.
+* ``--install-shell-extensions``: Install ``bootstrapper`` alias and auto completion for bash and zsh. See :doc:`cli-application-shell-extensions` for more info.
+* ``--help``, ``-h``: Show this help message and exit.
+* ``--version``, ``-V``: Print version information and exit.

docs/source/running-a-node.rst

@@ -53,68 +53,21 @@ The node can optionally be started with the following command-line options:
 * ``--clear-network-map-cache``, ``-c``: Clears local copy of network map, on node startup it will be restored from server or file system.
 * ``--config-file``, ``-f``: The path to the config file. Defaults to ``node.conf``.
 * ``--dev-mode``, ``-d``: Runs the node in developer mode. Unsafe in production. Defaults to true on MacOS and desktop versions of Windows. False otherwise.
-* ``--help``, ``-h``: Displays the help message and exits.
+* ``--initial-registration``: Start initial node registration with the compatibility zone to obtain a certificate from the Doorman.
-* ``--initial-registration``: Start initial node registration with Corda network to obtain certificate from the permissioning 
+* ``--just-generate-node-info``: Perform the node start-up task necessary to generate its nodeInfo, save it to disk, then
-  server.
-* ``--install-shell-extensions``: Installs an alias and auto-completion for users of ``bash`` or ``zsh``. See below for more information.
-* ``--just-generate-node-info``: Perform the node start-up task necessary to generate its nodeInfo, save it to disk, then 
   quit.
 * ``--just-generate-rpc-ssl-settings``: Generate the ssl keystore and truststore for a secure RPC connection.
-* ``--log-to-console``, ``--verbose``, ``-v``: If set, prints logging to the console as well as to a file.
-* ``--logging-level <[ERROR,WARN,INFO,DEBUG,TRACE]>``: Enable logging at this level and higher. Defaults to INFO.
 * ``--network-root-truststore``, ``-t``: Network root trust store obtained from network operator.
 * ``--network-root-truststore-password``, ``-p``: Network root trust store password obtained from network operator.
 * ``--no-local-shell``, ``-n``: Do not start the embedded shell locally.
 * ``--on-unknown-config-keys <[FAIL,WARN,INFO]>``: How to behave on unknown node configuration. Defaults to FAIL.
 * ``--sshd``: Enables SSH server for node administration.
 * ``--sshd-port``: Sets the port for the SSH server. If not supplied and SSH server is enabled, the port defaults to 2222.
-* ``--version``, ``-V``: Prints the version and exits.
+* ``--verbose``, ``--log-to-console``, ``-v``: If set, prints logging to the console as well as to a file.
-
+* ``--logging-level=<loggingLevel>``: Enable logging at this level and higher. Possible values: ERROR, WARN, INFO, DEBUG, TRACE. Default: INFO.
-.. _installing-shell-extensions:
+* ``--install-shell-extensions``: Install ``corda`` alias and auto completion for bash and zsh. See :doc:`cli-application-shell-extensions` for more info.
-
+* ``--help``, ``-h``: Show this help message and exit.
-Installing shell extensions
+* ``--version``, ``-V``: Print version information and exit.
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Users of ``bash`` or ``zsh`` can install an alias and command line completion for Corda. Run:
-
-.. code-block:: shell
-
-   java -jar corda.jar --install-shell-extensions
-
-Then, either restart your shell, or for ``bash`` users run:
-
-.. code-block:: shell
-
-   . ~/.bashrc
-
-Or, for ``zsh`` run:
-
-.. code-block:: shell
-
-   . ~/.zshrc
-
-You will now be able to run a Corda node from anywhere by running the following:
-
-.. code-block:: shell
-
-   corda --<option>
-
-Upgrading shell extensions
-~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-Once the shell extensions have been installed, you can upgrade them in one of two ways.
-
-1) Overwrite the existing ``corda.jar`` with the newer version. The next time you run Corda, it will automatically update
-   the completion file. Either restart the shell or see :ref:`above<installing-shell-extensions>` for instructions
-   on making the changes take effect immediately.
-2) If you wish to use a new ``corda.jar`` from a different directory, navigate to that directory and run:
-
-   .. code-block:: shell
-
-      java -jar corda.jar
-
-   Which will update the ``corda`` alias to point to the new location, and update command line completion functionality. Either
-   restart the shell or see :ref:`above<installing-shell-extensions>` for instructions on making the changes take effect immediately.
 
 .. _enabling-remote-debugging:
 

docs/source/shell.rst

@@ -94,6 +94,8 @@ Windows
 
 Windows does not provide a built-in SSH tool. An alternative such as PuTTY should be used.
 
+.. _standalone-shell:
+
 The standalone shell
 --------------------
 The standalone shell is a standalone application interacting with a Corda node via RPC calls.
@@ -119,26 +121,34 @@ Windows
 
 .. code:: bash
 
-    java -jar corda-tools-shell-cli-VERSION_NUMBER.jar [--config-file PATH | --cordpass-directory PATH --commands-directory PATH --host HOST --port PORT
+    corda-shell [-hvV] [--install-shell-extensions]
-             --user USER --password PASSWORD --sshd-port PORT --sshd-hostkey-directory PATH --keystore-password PASSWORD
+                [--logging-level=<loggingLevel>] [--password=<password>]
-             --keystore-file FILE --truststore-password PASSWORD --truststore-file FILE | --help]
+                [--sshd-hostkey-directory=<sshdHostKeyDirectory>]
+                [--sshd-port=<sshdPort>] [--truststore-file=<trustStoreFile>]
+                [--truststore-password=<trustStorePassword>]
+                [--truststore-type=<trustStoreType>] [--user=<user>] [-a=<host>]
+                [-c=<cordappDirectory>] [-f=<configFile>] [-o=<commandsDirectory>]
+                [-p=<port>]
 
 Where:
 
-* ``config-file`` is the path to config file, used instead of providing the rest of command line options
+* ``--config-file=<configFile>``, ``--f`` The path to the shell configuration file, used instead of providing the rest of the command line options.
-* ``cordpass-directory`` is the directory containing Cordapps jars, Cordapps are require when starting flows
+* ``--cordapp-directory=<cordappDirectory>``, ``-c`` The path to the directory containing CorDapp jars, CorDapps are required when starting flows.
-* ``commands-directory`` is the directory with additional CRaSH shell commands
+* ``--commands-directory=<commandsDirectory>``, ``-o`` The path to the directory containing additional CRaSH shell commands.
-* ``host`` is the Corda node's host
+* ``--host``, ``-a``: The host address of the Corda node.
-* ``port`` is the Corda node's port, specified in the ``node.conf`` file
+* ``--port``, ``-p``: The RPC port of the Corda node.
-* ``user`` is the RPC username, if not provided it will be requested at startup
+* ``--user=<user>``: The RPC user name.
-* ``password`` is the RPC user password, if not provided it will be requested at startup
+* ``--password=<password>`` The RPC user password. If not provided it will be prompted for on startup.
-* ``sshd-port`` instructs the standalone shell app to start SSH server on the given port, optional
+* ``--sshd-port=<sshdPort>`` Enables SSH server for shell.
-* ``sshd-hostkey-directory`` is the directory containing hostkey.pem file for SSH server
+* ``--sshd-hostkey-directory=<sshHostKeyDirectory``: The directory containing the hostkey.pem file for the SSH server.
-* ``keystore-password`` the password to unlock the KeyStore file containing the standalone shell certificate and private key, optional, unencrypted RPC connection without SSL will be used if the option is not provided
+* ``--truststore-password=<trustStorePassword>``: The password to unlock the TrustStore file.
-* ``keystore-file`` is the path to the KeyStore file
+* ``--truststore-file=<trustStoreFile>``: The path to the TrustStore file.
-* ``truststore-password`` the password to unlock the TrustStore file containing the Corda node certificate, optional, unencrypted RPC connection without SSL will be used if the option is not provided
+* ``--truststore-type=<trustStoreType>``: The type of the TrustStore (e.g. JKS).
-* ``truststore-file`` is the path to the TrustStore file
+* ``--verbose``, ``--log-to-console``, ``-v``: If set, prints logging to the console as well as to a file.
-* ``help`` prints Shell help
+* ``--logging-level=<loggingLevel>``: Enable logging at this level and higher. Possible values: ERROR, WARN, INFO, DEBUG, TRACE. Default: INFO.
+* ``--install-shell-extensions``: Install ``corda-shell`` alias and auto completion for bash and zsh. See :doc:`cli-application-shell-extensions` for more info.
+* ``--help``, ``-h``: Show this help message and exit.
+* ``--version``, ``-V``: Print version information and exit.
 
 The format of ``config-file``:
 

node/build.gradle

@@ -232,6 +232,8 @@ dependencies {
         exclude group: "io.netty", module: "netty-tcnative"
         exclude group: "ch.qos.logback", module: "logback-classic"
     }
+
+    testCompile(project(':test-cli'))
 }
 
 tasks.withType(JavaCompile) {

node/src/main/kotlin/net/corda/node/internal/NodeStartup.kt

@@ -259,66 +259,7 @@ open class NodeStartup: CordaCliWrapper("corda", "Runs a Corda Node") {
             return
         }
         if (cmdLineOptions.justGenerateRpcSslCerts) {
-            val (keyPair, cert) = createKeyPairAndSelfSignedTLSCertificate(conf.myLegalName.x500Principal)
+            generateRpcSslCertificates(conf)
-
-            val keyStorePath = conf.baseDirectory / "certificates" / "rpcsslkeystore.jks"
-            val trustStorePath = conf.baseDirectory / "certificates" / "export" / "rpcssltruststore.jks"
-
-            if (keyStorePath.exists() || trustStorePath.exists()) {
-                println("Found existing RPC SSL keystores. Command was already run. Exiting..")
-                exitProcess(0)
-            }
-
-            val console: Console? = System.console()
-
-            when (console) {
-            // In this case, the JVM is not connected to the console so we need to exit
-                null -> {
-                    println("Not connected to console. Exiting")
-                    exitProcess(1)
-                }
-            // Otherwise we can proceed normally
-                else -> {
-                    while (true) {
-                        val keystorePassword1 = console.readPassword("Enter the keystore password => ")
-                        val keystorePassword2 = console.readPassword("Re-enter the keystore password => ")
-                        if (!keystorePassword1.contentEquals(keystorePassword2)) {
-                            println("The keystore passwords don't match.")
-                            continue
-                        }
-                        saveToKeyStore(keyStorePath, keyPair, cert, String(keystorePassword1), "rpcssl")
-                        println("The keystore was saved to: $keyStorePath .")
-                        break
-                    }
-
-                    while (true) {
-                        val trustStorePassword1 = console.readPassword("Enter the truststore password => ")
-                        val trustStorePassword2 = console.readPassword("Re-enter the truststore password => ")
-                        if (!trustStorePassword1.contentEquals(trustStorePassword2)) {
-                            println("The truststore passwords don't match.")
-                            continue
-                        }
-
-                        saveToTrustStore(trustStorePath, cert, String(trustStorePassword1), "rpcssl")
-                        println("The truststore was saved to: $trustStorePath .")
-                        println("You need to distribute this file along with the password in a secure way to all RPC clients.")
-                        break
-                    }
-
-                    val dollar = '$'
-                    println("""
-                        |
-                        |The SSL certificates were generated successfully.
-                        |
-                        |Add this snippet to the "rpcSettings" section of your node.conf:
-                        |       useSsl=true
-                        |       ssl {
-                        |           keyStorePath=$dollar{baseDirectory}/certificates/rpcsslkeystore.jks
-                        |           keyStorePassword=the_above_password
-                        |       }
-                        |""".trimMargin())
-                }
-            }
             return
         }
 
@@ -358,6 +299,82 @@ open class NodeStartup: CordaCliWrapper("corda", "Runs a Corda Node") {
         node.run()
     }
 
+    private fun generateRpcSslCertificates(conf: NodeConfiguration) {
+        val (keyPair, cert) = createKeyPairAndSelfSignedTLSCertificate(conf.myLegalName.x500Principal)
+
+        val keyStorePath = conf.baseDirectory / "certificates" / "rpcsslkeystore.jks"
+        val trustStorePath = conf.baseDirectory / "certificates" / "export" / "rpcssltruststore.jks"
+
+        if (keyStorePath.exists() || trustStorePath.exists()) {
+            println("Found existing RPC SSL keystores. Command was already run. Exiting..")
+            exitProcess(0)
+        }
+
+        val console: Console? = System.console()
+
+        when (console) {
+        // In this case, the JVM is not connected to the console so we need to exit.
+            null -> {
+                println("Not connected to console. Exiting")
+                exitProcess(1)
+            }
+        // Otherwise we can proceed normally.
+            else -> {
+                while (true) {
+                    val keystorePassword1 = console.readPassword("Enter the RPC keystore password => ")
+                    // TODO: consider adding a password strength policy.
+                    if (keystorePassword1.isEmpty()) {
+                        println("The RPC keystore password cannot be an empty String.")
+                        continue
+                    }
+
+                    val keystorePassword2 = console.readPassword("Re-enter the RPC keystore password => ")
+                    if (!keystorePassword1.contentEquals(keystorePassword2)) {
+                        println("The RPC keystore passwords don't match.")
+                        continue
+                    }
+
+                    saveToKeyStore(keyStorePath, keyPair, cert, String(keystorePassword1), "rpcssl")
+                    println("The RPC keystore was saved to: $keyStorePath .")
+                    break
+                }
+
+                while (true) {
+                    val trustStorePassword1 = console.readPassword("Enter the RPC truststore password => ")
+                    // TODO: consider adding a password strength policy.
+                    if (trustStorePassword1.isEmpty()) {
+                        println("The RPC truststore password cannot be an empty String.")
+                        continue
+                    }
+
+                    val trustStorePassword2 = console.readPassword("Re-enter the RPC truststore password => ")
+                    if (!trustStorePassword1.contentEquals(trustStorePassword2)) {
+                        println("The RPC truststore passwords don't match.")
+                        continue
+                    }
+
+                    saveToTrustStore(trustStorePath, cert, String(trustStorePassword1), "rpcssl")
+                    println("The RPC truststore was saved to: $trustStorePath .")
+                    println("You need to distribute this file along with the password in a secure way to all RPC clients.")
+                    break
+                }
+
+                val dollar = '$'
+                println("""
+                            |
+                            |The SSL certificates for RPC were generated successfully.
+                            |
+                            |Add this snippet to the "rpcSettings" section of your node.conf:
+                            |       useSsl=true
+                            |       ssl {
+                            |           keyStorePath=$dollar{baseDirectory}/certificates/rpcsslkeystore.jks
+                            |           keyStorePassword=the_above_password
+                            |       }
+                            |""".trimMargin())
+            }
+        }
+    }
+
     protected open fun logStartupInfo(versionInfo: VersionInfo, conf: NodeConfiguration) {
         logger.info("Vendor: ${versionInfo.vendor}")
         logger.info("Release: ${versionInfo.releaseVersion}")
@@ -426,7 +443,7 @@ open class NodeStartup: CordaCliWrapper("corda", "Runs a Corda Node") {
         )
     }
 
-    open protected fun logLoadedCorDapps(corDapps: List<CordappImpl>) {
+    protected open fun logLoadedCorDapps(corDapps: List<CordappImpl>) {
         fun CordappImpl.Info.description() = "$shortName version $version by $vendor"
 
         Node.printBasicNodeInfo("Loaded ${corDapps.size} CorDapp(s)", corDapps.map { it.info }.joinToString(", ", transform = CordappImpl.Info::description))

node/src/main/resources/net.corda.node.internal.NodeStartup.yml

@@ -0,0 +1,141 @@
+- commandName: "<main class>"
+  positionalParams: []
+  params:
+  - parameterName: "--base-directory"
+    parameterType: "java.nio.file.Path"
+    required: false
+    multiParam: true
+    acceptableValues: []
+  - parameterName: "--bootstrap-raft-cluster"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--clear-network-map-cache"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--config-file"
+    parameterType: "java.nio.file.Path"
+    required: false
+    multiParam: true
+    acceptableValues: []
+  - parameterName: "--dev-mode"
+    parameterType: "java.lang.Boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--initial-registration"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--install-shell-extensions"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--just-generate-node-info"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--just-generate-rpc-ssl-settings"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--log-to-console"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--logging-level"
+    parameterType: "org.slf4j.event.Level"
+    required: false
+    multiParam: false
+    acceptableValues:
+    - "ERROR"
+    - "WARN"
+    - "INFO"
+    - "DEBUG"
+    - "TRACE"
+  - parameterName: "--network-root-truststore"
+    parameterType: "java.nio.file.Path"
+    required: false
+    multiParam: true
+    acceptableValues: []
+  - parameterName: "--network-root-truststore-password"
+    parameterType: "java.lang.String"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--no-local-shell"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--on-unknown-config-keys"
+    parameterType: "net.corda.nodeapi.internal.config.UnknownConfigKeysPolicy"
+    required: false
+    multiParam: false
+    acceptableValues:
+    - "FAIL"
+    - "WARN"
+    - "IGNORE"
+  - parameterName: "--sshd"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--sshd-port"
+    parameterType: "int"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--verbose"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "-b"
+    parameterType: "java.nio.file.Path"
+    required: false
+    multiParam: true
+    acceptableValues: []
+  - parameterName: "-c"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "-d"
+    parameterType: "java.lang.Boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "-f"
+    parameterType: "java.nio.file.Path"
+    required: false
+    multiParam: true
+    acceptableValues: []
+  - parameterName: "-n"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "-p"
+    parameterType: "java.lang.String"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "-t"
+    parameterType: "java.nio.file.Path"
+    required: false
+    multiParam: true
+    acceptableValues: []
+  - parameterName: "-v"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []

node/src/test/kotlin/net/corda/node/internal/NodeStartupCompatibilityTest.kt

@@ -0,0 +1,5 @@
+package net.corda.node.internal
+
+import net.corda.testing.CliBackwardsCompatibleTest
+
+class NodeStartupCompatibilityTest : CliBackwardsCompatibleTest(NodeStartup::class.java)

testing/test-cli/src/main/kotlin/net/corda/testing/CliBackwardsCompatibleTest.kt

@@ -1,16 +1,21 @@
 package net.corda.testing
 
 import junit.framework.AssertionFailedError
+import org.junit.Test
 
-open class CliBackwardsCompatibleTest {
+open class CliBackwardsCompatibleTest(val clazz: Class<*>) {
 
+    @Test
+    fun `should always be backwards compatible`() {
+        checkBackwardsCompatibility(clazz)
+    }
 
     fun checkBackwardsCompatibility(clazz: Class<*>) {
         val checker = CommandLineCompatibilityChecker()
         val checkResults = checker.checkCommandLineIsBackwardsCompatible(clazz)
 
         if (checkResults.isNotEmpty()) {
-            val exceptionMessage= checkResults.map { it.message }.joinToString(separator = "\n")
+            val exceptionMessage = checkResults.map { it.message }.joinToString(separator = "\n")
             throw AssertionFailedError("Command line is not backwards compatible:\n$exceptionMessage")
         }
     }

testing/test-cli/src/main/kotlin/net/corda/testing/CommandLineCompatibilityUtils.kt

@@ -11,6 +11,11 @@ import kotlin.collections.ArrayList
 
 
 class CommandLineCompatibilityChecker {
+    companion object {
+        fun printCommandLineYAML(clazz: Class<*>) {
+            CommandLineCompatibilityChecker().printCommandDescription(CommandLine(clazz.newInstance()))
+        }
+    }
 
     fun topoSort(commandLine: CommandLine): List<CommandDescription> {
         val toVisit = Stack<CommandLine>()
@@ -62,6 +67,7 @@ class CommandLineCompatibilityChecker {
         return Iterable::class.java.isAssignableFrom(clazz) || Array<Any>::class.java.isAssignableFrom(clazz)
     }
 
+
     fun printCommandDescription(commandLine: CommandLine) {
         val objectMapper = ObjectMapper(YAMLFactory()).registerKotlinModule()
         val results = topoSort(commandLine)
@@ -109,7 +115,7 @@ class CommandLineCompatibilityChecker {
             throw IllegalArgumentException("Commands must match (${old.commandName} != ${new.commandName})")
         }
         val oldSet = old.positionalParams.sortedBy { it.parameterName }.toSet()
-        val newSet = new.positionalParams.sortedBy { it.parameterName}.toSet()
+        val newSet = new.positionalParams.sortedBy { it.parameterName }.toSet()
         val newIsSuperSetOfOld = newSet.containsAll(oldSet)
         return if (!newIsSuperSetOfOld) {
             oldSet.filterNot { newSet.contains(it) }.map {
@@ -150,7 +156,9 @@ class CommandLineCompatibilityChecker {
         val commandLineToCheckName = commandLineToCheck.canonicalName
         val instance = commandLineToCheck.newInstance()
         val resourceAsStream = this.javaClass.classLoader.getResourceAsStream("$commandLineToCheckName.yml")
-                ?: throw IllegalStateException("no Descriptor for $commandLineToCheckName found on classpath")
+                ?: throw IllegalStateException("$commandLineToCheckName.yml not found on classpath").also {
+                    printCommandLineYAML(commandLineToCheck)
+                }
         val old = readCommandDescription(resourceAsStream)
         val new = topoSort(CommandLine(instance))
         return checkCommandLineIsBackwardsCompatible(old, new)
@@ -158,8 +166,8 @@ class CommandLineCompatibilityChecker {
 
 
     fun checkBackwardsCompatibility(old: CommandLine, new: CommandLine): List<CliBackwardsCompatibilityValidationCheck> {
-        val topoSortOld= topoSort(old)
+        val topoSortOld = topoSort(old)
-        val topoSortNew= topoSort(new)
+        val topoSortNew = topoSort(new)
         return checkCommandLineIsBackwardsCompatible(topoSortOld, topoSortNew)
     }
 

tools/blobinspector/build.gradle

@@ -5,7 +5,7 @@ apply plugin: 'com.jfrog.artifactory'
 
 dependencies {
     compile project(':client:jackson')
-    compile "info.picocli:picocli:$picocli_version"
+    compile project(':tools:cliutils')
     compile "org.slf4j:jul-to-slf4j:$slf4j_version"
     compile "org.apache.logging.log4j:log4j-slf4j-impl:$log4j_version"
     compile "com.jcabi:jcabi-manifests:$jcabi_manifests_version"
@@ -23,10 +23,9 @@ jar {
         exclude "META-INF/*.DSA"
         exclude "META-INF/*.RSA"
     }
-    archiveName = "blob-inspector-${corda_release_version}.jar"
+    baseName = "blob-inspector"
     manifest {
         attributes(
-                'Automatic-Module-Name': 'net.corda.blobinspector',
                 'Main-Class': 'net.corda.blobinspector.BlobInspectorKt'
         )
     }

tools/blobinspector/src/main/kotlin/net/corda/blobinspector/BlobInspector.kt

@@ -4,6 +4,9 @@ import com.fasterxml.jackson.core.JsonFactory
 import com.fasterxml.jackson.dataformat.yaml.YAMLFactory
 import com.jcabi.manifests.Manifests
 import net.corda.client.jackson.JacksonSupport
+import net.corda.cliutils.CordaCliWrapper
+import net.corda.cliutils.ExitCodes
+import net.corda.cliutils.start
 import net.corda.core.internal.isRegularFile
 import net.corda.core.internal.rootMessage
 import net.corda.core.serialization.SerializationContext
@@ -21,39 +24,23 @@ import net.corda.serialization.internal.SerializationFactoryImpl
 import net.corda.serialization.internal.amqp.AbstractAMQPSerializationScheme
 import net.corda.serialization.internal.amqp.DeserializationInput
 import net.corda.serialization.internal.amqp.amqpMagic
+import org.slf4j.event.Level
 import picocli.CommandLine
 import picocli.CommandLine.*
 import java.io.PrintStream
 import java.net.MalformedURLException
 import java.net.URL
 import java.nio.file.Paths
+import java.util.*
 import kotlin.system.exitProcess
 
 fun main(args: Array<String>) {
-    val main = BlobInspector()
+    BlobInspector().start(args)
-    try {
-        CommandLine.run(main, *args)
-    } catch (e: ExecutionException) {
-        val throwable = e.cause ?: e
-        if (main.verbose) {
-            throwable.printStackTrace()
-        } else {
-            System.err.println("*ERROR*: ${throwable.rootMessage}. Use --verbose for more details")
-        }
-        exitProcess(1)
-    }
 }
 
-@Command(
+class BlobInspector : CordaCliWrapper("blob-inspector", "Convert AMQP serialised binary blobs to text") {
-        name = "blob-inspector",
+    @Parameters(index = "*..0", paramLabel = "SOURCE", description = ["URL or file path to the blob"], converter = [SourceConverter::class])
-        versionProvider = CordaVersionProvider::class,
+    var source: MutableList<URL> = mutableListOf()
-        mixinStandardHelpOptions = true,   // add --help and --version options,
-        showDefaultValues = true,
-        description = ["Convert AMQP serialised binary blobs to text"]
-)
-class BlobInspector : Runnable {
-    @Parameters(index = "0", paramLabel = "SOURCE", description = ["URL or file path to the blob"], converter = [SourceConverter::class])
-    var source: URL? = null
 
     @Option(names = ["--format"], paramLabel = "type", description = ["Output format. Possible values: [YAML, JSON]"])
     private var formatType: OutputFormatType = OutputFormatType.YAML
@@ -68,17 +55,19 @@ class BlobInspector : Runnable {
     @Option(names = ["--schema"], description = ["Print the blob's schema first"])
     private var schema: Boolean = false
 
-    @Option(names = ["--verbose"], description = ["Enable verbose output"])
+    override fun runProgram() = run(System.out)
-    var verbose: Boolean = false
 
-    override fun run() = run(System.out)
+    override fun initLogging() {
-
-    fun run(out: PrintStream) {
         if (verbose) {
-            System.setProperty("logLevel", "trace")
+            loggingLevel = Level.TRACE
         }
+        val loggingLevel = loggingLevel.name.toLowerCase(Locale.ENGLISH)
+        System.setProperty("logLevel", loggingLevel) // This property is referenced from the XML config file.
+    }
 
-        val inputBytes = source!!.readBytes()
+    fun run(out: PrintStream): Int {
+        require(source.count() == 1) { "You must specify URL or file path to the blob" }
+        val inputBytes = source.first().readBytes()
         val bytes = parseToBinaryRelaxed(inputFormatType, inputBytes)
                 ?: throw IllegalArgumentException("Error: this input does not appear to be encoded in Corda's AMQP extended format, sorry.")
 
@@ -102,6 +91,9 @@ class BlobInspector : Runnable {
             val deserialized = bytes.deserialize<Any>(context = SerializationDefaults.STORAGE_CONTEXT)
             out.println(deserialized.javaClass.name)
             mapper.writeValue(out, deserialized)
+            return ExitCodes.SUCCESS
+        } catch(e: Exception) {
+            return ExitCodes.FAILURE
         } finally {
             _contextSerializationEnv.set(null)
         }

tools/blobinspector/src/test/kotlin/net/corda/blobinspector/BlobInspectorTest.kt

@@ -53,7 +53,7 @@ class BlobInspectorTest {
     }
 
     private fun run(resourceName: String): String {
-        blobInspector.source = javaClass.getResource(resourceName)
+        blobInspector.source = mutableListOf(javaClass.getResource(resourceName))
         val writer = StringWriter()
         blobInspector.run(PrintStream(WriterOutputStream(writer, UTF_8)))
         val output = writer.toString()

tools/bootstrapper/build.gradle

@@ -8,6 +8,12 @@ dependencies {
     compile project(':node-api')
     compile project(':tools:cliutils')
     compile "org.apache.logging.log4j:log4j-slf4j-impl:$log4j_version"
+
+    testCompile(project(':test-utils')) {
+        exclude group: 'org.apache.logging.log4j', module: 'log4j-slf4j-impl'
+    }
+
+    testCompile(project(':test-cli'))
 }
 
 processResources {

tools/bootstrapper/src/test/kotlin/net/corda/bootstrapper/NetworkBootstrapperRunnerTest.kt

@@ -0,0 +1,5 @@
+package net.corda.bootstrapper
+
+import net.corda.testing.CliBackwardsCompatibleTest
+
+class NetworkBootstrapperRunnerTest : CliBackwardsCompatibleTest(NetworkBootstrapperRunner::class.java)

tools/bootstrapper/src/test/resources/net.corda.bootstrapper.NetworkBootstrapperRunner.yml

@@ -0,0 +1,43 @@
+- commandName: "<main class>"
+  positionalParams: []
+  params:
+  - parameterName: "--dir"
+    parameterType: "java.nio.file.Path"
+    required: false
+    multiParam: true
+    acceptableValues: []
+  - parameterName: "--install-shell-extensions"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--log-to-console"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--logging-level"
+    parameterType: "org.slf4j.event.Level"
+    required: false
+    multiParam: false
+    acceptableValues:
+    - "ERROR"
+    - "WARN"
+    - "INFO"
+    - "DEBUG"
+    - "TRACE"
+  - parameterName: "--no-copy"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--verbose"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "-v"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []

tools/shell-cli/build.gradle

@@ -17,13 +17,15 @@ dependencies {
     testCompile(project(':test-utils')) {
         exclude group: 'org.apache.logging.log4j', module: 'log4j-slf4j-impl'
     }
+
+    testCompile(project(':test-cli'))
 }
 
 processResources {
     from file("$rootDir/config/dev/log4j2.xml")
 }
 
-shadowJar  {
+shadowJar {
     mergeServiceFiles()
 }
 

tools/shell-cli/src/test/kotlin/net/corda/tools/shell/StandaloneShellCompatibilityTest.kt

@@ -0,0 +1,6 @@
+package net.corda.tools.shell
+
+import net.corda.testing.CliBackwardsCompatibleTest
+
+class StandaloneShellCompatibilityTest : CliBackwardsCompatibleTest(StandaloneShell::class.java)
+

tools/shell-cli/src/test/resources/net.corda.tools.shell.StandaloneShell.yml

@@ -0,0 +1,119 @@
+- commandName: "<main class>"
+  positionalParams: []
+  params:
+  - parameterName: "--commands-directory"
+    parameterType: "java.nio.file.Path"
+    required: false
+    multiParam: true
+    acceptableValues: []
+  - parameterName: "--config-file"
+    parameterType: "java.nio.file.Path"
+    required: false
+    multiParam: true
+    acceptableValues: []
+  - parameterName: "--cordapp-directory"
+    parameterType: "java.nio.file.Path"
+    required: false
+    multiParam: true
+    acceptableValues: []
+  - parameterName: "--host"
+    parameterType: "java.lang.String"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--install-shell-extensions"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--log-to-console"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--logging-level"
+    parameterType: "org.slf4j.event.Level"
+    required: false
+    multiParam: false
+    acceptableValues:
+    - "ERROR"
+    - "WARN"
+    - "INFO"
+    - "DEBUG"
+    - "TRACE"
+  - parameterName: "--password"
+    parameterType: "java.lang.String"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--port"
+    parameterType: "java.lang.String"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--sshd-hostkey-directory"
+    parameterType: "java.nio.file.Path"
+    required: false
+    multiParam: true
+    acceptableValues: []
+  - parameterName: "--sshd-port"
+    parameterType: "java.lang.String"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--truststore-file"
+    parameterType: "java.nio.file.Path"
+    required: false
+    multiParam: true
+    acceptableValues: []
+  - parameterName: "--truststore-password"
+    parameterType: "java.lang.String"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--truststore-type"
+    parameterType: "java.lang.String"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--user"
+    parameterType: "java.lang.String"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "--verbose"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "-a"
+    parameterType: "java.lang.String"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "-c"
+    parameterType: "java.nio.file.Path"
+    required: false
+    multiParam: true
+    acceptableValues: []
+  - parameterName: "-f"
+    parameterType: "java.nio.file.Path"
+    required: false
+    multiParam: true
+    acceptableValues: []
+  - parameterName: "-o"
+    parameterType: "java.nio.file.Path"
+    required: false
+    multiParam: true
+    acceptableValues: []
+  - parameterName: "-p"
+    parameterType: "java.lang.String"
+    required: false
+    multiParam: false
+    acceptableValues: []
+  - parameterName: "-v"
+    parameterType: "boolean"
+    required: false
+    multiParam: false
+    acceptableValues: []
+