ExternalVendorCode/corda
1
0
Fork 0
mirror of https://github.com/corda/corda.git synced 2024-12-18 20:47:57 +00:00
Code Issues Actions 2 Packages Projects Releases Wiki Activity

CORDA-2998 fix network builder (#5265)

Browse Source 
CORDA-2998 fix network builder
This commit is contained in:
szymonsztuka 2019-07-02 12:32:48 +02:00 committed by Rick Parker
parent fee396efc3
commit f89008c070
59 changed files with 480 additions and 530 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
build.gradle
View File

@ -413,7 +413,8 @@ bintrayConfig {
'corda-tools-cliutils',
'corda-common-configuration-parsing',
'corda-common-validation',
'corda-common-logging'
'corda-common-logging',
'corda-tools-network-builder'
]
license {
name = 'Apache-2.0'

82
docs/source/network-builder.rst
View File

@ -8,11 +8,15 @@ containers to abstract the complexity of managing a distributed network away fro
.. image:: _static/images/network-builder-v4.png
The network you build will either be made up of local ``docker`` nodes *or* of nodes spread across Azure
containers. More backends may be added in future. The tool is open source, so contributions to add more
The network you build will either be made up of local ``Docker`` nodes *or* of nodes spread across Azure
containers.
For each node a separate Docker image is built based on `corda/corda-zulu-4.0 <https://hub.docker.com/r/corda/corda-zulu-4.0>`_.
Unlike the official image, a `node.conf` file and CorDapps are embedded into the image
(they are not externally provided to the running container via volumes/mount points).
More backends may be added in future. The tool is open source, so contributions to add more
destinations for the containers are welcome!
`Download the Corda Network Builder <https://ci-artifactory.corda.r3cev.com/artifactory/corda-releases/net/corda/corda-network-builder/3.2.1847-corda/corda-network-builder-3.2.1847-corda-executable.jar>`_.
`Download the Corda Network Builder <https://ci-artifactory.corda.r3cev.com/artifactory/corda-releases/net/corda/corda-tools-network-builder/|corda_version|/corda-tools-network-builder-|corda_version|.jar>`_.
.. _pre-requisites:
@ -41,9 +45,9 @@ the following layout:
An easy way to build a valid set of nodes is by running ``deployNodes``. In this document, we will be using
the output of running ``deployNodes`` for the `Example CorDapp <https://github.com/corda/cordapp-example>`_:
1. ``git clone https://github.com/corda/cordapp-example``
2. ``cd cordapp-example``
3. ``./gradlew clean deployNodes``
1. ``git clone https://github.com/corda/samples``
2. ``cd samples/cordapp-example``
3. ``./gradlew clean workflows-java:deployNodes``
Building a network via the command line
---------------------------------------
@ -54,24 +58,27 @@ Starting the nodes
Quickstart Local Docker
~~~~~~~~~~~~~~~~~~~~~~~
1. ``cd kotlin-source/build/nodes``
2. ``java -jar <path/to/network-builder-jar> -d .``
1. ``cd workflows-java/build/nodes``
2. ``java -jar <path/to/corda-tools-network-builder.jar> -d .``
If you run ``docker ps`` to see the running containers, the following output should be displayed:
.. sourcecode:: shell
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
406868b4ba69 node-partyc:corda-network "/run-corda.sh" 17 seconds ago Up 16 seconds 0.0.0.0:32902->10003/tcp, 0.0.0.0:32895->10005/tcp, 0.0.0.0:32898->10020/tcp, 0.0.0.0:32900->12222/tcp partyc0
4546a2fa8de7 node-partyb:corda-network "/run-corda.sh" 17 seconds ago Up 17 seconds 0.0.0.0:32896->10003/tcp, 0.0.0.0:32899->10005/tcp, 0.0.0.0:32901->10020/tcp, 0.0.0.0:32903->12222/tcp partyb0
c8c44c515bdb node-partya:corda-network "/run-corda.sh" 17 seconds ago Up 17 seconds 0.0.0.0:32894->10003/tcp, 0.0.0.0:32897->10005/tcp, 0.0.0.0:32892->10020/tcp, 0.0.0.0:32893->12222/tcp partya0
cf7ab689f493 node-notary:corda-network "/run-corda.sh" 30 seconds ago Up 31 seconds 0.0.0.0:32888->10003/tcp, 0.0.0.0:32889->10005/tcp, 0.0.0.0:32890->10020/tcp, 0.0.0.0:32891->12222/tcp notary0
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
406868b4ba69 node-partyc:corda-network "run-corda" 17 seconds ago Up 16 seconds 0.0.0.0:32902->10003/tcp, 0.0.0.0:32895->10005/tcp, 0.0.0.0:32898->10020/tcp, 0.0.0.0:32900->12222/tcp partyc0
4546a2fa8de7 node-partyb:corda-network "run-corda" 17 seconds ago Up 17 seconds 0.0.0.0:32896->10003/tcp, 0.0.0.0:32899->10005/tcp, 0.0.0.0:32901->10020/tcp, 0.0.0.0:32903->12222/tcp partyb0
c8c44c515bdb node-partya:corda-network "run-corda" 17 seconds ago Up 17 seconds 0.0.0.0:32894->10003/tcp, 0.0.0.0:32897->10005/tcp, 0.0.0.0:32892->10020/tcp, 0.0.0.0:32893->12222/tcp partya0
cf7ab689f493 node-notary:corda-network "run-corda" 30 seconds ago Up 31 seconds 0.0.0.0:32888->10003/tcp, 0.0.0.0:32889->10005/tcp, 0.0.0.0:32890->10020/tcp, 0.0.0.0:32891->12222/tcp notary0
Depending on you machine performance, even after all containers are reported as running,
the underlying Corda nodes may be still starting and SSHing to a node may be not available immediately.
Quickstart Remote Azure
~~~~~~~~~~~~~~~~~~~~~~~
1. ``cd kotlin-source/build/nodes``
2. ``java -jar <path/to/network-builder-jar> -b AZURE -d .``
2. ``java -jar <path/to/corda-tools-network-builder.jar> -b AZURE -d .``
.. note:: The Azure configuration is handled by the az-cli utility. See the :ref:`pre-requisites`.
@ -95,21 +102,26 @@ You can interact with the nodes by SSHing into them on the port that is mapped t
>>> run networkMapSnapshot
[
{ "addresses" : [ "partya0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyA, L=London, C=GB" ], "platformVersion" : 3, "serial" : 1532701330613 },
{ "addresses" : [ "notary0:10020" ], "legalIdentitiesAndCerts" : [ "O=Notary, L=London, C=GB" ], "platformVersion" : 3, "serial" : 1532701305115 },
{ "addresses" : [ "partyc0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyC, L=Paris, C=FR" ], "platformVersion" : 3, "serial" : 1532701331608 },
{ "addresses" : [ "partyb0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyB, L=New York, C=US" ], "platformVersion" : 3, "serial" : 1532701330118 }
{ "addresses" : [ "partya0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyA, L=London, C=GB" ], "platformVersion" : |platform_version|, "serial" : 1532701330613 },
{ "addresses" : [ "notary0:10020" ], "legalIdentitiesAndCerts" : [ "O=Notary, L=London, C=GB" ], "platformVersion" : |platform_version|, "serial" : 1532701305115 },
{ "addresses" : [ "partyc0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyC, L=Paris, C=FR" ], "platformVersion" : |platform_version|, "serial" : 1532701331608 },
{ "addresses" : [ "partyb0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyB, L=New York, C=US" ], "platformVersion" : |platform_version|, "serial" : 1532701330118 }
]
>>>
You can also run a flow from cordapp-example: ``flow start com.example.flow.ExampleFlow$Initiator iouValue: 20, otherParty: "PartyB"``
To verify it, connect into the ``partyb0`` node and run ``run vaultQuery contractStateType: "com.example.state.IOUState"``.
The ``partyb0`` vault should contain ``IOUState``.
Adding additional nodes
^^^^^^^^^^^^^^^^^^^^^^^
It is possible to add additional nodes to the network by reusing the nodes you built earlier. For example, to add a
node by reusing the existing ``PartyA`` node, you would run:
``java -jar <network-builder-jar> --add "PartyA=O=PartyZ,L=London,C=GB"``
``java -jar <path/to/corda-tools-network-builder.jar> --add "PartyA=O=PartyZ,L=London,C=GB"``
To confirm the node has been started correctly, run the following in the previously connected SSH session:
@ -117,18 +129,18 @@ To confirm the node has been started correctly, run the following in the previou
Tue Jul 17 15:47:14 GMT 2018>>> run networkMapSnapshot
[
{ "addresses" : [ "partya0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyA, L=London, C=GB" ], "platformVersion" : 3, "serial" : 1532701330613 },
{ "addresses" : [ "notary0:10020" ], "legalIdentitiesAndCerts" : [ "O=Notary, L=London, C=GB" ], "platformVersion" : 3, "serial" : 1532701305115 },
{ "addresses" : [ "partyc0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyC, L=Paris, C=FR" ], "platformVersion" : 3, "serial" : 1532701331608 },
{ "addresses" : [ "partyb0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyB, L=New York, C=US" ], "platformVersion" : 3, "serial" : 1532701330118 },
{ "addresses" : [ "partya1:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyZ, L=London, C=GB" ], "platformVersion" : 3, "serial" : 1532701630861 }
{ "addresses" : [ "partya0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyA, L=London, C=GB" ], "platformVersion" : |platform_version|, "serial" : 1532701330613 },
{ "addresses" : [ "notary0:10020" ], "legalIdentitiesAndCerts" : [ "O=Notary, L=London, C=GB" ], "platformVersion" : |platform_version|, "serial" : 1532701305115 },
{ "addresses" : [ "partyc0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyC, L=Paris, C=FR" ], "platformVersion" : |platform_version|, "serial" : 1532701331608 },
{ "addresses" : [ "partyb0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyB, L=New York, C=US" ], "platformVersion" : |platform_version|, "serial" : 1532701330118 },
{ "addresses" : [ "partya1:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyZ, L=London, C=GB" ], "platformVersion" : |platform_version|, "serial" : 1532701630861 }
]
Building a network in Graphical User Mode
-----------------------------------------
The Corda Network Builder also provides a GUI for when automated interactions are not required. To launch it, run
``java -jar <path/to/network-builder-jar> -g``.
``java -jar <path/to/corda-tools-network-builder.jar> -g``.
Starting the nodes
^^^^^^^^^^^^^^^^^^
@ -145,11 +157,11 @@ see the running containers, the following output should be displayed:
.. sourcecode:: shell
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
406868b4ba69 node-partyc:corda-network "/run-corda.sh" 17 seconds ago Up 16 seconds 0.0.0.0:32902->10003/tcp, 0.0.0.0:32895->10005/tcp, 0.0.0.0:32898->10020/tcp, 0.0.0.0:32900->12222/tcp partyc0
4546a2fa8de7 node-partyb:corda-network "/run-corda.sh" 17 seconds ago Up 17 seconds 0.0.0.0:32896->10003/tcp, 0.0.0.0:32899->10005/tcp, 0.0.0.0:32901->10020/tcp, 0.0.0.0:32903->12222/tcp partyb0
c8c44c515bdb node-partya:corda-network "/run-corda.sh" 17 seconds ago Up 17 seconds 0.0.0.0:32894->10003/tcp, 0.0.0.0:32897->10005/tcp, 0.0.0.0:32892->10020/tcp, 0.0.0.0:32893->12222/tcp partya0
cf7ab689f493 node-notary:corda-network "/run-corda.sh" 30 seconds ago Up 31 seconds 0.0.0.0:32888->10003/tcp, 0.0.0.0:32889->10005/tcp, 0.0.0.0:32890->10020/tcp, 0.0.0.0:32891->12222/tcp notary0
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
406868b4ba69 node-partyc:corda-network "run-corda" 17 seconds ago Up 16 seconds 0.0.0.0:32902->10003/tcp, 0.0.0.0:32895->10005/tcp, 0.0.0.0:32898->10020/tcp, 0.0.0.0:32900->12222/tcp partyc0
4546a2fa8de7 node-partyb:corda-network "run-corda" 17 seconds ago Up 17 seconds 0.0.0.0:32896->10003/tcp, 0.0.0.0:32899->10005/tcp, 0.0.0.0:32901->10020/tcp, 0.0.0.0:32903->12222/tcp partyb0
c8c44c515bdb node-partya:corda-network "run-corda" 17 seconds ago Up 17 seconds 0.0.0.0:32894->10003/tcp, 0.0.0.0:32897->10005/tcp, 0.0.0.0:32892->10020/tcp, 0.0.0.0:32893->12222/tcp partya0
cf7ab689f493 node-notary:corda-network "run-corda" 30 seconds ago Up 31 seconds 0.0.0.0:32888->10003/tcp, 0.0.0.0:32889->10005/tcp, 0.0.0.0:32890->10020/tcp, 0.0.0.0:32891->12222/tcp notary0
Interacting with the nodes
^^^^^^^^^^^^^^^^^^^^^^^^^^
@ -173,11 +185,11 @@ node has been started correctly, run the following in the previously connected S
Tue Jul 17 15:47:14 GMT 2018>>> run networkMapSnapshot
[
{ "addresses" : [ "partya0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyA, L=London, C=GB" ], "platformVersion" : 3, "serial" : 1532701330613 },
{ "addresses" : [ "notary0:10020" ], "legalIdentitiesAndCerts" : [ "O=Notary, L=London, C=GB" ], "platformVersion" : 3, "serial" : 1532701305115 },
{ "addresses" : [ "partyc0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyC, L=Paris, C=FR" ], "platformVersion" : 3, "serial" : 1532701331608 },
{ "addresses" : [ "partyb0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyB, L=New York, C=US" ], "platformVersion" : 3, "serial" : 1532701330118 },
{ "addresses" : [ "partya1:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyZ, L=London, C=GB" ], "platformVersion" : 3, "serial" : 1532701630861 }
{ "addresses" : [ "partya0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyA, L=London, C=GB" ], "platformVersion" : |platform_version|, "serial" : 1532701330613 },
{ "addresses" : [ "notary0:10020" ], "legalIdentitiesAndCerts" : [ "O=Notary, L=London, C=GB" ], "platformVersion" : |platform_version|, "serial" : 1532701305115 },
{ "addresses" : [ "partyc0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyC, L=Paris, C=FR" ], "platformVersion" : |platform_version|, "serial" : 1532701331608 },
{ "addresses" : [ "partyb0:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyB, L=New York, C=US" ], "platformVersion" : |platform_version|, "serial" : 1532701330118 },
{ "addresses" : [ "partya1:10020" ], "legalIdentitiesAndCerts" : [ "O=PartyZ, L=London, C=GB" ], "platformVersion" : |platform_version|, "serial" : 1532701630861 }
]
Shutting down the nodes

2
settings.gradle
View File

@ -52,7 +52,7 @@ include 'tools:bootstrapper'
include 'tools:blobinspector'
include 'tools:shell'
include 'tools:shell-cli'
include 'tools:network-bootstrapper'
include 'tools:network-builder'
include 'tools:cliutils'
include 'tools:worldmap'
include 'example-code'

49
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/nodes/NodeBuilder.kt
View File

@ -1,49 +0,0 @@
package net.corda.bootstrapper.nodes
import com.github.dockerjava.core.command.BuildImageResultCallback
import com.typesafe.config.Config
import com.typesafe.config.ConfigFactory
import com.typesafe.config.ConfigValueFactory
import net.corda.bootstrapper.docker.DockerUtils
import net.corda.common.configuration.parsing.internal.Configuration
import net.corda.common.validation.internal.Validated
import net.corda.node.services.config.NodeConfiguration
import net.corda.node.services.config.parseAsNodeConfiguration
import org.slf4j.LoggerFactory
import java.io.File
open class NodeBuilder {
companion object {
val LOG = LoggerFactory.getLogger(NodeBuilder::class.java)
}
fun buildNode(copiedNode: CopiedNode): BuiltNode {
val localDockerClient = DockerUtils.createLocalDockerClient()
val copiedNodeConfig = copiedNode.copiedNodeConfig
val nodeDir = copiedNodeConfig.parentFile
if (!copiedNodeConfig.exists()) {
throw IllegalStateException("There is no nodeConfig for dir: $copiedNodeConfig")
}
val nodeConfig = ConfigFactory.parseFile(copiedNodeConfig)
LOG.info("starting to build docker image for: $nodeDir")
val nodeImageId = localDockerClient.buildImageCmd()
.withDockerfile(File(nodeDir, "Dockerfile"))
.withBaseDirectory(nodeDir)
.exec(BuildImageResultCallback()).awaitImageId()
LOG.info("finished building docker image for: $nodeDir with id: $nodeImageId")
val config = nodeConfig.parseAsNodeConfigWithFallback(ConfigFactory.parseFile(copiedNode.configFile)).value()
return copiedNode.builtNode(config, nodeImageId)
}
}
fun Config.parseAsNodeConfigWithFallback(preCopyConfig: Config): Validated<NodeConfiguration, Configuration.Validation.Error> {
val nodeConfig = this
.withValue("baseDirectory", ConfigValueFactory.fromAnyRef(""))
.withFallback(ConfigFactory.parseResources("reference.conf"))
.withFallback(preCopyConfig)
.resolve()
return nodeConfig.parseAsNodeConfiguration()
}

32
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/nodes/NodeFinder.kt
View File

@ -1,32 +0,0 @@
package net.corda.bootstrapper.nodes
import com.typesafe.config.ConfigException
import com.typesafe.config.ConfigFactory
import net.corda.bootstrapper.Constants
import net.corda.core.utilities.contextLogger
import java.io.File
class NodeFinder(private val scratchDir: File) {
fun findNodes(): List<FoundNode> {
return scratchDir.walkBottomUp().filter { it.name == "node.conf" && !it.absolutePath.contains(Constants.BOOTSTRAPPER_DIR_NAME) }.map {
try {
ConfigFactory.parseFile(it) to it
} catch (e: ConfigException) {
null
}
}.filterNotNull()
.filter { !it.first.hasPath("notary") }
.map { (_, nodeConfigFile) ->
LOG.info("We've found a node with name: ${nodeConfigFile.parentFile.name}")
FoundNode(nodeConfigFile, nodeConfigFile.parentFile)
}.toList()
}
companion object {
val LOG = contextLogger()
}
}

26
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/notaries/NotaryFinder.kt
View File

@ -1,26 +0,0 @@
package net.corda.bootstrapper.notaries
import com.typesafe.config.ConfigException
import com.typesafe.config.ConfigFactory
import net.corda.bootstrapper.Constants
import net.corda.bootstrapper.nodes.FoundNode
import java.io.File
class NotaryFinder(private val dirToSearch: File) {
fun findNotaries(): List<FoundNode> {
return dirToSearch.walkBottomUp().filter { it.name == "node.conf" && !it.absolutePath.contains(Constants.BOOTSTRAPPER_DIR_NAME) }
.map {
try {
ConfigFactory.parseFile(it) to it
} catch (e: ConfigException) {
null
}
}.filterNotNull()
.filter { it.first.hasPath("notary") }
.map { (_, nodeConfigFile) ->
FoundNode(nodeConfigFile)
}.toList()
}
}

37
tools/network-bootstrapper/src/main/resources/node-Dockerfile
View File

@ -1,37 +0,0 @@
# Base image from (http://phusion.github.io/baseimage-docker)
FROM openjdk:8u151-jre-alpine
RUN apk upgrade --update && \
apk add --update --no-cache bash iputils && \
rm -rf /var/cache/apk/* && \
# Add user to run the app && \
addgroup corda && \
adduser -G corda -D -s /bin/bash corda && \
# Create /opt/corda directory && \
mkdir -p /opt/corda/plugins && \
mkdir -p /opt/corda/logs && \
mkdir -p /opt/corda/additional-node-infos && \
mkdir -p /opt/node-setup
# Copy corda files
ADD --chown=corda:corda corda.jar /opt/corda/corda.jar
ADD --chown=corda:corda node.conf /opt/corda/node.conf
ADD --chown=corda:corda cordapps/ /opt/corda/cordapps
# Copy node info watcher script
ADD --chown=corda:corda node_info_watcher.sh /opt/corda/
COPY run-corda.sh /run-corda.sh
RUN chmod +x /run-corda.sh && \
chmod +x /opt/corda/node_info_watcher.sh && \
sync && \
chown -R corda:corda /opt/corda
# Working directory for Corda
WORKDIR /opt/corda
ENV HOME=/opt/corda
USER corda
# Start it
CMD ["/run-corda.sh"]

39
tools/network-bootstrapper/src/main/resources/notary-Dockerfile
View File

@ -1,39 +0,0 @@
# Base image from (http://phusion.github.io/baseimage-docker)
FROM openjdk:8u151-jre-alpine
RUN apk upgrade --update && \
apk add --update --no-cache bash iputils && \
rm -rf /var/cache/apk/* && \
# Add user to run the app && \
addgroup corda && \
adduser -G corda -D -s /bin/bash corda && \
# Create /opt/corda directory && \
mkdir -p /opt/corda/plugins && \
mkdir -p /opt/corda/logs && \
mkdir -p /opt/corda/additional-node-infos && \
mkdir -p /opt/node-setup
# Copy corda files
ADD --chown=corda:corda corda.jar /opt/corda/corda.jar
ADD --chown=corda:corda node.conf /opt/corda/node.conf
ADD --chown=corda:corda cordapps/ /opt/corda/cordapps
ADD --chown=corda:corda certificates/ /opt/corda/certificates
#ADD --chown=corda:corda nodeInfo-* /opt/corda/
# Copy node info watcher script
ADD --chown=corda:corda node_info_watcher.sh /opt/corda/
COPY run-corda.sh /run-corda.sh
RUN chmod +x /run-corda.sh && \
chmod +x /opt/corda/node_info_watcher.sh && \
sync && \
chown -R corda:corda /opt/corda
# Working directory for Corda
WORKDIR /opt/corda
ENV HOME=/opt/corda
USER corda
# Start it
CMD ["/run-corda.sh"]

30
tools/network-bootstrapper/build.gradle → tools/network-builder/build.gradle
View File

@ -7,9 +7,11 @@ apply plugin: 'idea'
apply plugin: 'java'
apply plugin: 'application'
// We need to set mainClassName before applying the shadow plugin.
mainClassName = 'net.corda.bootstrapper.Main'
mainClassName = 'net.corda.networkbuilder.Main'
apply plugin: 'com.github.johnrengelman.shadow'
apply plugin: 'net.corda.plugins.publish-utils'
apply plugin: 'com.jfrog.artifactory'
configurations {
compile {
@ -49,14 +51,32 @@ tasks.withType(JavaCompile) {
options.compilerArgs << '-proc:none'
}
jar.enabled = false
processResources {
from file("$rootDir/config/dev/log4j2.xml")
}
shadowJar {
baseName = 'network-bootstrapper'
baseName = 'network-builder'
classifier = null
version = null
zip64 true
}
task buildNetworkBootstrapper(dependsOn: shadowJar)
assemble.dependsOn buildNetworkBootstrapper
task buildNetworkBuilder(dependsOn: shadowJar)
assemble.dependsOn buildNetworkBuilder
artifacts {
publish shadowJar {
classifier = ""
}
}
jar {
classifier "ignore"
enabled = false
}
publish {
disableDefaultJar = true
name 'corda-tools-network-builder'
}

3
tools/network-bootstrapper/src/main/java/net/corda/bootstrapper/GuiUtils.java → tools/network-builder/src/main/java/net/corda/networkbuilder/GuiUtils.java
View File

@ -1,4 +1,4 @@
package net.corda.bootstrapper;
package net.corda.networkbuilder;
import javafx.application.Platform;
import javafx.embed.swing.JFXPanel;
@ -56,5 +56,4 @@ public class GuiUtils {
} catch (InterruptedException e) {
}
}
}

16
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/Constants.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/Constants.kt
View File

@ -1,4 +1,4 @@
package net.corda.bootstrapper
package net.corda.networkbuilder
import com.fasterxml.jackson.core.JsonGenerator
import com.fasterxml.jackson.core.JsonParser
@ -12,12 +12,12 @@ import com.microsoft.azure.management.resources.fluentcore.arm.Region
class Constants {
companion object {
val NODE_P2P_PORT = 10020
val NODE_SSHD_PORT = 12222
val NODE_RPC_PORT = 10003
val NODE_RPC_ADMIN_PORT = 10005
const val NODE_P2P_PORT = 10020
const val NODE_SSHD_PORT = 12222
const val NODE_RPC_PORT = 10003
const val NODE_RPC_ADMIN_PORT = 10005
val BOOTSTRAPPER_DIR_NAME = ".bootstrapper"
const val BOOTSTRAPPER_DIR_NAME = ".bootstrapper"
fun getContextMapper(): ObjectMapper {
val objectMapper = ObjectMapper(YAMLFactory()).registerKotlinModule()
@ -39,12 +39,10 @@ class Constants {
val ALPHA_NUMERIC_ONLY_REGEX = "[^\\p{IsAlphabetic}\\p{IsDigit}]".toRegex()
val ALPHA_NUMERIC_DOT_AND_UNDERSCORE_ONLY_REGEX = "[^\\p{IsAlphabetic}\\p{IsDigit}._]".toRegex()
val REGION_ARG_NAME = "REGION"
const val REGION_ARG_NAME = "REGION"
fun ResourceGroup.restFriendlyName(): String {
return this.name().replace(ALPHA_NUMERIC_ONLY_REGEX, "").toLowerCase()
}
}
}

18
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/Main.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/Main.kt
View File

@ -1,16 +1,16 @@
@file:JvmName("Main")
package net.corda.bootstrapper
package net.corda.networkbuilder
import javafx.application.Application
import net.corda.bootstrapper.backends.Backend
import net.corda.bootstrapper.backends.Backend.BackendType.AZURE
import net.corda.bootstrapper.cli.AzureParser
import net.corda.bootstrapper.cli.CliParser
import net.corda.bootstrapper.cli.CommandLineInterface
import net.corda.bootstrapper.docker.DockerUtils
import net.corda.bootstrapper.gui.Gui
import net.corda.bootstrapper.serialization.SerializationEngine
import net.corda.networkbuilder.backends.Backend
import net.corda.networkbuilder.backends.Backend.BackendType.AZURE
import net.corda.networkbuilder.cli.AzureParser
import net.corda.networkbuilder.cli.CliParser
import net.corda.networkbuilder.cli.CommandLineInterface
import net.corda.networkbuilder.docker.DockerUtils
import net.corda.networkbuilder.gui.Gui
import net.corda.networkbuilder.serialization.SerializationEngine
import picocli.CommandLine
import javax.ws.rs.ProcessingException
import kotlin.system.exitProcess

84
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/NetworkBuilder.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/NetworkBuilder.kt
View File

@ -1,10 +1,9 @@
package net.corda.bootstrapper
package net.corda.networkbuilder
import net.corda.bootstrapper.backends.Backend
import net.corda.bootstrapper.context.Context
import net.corda.bootstrapper.nodes.*
import net.corda.bootstrapper.notaries.NotaryCopier
import net.corda.bootstrapper.notaries.NotaryFinder
import net.corda.networkbuilder.backends.Backend
import net.corda.networkbuilder.context.Context
import net.corda.networkbuilder.nodes.*
import net.corda.networkbuilder.notaries.NotaryCopier
import java.io.File
import java.util.concurrent.CompletableFuture
@ -21,8 +20,8 @@ interface NetworkBuilder {
fun onNodeBuild(callback: (BuiltNode) -> Unit): NetworkBuilder
fun onNodePushed(callback: (PushedNode) -> Unit): NetworkBuilder
fun onNodeInstance(callback: (NodeInstance) -> Unit): NetworkBuilder
fun withNetworkName(networtName: String): NetworkBuilder
/** Sets network name */
fun withNetworkName(networkName: String): NetworkBuilder
fun withBasedir(baseDir: File): NetworkBuilder
fun withBackend(backendType: Backend.BackendType): NetworkBuilder
fun withBackendOptions(options: Map<String, String>): NetworkBuilder
@ -31,12 +30,10 @@ interface NetworkBuilder {
fun onNodeStartBuild(callback: (FoundNode) -> Unit): NetworkBuilder
fun onNodePushStart(callback: (BuiltNode) -> Unit): NetworkBuilder
fun onNodeInstancesRequested(callback: (List<NodeInstanceRequest>) -> Unit): NetworkBuilder
}
private class NetworkBuilderImpl : NetworkBuilder {
@Volatile
private var onNodeLocatedCallback: ((FoundNode) -> Unit) = {}
@Volatile
@ -76,10 +73,9 @@ private class NetworkBuilderImpl : NetworkBuilder {
return this
}
override fun onNodeStartBuild(callback: (FoundNode) -> Unit): NetworkBuilder {
this.onNodeBuildStartCallback = callback
return this;
return this
}
override fun onNodeBuild(callback: (BuiltNode) -> Unit): NetworkBuilder {
@ -98,12 +94,12 @@ private class NetworkBuilderImpl : NetworkBuilder {
}
override fun onNodeInstance(callback: (NodeInstance) -> Unit): NetworkBuilder {
this.onNodeInstanceCallback = callback;
this.onNodeInstanceCallback = callback
return this
}
override fun withNetworkName(networtName: String): NetworkBuilder {
this.networkName = networtName
override fun withNetworkName(networkName: String): NetworkBuilder {
this.networkName = networkName
return this
}
@ -123,19 +119,19 @@ private class NetworkBuilderImpl : NetworkBuilder {
}
override fun onNodePushStart(callback: (BuiltNode) -> Unit): NetworkBuilder {
this.onNodePushStartCallback = callback;
return this;
this.onNodePushStartCallback = callback
return this
}
override fun build(): CompletableFuture<Pair<List<NodeInstance>, Context>> {
val cacheDir = File(workingDir, cacheDirName)
val baseDir = workingDir!!
val context = Context(networkName, backendType, backendOptions)
if (cacheDir.exists()) cacheDir.deleteRecursively()
val (containerPusher, instantiator, volume) = Backend.fromContext(context, cacheDir)
val nodeFinder = NodeFinder(baseDir)
val notaryFinder = NotaryFinder(baseDir)
val notaryFinder = NodeFinder(baseDir)
val notaryCopier = NotaryCopier(cacheDir)
val nodeInstantiator = NodeInstantiator(instantiator, context)
@ -163,12 +159,21 @@ private class NetworkBuilderImpl : NetworkBuilder {
val notariesFuture = notaryDiscoveryFuture.thenCompose { copiedNotaries ->
copiedNotaries
.map { copiedNotary ->
nodeBuilder.buildNode(copiedNotary).also(onNodeBuiltCallback)
}.map { builtNotary ->
onNodePushStartCallback(builtNotary)
nodePusher.pushNode(builtNotary).thenApply { it.also(onNodePushedCallback) }
nodeBuilder.buildNode(copiedNotary).thenAlsoAsync {
onNodeBuildStartCallback.invoke(it)
}
}.map { builtNotaryFuture ->
builtNotaryFuture.thenComposeAsync { builtNotary ->
onNodeBuiltCallback(builtNotary)
onNodePushStartCallback(builtNotary)
nodePusher.pushNode(builtNotary).thenAlsoAsync { pushedNotary ->
onNodePushedCallback(pushedNotary)
}
}
}.map { pushedNotary ->
pushedNotary.thenApplyAsync { nodeInstantiator.createInstanceRequest(it).also { onNodeInstanceRequestedCallback.invoke(listOf(it)) } }
pushedNotary.thenApplyAsync {
nodeInstantiator.createInstanceRequest(it).also { onNodeInstanceRequestedCallback(listOf(it)) }
}
}.map { instanceRequest ->
instanceRequest.thenComposeAsync { request ->
nodeInstantiator.instantiateNotaryInstance(request).thenApply { it.also(onNodeInstanceCallback) }
@ -185,17 +190,16 @@ private class NetworkBuilderImpl : NetworkBuilder {
}
}.map { copiedNode: CopiedNode ->
onNodeBuildStartCallback.invoke(copiedNode)
nodeBuilder.buildNode(copiedNode).let {
onNodeBuiltCallback.invoke(it)
it
nodeBuilder.buildNode(copiedNode)
}.map { builtNodeFuture ->
builtNodeFuture.thenComposeAsync { builtNode ->
onNodeBuiltCallback.invoke(builtNode)
nodePusher.pushNode(builtNode).thenAlsoAsync { pushedNode ->
onNodePushedCallback.invoke(pushedNode)
}
}
}.map { builtNode ->
nodePusher.pushNode(builtNode).thenApplyAsync {
onNodePushedCallback.invoke(it)
it
}
}.map { pushedNode ->
pushedNode.thenApplyAsync {
}.map { pushedNodeFuture ->
pushedNodeFuture.thenApplyAsync {
nodeInstantiator.createInstanceRequests(it, nodeCount).also(onNodeInstanceRequestedCallback)
}
@ -213,10 +217,10 @@ private class NetworkBuilderImpl : NetworkBuilder {
}.toSingleFuture()
}.thenCompose { it }.thenApplyAsync { it.flatten() }
return notariesFuture.thenCombineAsync(nodesFuture, { _, nodeInstances ->
return notariesFuture.thenCombineAsync(nodesFuture) { _, nodeInstances ->
context.networkInitiated = true
nodeInstances to context
})
}
}
}
@ -225,3 +229,11 @@ fun <T> List<CompletableFuture<T>>.toSingleFuture(): CompletableFuture<List<T>>
this.map { it.getNow(null) }
}
}
/** Asynchronously runs `consumer` on the receiver CompletableFuture.*/
fun <T> CompletableFuture<T>.thenAlsoAsync(consumer: (T) -> Unit): CompletableFuture<T> {
return this.thenApplyAsync {
consumer(it)
it
}
}

24
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/backends/AzureBackend.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/backends/AzureBackend.kt
View File

@ -1,15 +1,15 @@
package net.corda.bootstrapper.backends
package net.corda.networkbuilder.backends
import com.microsoft.azure.CloudException
import com.microsoft.azure.credentials.AzureCliCredentials
import com.microsoft.azure.management.Azure
import com.microsoft.rest.LogLevel
import net.corda.bootstrapper.Constants
import net.corda.bootstrapper.containers.instance.azure.AzureInstantiator
import net.corda.bootstrapper.containers.push.azure.AzureContainerPusher
import net.corda.bootstrapper.containers.push.azure.RegistryLocator
import net.corda.bootstrapper.context.Context
import net.corda.bootstrapper.volumes.azure.AzureSmbVolume
import net.corda.networkbuilder.Constants
import net.corda.networkbuilder.containers.instance.azure.AzureInstantiator
import net.corda.networkbuilder.containers.push.azure.AzureContainerPusher
import net.corda.networkbuilder.containers.push.azure.RegistryLocator
import net.corda.networkbuilder.context.Context
import net.corda.networkbuilder.volumes.azure.AzureSmbVolume
import org.slf4j.LoggerFactory
import java.util.concurrent.CompletableFuture
@ -49,14 +49,12 @@ data class AzureBackend(override val containerPusher: AzureContainerPusher,
RegistryLocator(azure, resourceGroup)
}
val containerPusherFuture = registryLocatorFuture.thenApplyAsync {
AzureContainerPusher(azure, it.registry)
AzureContainerPusher(it.registry)
}
val azureNetworkStore = CompletableFuture.supplyAsync { AzureSmbVolume(azure, resourceGroup) }
val azureInstantiatorFuture = azureNetworkStore.thenCombine(registryLocatorFuture,
{ azureVolume, registryLocator ->
AzureInstantiator(azure, registryLocator.registry, azureVolume, resourceGroup)
}
)
val azureInstantiatorFuture = azureNetworkStore.thenCombine(registryLocatorFuture) { azureVolume, registryLocator ->
AzureInstantiator(azure, registryLocator.registry, azureVolume, resourceGroup)
}
return AzureBackend(containerPusherFuture.get(), azureInstantiatorFuture.get(), azureNetworkStore.get())
}
}

17
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/backends/Backend.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/backends/Backend.kt
View File

@ -1,11 +1,11 @@
package net.corda.bootstrapper.backends
package net.corda.networkbuilder.backends
import net.corda.bootstrapper.backends.Backend.BackendType.AZURE
import net.corda.bootstrapper.backends.Backend.BackendType.LOCAL_DOCKER
import net.corda.bootstrapper.containers.instance.Instantiator
import net.corda.bootstrapper.containers.push.ContainerPusher
import net.corda.bootstrapper.context.Context
import net.corda.bootstrapper.volumes.Volume
import net.corda.networkbuilder.backends.Backend.BackendType.AZURE
import net.corda.networkbuilder.backends.Backend.BackendType.LOCAL_DOCKER
import net.corda.networkbuilder.containers.instance.Instantiator
import net.corda.networkbuilder.containers.push.ContainerPusher
import net.corda.networkbuilder.context.Context
import net.corda.networkbuilder.volumes.Volume
import java.io.File
interface Backend {
@ -26,12 +26,9 @@ interface Backend {
AZURE("Azure Containers"), LOCAL_DOCKER("Local Docker");
override fun toString(): String {
return this.displayName
}
}
operator fun component1(): ContainerPusher {

14
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/backends/DockerBackend.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/backends/DockerBackend.kt
View File

@ -1,16 +1,15 @@
package net.corda.bootstrapper.backends
package net.corda.networkbuilder.backends
import net.corda.bootstrapper.containers.instance.docker.DockerInstantiator
import net.corda.bootstrapper.containers.push.docker.DockerContainerPusher
import net.corda.bootstrapper.context.Context
import net.corda.bootstrapper.volumes.docker.LocalVolume
import net.corda.networkbuilder.containers.instance.docker.DockerInstantiator
import net.corda.networkbuilder.containers.push.docker.DockerContainerPusher
import net.corda.networkbuilder.context.Context
import net.corda.networkbuilder.volumes.docker.LocalVolume
import java.io.File
class DockerBackend(override val containerPusher: DockerContainerPusher,
override val instantiator: DockerInstantiator,
override val volume: LocalVolume) : Backend {
companion object {
fun fromContext(context: Context, baseDir: File): DockerBackend {
val dockerContainerPusher = DockerContainerPusher()
@ -19,7 +18,4 @@ class DockerBackend(override val containerPusher: DockerContainerPusher,
return DockerBackend(dockerContainerPusher, dockerInstantiator, localVolume)
}
}
}

24
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/cli/CommandLineInterface.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/cli/CommandLineInterface.kt
View File

@ -1,13 +1,13 @@
package net.corda.bootstrapper.cli
package net.corda.networkbuilder.cli
import com.fasterxml.jackson.databind.ObjectMapper
import net.corda.bootstrapper.Constants
import net.corda.bootstrapper.NetworkBuilder
import net.corda.bootstrapper.backends.Backend
import net.corda.bootstrapper.context.Context
import net.corda.bootstrapper.nodes.NodeAdder
import net.corda.bootstrapper.nodes.NodeInstantiator
import net.corda.bootstrapper.toSingleFuture
import net.corda.networkbuilder.Constants
import net.corda.networkbuilder.NetworkBuilder
import net.corda.networkbuilder.backends.Backend
import net.corda.networkbuilder.context.Context
import net.corda.networkbuilder.nodes.NodeAdder
import net.corda.networkbuilder.nodes.NodeInstantiator
import net.corda.networkbuilder.toSingleFuture
import net.corda.core.identity.CordaX500Name
import net.corda.core.utilities.getOrThrow
import java.io.File
@ -44,13 +44,12 @@ class CommandLineInterface {
}.toSingleFuture().getOrThrow()
persistContext(contextFile, objectMapper, context)
}
}
private fun setupContextFromExisting(contextFile: File, objectMapper: ObjectMapper): Context {
return contextFile.let {
if (it.exists()) {
it.inputStream().use {
return contextFile.let { file ->
if (file.exists()) {
file.inputStream().use {
objectMapper.readValue(it, Context::class.java)
}
} else {
@ -59,7 +58,6 @@ class CommandLineInterface {
}
}
private fun persistContext(contextFile: File, objectMapper: ObjectMapper, context: Context?) {
contextFile.outputStream().use {
objectMapper.writeValue(it, context)

6
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/cli/CommandParsers.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/cli/CommandParsers.kt
View File

@ -1,8 +1,8 @@
package net.corda.bootstrapper.cli
package net.corda.networkbuilder.cli
import com.microsoft.azure.management.resources.fluentcore.arm.Region
import net.corda.bootstrapper.Constants
import net.corda.bootstrapper.backends.Backend
import net.corda.networkbuilder.Constants
import net.corda.networkbuilder.backends.Backend
import picocli.CommandLine
import picocli.CommandLine.Option
import java.io.File

2
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/containers/instance/InstanceInfo.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/containers/instance/InstanceInfo.kt
View File

@ -1,4 +1,4 @@
package net.corda.bootstrapper.containers.instance
package net.corda.networkbuilder.containers.instance
data class InstanceInfo(val groupId: String,
val instanceName: String,

6
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/containers/instance/Instantiator.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/containers/instance/Instantiator.kt
View File

@ -1,17 +1,15 @@
package net.corda.bootstrapper.containers.instance
package net.corda.networkbuilder.containers.instance
import java.util.concurrent.CompletableFuture
interface Instantiator {
fun instantiateContainer(imageId: String,
portsToOpen: List<Int>,
instanceName: String,
env: Map<String, String>? = null): CompletableFuture<Pair<String, Map<Int, Int>>>
companion object {
val ADDITIONAL_NODE_INFOS_PATH = "/opt/corda/additional-node-infos"
const val ADDITIONAL_NODE_INFOS_PATH = "/opt/corda/additional-node-infos"
}
fun getExpectedFQDN(instanceName: String): String

22
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/containers/instance/azure/AzureInstantiator.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/containers/instance/azure/AzureInstantiator.kt
View File

@ -1,4 +1,4 @@
package net.corda.bootstrapper.containers.instance.azure
package net.corda.networkbuilder.containers.instance.azure
import com.microsoft.azure.management.Azure
import com.microsoft.azure.management.containerinstance.ContainerGroup
@ -6,11 +6,11 @@ import com.microsoft.azure.management.containerinstance.ContainerGroupRestartPol
import com.microsoft.azure.management.containerregistry.Registry
import com.microsoft.azure.management.resources.ResourceGroup
import com.microsoft.rest.ServiceCallback
import net.corda.bootstrapper.Constants.Companion.restFriendlyName
import net.corda.bootstrapper.containers.instance.Instantiator
import net.corda.bootstrapper.containers.instance.Instantiator.Companion.ADDITIONAL_NODE_INFOS_PATH
import net.corda.bootstrapper.containers.push.azure.RegistryLocator.Companion.parseCredentials
import net.corda.bootstrapper.volumes.azure.AzureSmbVolume
import net.corda.networkbuilder.Constants.Companion.restFriendlyName
import net.corda.networkbuilder.containers.instance.Instantiator
import net.corda.networkbuilder.containers.instance.Instantiator.Companion.ADDITIONAL_NODE_INFOS_PATH
import net.corda.networkbuilder.containers.push.azure.RegistryLocator.Companion.parseCredentials
import net.corda.networkbuilder.volumes.azure.AzureSmbVolume
import org.slf4j.LoggerFactory
import java.util.concurrent.CompletableFuture
@ -28,9 +28,9 @@ class AzureInstantiator(private val azure: Azure,
LOG.info("Starting instantiation of container: $instanceName using $imageId")
val registryAddress = registry.loginServerUrl()
val (username, password) = registry.parseCredentials();
val (username, password) = registry.parseCredentials()
val mountName = "node-setup"
val future = CompletableFuture<Pair<String, Map<Int, Int>>>().also {
return CompletableFuture<Pair<String, Map<Int, Int>>>().also {
azure.containerGroups().define(buildIdent(instanceName))
.withRegion(resourceGroup.region())
.withExistingResourceGroup(resourceGroup)
@ -56,11 +56,10 @@ class AzureInstantiator(private val azure: Azure,
override fun success(result: ContainerGroup) {
val fqdn = result.fqdn()
LOG.info("Completed instantiation: $instanceName is running at $fqdn with port(s) $portsToOpen exposed")
it.complete(result.fqdn() to portsToOpen.map { it to it }.toMap())
it.complete(result.fqdn() to portsToOpen.map { port -> port to port }.toMap())
}
})
}
return future
}
private fun buildIdent(instanceName: String) = "$instanceName-${resourceGroup.restFriendlyName()}"
@ -75,11 +74,10 @@ class AzureInstantiator(private val azure: Azure,
LOG.info("Found an existing instance of: $containerName destroying ContainerGroup")
azure.containerGroups().deleteByResourceGroup(resourceGroup.name(), containerName)
}
return existingContainer;
return existingContainer
}
companion object {
val LOG = LoggerFactory.getLogger(AzureInstantiator::class.java)
}
}

28
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/containers/instance/docker/DockerInstantiator.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/containers/instance/docker/DockerInstantiator.kt
View File

@ -1,19 +1,18 @@
package net.corda.bootstrapper.containers.instance.docker
package net.corda.networkbuilder.containers.instance.docker
import com.github.dockerjava.api.model.*
import net.corda.bootstrapper.Constants
import net.corda.bootstrapper.containers.instance.Instantiator
import net.corda.bootstrapper.context.Context
import net.corda.bootstrapper.docker.DockerUtils
import net.corda.bootstrapper.volumes.docker.LocalVolume
import net.corda.networkbuilder.Constants
import net.corda.networkbuilder.containers.instance.Instantiator
import net.corda.networkbuilder.context.Context
import net.corda.networkbuilder.docker.DockerUtils
import net.corda.networkbuilder.volumes.docker.LocalVolume
import org.slf4j.LoggerFactory
import java.util.concurrent.CompletableFuture
class DockerInstantiator(private val volume: LocalVolume,
private val context: Context) : Instantiator {
val networkId = setupNetwork();
private val networkId = setupNetwork()
override fun instantiateContainer(imageId: String,
portsToOpen: List<Int>,
@ -43,7 +42,9 @@ class DockerInstantiator(private val volume: LocalVolume,
}
LOG.info("starting local docker instance of: $imageId with name $instanceName and env: $env")
val ports = (portsToOpen + Constants.NODE_RPC_ADMIN_PORT).map { ExposedPort.tcp(it) }.map { PortBinding(null, it) }.let { Ports(*it.toTypedArray()) }
val ports = (portsToOpen + Constants.NODE_RPC_ADMIN_PORT).map { ExposedPort.tcp(it) }
.map { PortBinding(null, it) }
.let { Ports(*it.toTypedArray()) }
val createCmd = localClient.createContainerCmd(imageId)
.withName(instanceName)
.withVolumes(nodeInfosVolume)
@ -56,25 +57,20 @@ class DockerInstantiator(private val volume: LocalVolume,
localClient.startContainerCmd(createCmd.id).exec()
val foundContainer = localClient.listContainersCmd().exec()
.filter { it.id == (createCmd.id) }
.firstOrNull()
.firstOrNull { it.id == createCmd.id }
val portMappings = foundContainer?.ports?.map {
(it.privatePort ?: 0) to (it.publicPort ?: 0)
}?.toMap()?.toMap()
?: portsToOpen.map { it to it }.toMap()
return CompletableFuture.completedFuture(("localhost") to portMappings)
}
private fun buildDockerEnv(env: Map<String, String>?) =
(env ?: emptyMap()).entries.map { (key, value) -> "$key=$value" }.toList()
override fun getExpectedFQDN(instanceName: String): String {
return instanceName
}
override fun getExpectedFQDN(instanceName: String): String = instanceName
private fun setupNetwork(): String {
val createLocalDockerClient = DockerUtils.createLocalDockerClient()

2
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/containers/push/ContainerPusher.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/containers/push/ContainerPusher.kt
View File

@ -1,4 +1,4 @@
package net.corda.bootstrapper.containers.push
package net.corda.networkbuilder.containers.push
import java.util.concurrent.CompletableFuture

16
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/containers/push/azure/AzureContainerPusher.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/containers/push/azure/AzureContainerPusher.kt
View File

@ -1,25 +1,21 @@
package net.corda.bootstrapper.containers.push.azure
package net.corda.networkbuilder.containers.push.azure
import com.github.dockerjava.api.async.ResultCallback
import com.github.dockerjava.api.model.PushResponseItem
import com.microsoft.azure.management.Azure
import com.microsoft.azure.management.containerregistry.Registry
import net.corda.bootstrapper.containers.push.ContainerPusher
import net.corda.bootstrapper.containers.push.azure.RegistryLocator.Companion.parseCredentials
import net.corda.bootstrapper.docker.DockerUtils
import net.corda.networkbuilder.containers.push.ContainerPusher
import net.corda.networkbuilder.containers.push.azure.RegistryLocator.Companion.parseCredentials
import net.corda.networkbuilder.docker.DockerUtils
import org.slf4j.LoggerFactory
import java.io.Closeable
import java.util.concurrent.CompletableFuture
class AzureContainerPusher(private val azure: Azure, private val azureRegistry: Registry) : ContainerPusher {
class AzureContainerPusher(private val azureRegistry: Registry) : ContainerPusher {
override fun pushContainerToImageRepository(localImageId: String,
remoteImageName: String,
networkName: String): CompletableFuture<String> {
val (registryUser, registryPassword) = azureRegistry.parseCredentials()
val dockerClient = DockerUtils.createDockerClient(
azureRegistry.loginServerUrl(),
@ -57,6 +53,4 @@ class AzureContainerPusher(private val azure: Azure, private val azureRegistry:
companion object {
val LOG = LoggerFactory.getLogger(AzureContainerPusher::class.java)
}
}

22
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/containers/push/azure/AzureRegistryLocator.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/containers/push/azure/AzureRegistryLocator.kt
View File

@ -1,37 +1,34 @@
package net.corda.bootstrapper.containers.push.azure
package net.corda.networkbuilder.containers.push.azure
import com.microsoft.azure.management.Azure
import com.microsoft.azure.management.containerregistry.AccessKeyType
import com.microsoft.azure.management.containerregistry.Registry
import com.microsoft.azure.management.resources.ResourceGroup
import net.corda.bootstrapper.Constants.Companion.restFriendlyName
import net.corda.bootstrapper.containers.instance.azure.AzureInstantiator
import net.corda.networkbuilder.Constants.Companion.restFriendlyName
import net.corda.networkbuilder.containers.instance.azure.AzureInstantiator
import org.slf4j.LoggerFactory
class RegistryLocator(private val azure: Azure,
private val resourceGroup: ResourceGroup) {
val registry: Registry = locateRegistry()
private fun locateRegistry(): Registry {
LOG.info("Attempting to find existing registry with name: ${resourceGroup.restFriendlyName()}")
val found = azure.containerRegistries().getByResourceGroup(resourceGroup.name(), resourceGroup.restFriendlyName())
if (found == null) {
return if (found == null) {
LOG.info("Did not find existing container registry - creating new registry with name ${resourceGroup.restFriendlyName()}")
return azure.containerRegistries()
azure.containerRegistries()
.define(resourceGroup.restFriendlyName())
.withRegion(resourceGroup.region().name())
.withExistingResourceGroup(resourceGroup)
.withBasicSku()
.withRegistryNameAsAdminUser()
.create()
} else {
LOG.info("found existing registry with name: ${resourceGroup.restFriendlyName()} reusing")
return found
found
}
}
@ -44,12 +41,5 @@ class RegistryLocator(private val azure: Azure,
}
val LOG = LoggerFactory.getLogger(AzureInstantiator::class.java)
}
}

7
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/containers/push/docker/DockerContainerPusher.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/containers/push/docker/DockerContainerPusher.kt
View File

@ -1,12 +1,11 @@
package net.corda.bootstrapper.containers.push.docker
package net.corda.networkbuilder.containers.push.docker
import net.corda.bootstrapper.containers.push.ContainerPusher
import net.corda.bootstrapper.docker.DockerUtils
import net.corda.networkbuilder.containers.push.ContainerPusher
import net.corda.networkbuilder.docker.DockerUtils
import java.util.concurrent.CompletableFuture
class DockerContainerPusher : ContainerPusher {
override fun pushContainerToImageRepository(localImageId: String, remoteImageName: String, networkName: String): CompletableFuture<String> {
val dockerClient = DockerUtils.createLocalDockerClient()
dockerClient.tagImageCmd(localImageId, remoteImageName, networkName).withForce().exec()

12
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/context/Context.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/context/Context.kt
View File

@ -1,15 +1,14 @@
package net.corda.bootstrapper.context
package net.corda.networkbuilder.context
import net.corda.bootstrapper.Constants
import net.corda.bootstrapper.backends.Backend
import net.corda.bootstrapper.nodes.NodeInstanceRequest
import net.corda.networkbuilder.Constants
import net.corda.networkbuilder.backends.Backend
import net.corda.networkbuilder.nodes.NodeInstanceRequest
import net.corda.core.identity.CordaX500Name
import org.apache.activemq.artemis.utils.collections.ConcurrentHashSet
import java.util.concurrent.ConcurrentHashMap
class Context(val networkName: String, val backendType: Backend.BackendType, backendOptions: Map<String, String> = emptyMap()) {
@Volatile
var safeNetworkName: String = networkName.replace(Constants.ALPHA_NUMERIC_ONLY_REGEX, "").toLowerCase()
@ -30,7 +29,6 @@ class Context(val networkName: String, val backendType: Backend.BackendType, bac
registerNode(request.name, request)
}
data class PersistableNodeInstance(
val groupName: String,
val groupX500: CordaX500Name?,
@ -43,7 +41,6 @@ class Context(val networkName: String, val backendType: Backend.BackendType, bac
val rpcUser: String,
val rpcPassword: String)
companion object {
fun fromInstanceRequest(nodeInstanceRequest: NodeInstanceRequest): PersistableNodeInstance {
return PersistableNodeInstance(
@ -58,7 +55,6 @@ class Context(val networkName: String, val backendType: Backend.BackendType, bac
"",
""
)
}
}

4
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/docker/DockerUtils.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/docker/DockerUtils.kt
View File

@ -1,4 +1,4 @@
package net.corda.bootstrapper.docker
package net.corda.networkbuilder.docker
import com.github.dockerjava.api.DockerClient
import com.github.dockerjava.core.DefaultDockerClientConfig
@ -30,6 +30,4 @@ object DockerUtils {
.withRegistryPassword(password)
.build()
}
}

40
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/gui/BootstrapperView.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/gui/BootstrapperView.kt
View File

@ -1,6 +1,7 @@
package net.corda.bootstrapper.gui
package net.corda.networkbuilder.gui
import com.microsoft.azure.management.resources.fluentcore.arm.Region
import javafx.beans.binding.Bindings
import javafx.beans.property.SimpleObjectProperty
import javafx.beans.property.SimpleStringProperty
import javafx.collections.ObservableListBase
@ -13,14 +14,13 @@ import javafx.scene.layout.HBox
import javafx.scene.layout.Priority
import javafx.scene.layout.VBox
import javafx.stage.DirectoryChooser
import net.corda.bootstrapper.Constants
import net.corda.bootstrapper.GuiUtils
import net.corda.bootstrapper.NetworkBuilder
import net.corda.bootstrapper.backends.Backend
import net.corda.bootstrapper.baseArgs
import net.corda.bootstrapper.context.Context
import net.corda.bootstrapper.nodes.*
import net.corda.bootstrapper.notaries.NotaryFinder
import net.corda.networkbuilder.Constants
import net.corda.networkbuilder.GuiUtils
import net.corda.networkbuilder.NetworkBuilder
import net.corda.networkbuilder.backends.Backend
import net.corda.networkbuilder.baseArgs
import net.corda.networkbuilder.context.Context
import net.corda.networkbuilder.nodes.*
import net.corda.core.identity.CordaX500Name
import org.apache.commons.lang3.RandomStringUtils
import org.controlsfx.control.SegmentedButton
@ -50,7 +50,7 @@ class BootstrapperView : View("Corda Network Builder") {
visuallyTweakBackendSelector()
buildButton.run {
enableWhen { controller.baseDir.isNotNull }
enableWhen { controller.hasNodesOrNotaries.and(controller.baseDir.isNotNull) }
action {
var networkName = "corda-network"
@ -104,7 +104,6 @@ class BootstrapperView : View("Corda Network Builder") {
override fun get(index: Int): String {
return controller.foundNodes[index].id
}
override val size: Int
get() = controller.foundNodes.size
}
@ -139,7 +138,8 @@ class BootstrapperView : View("Corda Network Builder") {
it.instanceName,
it.instanceAddress,
it.reachableAddress,
it.portMapping[Constants.NODE_P2P_PORT] ?: Constants.NODE_P2P_PORT,
it.portMapping[Constants.NODE_P2P_PORT]
?: Constants.NODE_P2P_PORT,
it.portMapping[Constants.NODE_SSHD_PORT]
?: Constants.NODE_SSHD_PORT))
}
@ -160,7 +160,7 @@ class BootstrapperView : View("Corda Network Builder") {
columnResizePolicy = TableView.CONSTRAINED_RESIZE_POLICY
hgrow = Priority.ALWAYS
onMouseClicked = EventHandler<MouseEvent> { _ ->
onMouseClicked = EventHandler<MouseEvent> {
val selectedItem: NodeTemplateInfo = selectionModel.selectedItem ?: return@EventHandler
infoTextArea.text = YAML_MAPPER.writeValueAsString(translateForPrinting(selectedItem))
}
@ -221,17 +221,17 @@ class BootstrapperView : View("Corda Network Builder") {
nodeFinder.findNodes()
}
val foundNotaries = CompletableFuture.supplyAsync {
val notaryFinder = NotaryFinder(dir)
val notaryFinder = NodeFinder(dir)
notaryFinder.findNotaries()
}
foundNodes.thenCombine(foundNotaries) { nodes, notaries ->
notaries to nodes
}.thenAcceptAsync({ (notaries: List<FoundNode>, nodes: List<FoundNode>) ->
}.thenAcceptAsync { (notaries: List<FoundNode>, nodes: List<FoundNode>) ->
runLater {
controller.foundNodes(nodes)
controller.notaries(notaries)
}
})
}
}
class NodeTemplateInfo(templateId: String, type: NodeType) {
@ -260,7 +260,7 @@ class BootstrapperView : View("Corda Network Builder") {
val foundNodes = Collections.synchronizedList(ArrayList<FoundNodeTableEntry>()).observable()
val foundNotaries = Collections.synchronizedList(ArrayList<FoundNode>()).observable()
val networkContext = SimpleObjectProperty<Context>(null)
var hasNodesOrNotaries = Bindings.size(foundNotaries).greaterThan(0).or(Bindings.size(foundNotaries).greaterThan(0))
val unsortedNodes = Collections.synchronizedList(ArrayList<NodeTemplateInfo>()).observable()
val sortedNodes = SortedList(unsortedNodes, Comparator<NodeTemplateInfo> { o1, o2 ->
compareValues(o1.nodeType.toString() + o1.templateId, o2.nodeType.toString() + o2.templateId) * -1
@ -324,8 +324,10 @@ class BootstrapperView : View("Corda Network Builder") {
nodeInstance.nodeInstanceName,
nodeInstance.expectedFqName,
nodeInstance.reachableAddress,
nodeInstance.portMapping[Constants.NODE_P2P_PORT] ?: Constants.NODE_P2P_PORT,
nodeInstance.portMapping[Constants.NODE_SSHD_PORT] ?: Constants.NODE_SSHD_PORT)
nodeInstance.portMapping[Constants.NODE_P2P_PORT]
?: Constants.NODE_P2P_PORT,
nodeInstance.portMapping[Constants.NODE_SSHD_PORT]
?: Constants.NODE_SSHD_PORT)
)
}

2
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/gui/Gui.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/gui/Gui.kt
View File

@ -1,4 +1,4 @@
package net.corda.bootstrapper.gui
package net.corda.networkbuilder.gui
import javafx.stage.Stage
import tornadofx.App

3
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/nodes/BuiltNode.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/nodes/BuiltNode.kt
View File

@ -1,4 +1,4 @@
package net.corda.bootstrapper.nodes
package net.corda.networkbuilder.nodes
import net.corda.node.services.config.NodeConfiguration
import java.io.File
@ -7,7 +7,6 @@ open class BuiltNode(configFile: File, baseDirectory: File,
copiedNodeConfig: File, copiedNodeDir: File,
val nodeConfig: NodeConfiguration, val localImageId: String) : CopiedNode(configFile, baseDirectory, copiedNodeConfig, copiedNodeDir) {
override fun toString(): String {
return "BuiltNode(" +
"nodeConfig=$nodeConfig," +

8
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/nodes/CopiedNode.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/nodes/CopiedNode.kt
View File

@ -1,4 +1,4 @@
package net.corda.bootstrapper.nodes
package net.corda.networkbuilder.nodes
import net.corda.node.services.config.NodeConfiguration
import java.io.File
@ -12,11 +12,11 @@ open class CopiedNode(configFile: File, baseDirectory: File,
)
operator fun component4(): File {
return copiedNodeDir;
return copiedNodeDir
}
operator fun component5(): File {
return copiedNodeConfig;
return copiedNodeConfig
}
@ -35,6 +35,4 @@ open class CopiedNode(configFile: File, baseDirectory: File,
fun toBuiltNode(nodeConfig: NodeConfiguration, localImageId: String): BuiltNode {
return BuiltNode(this.configFile, this.baseDirectory, this.copiedNodeConfig, this.copiedNodeDir, nodeConfig, localImageId)
}
}

16
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/nodes/FoundNode.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/nodes/FoundNode.kt
View File

@ -1,22 +1,22 @@
package net.corda.bootstrapper.nodes
package net.corda.networkbuilder.nodes
import net.corda.networkbuilder.Constants
import java.io.File
open class FoundNode(open val configFile: File,
open val baseDirectory: File = configFile.parentFile,
val name: String = configFile.parentFile.name.toLowerCase().replace(net.corda.bootstrapper.Constants.ALPHA_NUMERIC_ONLY_REGEX, "")) {
val name: String = configFile.parentFile.name.toLowerCase().replace(Constants.ALPHA_NUMERIC_ONLY_REGEX, "")) {
operator fun component1(): File {
return baseDirectory;
return baseDirectory
}
operator fun component2(): File {
return configFile;
return configFile
}
operator fun component3(): String {
return name;
return name
}
override fun equals(other: Any?): Boolean {
@ -46,8 +46,4 @@ open class FoundNode(open val configFile: File,
fun toCopiedNode(copiedNodeConfig: File, copiedNodeDir: File): CopiedNode {
return CopiedNode(this.configFile, this.baseDirectory, copiedNodeConfig, copiedNodeDir)
}
}

6
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/nodes/NodeAdder.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/nodes/NodeAdder.kt
View File

@ -1,7 +1,7 @@
package net.corda.bootstrapper.nodes
package net.corda.networkbuilder.nodes
import net.corda.bootstrapper.containers.instance.InstanceInfo
import net.corda.bootstrapper.context.Context
import net.corda.networkbuilder.containers.instance.InstanceInfo
import net.corda.networkbuilder.context.Context
import net.corda.core.identity.CordaX500Name
import java.util.concurrent.CompletableFuture

71
tools/network-builder/src/main/kotlin/net/corda/networkbuilder/nodes/NodeBuilder.kt Normal file
View File

@ -0,0 +1,71 @@
package net.corda.networkbuilder.nodes
import com.github.dockerjava.api.model.BuildResponseItem
import com.github.dockerjava.core.async.ResultCallbackTemplate
import com.github.dockerjava.core.command.BuildImageResultCallback
import com.typesafe.config.Config
import com.typesafe.config.ConfigFactory
import com.typesafe.config.ConfigValueFactory
import net.corda.networkbuilder.docker.DockerUtils
import net.corda.common.configuration.parsing.internal.Configuration
import net.corda.common.validation.internal.Validated
import net.corda.node.services.config.NodeConfiguration
import net.corda.node.services.config.parseAsNodeConfiguration
import org.slf4j.LoggerFactory
import java.io.File
import java.util.concurrent.CompletableFuture
/** Builds a Docker image. */
open class NodeBuilder {
companion object {
val LOG = LoggerFactory.getLogger(NodeBuilder::class.java)
}
fun buildNode(copiedNode: CopiedNode): CompletableFuture<BuiltNode> {
val future: CompletableFuture<BuiltNode> = CompletableFuture()
val localDockerClient = DockerUtils.createLocalDockerClient()
val copiedNodeConfig = copiedNode.copiedNodeConfig
val nodeDir = copiedNodeConfig.parentFile
if (!copiedNodeConfig.exists()) {
throw IllegalStateException("There is no nodeConfig for dir: $copiedNodeConfig")
}
val nodeConfig = ConfigFactory.parseFile(copiedNodeConfig)
LOG.info("starting to build docker image for: $nodeDir")
localDockerClient.buildImageCmd()
.withDockerfile(File(nodeDir, "Dockerfile"))
.withBaseDirectory(nodeDir).exec(object : ResultCallbackTemplate<BuildImageResultCallback, BuildResponseItem>() {
var result: BuildResponseItem? = null
override fun onNext(`object`: BuildResponseItem?) {
this.result = `object`
}
override fun onError(throwable: Throwable?) {
future.completeExceptionally(throwable)
}
override fun onComplete() {
super.onComplete()
LOG.info("finished building docker image for: $nodeDir with id: ${result?.imageId}")
if (result == null || (result?.id == null && result?.errorDetail != null)) {
future.completeExceptionally(IllegalStateException("Could not build image for: $nodeDir, reason: ${result?.errorDetail}"))
}
val config = nodeConfig.parseAsNodeConfigWithFallback(ConfigFactory.parseFile(copiedNode.configFile)).value()
future.complete(copiedNode.builtNode(config, result?.imageId!!))
}
})
return future
}
}
fun Config.parseAsNodeConfigWithFallback(preCopyConfig: Config): Validated<NodeConfiguration, Configuration.Validation.Error> {
val nodeConfig = this
.withValue("baseDirectory", ConfigValueFactory.fromAnyRef(""))
.withFallback(ConfigFactory.parseResources("reference.conf"))
.withFallback(preCopyConfig)
.resolve()
return nodeConfig.parseAsNodeConfiguration()
}

24
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/nodes/NodeCopier.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/nodes/NodeCopier.kt
View File

@ -1,29 +1,32 @@
package net.corda.bootstrapper.nodes
package net.corda.networkbuilder.nodes
import com.typesafe.config.ConfigFactory
import com.typesafe.config.ConfigRenderOptions
import com.typesafe.config.ConfigValue
import net.corda.core.internal.div
import org.slf4j.LoggerFactory
import java.io.File
import java.nio.file.Files
import java.nio.file.Path
open class NodeCopier(private val cacheDir: File) {
fun copyNode(foundNode: FoundNode): CopiedNode {
val nodeCacheDir = File(cacheDir, foundNode.baseDirectory.name)
nodeCacheDir.deleteRecursively()
LOG.info("copying: ${foundNode.baseDirectory} to $nodeCacheDir")
foundNode.baseDirectory.copyRecursively(nodeCacheDir, overwrite = true)
//docker-java lib doesn't copy an empty folder, so if it's empty add a dummy file
ensureDirectoryIsNonEmpty(nodeCacheDir.toPath()/("cordapps"))
copyBootstrapperFiles(nodeCacheDir)
val configInCacheDir = File(nodeCacheDir, "node.conf")
LOG.info("Applying precanned config " + configInCacheDir)
LOG.info("Applying precanned config $configInCacheDir")
val rpcSettings = getDefaultRpcSettings()
val sshSettings = getDefaultSshSettings();
val sshSettings = getDefaultSshSettings()
mergeConfigs(configInCacheDir, rpcSettings, sshSettings)
return CopiedNode(foundNode, configInCacheDir, nodeCacheDir)
}
fun copyBootstrapperFiles(nodeCacheDir: File) {
this.javaClass.classLoader.getResourceAsStream("node-Dockerfile").use { nodeDockerFileInStream ->
val nodeDockerFile = File(nodeCacheDir, "Dockerfile")
@ -88,14 +91,21 @@ open class NodeCopier(private val cacheDir: File) {
}
}
/** Adds a dummy file if the directory is empty. Creates a directory if it doesn't exists. */
internal fun ensureDirectoryIsNonEmpty(path: Path, dummyFileName: String = "dummy.txt") {
if (!Files.exists(path)) {
Files.createDirectories(path)
}
if (Files.list(path).noneMatch { !Files.isDirectory(it) }) {
Files.createFile(path / dummyFileName)
}
}
internal enum class Mode {
NOTARY, NODE
}
companion object {
val LOG = LoggerFactory.getLogger(NodeCopier::class.java)
}
}

38
tools/network-builder/src/main/kotlin/net/corda/networkbuilder/nodes/NodeFinder.kt Normal file
View File

@ -0,0 +1,38 @@
package net.corda.networkbuilder.nodes
import com.typesafe.config.Config
import com.typesafe.config.ConfigException
import com.typesafe.config.ConfigFactory
import net.corda.networkbuilder.Constants
import net.corda.core.utilities.contextLogger
import java.io.File
class NodeFinder(private val dirToSearch: File) {
fun findNodes(): List<FoundNode> = findNodes { config -> !isNotary(config) }
fun findNotaries(): List<FoundNode> = findNodes { config -> isNotary(config) }
private fun isNotary(config: Config) = config.hasPath("notary")
private fun findNodes(filter: (Config) -> Boolean): List<FoundNode> {
return dirToSearch.walkBottomUp().filter {
it.name == "node.conf" && !it.absolutePath.contains(Constants.BOOTSTRAPPER_DIR_NAME)
}.mapNotNull {
try {
ConfigFactory.parseFile(it) to it
} catch (e: ConfigException) {
null
}
}.filter {
filter(it.first)
}.map { (_, nodeConfigFile) ->
LOG.info("We've found a node with name: ${nodeConfigFile.parentFile.name}")
FoundNode(nodeConfigFile, nodeConfigFile.parentFile)
}.toList()
}
companion object {
val LOG = contextLogger()
}
}

6
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/nodes/NodeInstance.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/nodes/NodeInstance.kt
View File

@ -1,4 +1,4 @@
package net.corda.bootstrapper.nodes
package net.corda.networkbuilder.nodes
import net.corda.node.services.config.NodeConfiguration
import java.io.File
@ -26,6 +26,4 @@ class NodeInstance(configFile: File,
nodeInstanceName,
actualX500,
expectedFqName
) {
}
)

7
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/nodes/NodeInstanceRequest.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/nodes/NodeInstanceRequest.kt
View File

@ -1,4 +1,4 @@
package net.corda.bootstrapper.nodes
package net.corda.networkbuilder.nodes
import net.corda.node.services.config.NodeConfiguration
import java.io.File
@ -17,8 +17,7 @@ open class NodeInstanceRequest(configFile: File, baseDirectory: File,
}
fun toNodeInstance(reachableAddress: String, portMapping: Map<Int, Int>): NodeInstance {
return NodeInstance(configFile, baseDirectory, copiedNodeConfig, copiedNodeDir, nodeConfig, localImageId, remoteImageName, nodeInstanceName, actualX500, expectedFqName, reachableAddress, portMapping)
return NodeInstance(configFile, baseDirectory, copiedNodeConfig, copiedNodeDir, nodeConfig,
localImageId, remoteImageName, nodeInstanceName, actualX500, expectedFqName, reachableAddress, portMapping)
}
}

14
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/nodes/NodeInstantiator.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/nodes/NodeInstantiator.kt
View File

@ -1,16 +1,15 @@
package net.corda.bootstrapper.nodes
package net.corda.networkbuilder.nodes
import net.corda.bootstrapper.Constants
import net.corda.bootstrapper.containers.instance.InstanceInfo
import net.corda.bootstrapper.containers.instance.Instantiator
import net.corda.bootstrapper.context.Context
import net.corda.networkbuilder.Constants
import net.corda.networkbuilder.containers.instance.InstanceInfo
import net.corda.networkbuilder.containers.instance.Instantiator
import net.corda.networkbuilder.context.Context
import net.corda.core.identity.CordaX500Name
import java.util.concurrent.CompletableFuture
class NodeInstantiator(val instantiator: Instantiator,
val context: Context) {
fun createInstanceRequests(pushedNode: PushedNode, nodeCount: Map<FoundNode, Int>): List<NodeInstanceRequest> {
val namedMap = nodeCount.map { it.key.name.toLowerCase() to it.value }.toMap()
@ -30,7 +29,6 @@ class NodeInstantiator(val instantiator: Instantiator,
return createInstanceRequest(node, 0)
}
private fun buildX500(baseX500: CordaX500Name, i: Int): String {
if (i == 0) {
return baseX500.toString()
@ -87,6 +85,4 @@ class NodeInstantiator(val instantiator: Instantiator,
fun expectedFqdn(newInstanceName: String): String {
return instantiator.getExpectedFQDN(newInstanceName)
}
}

7
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/nodes/NodePusher.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/nodes/NodePusher.kt
View File

@ -1,13 +1,12 @@
package net.corda.bootstrapper.nodes
package net.corda.networkbuilder.nodes
import net.corda.bootstrapper.containers.push.ContainerPusher
import net.corda.bootstrapper.context.Context
import net.corda.networkbuilder.containers.push.ContainerPusher
import net.corda.networkbuilder.context.Context
import java.util.concurrent.CompletableFuture
class NodePusher(private val containerPusher: ContainerPusher,
private val context: Context) {
fun pushNode(builtNode: BuiltNode): CompletableFuture<PushedNode> {
val localImageId = builtNode.localImageId

4
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/nodes/PushedNode.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/nodes/PushedNode.kt
View File

@ -1,4 +1,4 @@
package net.corda.bootstrapper.nodes
package net.corda.networkbuilder.nodes
import net.corda.node.services.config.NodeConfiguration
import java.io.File
@ -20,6 +20,4 @@ open class PushedNode(configFile: File, baseDirectory: File,
override fun toString(): String {
return "PushedNode(remoteImageName='$remoteImageName') ${super.toString()}"
}
}

7
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/notaries/CopiedNotary.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/notaries/CopiedNotary.kt
View File

@ -1,12 +1,11 @@
package net.corda.bootstrapper.notaries
package net.corda.networkbuilder.notaries
import net.corda.bootstrapper.nodes.CopiedNode
import net.corda.networkbuilder.nodes.CopiedNode
import java.io.File
class CopiedNotary(configFile: File, baseDirectory: File,
copiedNodeConfig: File, copiedNodeDir: File, val nodeInfoFile: File) :
CopiedNode(configFile, baseDirectory, copiedNodeConfig, copiedNodeDir) {
}
CopiedNode(configFile, baseDirectory, copiedNodeConfig, copiedNodeDir)
fun CopiedNode.toNotary(nodeInfoFile: File): CopiedNotary {

24
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/notaries/NotaryCopier.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/notaries/NotaryCopier.kt
View File

@ -1,23 +1,26 @@
package net.corda.bootstrapper.notaries
package net.corda.networkbuilder.notaries
import net.corda.bootstrapper.nodes.CopiedNode
import net.corda.bootstrapper.nodes.FoundNode
import net.corda.bootstrapper.nodes.NodeCopier
import net.corda.core.internal.div
import net.corda.networkbuilder.nodes.CopiedNode
import net.corda.networkbuilder.nodes.FoundNode
import net.corda.networkbuilder.nodes.NodeCopier
import org.slf4j.LoggerFactory
import java.io.File
class NotaryCopier(val cacheDir: File) : NodeCopier(cacheDir) {
class NotaryCopier(private val cacheDir: File) : NodeCopier(cacheDir) {
fun copyNotary(foundNotary: FoundNode): CopiedNotary {
val nodeCacheDir = File(cacheDir, foundNotary.baseDirectory.name)
nodeCacheDir.deleteRecursively()
LOG.info("copying: ${foundNotary.baseDirectory} to $nodeCacheDir")
foundNotary.baseDirectory.copyRecursively(nodeCacheDir, overwrite = true)
//docker-java lib doesn't copy an empty folder, so if it's empty add a dummy file
ensureDirectoryIsNonEmpty(nodeCacheDir.toPath()/("cordapps"))
copyNotaryBootstrapperFiles(nodeCacheDir)
val configInCacheDir = File(nodeCacheDir, "node.conf")
LOG.info("Applying precanned config " + configInCacheDir)
LOG.info("Applying precanned config $configInCacheDir")
val rpcSettings = getDefaultRpcSettings()
val sshSettings = getDefaultSshSettings();
val sshSettings = getDefaultSshSettings()
mergeConfigs(configInCacheDir, rpcSettings, sshSettings, Mode.NOTARY)
val generatedNodeInfo = generateNodeInfo(nodeCacheDir)
return CopiedNode(foundNotary, configInCacheDir, nodeCacheDir).toNotary(generatedNodeInfo)
@ -32,10 +35,9 @@ class NotaryCopier(val cacheDir: File) : NodeCopier(cacheDir) {
val exitCode = nodeInfoGeneratorProcess.waitFor()
if (exitCode != 0) {
throw IllegalStateException("Failed to generate nodeInfo for notary: " + dirToGenerateFrom)
throw IllegalStateException("Failed to generate nodeInfo for notary: $dirToGenerateFrom")
}
val nodeInfoFile = dirToGenerateFrom.listFiles().filter { it.name.startsWith("nodeInfo") }.single()
return nodeInfoFile;
return dirToGenerateFrom.listFiles().single { it.name.startsWith("nodeInfo") }
}
private fun copyNotaryBootstrapperFiles(nodeCacheDir: File) {
@ -64,6 +66,4 @@ class NotaryCopier(val cacheDir: File) : NodeCopier(cacheDir) {
companion object {
val LOG = LoggerFactory.getLogger(NotaryCopier::class.java)
}
}

3
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/serialization/SerializationHelper.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/serialization/SerializationHelper.kt
View File

@ -1,4 +1,4 @@
package net.corda.bootstrapper.serialization
package net.corda.networkbuilder.serialization
import net.corda.core.serialization.internal.SerializationEnvironment
import net.corda.core.serialization.internal.nodeSerializationEnv
@ -22,7 +22,6 @@ class SerializationEngine {
p2pContext = AMQP_P2P_CONTEXT.withClassLoader(classloader),
rpcServerContext = AMQP_P2P_CONTEXT.withClassLoader(classloader),
storageContext = AMQP_STORAGE_CONTEXT.withClassLoader(classloader),
checkpointContext = KRYO_CHECKPOINT_CONTEXT.withClassLoader(classloader),
checkpointSerializer = KryoCheckpointSerializer
)

10
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/volumes/Volume.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/volumes/Volume.kt
View File

@ -1,9 +1,9 @@
package net.corda.bootstrapper.volumes
package net.corda.networkbuilder.volumes
import com.microsoft.azure.storage.file.CloudFile
import com.typesafe.config.ConfigFactory
import net.corda.bootstrapper.notaries.CopiedNotary
import net.corda.bootstrapper.serialization.SerializationEngine
import net.corda.networkbuilder.notaries.CopiedNotary
import net.corda.networkbuilder.serialization.SerializationEngine
import net.corda.core.node.NetworkParameters
import net.corda.core.node.NotaryInfo
import net.corda.core.serialization.deserialize
@ -26,7 +26,6 @@ interface Volume {
internal val networkMapCa = createDevNetworkMapCa(DEV_ROOT_CA)
internal val networkMapCert: X509Certificate = networkMapCa.certificate
internal val keyPair = networkMapCa.keyPair
}
@ -34,7 +33,6 @@ interface Volume {
this.uploadFromByteArray(array, 0, array.size)
}
fun convertNodeIntoToNetworkParams(notaryFiles: List<Pair<File, File>>): NetworkParameters {
val notaryInfos = notaryFiles.map { (configFile, nodeInfoFile) ->
val validating = ConfigFactory.parseFile(configFile).getConfig("notary").getBooleanCaseInsensitive("validating")
@ -52,6 +50,4 @@ interface Volume {
whitelistedContractImplementations = emptyMap())
}
}
}

21
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/volumes/azure/AzureSmbVolume.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/volumes/azure/AzureSmbVolume.kt
View File

@ -1,14 +1,14 @@
package net.corda.bootstrapper.volumes.azure
package net.corda.networkbuilder.volumes.azure
import com.microsoft.azure.management.Azure
import com.microsoft.azure.management.resources.ResourceGroup
import com.microsoft.azure.management.storage.StorageAccount
import com.microsoft.azure.storage.CloudStorageAccount
import net.corda.bootstrapper.Constants.Companion.restFriendlyName
import net.corda.bootstrapper.notaries.CopiedNotary
import net.corda.bootstrapper.volumes.Volume
import net.corda.bootstrapper.volumes.Volume.Companion.keyPair
import net.corda.bootstrapper.volumes.Volume.Companion.networkMapCert
import net.corda.networkbuilder.Constants.Companion.restFriendlyName
import net.corda.networkbuilder.notaries.CopiedNotary
import net.corda.networkbuilder.volumes.Volume
import net.corda.networkbuilder.volumes.Volume.Companion.keyPair
import net.corda.networkbuilder.volumes.Volume.Companion.networkMapCert
import net.corda.core.internal.signWithCert
import net.corda.core.serialization.serialize
import net.corda.nodeapi.internal.network.NETWORK_PARAMS_FILE_NAME
@ -38,7 +38,6 @@ class AzureSmbVolume(private val azure: Azure, private val resourceGroup: Resour
val storageAccountKey: String
get() = accKeys.value()
init {
while (true) {
try {
@ -64,18 +63,12 @@ class AzureSmbVolume(private val azure: Azure, private val resourceGroup: Resour
override fun notariesForNetworkParams(notaries: List<CopiedNotary>) {
val networkParamsFile = networkParamsFolder.getFileReference(NETWORK_PARAMS_FILE_NAME)
networkParamsFile.deleteIfExists()
LOG.info("Storing network-params in AzureFile location: " + networkParamsFile.uri)
LOG.info("Storing network-params in AzureFile location: ${networkParamsFile.uri}")
val networkParameters = convertNodeIntoToNetworkParams(notaries.map { it.configFile to it.nodeInfoFile })
networkParamsFile.uploadFromByteArray(networkParameters.signWithCert(keyPair.private, networkMapCert).serialize().bytes)
}
companion object {
val LOG = LoggerFactory.getLogger(AzureSmbVolume::class.java)
}
}

9
tools/network-bootstrapper/src/main/kotlin/net/corda/bootstrapper/volumes/docker/LocalVolume.kt → tools/network-builder/src/main/kotlin/net/corda/networkbuilder/volumes/docker/LocalVolume.kt
View File

@ -1,8 +1,8 @@
package net.corda.bootstrapper.volumes.docker
package net.corda.networkbuilder.volumes.docker
import net.corda.bootstrapper.context.Context
import net.corda.bootstrapper.notaries.CopiedNotary
import net.corda.bootstrapper.volumes.Volume
import net.corda.networkbuilder.context.Context
import net.corda.networkbuilder.notaries.CopiedNotary
import net.corda.networkbuilder.volumes.Volume
import net.corda.core.internal.signWithCert
import net.corda.core.serialization.serialize
import net.corda.nodeapi.internal.network.NETWORK_PARAMS_FILE_NAME
@ -24,7 +24,6 @@ class LocalVolume(scratchDir: File, context: Context) : Volume {
LOG.info("wrote network params to local file: ${networkParamsFile.absolutePath}")
}
fun getPath(): String {
return volumeDir.absolutePath
}

21
tools/network-builder/src/main/resources/node-Dockerfile Normal file
View File

@ -0,0 +1,21 @@
FROM corda/corda-zulu-4.0
# Copy corda files
ADD --chown=corda:corda node.conf /opt/corda/node.conf
ADD --chown=corda:corda cordapps/ /opt/corda/cordapps
# Copy node info watcher script
ADD --chown=corda:corda node_info_watcher.sh /opt/corda/
COPY --chown=corda:corda run-corda.sh /opt/corda/bin/run-corda
RUN chmod +x /opt/corda/bin/run-corda && \
chmod +x /opt/corda/node_info_watcher.sh && \
sync
# Working directory for Corda
WORKDIR /opt/corda
ENV HOME=/opt/corda
# Start it
CMD ["run-corda"]

0
tools/network-bootstrapper/src/main/resources/node_info_watcher.sh → tools/network-builder/src/main/resources/node_info_watcher.sh Executable file → Normal file
View File

0
tools/network-bootstrapper/src/main/resources/node_killer.sh → tools/network-builder/src/main/resources/node_killer.sh Executable file → Normal file
View File

22
tools/network-builder/src/main/resources/notary-Dockerfile Normal file
View File

@ -0,0 +1,22 @@
FROM corda/corda-zulu-4.0
# Copy corda files
ADD --chown=corda:corda node.conf /opt/corda/node.conf
ADD --chown=corda:corda cordapps/ /opt/corda/cordapps
ADD --chown=corda:corda certificates/ /opt/corda/certificates
# Copy node info watcher script
ADD --chown=corda:corda node_info_watcher.sh /opt/corda/
COPY --chown=corda:corda run-corda.sh /opt/corda/bin/run-corda
RUN chmod +x /opt/corda/bin/run-corda && \
chmod +x /opt/corda/node_info_watcher.sh && \
sync
# Working directory for Corda
WORKDIR /opt/corda
ENV HOME=/opt/corda
# Start it
CMD ["run-corda"]

0
tools/network-bootstrapper/src/main/resources/rpc-settings.conf → tools/network-builder/src/main/resources/rpc-settings.conf
View File

2
tools/network-bootstrapper/src/main/resources/run-corda-node.sh → tools/network-builder/src/main/resources/run-corda-node.sh Executable file → Normal file
View File

@ -18,7 +18,7 @@ bash node_info_watcher.sh &
cd ${CORDA_HOME}
if java ${JAVA_OPTIONS} -jar ${CORDA_HOME}/corda.jar 2>&1 ; then
if java ${JAVA_OPTIONS} -jar ${CORDA_HOME}/bin/corda.jar 2>&1 ; then
echo "Corda exited with zero exit code"
else
echo "Corda exited with nonzero exit code, sleeping to allow log collection"

2
tools/network-bootstrapper/src/main/resources/run-corda-notary.sh → tools/network-builder/src/main/resources/run-corda-notary.sh Executable file → Normal file
View File

@ -14,7 +14,7 @@ bash node_info_watcher.sh &
cd ${CORDA_HOME}
if java ${JAVA_OPTIONS} -jar ${CORDA_HOME}/corda.jar 2>&1 ; then
if java ${JAVA_OPTIONS} -jar ${CORDA_HOME}/bin/corda.jar 2>&1 ; then
echo "Corda exited with zero exit code"
else
echo "Corda exited with nonzero exit code, sleeping to allow log collection"

0
tools/network-bootstrapper/src/main/resources/ssh.conf → tools/network-builder/src/main/resources/ssh.conf
View File

0
tools/network-bootstrapper/src/main/resources/views/bootstrapper.css → tools/network-builder/src/main/resources/views/bootstrapper.css vendored
View File

0
tools/network-bootstrapper/src/main/resources/views/cordalogo.png → tools/network-builder/src/main/resources/views/cordalogo.png
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 32 KiB

After

Width:  |  Height:  |  Size: 32 KiB

0
tools/network-bootstrapper/src/main/resources/views/mainPane.fxml → tools/network-builder/src/main/resources/views/mainPane.fxml
View File