diff --git a/docs/source/api-flows.rst b/docs/source/api-flows.rst index 780c2572da..e3da7f5ad0 100644 --- a/docs/source/api-flows.rst +++ b/docs/source/api-flows.rst @@ -744,3 +744,35 @@ We then update the progress tracker's current step as we progress through the fl :start-after: DOCSTART 18 :end-before: DOCEND 18 :dedent: 12 + + +Concurrency, Locking and Waiting +-------------------------------- +This is an advanced topic. Because Corda is designed to: + +* run many flows in parallel, +* may persist flows to storage and resurrect those flows much later, +* (in the future) migrate flows between JVMs, + +flows should avoid use of locks and typically not even attempt to interact with objects shared between flows (except +``ServiceHub`` and other carefully crafted services such as Oracles. See :doc:`oracles`). +Locks will significantly reduce the scalability of the node, in the best case, and can cause the node to deadlock if they +remain locked across flow context switch boundaries (such as sending and receiving +from peers discussed above, and the sleep discussed below). + +If you need activities that are scheduled, you should investigate the use of ``SchedulableState``. +However, we appreciate that Corda support for some more advanced patterns is still in the future, and if there is a need +for brief pauses in flows then you should use ``FlowLogic.sleep`` in place of where you might have used ``Thread.sleep``. +Flows should expressly not use ``Thread.sleep``, since this will prevent the node from processing other flows +in the meantime, significantly impairing the performance of the node. +Even ``FlowLogic.sleep`` is not to be used to create long running flows, since the Corda ethos is for short lived flows +(otherwise upgrading nodes or CorDapps is much more complicated), or as a substitute to using the ``SchedulableState`` scheduler. + +Currently the ``finance`` package uses ``FlowLogic.sleep`` to make several attempts at coin selection, where necessary, +when many states are soft locked and we wish to wait for those, or other new states in their place, to become unlocked. + + .. literalinclude:: ../../finance/src/main/kotlin/net/corda/finance/contracts/asset/cash/selection/AbstractCashSelection.kt + :language: kotlin + :start-after: DOCSTART CASHSELECT 1 + :end-before: DOCEND CASHSELECT 1 + :dedent: 8 diff --git a/docs/source/corda-configuration-file.rst b/docs/source/corda-configuration-file.rst index 3c41f75d5b..9f954d3bb5 100644 --- a/docs/source/corda-configuration-file.rst +++ b/docs/source/corda-configuration-file.rst @@ -1,64 +1,47 @@ Node configuration ================== +.. contents:: + File location ------------- +When starting a node, the ``corda.jar`` file defaults to reading the node's configuration from a ``node.conf`` file in +the directory from which the command to launch Corda is executed. There are two command-line options to override this +behaviour: -The Corda all-in-one ``corda.jar`` file is generated by the ``gradle buildCordaJAR`` task and defaults to reading configuration -from a ``node.conf`` file in the present working directory. This behaviour can be overidden using the ``--config-file`` -command line option to target configuration files with different names, or different file location (relative paths are -relative to the current working directory). Also, the ``--base-directory`` command line option alters the Corda node -workspace location and if specified a ``node.conf`` configuration file is then expected in the root of the workspace. +* The ``--config-file`` command line option allows you to specify a configuration file with a different name, or at + different file location. Paths are relative to the current working directory -The configuration file templates used for the ``gradle deployNodes`` task are to be found in the ``/config/dev`` folder. -Also note that there is a basic set of defaults loaded from the built in resource file ``/node/src/main/resources/reference.conf`` -of the ``:node`` gradle module. All properties in this can be overidden in the file configuration and for rarely changed -properties this defaulting allows the property to be excluded from the configuration file. +* The ``--base-directory`` command line option allows you to specify the node's workspace location. A ``node.conf`` + configuration file is then expected in the root of this workspace + +If you specify both command line arguments at the same time, the node will fail to start. Format ------ +The Corda configuration file uses the HOCON format which is superset of JSON. Please visit +``_ for further details. -The Corda configuration file uses the HOCON format which is superset of JSON. It has several features which makes it -very useful as a configuration format. Please visit their `page `_ -for further details. - -Examples +Defaults -------- +A set of default configuration options are loaded from the built-in resource file ``/node/src/main/resources/reference.conf``. +This file can be found in the ``:node`` gradle module of the `Corda repository `_. Any +options you do not specify in your own ``node.conf`` file will use these defaults. -General node configuration file for hosting the IRSDemo services. +Here are the contents of the ``reference.conf`` file: -.. literalinclude:: example-code/src/main/resources/example-node.conf +.. literalinclude:: ../../node/src/main/resources/reference.conf :language: javascript -Simple Notary configuration file. - -.. parsed-literal:: - - myLegalName : "O=Notary Service,OU=corda,L=London,C=GB" - keyStorePassword : "cordacadevpass" - trustStorePassword : "trustpass" - p2pAddress : "localhost:12345" - rpcSettings = { - useSsl = false - standAloneBroker = false - address : "my-corda-node:10003" - adminAddress : "my-corda-node:10004" - } - webAddress : "localhost:12347" - notary : { - validating : false - } - devMode : true - compatibilityZoneURL : "https://cz.corda.net" - Fields ------ +The available config fields are listed below. ``baseDirectory`` is available as a substitution value and contains the +absolute path to the node's base directory. -The available config fields are listed below. ``baseDirectory`` is available as a substitution value, containing the absolute -path to the node's base directory. - -:myLegalName: The legal identity of the node acts as a human readable alias to the node's public key and several demos use - this to lookup the NodeInfo. +:myLegalName: The legal identity of the node. This acts as a human-readable alias to the node's public key and can be used with + the network map to look up the node's info. This is the name that is used in the node's certificates (either when requesting them + from the doorman, or when auto-generating them in dev mode). At runtime, Corda checks whether this name matches the + name in the node's certificates. :keyStorePassword: The password to unlock the KeyStore file (``/certificates/sslkeystore.jks``) containing the node certificate and private key. @@ -195,4 +178,33 @@ path to the node's base directory. Otherwise defaults to 10MB :attachmentCacheBound: Optionally specify how many attachments should be cached locally. Note that this includes only the key and - metadata, the content is cached separately and can be loaded lazily. Defaults to 1024. \ No newline at end of file + metadata, the content is cached separately and can be loaded lazily. Defaults to 1024. + +Examples +-------- + +General node configuration file for hosting the IRSDemo services: + +.. literalinclude:: example-code/src/main/resources/example-node.conf + :language: javascript + +Simple notary configuration file: + +.. parsed-literal:: + + myLegalName : "O=Notary Service,OU=corda,L=London,C=GB" + keyStorePassword : "cordacadevpass" + trustStorePassword : "trustpass" + p2pAddress : "localhost:12345" + rpcSettings = { + useSsl = false + standAloneBroker = false + address : "my-corda-node:10003" + adminAddress : "my-corda-node:10004" + } + webAddress : "localhost:12347" + notary : { + validating : false + } + devMode : true + compatibilityZoneURL : "https://cz.corda.net" \ No newline at end of file diff --git a/docs/source/node-database.rst b/docs/source/node-database.rst index e38cde81c9..c429abc082 100644 --- a/docs/source/node-database.rst +++ b/docs/source/node-database.rst @@ -1,16 +1,14 @@ Node database ============= -Currently, nodes store their data in an H2 database. In the future, we plan to support a wide range of databases. - -You can connect directly to a running node's database to see its stored states, transactions and attachments as -follows: +Default in-memory database +-------------------------- +By default, nodes store their data in an H2 database. You can connect directly to a running node's database to see its +stored states, transactions and attachments as follows: * Download the `h2 platform-independent zip `_, unzip the zip, and navigate in a terminal window to the unzipped folder -* Change directories to the bin folder: - - ``cd h2/bin`` +* Change directories to the bin folder: ``cd h2/bin`` * Run the following command to open the h2 web console in a web browser tab: @@ -25,4 +23,34 @@ follows: * Paste this string into the JDBC URL field and click ``Connect``, using the default username and password. You will be presented with a web interface that shows the contents of your node's storage and vault, and provides an -interface for you to query them using SQL. \ No newline at end of file +interface for you to query them using SQL. + +PostgreSQL +---------- +Nodes also have untested support for PostgreSQL 9.6, using PostgreSQL JDBC Driver 42.1.4. + +.. warning:: This is an experimental community contribution, and is currently untested. We welcome pull requests to add + tests and additional support for this feature. + +Configuration +~~~~~~~~~~~~~ +Here is an example node configuration for PostgreSQL: + +.. sourcecode:: groovy + + dataSourceProperties = { + dataSourceClassName = "org.postgresql.ds.PGSimpleDataSource" + dataSource.url = "jdbc:postgresql://[HOST]:[PORT]/postgres" + dataSource.user = [USER] + dataSource.password = [PASSWORD] + } + database = { + transactionIsolationLevel = READ_COMMITTED + schema = [SCHEMA] + } + +Note that: + +* The ``database.schema`` property is optional +* The value of ``database.schema`` is not wrapped in double quotes and Postgres always treats it as a lower-case value + (e.g. ``AliceCorp`` becomes ``alicecorp``) \ No newline at end of file diff --git a/docs/source/quickstart-index.rst b/docs/source/quickstart-index.rst index fa884b229d..c0b655e7aa 100644 --- a/docs/source/quickstart-index.rst +++ b/docs/source/quickstart-index.rst @@ -1,10 +1,7 @@ Quickstart ========== -.. toctree:: - :maxdepth: 1 - - getting-set-up - tutorial-cordapp - Other CorDapps - Utilities \ No newline at end of file +* :doc:`Set up your machine for CorDapp development ` +* :doc:`Run the Example CorDapp ` +* `View CorDapps in Corda Explore `_ +* `Download sample CorDapps `_ \ No newline at end of file diff --git a/docs/source/shell.rst b/docs/source/shell.rst index 86833f002b..90edacf5ff 100644 --- a/docs/source/shell.rst +++ b/docs/source/shell.rst @@ -184,7 +184,7 @@ Starting a flow We would start the ``CashIssue`` flow as follows: -``flow start CashIssue amount: $1000, issueRef: 1234, recipient: "O=Bank A,L=London,C=GB", notary: "O=Notary Service,OU=corda,L=London,C=GB"`` +``flow start CashIssueFlow amount: $1000, issuerBankPartyRef: 1234, notary: "O=Controller, L=London, C=GB"`` This breaks down as follows: diff --git a/finance/src/main/kotlin/net/corda/finance/contracts/asset/cash/selection/AbstractCashSelection.kt b/finance/src/main/kotlin/net/corda/finance/contracts/asset/cash/selection/AbstractCashSelection.kt index eff93f6d3f..2fc04a2d46 100644 --- a/finance/src/main/kotlin/net/corda/finance/contracts/asset/cash/selection/AbstractCashSelection.kt +++ b/finance/src/main/kotlin/net/corda/finance/contracts/asset/cash/selection/AbstractCashSelection.kt @@ -12,7 +12,10 @@ import net.corda.core.node.ServiceHub import net.corda.core.node.services.StatesNotAvailableException import net.corda.core.utilities.* import net.corda.finance.contracts.asset.Cash -import java.sql.* +import java.sql.Connection +import java.sql.DatabaseMetaData +import java.sql.ResultSet +import java.sql.SQLException import java.util.* import java.util.concurrent.atomic.AtomicReference import java.util.concurrent.locks.ReentrantLock @@ -101,6 +104,7 @@ abstract class AbstractCashSelection { withIssuerRefs: Set = emptySet()): List> { val stateAndRefs = mutableListOf>() + // DOCSTART CASHSELECT 1 for (retryCount in 1..MAX_RETRIES) { if (!attemptSpend(services, amount, lockId, notary, onlyFromIssuerParties, withIssuerRefs, stateAndRefs)) { log.warn("Coin selection failed on attempt $retryCount") @@ -116,6 +120,7 @@ abstract class AbstractCashSelection { break } } + // DOCEND CASHSELECT 1 return stateAndRefs } diff --git a/node-api/src/main/kotlin/net/corda/nodeapi/internal/ClassloaderUtils.kt b/node-api/src/main/kotlin/net/corda/nodeapi/internal/ClassloaderUtils.kt index 774e30e2c0..49739303a1 100644 --- a/node-api/src/main/kotlin/net/corda/nodeapi/internal/ClassloaderUtils.kt +++ b/node-api/src/main/kotlin/net/corda/nodeapi/internal/ClassloaderUtils.kt @@ -26,6 +26,7 @@ fun scanJarForContracts(cordappJarPath: String): List { // Only keep instantiable contracts val classLoader = URLClassLoader(arrayOf(File(cordappJarPath).toURL()), currentClassLoader) val concreteContracts = contracts.map(classLoader::loadClass).filter { !it.isInterface && !Modifier.isAbstract(it.modifiers) } + classLoader.close() return concreteContracts.map { it.name } } diff --git a/node/src/main/kotlin/net/corda/node/internal/ServiesForResolutionImpl.kt b/node/src/main/kotlin/net/corda/node/internal/ServicesForResolutionImpl.kt similarity index 94% rename from node/src/main/kotlin/net/corda/node/internal/ServiesForResolutionImpl.kt rename to node/src/main/kotlin/net/corda/node/internal/ServicesForResolutionImpl.kt index c653a689b0..369bf89a33 100644 --- a/node/src/main/kotlin/net/corda/node/internal/ServiesForResolutionImpl.kt +++ b/node/src/main/kotlin/net/corda/node/internal/ServicesForResolutionImpl.kt @@ -23,10 +23,10 @@ data class ServicesForResolutionImpl( @Throws(TransactionResolutionException::class) override fun loadStates(stateRefs: Set): Set> { - return stateRefs.groupBy { it.txhash }.map { + return stateRefs.groupBy { it.txhash }.flatMap { val stx = validatedTransactions.getTransaction(it.key) ?: throw TransactionResolutionException(it.key) val baseTx = stx.resolveBaseTransaction(this) it.value.map { StateAndRef(baseTx.outputs[it.index], it) } - }.flatMap { it }.toSet() + }.toSet() } -} \ No newline at end of file +} diff --git a/node/src/main/kotlin/net/corda/node/services/transactions/BFTSMaRt.kt b/node/src/main/kotlin/net/corda/node/services/transactions/BFTSMaRt.kt index 899ecde450..aa3027559b 100644 --- a/node/src/main/kotlin/net/corda/node/services/transactions/BFTSMaRt.kt +++ b/node/src/main/kotlin/net/corda/node/services/transactions/BFTSMaRt.kt @@ -248,7 +248,7 @@ object BFTSMaRt { /** Generates a transaction signature over the specified transaction [txId]. */ protected fun sign(txId: SecureHash): TransactionSignature { val signableData = SignableData(txId, SignatureMetadata(services.myInfo.platformVersion, Crypto.findSignatureScheme(notaryIdentityKey).schemeNumberID)) - return services.keyManagementService.sign(signableData, notaryIdentityKey) + return services.database.transaction { services.keyManagementService.sign(signableData, notaryIdentityKey) } } // TODO: diff --git a/node/src/test/kotlin/net/corda/node/ArgsParserTest.kt b/node/src/test/kotlin/net/corda/node/ArgsParserTest.kt index 8ddb9707e9..bb664bb4a7 100644 --- a/node/src/test/kotlin/net/corda/node/ArgsParserTest.kt +++ b/node/src/test/kotlin/net/corda/node/ArgsParserTest.kt @@ -5,15 +5,29 @@ import net.corda.core.internal.div import net.corda.nodeapi.internal.crypto.X509KeyStore import org.assertj.core.api.Assertions.assertThat import org.assertj.core.api.Assertions.assertThatExceptionOfType +import org.junit.BeforeClass import org.junit.Test import org.slf4j.event.Level +import java.nio.file.Files +import java.nio.file.Path import java.nio.file.Paths import kotlin.test.assertEquals import kotlin.test.assertNotNull class ArgsParserTest { private val parser = ArgsParser() - private val workingDirectory = Paths.get(".").normalize().toAbsolutePath() + + companion object { + private lateinit var workingDirectory: Path + private lateinit var buildDirectory: Path + + @BeforeClass + @JvmStatic + fun initDirectories() { + workingDirectory = Paths.get(".").normalize().toAbsolutePath() + buildDirectory = workingDirectory.resolve("build") + } + } @Test fun `no command line arguments`() { @@ -113,17 +127,21 @@ class ArgsParserTest { @Test fun `initial-registration`() { - val truststorePath = workingDirectory / "truststore" / "file.jks" + // Create this temporary file in the "build" directory so that "clean" can delete it. + val truststorePath = buildDirectory / "truststore" / "file.jks" assertThatExceptionOfType(IllegalArgumentException::class.java).isThrownBy { parser.parse("--initial-registration", "--network-root-truststore", "$truststorePath", "--network-root-truststore-password", "password-test") }.withMessageContaining("Network root trust store path").withMessageContaining("doesn't exist") X509KeyStore.fromFile(truststorePath, "dummy_password", createNew = true) - - val cmdLineOptions = parser.parse("--initial-registration", "--network-root-truststore", "$truststorePath", "--network-root-truststore-password", "password-test") - assertNotNull(cmdLineOptions.nodeRegistrationConfig) - assertEquals(truststorePath.toAbsolutePath(), cmdLineOptions.nodeRegistrationConfig?.networkRootTrustStorePath) - assertEquals("password-test", cmdLineOptions.nodeRegistrationConfig?.networkRootTrustStorePassword) + try { + val cmdLineOptions = parser.parse("--initial-registration", "--network-root-truststore", "$truststorePath", "--network-root-truststore-password", "password-test") + assertNotNull(cmdLineOptions.nodeRegistrationConfig) + assertEquals(truststorePath.toAbsolutePath(), cmdLineOptions.nodeRegistrationConfig?.networkRootTrustStorePath) + assertEquals("password-test", cmdLineOptions.nodeRegistrationConfig?.networkRootTrustStorePassword) + } finally { + Files.delete(truststorePath) + } } @Test diff --git a/samples/irs-demo/src/system-test/kotlin/IRSDemoDockerTest.kt b/samples/irs-demo/src/system-test/kotlin/IRSDemoDockerTest.kt index b40e829c9f..cce25fa4f3 100644 --- a/samples/irs-demo/src/system-test/kotlin/IRSDemoDockerTest.kt +++ b/samples/irs-demo/src/system-test/kotlin/IRSDemoDockerTest.kt @@ -72,15 +72,9 @@ class IRSDemoDockerTest { //Wait for deals to appear in a rows table val dealsList = driverWait.until({ - makeScreenshot(driver, "second") it?.findElement(By.cssSelector("table#deal-list tbody tr")) }) assertNotNull(dealsList) } - - private fun makeScreenshot(driver: PhantomJSDriver, name: String) { - val screenshotAs = driver.getScreenshotAs(OutputType.FILE) - Files.copy(screenshotAs.toPath(), Paths.get("/Users", "maksymilianpawlak", "phantomjs", name + System.currentTimeMillis() + ".png"), StandardCopyOption.REPLACE_EXISTING) - } }