* migrate PersistentIdentityService to use key.toShortString()
update definition of PublicKeyToExternalId mapping to allow fast lookup by externalId/publicKey
* fix misspelled table name
* add test of migration script
* add design document for proposal to move IdentityService to using the correct PK.toStringShort() method for hashing a publickey
* add enterprise testing considerations to design
* address review comments
* fix compilation errors
* modify PublicKeyToOwningIdentityCache to use toStringShort() as it's lookup key
* address syzmon's code review comments
* CORDA-2617: Add failover listeners to terminate node process
This is a backport of changes done in Corda Enterprise.
It will be triggered in case of:
a) Loss of connectivity to in-built Artemis for Bridge Control;
b) Loss of connectivity to in-built Artemis for P2P connectivity.
Note on merge to CE: Disregard these changes and take whatever CE already has.
* CORDA-2617: Update documentation on stability of Corda Node
* CORDA-2617: Documentation update after discussion with @mnesbit
* CORDA-2216 Restrict extended key usage of certificate types
- Remove `anyExtendedKeyUsage` from all certificates
- Restrict to server and client auth for all certificates except
`ROOT_CA` which has had all extended key usages removed
* CORDA-2216 Update certificate types
- Remove server authentication from all certificates except for TLS
- Add client authentication to all certificates except for ROOT
* Fix test execution flakiness on fast hardware: set node info polling interval to 1 second in DriverDSL Node Startup (was only being set to 1 sec in Node Registration). Follow-up to https://github.com/corda/corda/pull/5240
* Diagnostics: add additional logging to NodeInfoFilersCopier.
* Diagnostics: add additional logging to NodeInfoWatcher
* Downgrade logging level severity to debug.
* Make atomic counter unique across instances.
* CORDA-3021: Introduce `SignOnlyCryptoService` and use it whenever possible
Also modify `CryptoServiceFactory` to show how sign only implementation can be created.
* CORDA-3021: Undo some of my earlier changes
Which after discussion with @dimosr and @fowlerrr proven to be contradictory.
* ENT-3642: move the crypto service builder method to node-api
* ENT-3642: add arg for different crypto services
* ENT-3642: add arg for cryptoservice config
* ENT-3642: add null check for BCC requirement
* ENT-3642: remove double-bang operator
* ENT-3642: cryptoservice factory method moved to its own class, improve API
* ENT-3642: remove import
* CORDA-2882: Added ability to specify signature scheme when signing.
* CORDA-2882: Sign operation with algo specified does not now use Crypto service.
* CORDA-2882: Added jvmoverloads for sign operation.
* CORDA-2882: Removed unused imports.
* CORDA-2651 Check if resources are in classpath before passing them to Liquibase
* CORDA-2651 Add missing stop
* CORDA-2651 Change exception type. Improve exception log message.
* CORDA-2651 Add null check when getting resources from class loader
* CORDA-2651 Do not include null values in the changelog list
The test is currently disabled till we move to Java 11 (or beyond) when TLS 1.3 becomes available as part of JDK.
Local testing been performed with Open JDK 12 (12+33) and the test is passing.
* Plumb through the crlCheckSoftFail configuration option to bridge manager
* Add crlCheckSoftFail test to bridge manager and fix equivalent proton wrapper test
* Update documentation and set the node configuration default to true
* Revert default change and clarify consequences of setting option to false
* Remove NodeConfiguration default to leave only AMQPConfiguration default
* NetworkBoostrapper can optionally whitelist contracts from signed jars based on include_whitelist.txt file.
* refactoring, docs
* logs
* add ne parameters to the generateWhitelist method at the end
* Addressing review comments.
* CORDA-2577 disable non-downgrade rule - test fix and docs
* ENT-3165 Kotlin toList() does not work on concurrent collections. OS backport.
ENT-3165 Added comment.
* ENT-3187 Additional use of toList() on concurrent data structure.
* CORDA-2669 - pendingFlowsCount not in public API
Reintroduce `pendingFlowsCount` to public API (as deprecated). Advise
to use the `gracefulShutdown` command in the shell instead.
* CORDA-2669 - Add pendingFlowsCount to api-current.txt
* ENT-3053 Database connection pools leaking memory on every checkpoint. Flip in the thread local from the thread into the fiber.
* Back port to OS (needs some gradle changes) and added TODO, ability for it to avoid erroring if not using Hikari.
* Review feedback to remove warning.
* Handle case where file containing network parameters is missing
* Improve logging if attachments are not available and the file path doesn't exist
* Get logs the right way around
* Fix build error in tests
* Create a new custom migration for populating the state_party table and the relevancy_status column within the vault_states table when migrating from V3 to V4.
* Add a corresponding unit test suite.
* Small refactorings to expose data required by the migration: the isRelevant function in the NodeVaultService, and the node's legal name from the configuration.
* CORDA-2547: WIP Migrate old notary schema correctly.
Modify liquibase scripts to rename the old committed state table instead of creating a new one.
* Rename liquibase change sets
* Change 3.3 baseline before Liquibase.
* Aligin with ENT, orginal node-notary.changelog-v1.xml moved to node-notary.changelog-v2.xml and it's compatible with ENT (which already has changes from v2).
* Aligin with ENT, orginal node-notary.changelog-v1.xml moved to node-notary.changelog-v2.xml and it's compatible with ENT (which already has changes from v2) - adding v2 to master file.
* Ensure setting up Liquibase on pre-existing pre-Liquibase database
* Allow for duplicate (contract class, version) signed attachments in devMode.
* Code clean-up.
* Fix compilation error in test code.
* Additional gating/warning and added Unit test for development mode behaviour.
* Start pooling classpath scanning
Quickly patch synchronisation of attachment class loader cache. Needs a revisit but more complicated due to DJVM.
Annotate away for DJVM
Take ClassGraph utils into their own file so we can exclude for DJVM
Clean up a little
* Daemonize the threads
* Seems to be some concurrency problems with use of ClassGraph. Using a mutex for now to work around.
Upgrade from node 3.0 to 4.0 fails to create versions of whitelisted JARs from networkParameters - read parameters from file at first as in Corda 3.0 there no relevant table, then try from the table.
As this is migration, the code will run only once on each node, she increased log level to info for messages. Tested using https://r3-cev.atlassian.net/browse/R3T-1549
The version of contract attachments that are whitelisted should be read from NetworkParameters.whitelistedContractImplementations.
It use the lattes network map from db with the highest epoch.
Corda Node ensures a given contract class and version can be sourced from only one signed and trusted Attachment (JAR).
An attempt to import a signed JAR as a trusted uploader (or promote to be trusted) with a class and version already present in the other trusted Attachment will raise DuplicateContractClassException.
Minor fixes to Hibernate Attachment Query parser (original query to select attachment without signers would always return no attachments)
Move Raft and BFT-Smart notaries back into node to preserve backwards compatibility.
* Allow overriding full node config when using internal mock network parameters.
* Make BFT-Smart notary start up in prod mode as well
* Move raft & bftsmart notaries to net.corda.notary.experimental package
* Make sure Raft notary handles reference state edge cases correctly.
* Make sure BFT-Smart notary handles reference state edge cases correctly.
* Include notary schemas in node internal schemas
* Undo Raft notary table schema changes to maintain compatibility.
* Add cordapp code signing dev key to production blacklist.
* Remove code used to generate the actual PK hash (it is now attached to the JIRA story)
* Remove wiki page reference.
* Remove the non redundant cash transfer.
* ENT-2923 - remove db access code from the verification thread pool
* Remove worker pool for tx verification and disable db access.
* Address code review comments
The overload that takes in a String does NOT check that the exception thrown has that message, which is what these tests are assuming. Rather it's the assertion message when the test fails.
TestCordapp has now two implementations to clearly separate the two use cases it has in the Corda repo:
* TestCordappImpl which implements the revised public API of TestCordapp; namely that a TestCordapp instance references a real CorDapp jar on the classpath. This is either an external dependency jar in which case it’s taken as is and given to the node, or it’s a local gradle project in which case it’s compiled using the gradle “jar” task to generate the CorDapp jar. This approach means the jar has all the original CorDapp versioning information, which is important that it’s correct when testing. To this end, TestCordapp only needs to expose the ability to specify the app’s config. All the remaining properties have moved to CustomCordapp.
* CustomCordapp for creating arbitrary custom CorDapps, including specifying the jar’s MANIFEST values. This is internal API and only used for testing the platform. Technically this shouldn’t implement TestCordapp but does so to reduce the complexity of the driver and mock network.
* Refactor into attachment service
Fix up mock service
First caching version, but with no invalidation currently
Set cache size
Fix up after rebase
Cache invalidation
Formatting tidy up
Sort out some nullability
Add kdocs.
Unit tests
More unit tests
Fix TODO
Unit test fixes
Unit test fixes
Fixed concurrent invalidating transaction support.
* Correct some transaction concurrency bug, including unit test.
* Added some unit tests for the method I added to persistence.
* Remove some blank lines
* Review feedback
* Fix imports
* Implementation of Contract and Workflow attribute identifiers.
* Fixes following rebase from master.
* Fix broken JUnit test.
* Fix broken JUnit test.
* Fix broken JUnit test.
* Added missing constants.
* Further clean-up.
* Updated documentation.
* Added changelog entry.
* Updated all samples (using new Gradle Plugin 4.0.37 functionality)
* Temporarily resolve gradle plugins from latest published snapshot.
* Temporarily resolve gradle plugins from latest published snapshot.
* Updates following feedback from PR review.
* Move constants into CordappInfo companion object.
* Contract and Workflow attribute `version` to `versionId` (as version is a reserved gradle variable)
* Clarified warning message on incorrect version identifier.
* Align version identifier processing logic with gradle cordapp plugin.
* Updated comment.
* Minor fixes following rebase from master.
* Fixed broken unit test.
* Improved exception reporting.
* Update to use 4.0.37 of Gradle Plugins.
* Added support for combined Contract and Workflow CorDapp info.
* Updated following discussions with Shams + cleanup.
* Updated following Shams PR review.
* Minor API improvements.
* Added missing cordapp info causing deployNodes to fail.
New property database.initialiseAppSchema introduced that allows to turn-off Hibernate DDL generation while leaving Node-specific Liquibase handling enabled.
* ENT-2835: handle uncaught exception when bridge tries to send a message using a disconnected amqp client
* ENT-2835: remove emptyline
* ENT-2835: added comment explaining the exception handling
Contract class version non-downgrade rule is check by LedgerTransaction.verify().
TransactionBuilder.toWireTransaction(services: ServicesForResolution) selects attachments for the transaction which obey non downgrade rule.
New ServiceHub method loadAttachmentConstraint(stateRef: StateRef, forContractClassName: ContractClassName? = null) retrieves the attachment contract related to transaction output states of given contract class name.
Finance CorDapp v3.0 and core node 3.0 database tables doesn't have Liquibase migration scripts, now in Corda v4.0 the Liquibase has been introduced. Allow older Finance Cordapp v3.0 which doesn't have Liquibase to run in node v4.0 and create Liquibase log entries for FinanceApp only if it has schema migration (so it’s of v4.0).
At implementation level: there is new case when database has already Liquibase control tables however it doesn’t contains entries related to tables created by FInnaceApp and if the FinaceApp has Liquibase scheme it means it needs to be added to Liquibase logs.
TransactionBuilder loads attachment using attachment storage instead of CordappLoader,
contract class version is now Integer (format and stored in db as Integer).
* Make the default schemes for TLS and identity available from the CryptoService interface.
* Change CryptoService.generateKeyPair to accept SignatureScheme instead of Int.
* CORDA-2089 - network parameters tags - part
Data structures changes, storage and notarisation.
Tag transactions with network parameters hash that was in force when tx
was created. Add component group on all core transactions and resolved
parameters on full transactions. The hash should be always visible on
the filtered versions of transactions. Add
notarisation check that the parameters are current.
Implement network parameters storage on services for resolution.
This is only part of the work, next PR will include changes to
ResolveTransactionsFlow to make sure that parameters in the transaction
graph are ordered (this is to prevent the downgrade attack, when the
malicious notary and participants sign transaction that shouldn't be
notarised otherwise).
Probably on network services side we need the default parameters
endpoint for the
transactions that were created before this change - for now it's default
to the current ones.
* Make parameters storage agnostic to cert hierarchy
Test fixes
* Address most PR comments
* Fixes after rebase
* Fixes. Add epoch column to parameters storage.
* Address part of review comments
* Some more comments
* Hopefully fixing what I broke doing rebse
* Address Kostas comments
* Further fixes
* Save all parameters from updates to storage
* Fix integration test
* Address comments
* Fixes after rebase
* Fix test
* Fixes
* Add wrapper for filtering around parameters hash
* API stability fixes
* Add NetworkParametersStorageInternal
* Rename
* Temp commit
* Print the error message first by default, makes error output more natural.
* Polishing
* Further modifications after testing
* Documentation updates
* Couple of fixes after review
* Removing unnecessary tests
* Fix broken test
* Add interface to bootstrapper for testign
* Added unit tests
* Remove unused class
* Fix up bootstrapper unit tests and add a couple more
* Refactor the tests slightly
* Review comments
* Couple of minor tweaks
* [CORDA-2219] Show message if CorDapp already exists
* Update definition of `net.corda.core.node.services.Vault$StateMetadata` in `api-current.txt` or else diff tool gets very confused.
* First pass
* Update test.
* Address review comments.
* Added docs and kdocs.
* Clean-up.
* Add extra test.
* Changes to docsite.
* Added try/catch block as recommended by Andras.
* Removed try catch block. It's not required as the checkpoint serialiser deals with this.
* Re-used existing DB session instead of creating a new session.
* Entity manager auto flushes.
* Added java friendly api.
* Addressed review comments.
* print node-gen log when nodeInfo generation fails during bootstrapping
* add logic to print out the legal name of the node which failed to generate nodeInfo
By default Cordaps build by corda-gradle-plugins are signed by Corda development key.
In dev mode any key can be used to sign Cordapp JAR .
In production node Corda dev keys were forbidden. This code change allows to opt-out by setting node option cordappSignerKeyFingerprintBlacklist=[] or specify more public keys to blacklist.
The option is used in production only mode.
* Tidy up
* Add install-shell-extensions command
* Make cli tests use same version of picocli as everything else
* Remove initLogging from NodeStartup, it is ran earlier by CordaCLIWrapper
* Use picocli snapshot for testing
* Use RunLast() parser to invoke correct subcommands
* Deprecate old clear-network-map-cache parameter
* Restructure NodeStartup for commands
* Get rid of -c option since the flag method has been deprecated and that didn't exist in last release
* Update documentation
* Update backwards compatibility test
* Get all subcommands working
* Refactor sub commands into seperate classes
* Update docs and fix some tests
* Docs changes
* Fix merge conflicts with master
* Fix renamed parameters
* Fix test failure
* Fix compatibility tests
* Add missing compatibility test for blob inspector
* Remove blob inspector compatibility test as there are import conflicts
* Assorted doc fixes
* Addressing review comments
* More review comments
* Couple more bits
* Fix broken tests
* Fix compilation error
* More merge conflicts
* Make startup logging function a bit more sensible
* Fix broken shell extensions
* Make shell extensions work with subcommands
* Make sure parameters for deprecated options are carried through
* More review comments
* Adding some s's
* One last go
* Fix compilation error on Windows
* Revert logging changes
* Revert docs back to their original imperatively moody state
Related to CORDA-1915 Signing CorDapp JARs - Corda node rejects CorDapps signed by our development keys when running in production mode. This prevents Cordapps signed by our dev key (by default) running in production (node devMode=false).
The configuration objects for specific notary implementations have been replaced
by a single untyped "extraConfig" Config object that is left to the notary service
itself to parse.
* Remove the raft bootstrapping command from node, we'll need a different
mechanism for that.
* Remove pre-generated identity config value.
* Split up obtainIdentity() in AbstractNode to make it easier to read.
* A temporary workaround for the bootstrapper tool to support BFT notaries.
* Update docs
* Add upgrade notes
* Fix rebase issue
* Add a config diff for the bft notary as well
* ENT-2610: Separate passwords for store and for private keys in Corda OS.
When it comes to KeyStores there are *2* passwords: 1 for the keyStore as a whole and separately there is one private keys within this keyStore.
Unfortunately, those 2 passwords have to be the same due to Artemis limitation, for more details please see:
`org.apache.activemq.artemis.core.remoting.impl.ssl.SSLSupport.loadKeyManagerFactory`
where it is calling `KeyManagerFactory.init()` with store password.
Before change in this PR, throughout our codebase there are multiple places where we assume that storePassword is the same as keyPassword, even in the classes that have nothing to do with Artemis.
This is of course less than ideal as TLS communication may be used not only for Artemis connectivity (e.g. Bridge/Float interaction in Ent) and it is unfair to impose same passwords constraint on that communication channel.
Therefore this PR is removing this limitation and properly separating storePassword from keyPassword.
Linked Jira(https://r3-cev.atlassian.net/browse/ENT-2610) has for more background info.
Suggest to start review from `net.corda.core.crypto.X509NameConstraintsTest` to get an idea about the nature of the changes made.
* ENT-2610: Address PR input from @kchalkias
* ENT-2610: Address PR input from @kchalkias, s/privateKeyPassword/entryPassword/
* ENT-2610: Address PR input from @kchalkias, s/keyPassword/entryPassword/
In the implementation of `CertificateStoreSupplier`
The entry point to the API has been simplified to just requireing a list of packages to scan, with sensible defaults provided for the metadata. Because of the wither methods, having parameters for the metadata (with default values) seems unnecessary. Also the ability to scan just individual classes has been made internal, as it seems unlikely app developers would need that level of control when testing their apps.
TestCordappImpl is a data class and thus acts as a natural key for the Jar caching, where previously the key was the package names. This fixes an issue where it was not possible to create two CorDapp Jars of the same package but different metadata.
* Provide an optional configuration setting to specify the minimum platform version to use in the network params file.
* Leave Cordform signature intact.
* Leave previous Gradle Plugin called signature intact.
* Incorporating feedback from PR review.
* Added minimum platform version validation check.
* Removed final 2 references to "default"
* Added changelog entry.
* Add test for SNI header to prevent changing it accidentally.
* added hardcoded values test to ensure hashing function and corda x500 name format can't be changed
The cordapp and cordformation plugins (from v4.0.30) are going to have ability to sign JARs (in cordformation signing will be by default), to enable signature constraints to work out of box Network Bootstrapper will not whitelist contracts form signed JARs.
For unsigned JARs the Network Bootstrapper behaviour is unchanged.
* CORDA-2001: added SNI header to TLS connections based on hashed CordaX500
* CORDA-2001: added newline
* CORDA-2001: truncate hashed x500 to pass IDN validations
* CORDA-2001: convert hostname to lower case
* CORDA-2001: to lower after truncating, use of better suited method
* CORDA-2001: to lower after truncating, use of better suited method
* CORDA-2001: correctly set ssl parameters in the engine
* use constant for default platform version value when tests with out of process nodes are run from Intellij.
* node will use constant for platform version instead of manifest file(RPC already uses the constant, no sense in having 2 sources for it)
* fix issues caused by merge
Allow configuration in node for additional advertised addresses.
fix logic error
Use empty list as default config not null
Allow multiple addresses in NodeInfo
Describe new additionalP2PAddresses property in docs.
Add integration test of additionalP2PAddress feature
Fixup after rebase
Address PR comment
Address PR comments by removing unused element of NodeAddress
* Bugfix for corda.jar manually deployed without coping the matching corda-finance-VERSION.jar.
If the older finance cordapp version is detected (which doesn't have Liquibase migration scripts) fail node at startup and print message: "Could not create the DataSource: Detected incompatible corda-finance cordapp without database migration scripts, replace the existing corda-finance-VERSION.jar with the latest one."
Since coda-finance is an optional cordapp,the presence of Liquibase scripts is only checked if corda-finance-VERSION.jar is present in cordapps folder.
* Allow to start using Liquibase from any point of 4.0-SNAPSHOT before Liquibase was introduced (not only from 3.0/3.X versions) - 2 database changes introduced after 3.2 but before Liquibase are now conditional database changes.
* Create constraint, extract Jar signature collection
* Extract JarSignatureCollector into its own file
* Jar signature collection throws exception if signatures are inconsistent
* Focus testing in Jar signature collection
* Extract some helper functions in test
* Patch tests with mock attachment storage
* Assert that generated constraint is satisfied by signed attachment
* Clarify constraint selection logic
* Explicit return types on extension methods
* Link to docsite Signature Contrainsts documentation
* Fix issue with shared JAR reading buffer
* Add named caches and apply to NonInvalidingUnboundCache and all usages.
* Add named caches and apply to NonInvalidingCache and all usages.
* Add named caches and apply to NonInvalidingWeightBasedCache and all usages.
* Move NamedCache to core/internal
* Remove type `NamedCache` and `NamedLoadingCache`
* Suppressed 'name not used' warning, added comment, and fixed generic parameters on the buildNamed functions.
* Use `buildNamed` in all caffeine instances in production code. Not using it for caches that are created in test code.
* Add checks for the cache name
* Formatting
* Minor code review revisions
Internal tables (the tables from node and finance modules) are now tracked /created by Liquibase script.
Tables backing MappedSchemma in Cordapps are created by Hibernate (as before).
The PR scope added Liquibase library, setup code SchemaMigration and XML scripts and from Enterprise.
For existing database installation - the node will auto-upgrade to use Liquibase.
Method migrateOlderDatabaseToUseLiquibase checks for any 3.X existing Corda database to upgrade database to use Liquibase. When the existing database without Liquibase integral tables is detected, the node (at startup) will create Liquibase tracking tables and fill them with all migration scripts (marked as done), this ensure the database will look as it would use Liquibase from the beginning.
The database changes gradually introduced by the subsequent 3.X releases (3.1, 3.2) are conditionally run by Liquibase.
* CORDA-1813 fix Postgres db bloat issue
* CORDA-1813 merge fixes
* CORDA-1813 change column type and size to a standard corda type
* CORDA-1813 docs
* CORDA-1813 create custom hibernate type for the checkpoint blob and align with enterprise
* CORDA-1813 Remove max col size
* CORDA-1813 Remove max col size
* CORDA-1813 Fix merge
* CORDA-1813 Remove buggy :serverNameTablePrefix: configuration
