* CORDA-3273: Replace deprecated use of Class.newInstance() for sake of DJVM. (#5522)
* ENT-4090: move startFlow into try block so exception is caught and managed (#5519)
* CORDA-3091: Move executor thread management into CordaRPCConnection (#5491)
* CORDA-3184 - Vault Query API enhancement: strict participants matching (#5524)
* Implementation of exact participant parsing for common query criteria.
* Added more unit tests + fixed a minor bug.
* Additional clean-up.
* Add documentation and changelog entry.
* Revert debug level SQL logging.
* Fix detekt errors.
* Fix docs formatting in code samples.
* Updating baseline (false positives being reported).
* Fix "Name shadowed: subRoot" code quality check.
* Fix API stability checker failures.
* Update baseline.
* TM-43 updating code style docs to reflect the addition of Detekt (#5521)
* CORDA-3226 Fix dba migration for PostgreSQL following changes in CORDA-3009 and ENT-4192 (#5529)
* Fix dta migration for PostgreSQL following changes for CORDA-3009 Invalid hash function used for PersistentIdentity in PersistentIdentityService
and ENT-4192 Identity service refactor for confidential-identities and accounts.
* Different table definition for PostgreSQL and other dbs in one changeset instead of running generic DDL and the specifically fix table in Postgres (in relation to CORDA-3009 Invalid hash function used for PersistentIdentity in PersistentIdentityService and ENT-4192 Identity service refactor for confidential-identities and accounts).
NotaryFlow.Client flow assumes the passed transaction is verified, which is
the case when it's invoked as a sub-flow in FinalityFlow. However,
NotaryFlow.Client can also be called directly from any custom flow, so
we need to ensure it performs transaction verification to avoid accidentally
sending an invalid transaction to a non-validating notary.
* NOTICK: Corda 4.3-RC01
Created first release candidate of Corda 4.3 - RC01.
* CORDA-3141: Add GracefulReconnect callbacks which allow logic to be performed when RPC disconnects unexpectedly (#5430)
Also removed potential for growing stack trace on reconnects.
* CORDA-2050 Upgrade Corda to Java 11 (compatibility mode) (#5356)
Upgrade Corda to run with Java 11 (compatibility mode) - see https://github.com/corda/corda/pull/5356
* ENT-4198 Adding legal text
Signed-off-by: Ed Prosser <edward.prosser@r3.com>
* TM-29 new baseline for 4.3 since new debt has been added with the last few commits (#5487)
* TM-23 compileAll task to compile all code (#5490)
* Add simple compileAll task to be used by warning check
* lazy configure compileAll
* TM-32 Merge OS 4.3 into 4.4
* TM-32 fixed detekt issue
* Downgrade Dokka back to 0.9.17 due to failing docs_builder.
* add ability to group test types together (#5459)
* add ability to group test types together
* add ability to specify podCount for use in parallel testing
* remove compiler xml
* add Jenkinsfile to enable scanning
* trigger build
* add ability to specify what docker tag to use from outside of the build
* fix docker work dir
* fix pipeline syntax issues
* use environment rather than `def`
* move agent restrictor outside of stages block
* use steps block
* more pipeline syntax fixes
* even more pipeline syntax fixes
* even more pipeline syntax fixes
* add kubenetize as property to image build
* move clear of docker image to end of build rather than start to prevent colocated builds
* escape dollar on docker image remove command
* attempt to kill all existing jobs
* fix compile issue due to killall_jobs
* fix compile issue due to killall_jobs pt2
* fix spelling
* make all variables environment variables
* add logic to delete images locally after pushing
* wrap testing phase with try / finally so that junit reports are always evaluated
* change the behaviour around post build actions
* break implicit link between testing phase and image building phase, allowing testing to occur without a rebuild and push of image
* prepend registry name to provided tag
* allow tasks to specify whether they wish to stream output from containers
* add timestamps directive to Jenkinsfile to have timing info on output
* make KubesTest resilient against transient pod failures in k8s
* increase CPU request
* add logic to allow specifying container resource requests
* attempt to run unit and integration tests in parallel
* change unit tests to use 3 cores to allow co-location on 8c machines
* join grouped tests together to give pod meaningful name
* add step to renew token with GKE
* change renew step to use pods instead of nodes
* fix bug where memory request is not correctly passed to pod
* disable unit tests for now
* [CORDA-2368] Added exception handling for missing files that displays appropriate messages rather than defaulting to file names. (#5472)
* NOTIK Minor adjustments to Detekt rules to reflect current working practises (#5498)
* Minor adjustments to rules to reflect current working practises (including IntelliJ code style alignment)
* Adjust another rule in line with existing code style.
* rebaseline with changed detekt ruleset
* rebaseline with NodeStartup changes
Updating FinalityFlow with functionality to allow the initiator to indicate the appropriate StatesToRecord. This allows the initiating party to record states in transactions which they are proposing but are not necessarily participants of. An example would be the issuance of a token where the only participant is the holder NOT the issuer.
* Removed IdentityServiceInternal as it is no longer used.
* Removed externalIdForPublicKey API from KMS and added it to IdentityService.
Added a registerKeyToExternalId API on IdentityService.
* Fix remaining compile errors.
* Removed "registerKeyToParty" and in its place added a new registerKey method which takes a PublicKey, Party and optionally a UUID.
Added a cache to the "PersistentIdentityService" to store other node's public keys.
Added the cache and new hibernate entity to all teh places where one needs to add them.
New keys created by teh node now automatically get associated entries in the KEY -> PARTY map and optionally the KEy -> EXT ID map.
Added a test.
* Removed old comments and TODOs.
* Fixed broken test.
Added comments/explanations for what's going on in IdentityService.
Updated kdocs.
* First try at Implementing publicKeysForExternalId.
* Fixed broken test.
* Added migration.
Amended existing persistent identity service migration to handle new migration.
Addressed some review comments.
* Fixed broken test - whoops!
* Implemented mock identity service methods.
* Added back exception when remapping a key to a different party.
* Fixed compile errors.
Fixed broken tests.
* Use set instead of first entry in ourNames.
* CORDA-3133 [v1]
This proposal is a little more flexible by design. It adds a property to the state pointer class, allowing them to be referenced in transactions on a per-state-pointer basis.
* CORDA-3133
- Updated `resolveAtTransaction` to `isResolved`.
- Moved `isResolved` out of the constructor into an abstract property.
- Added deprecation constructor declaration for backwards compatibility.
* CORDA-3133 (version 2)
- Added required changes to state pointers as per PR comments.
- Added unit tests to ensure isResolved can be used to configure when state pointers should be resolved to reference inputs.
* CORDA-3133 (version 2)
- fixed unit tests.
- added comment to changelog.rst.
- added helper functions to `StatePointer` to allow easier creation of static and linear pointers.
Allow node operators to blacklist signing keys (using blacklistedAttachmentSigningKeys config option). These blacklisted keys prevent attachments that are received over the network from being trusted. The docs have been updated to detail how to generate the key hashes that the config requires.
A new shell command attachments trustRoots has been added to see what attachments exist on the node along with information about their trust and where it comes from.
run dumpCheckpoints has been replaced by checkpoints dump as InternalCordaRPCOps needed to change to prevent a function that is meant to be internal from being visible on the shell.
* Split integration tests
* add simple example of printing all methods annotated with @Test
* add docker plugin to root project
remove docker plugin from child projects
add Dockerfile for image to use when testing
add task to build testing image to root project
* add comment describing proposed testing workflow
* simple attempt at running tests in docker container
* add my first k8s interaction script
* add fabric8 as dependnency to buildSrc
* before adding classpath
* collect reports from containers and run through testReports
* re-enable kubes backed testing
* for each project
1. add a list tests task
2. use this list tests task to modify the included tests
3. add a parallel version of the test task
* tweak logic for downloading test report XML files
* use output of parallel testing tasks in report tasks to determine build resultCode
* prepare for jenkins test
* prepare for jenkins test
* make docker reg password system property
* add logging to print out docker reg creds
* enable docker build
* fix gradle build file
* gather xml files into root project
* change log level for gradle modification
* stop printing gradle docker push passwd
* tidy up report generation
* fix compilation errors
* split signature constraints test into two
* change Sig constraint tests type hierarchy
* tidy up build.gradle
* try method based test includes
* add unit test for test listing
* fix bug with test slicing
* stop filtering ignored tests to make the numbers match existing runs
* change log level to ensure print out
* move all plugin logic to buildSrc files
* tidy up test modification
add comments to explain what DistributedTesting plugin does
* move new plugins into properly named packages
* tidy up runConfigs
* fix compile errors due to merge with slow-integration-test work
* add system parameter to enable / disable build modification
* add -Dkubenetise to build command
* address review comments
* type safe declaration of parameters in KubesTest
* Added an additional property on VaultQueryCriteria to specify an externalId/accountId when performing a vault query.
Added logic in hibernate query criteria parser to handle the exernalId join and lookup.
Added a test.
* Fixed error in test.
Fixed backwards incompatible changes.
* Updated changelog.
* Updated docs to remove incorrect instructions for querying by external ID.
* migrate PersistentIdentityService to use key.toShortString()
update definition of PublicKeyToExternalId mapping to allow fast lookup by externalId/publicKey
* fix misspelled table name
* add test of migration script
* add design document for proposal to move IdentityService to using the correct PK.toStringShort() method for hashing a publickey
* add enterprise testing considerations to design
* address review comments
* fix compilation errors
* modify PublicKeyToOwningIdentityCache to use toStringShort() as it's lookup key
* address syzmon's code review comments
* CORDA-3018 Allow loading an untrusted contract jar if another attachment exists that was signed with the same keys and uploaded by a trusted uploader
`TransactionUtils.isAttachmentTrusted` requirements have been relaxed
to allow an untrusted attachment to be loaded as long as another
attachment exists that is signed by the same keys and was uploaded
by a trusted uploader.
The requirement of containing the same contract classes has been
removed. Therefore the contents of the existing trusted attachment
no longer matters.
* CORDA-3018 Allow a subset/intersection of signers in `isAttachmentTrusted`
Allow a subset/intersection of signers to satisfy the signer
requirements of `isAttachmentTrusted`. This allows an "untrusted"
attachment that is signed by one or more keys to be "trusted" as long
as another trusted attachment already exists that is signed by at least
one of the "untrusted" attachments signers.
A cache of trusted and untrusted public keys is now held (replacing the
previous cache of `List<PublicKey>`.
Tests have been added to `NodeAttachmentServiceTest` to confirm that
an attachment query using an `EQUAL` statement will actually return
attachments that are signed by any of the keys passed into the query.
Confirming this allowed an `EQUAL` query to satisfy the search that
had to be done as part of this change.
`MockAttachmentStorage`'s query criteria was updated to better match
the real `NodeAttachmentService` implementation.
* CORDA-3018 Update cache name and kdoc on `isAttachmentTrusted`
* CORDA-3018 Verify that chains of trust do not occur
* CORDA-3018 Switch keys around to improve chain of trust tests
* [ENT-3801] Store transactions in the database during transaction resolution (#2305)
* ENT-3801: Store downloaded txns as part of the backchain resolution into the db rather than the checkpoint
It's very inefficient to store the downloaded backchain in the checkpoint as more of it downloaded. Instead, if a threshold is reached (which currently defaults at 0) then the backchain is stored in the transactions table as unverified. A new is_verified column has been added to track this. Initially testing on the OS codebase has been very promising but unfortunately this current code is not quite ready. I had to quickly port it to ENT as this is meant to be an ENT-only optimisation.
To that effect, there is a TransactionResolver abstraction with two implementations: an in-memory one which has the old behaviour, and which will be the behaviour for OS, and a db one.
DBTransactionStorage hasn't been fully updated and I had to comment out the optimistic path for now.
Most of these changes will need to be ported to OS to keep the merge conflicts in check, but obviously not DbTransactionsResolver and the "is_verified" changes in DBTransactionStorage. DBTransactionStorage does have other refactoring which will make sense to port though.
* [ENT-3801] Start work on allowing modifications in AppendOnlyPersistentMap
* [ENT-3801] Add transaction resolver tests
* [ENT-3801] Adjust suspendable annotations
* [ENT-3801] Fix the ResolveTransactionFlow tests
* [ENT-3801] Update ResolveTransactionsFlow tests
* [ENT-3801] Add a liquibase migration script for isVerified
* [ENT-3801] Ensure the migration runs in the correct place
* [ENT-3801] Handle resolution of already present transactions
* [ENT-3801] Fix compile error in performance test app
* [ENT-3801] Logging and comment updates, plus a test case
* [ENT-3801] Add a notary change resolution test
* [ENT-3801] Add a contract upgrade transaction test
* [ENT-3801] Change new column to be a character based status
* [ENT-3801] Migration script type change
* [ENT-3801] Address first round of review comments
* [ENT-3801] Update variable names in AppendOnlyPersistentMap
* [ENT-3801] Another variable name clarification
* [ENT-3801] Fix missing name changes
* [ENT-3801] Make the signature list immutable when constructing cache value
* [ENT-3801] Add a locking strategy for unverified transactions
* [ENT-3801] Address tidying up review comments
* [ENT-3801] First attempt at ensuring locks are released after commit
* [ENT-3801] Remove references to old cache name
* [ENT-3801] Update locking logic
* [ENT-3801] Fix potential deadlock with read/write transaction locks
* [ENT-3801] Remove read locks, and ensure minimal extra suspends
* [ENT-3801] Fix build issues in tests
* [ENT-3801] Use the correct clock when calculating sleep durations
* [ENT-3801] Add a pessimism flag for writing verified transactions
* [ENT-3801] Change logging statement to debug
(cherry picked from commit 8ab6a55e17)
* [NOTICK] Fix up imports for some changed files
* [NOTICK] Fix transaction resolution tests
* [NOTICK] Reinstate the DBTransactionsResolver
* [NOTICK] Add the topological sort back to recordTransactions
* [NOTICK] Adjust test case to remove dependency on query ordering
* [NOTICK] Make test code match that in ENT
Allow Hash Constraints to propagate to Signature Constraints. When
propagating, the new jar is added to the transaction instead of the
original contract attachment (jar).
The following requirements must be met to do so:
- System property "net.corda.node.disableHashConstraints" must be set
to `true`
- The min platform version must be 4 or greater
- There must be an input state with a hash constraint
- There must be no output states with a hash constraint
- The new jar must be signed
If these requirements are not met, the original Hash
Constraint will be kept and the contract attachment related to it is
also used.
This transition is done at the start of `handleContract` as it is not
the normal path a transition would follow. It is considered a backdoor
and should be treated separately from the rest of the attachment and
constraint logic. Furthermore, it will only work in private network
since all nodes must set the special `disableHashConstraints` flag.
As a general purpose API, allPersisted should not be loading the entire contents of the database table into memory. Instead now it returns a Stream for processing of elements.
* Moved core.flows and core.contract tests
* move internal tests
* move node, schemas and serialization tests
* Move transactions tests
* Move test access helper into test package
* Move remaining tests, and create access helpers to test internals.
* Fix test dependencies.
* Delete some cruft that flagged up on the API checker
* Add readme explaing where tests need to go.
* fix typo
* add a shared memory port allocator to allow multiple processes to share a single allocation pool
* remove dangerous reset function on port allocator
* set forkCount = 2 in node integration test
* only allow one build of a cordapp at any given time for Driver tests
* make all portallocation requests use same starting point
* globally set forks to 6
* tweak forking parameters to allow parallel builds
* tweak unit test parallelism
* 2 workers for integrationTest
* some more tweaks for parallel builds
* some more tweaks for parallel builds
* seems that 49K is not the start of ephemeral ports on all kernels
* tweak parallel settings
* try fix RPC shutdown test in parallel env
* add some logging for RPC shutdown test
* added some logging around PortAllocation tests - try figure out where they are getting stuck
* added some logging around PortAllocation tests - try figure out where they are getting stuck
* fix api-scanner tests
* minimize api changes
* revert to complying with existing API
* add the AtomicInteger for api compatibility reasons
* make sizing script executable
* address review comments pt1
* address review comments pt2
* fix compile errors after review comments
* return to using home dir as temp dir seemed to interact badly with gradle
Dumps all the node's checkpoints as JSON into a single zip file in the node's directory. The output contains:
* All the fields for the top-level flow
* The current sub-flow call stack, along with the current progress tracker step for each sub-flow
* The event that suspended the flow, which if it's a send or sendAndReceive will show the payload that was sent
* Low level information on the active sessions with other peers
Party and AnonymousParty have been retrofitted to implement this interface and are currently the only supported types. A new FlowLogic.initiateFlow(Destination) method overload to easily support the addition of new destination types in future versions.
To prevent making `dumpCheckpoints` part of the public API a new
interface, `InternalCordaRPCOps` has been created and the function
has been moved there. `InternalCordaRPCOps` inherits from
`CordaRPCOps`.
`CordaRPCOpsImpl` now implements `InternalCordaRPCOps`.
`RunShellCommand` and `StringToMethodCallParser` required additional
changes due to issues handling inherited functions. This has only been
raised now due to `InternalCordaRPCOps` inheriting from `CordaRPCOps`.
Many classes have had references to `CordaRPCOps` changed to
`InternalCordaRPCOps`.
* Reduce test execution times by explicitly configure quasar package exclusions (using new quasar util extension introduced in Corda gradle plugins 5.0.1-SNAPSHOT).
* Remove SNAPSHOT from Corda gradle plugins version identifier.
* Incorporating PR review feedback from CR.
* Minor consolidation clean-up.
* Minor consolidation clean-up.
* Minor consolidation clean-up.
Dumps all the node's checkpoints as JSON into a single zip file in the node's directory. The output contains:
* All the fields for the top-level flow
* The current sub-flow call stack, along with the current progress tracker step for each sub-flow
* The event that suspended the flow, which if it's a send or sendAndReceive will show the payload that was sent
* Low level information on the active sessions with other peers
* What went wrong:
Execution failed for task ':core:test'.
> failed to read class file /Users/josecoll/IdeaProjects/corda-jdk11/core/build/classes/java/test/net/corda/core/flows/FlowsInJavaTest$PrimitiveReceiveFlow.class
* Upgrade gradle wrapper to Gradle 5.0
* Upgrade to use locally deployed version of Capsule plugin (using Gradle 5.0)
* Upgrade to use Corda Gradle Plugins 5.x (inclusive of Gradle 5.0 compatibility fixes)
* Fix compile-time problems resolving log4j packages.
* Update to use Artifactory deployed version of Capsule plugin (using Gradle 5.0)
* Upgrade to use Gradle 4.2.1 (excluding default memory setting change: command line client now starts with 64MB of heap instead of 1GB.)
* Added explicit compile-time dependency on 'de.jensd:fontawesomefx-commons:8.13' (compile-time error in tools:explorer without)
* Update "wrapper" gradleVersion to 5.4.1
* Update Capsule plugin version label to reference R3 forked build.
* Commit all gradle upgrade changes following: ./gradlew wrapper --gradle-version 5.4.1
* Increase maximum heap memory for Test tasks to 1GB, and upgrade build-scan plugin to 2.2.1.
* Increase Test tasks' maximum memory to 1.5GB - what could go wrong?
* Up maxHeapSize to 2g for gradle test runner (global setting).
* Added explicit compile-time dependency on 'de.jensd:fontawesomefx-commons:8.13' (compile-time error in tools:demobench without)
* Added forkEvery for node Unit tests.
* Up :node test task 'forkEvery' to 10.
* TC test execution tuning (:core, :node => forkEvery 10, default JVM heap size)
* TC test execution tuning (bump JVM heap size up to 1g for :node)
* TC test execution tuning (bump JVM heap size up to 1.5g for :node)
* TC test execution tuning (re-instate global JVM heap size of 1Gb)
* TC test execution tuning (re-instate JVM heap size of 2Gb for :node)
* Update Corda Gradle plugins to 5.0.0
* Updated plugin resolution order + renamed artifactory URL to use "software.r3.com"
* Reorder plugin resolution such that mavenlocal() is always queried first.
This allows a different signed version of the same CorDapp to be automatically trusted.
This reverts "[CORDA-2575] Allow users to whitelist attachments by public key config (#5035)"
The Signature Constraint documentation in `api-contract-constraints`
was very limited and referred to the design doc for most information.
Information was extracted from the design doc and added to the main
documentation.
If a single whitelisted constraint is being used by input states and the version of the cordapp changes + is signed, then the constraint will transition to a signature constraint.
When a `UnexpectedFlowEndException` or a `FlowException` is received the
peer that the exception was thrown from will be added to the stacktrace.
This is due to it being easier to see and a field that developers
are much less likely to override.
A nullable field `peer` has been added to `FlowException` and
`UnexpectedFlowEndException`. This is read later on (when peer info
is not available) to append the peer info to the stacktrace.
* CORDA-2817 Revert CORDA-2162 but modify Cash move to allow multiple move commands and thus multiple generateSpends in the same transaction.
* CORDA-2817 Remove API changes and internalise into Cash.
* Tests for custom registry restrictions
* ENT-3121 restrict custom serialisation
* Remove redundant code
* Only count declared annotations
* Check annotation on superclasses, remove annotation from ByteArray
* Forbid custom serialization of primitive types
* Remove @CordaSerializable from another class that is always handled by custom serialisation
* Add log warnings to aid diagnosis of custom serialization issues
* Remove another annotation
* Remove another annotation
* Remove another annotation
* Remove another annotation
* Fixup api-current
* Fixup api-current
* KDocs on exceptions
In Corda 4, FinalityFlow was updated to become an initiated flow, in order to ensure a node does not have to accept any signed transaction it receives without being able to check it first. The old behaviour of FinalityFlow was gated behind a targetPlatformVersion check, to prevent apps targeting V4 from using the old behaviour.
This is problematic for a few reasons. For an app wishing to be backwards compatible with a version running on V3, this forces the app to set targetPlatformVersion = 3, even if the app is thoroughly tested against V4. This goes against the purpose of the targetPlatformVersion. Another consequence is that an app remains pinned to targetPlatformVersion = 3 until it is sure that there are no other apps running at a lower version in the network, which would prevent newer versions of the app from taking advantage of features gated behind targetPlatformVersion checks. (Note that the restriction only prevents a new version of the app from initiating FinalityFlow with the old version - the old version is able to initiate a FinalityFlow and the new version will handle it, assuming the app has been written correctly.)
This fix removes the targetPlatformVersion check from FinalityFlow, and also provides a few documentation updates to clarify what level of testing would be expected to set a targetPlatformVersion.
