* CORDA-4045 Disable flaky test (#6792)
Disable `Restart does not set senderUUID` as it is unstable.
* INFRA-965: forward merge from 4.5 (2020-11-15) (#6806)
* EG-4168 Updating contributors.md list for OS 4.5 release branch (#6784)
* INFRA-965: forward merge from 4.4 (2020-11-14) (#6805)
* EG-4168 Updating contributors.md list for OS 4.4 release branch (#6785)
* INFRA-965: forward merge from 4.3 (2020-11-13) (#6803)
* INFRA-965: Jenkins/NexusIQ integration should target patches (#6802)
as well as major/minor releases
* Updated nightly builds to match changes for release branch builds
* Updated JDK11 builds to match changes for release branch builds
Co-authored-by: ivanterziev <61829352+ivanterziev-r3@users.noreply.github.com>
* Backporting updates for check-pr-title workflow
Co-authored-by: ivanterziev <61829352+ivanterziev-r3@users.noreply.github.com>
Co-authored-by: Ross Nicoll <ross.nicoll@r3.com>
Co-authored-by: ivanterziev <61829352+ivanterziev-r3@users.noreply.github.com>
* EG-4168 Updating contributors.md list for OS 4.5 release branch (#6784)
* INFRA-965: forward merge from 4.4 (2020-11-14) (#6805)
* EG-4168 Updating contributors.md list for OS 4.4 release branch (#6785)
* INFRA-965: forward merge from 4.3 (2020-11-13) (#6803)
* INFRA-965: Jenkins/NexusIQ integration should target patches (#6802)
as well as major/minor releases
* Updated nightly builds to match changes for release branch builds
* Updated JDK11 builds to match changes for release branch builds
Co-authored-by: ivanterziev <61829352+ivanterziev-r3@users.noreply.github.com>
* Backporting updates for check-pr-title workflow
Co-authored-by: ivanterziev <61829352+ivanterziev-r3@users.noreply.github.com>
* EG-4168 Updating contributors.md list for OS 4.4 release branch (#6785)
* INFRA-965: forward merge from 4.3 (2020-11-13) (#6803)
* INFRA-965: Jenkins/NexusIQ integration should target patches (#6802)
as well as major/minor releases
* Updated nightly builds to match changes for release branch builds
* Updated JDK11 builds to match changes for release branch builds
Co-authored-by: ivanterziev <61829352+ivanterziev-r3@users.noreply.github.com>
* INFRA-965: Jenkins/NexusIQ integration should target patches (#6802)
as well as major/minor releases
* Updated nightly builds to match changes for release branch builds
* wip
* wip
* wip (need to review IEE comments)
* wip
* wip
* Small refactoring, fixed network-verifier's TestNotaryFlow
* Added command line option to explicitly enable hash agility support
* wip-do-not-push
* wip
* wip
* wip
* aligned merkletree/transaction hash algorithms
* wip
* Added mixed algorithm support for nodes vs leaves and corrected mixed algorithm tests
* moved global computeNonce and componentHash to DigestService
* added comment for failing test to fix
* wip
* Minor cleanups, added deprecated componentHash/computeNonce
* restored exploratory changes to failing SignedTransaction test
* cleaned up and minor rafactoring
* Fixed some tests with hardcoded hash algorithm
* some changes and cleanups following code review
* WIP commit before large change
* WIP Fixed 3 tests
* WIP removed direct references to randomSHA256() and sha256()
* Updated/added liquibase migrations to support larger hash algorithms
* Reviewed, cleanups, comments, fixes
* removing direct references to sha256()
* WIP verifying obligations test errors
* reviewing obligation/attachment issues with sha3_256
* Full review before PR - intermediate commits
* Reviewed and cleaned up
* Futher cleanup
* Fixed partial tree backward compatible json and cleanups
* all tests passing
* Removed couple of unused imports
* Reworked global componentHash function to avoid deprecated warnings
* replaced SHA3s with some alternate SHA2s
* Removed SHA3-256 and SHA3-512 references
* fixed some tests using non ubiquitous hash algorithms
* Fixed ABI compatibility (not for TransactionBuilder)
* Fixed ABI compatibility to TransactionBuilder
* couple of fixes
* fixed DigestService's randomHash
* Removed constructor with loosely typed args for private constructor of LedgerTransaction class (API removal)
* re-introduced LedgerTransaction deprecated ctor for deserialization
* Add possibility to load CustomMessageDigest bypassing JCA (#6798)
* Change api-current for DigestAlgorithm
* disable flaky tests
Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
* CORDA-4034 Reduce forkEvery to 15 to attempt to mitigate memory leak.
* ENT-5679 Disable test which triggers OOM
* Run tests on two Jenkins agents
* Fixed processing JUnit test results by Allure
* Add timeouts to VaultObserverExceptionTest
* Revert "CORDA-3845: Update BC, log4j, slf4j (#6699)" to eliminate introduced memory leaks
Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
* Do not use Security.addProvider(BouncyCastleProvider()) in tests, to avoid disruptions of other tests.
* Forcibly register security providers before starting Jimfs, to resolve a sequencing problem where Jimfs triggers loading of the SFTP filesystem provider, which in turn registers the standard BouncyCastle provider rather than the patched version Corda needs.
* INFRA-683 Move Corda OS release branch builds to serial (#6703)
Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
* Add support for handling release candidate for Docker publishing
* Re-enabled Sonatype check
* Artifactory build name includes branch / tag name
* Environment variables sorted alphabetically
* CORDA-3908: Update corda-4.7 .ci/api-current.txt to match the previous release (corda-4.6)
* Reapply changes from PR 6644
* NodeParameters gained a property in PR 6644
* Remove duplicate class from bad merge
* INFRA-424 linux1 jenkinsfile
* INFRA-424 full run
* INFRA-424 bigger heap size
* Upgraded DJVM to handle BC - latest version of BC is a multirelease JAR.
When reading JKS keystore if a BC EdDSAPrivateKey is returned then swap for a net.i2p EdDSA private key.
* Temporary downgrade of BC
* Removed the BC EdDSA conversion
* INFRA-424 bigger heap size
* Upgrading Quasar to handle openJ9 different fields.
* INFRA-424: Handle lack of SUPPRESSED_SENTINEL in openj9.
* INFRA-424: If BCEdDSA public or private key is generated convert to net.i2p EdDSA form.
* INFRA-424 bigger heap size
* INFRA-424: On openJ9 only getting upto milli resolution.
* INFRA-424: Handle keystore returning a BCEdDSAPrivateKey.
* INFRA-424: Disable test on JDK11, as it requires the custom cordapp to generate JDK8 contract code, which we now check for.
* INFRA-424: Truncated time test to resolution of millis for openj9.
* INFRA-424 disabling log intensive tests until a fix is developed
* INFRA-424 one more test disabled
* INFRA-424: Disabled a couple of tests failing on openj9.
* INFRA-424: Disabling failing openj9 tests.
* INFRA-424: Disabling test failing on openj9.
* INFRA-424: Ignoring another flaky sleep test on openj9.
* INFRA-424 run integrationTests
* INFRA-424 set timeout to 4 hours
* INFRA-424: Cope with exception message from openj9.
* INFRA-424: Handle the coloured text characters openj9 adds.
* INFRA-424: Disabling test as it is generating JDK11 contract code under JDK11. Currently on JDK8 contract code allowed.
* INFRA-424: Commenting test out for openj9. Output of the processs thats read by the test is sometimes garbled.
* INFRA-424 switching to smoke tests
* INFRA-424 switching to slow integration tests
* INFRA-424 full run
* INFRA-424 moving jenkinsfile
* INFRA-424 removing references
* INFRA-424: Created common IS_OPENJ9 func for ignoring tests.
Co-authored-by: Schife <razvan.codreanu@r3.com>
* CORDA-3908: Update corda-4.5 .ci/api-current.txt to match the previous release (corda-4.4)
* Reapply changes from PR 5998
https://github.com/corda/corda/pull/5998
* CORDA-3960: Port MultiRPCClient to OS
* CORDA-3960: Carefully restore serialisation environment in `MultiRpcClientTest` to eliminate side effects on other tests
* CORDA-3960: Move ThreadDumpUtils.kt to `core-utils`
* CORDA-3917 Update to Jackson 2.9.8 (#6493)
* Update to Jackson 2.9.8 to address multiple security issues, and update warning note about
updates to clarify that it refers to 2.10+. When the note was added 2.9.7 as the highest
available version in the 2.9.x series.
* Add PR code checks Jenkinsfile
* CORDA-3916 Update to BouncyCastle 1.61 (#6492)
Update to BouncyCastle 1.61. Updating one version at a time to mitigate risk of a complex breaking change being introduced.
* Added missing collection of JUnit tests and logs
Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
* CORDA-3717: Apply custom serializers to checkpoints
* Remove try/catch to fix TooGenericExceptionCaught detekt rule
* Rename exception
* Extract method
* Put calls to the userSerializer on their own lines to improve readability
* Remove unused constructors from exception
* Remove unused proxyType field
* Give field a descriptive name
* Explain why we are looking for two type parameters when we only use one
* Tidy up the fetching of types
* Use 0 seconds when forcing a flow checkpoint inside test
* Add test to check references are restored correctly
* Add CheckpointCustomSerializer interface
* Wire up the new CheckpointCustomSerializer interface
* Use kryo default for abstract classes
* Remove unused imports
* Remove need for external library in tests
* Make file match original to remove from diff
* Remove maySkipCheckpoint from calls to sleep
* Add newline to end of file
* Test custom serializers mapped to interfaces
* Test serializer configured with abstract class
* Move test into its own package
* Rename test
* Move flows and serializers into their own source file
* Move broken map into its own source file
* Delete comment now source file is simpler
* Rename class to have a shorter name
* Add tests that run the checkpoint serializer directly
* Check serialization of final classes
* Register as default unless the target class is final
* Test PublicKey serializer has not been overridden
* Add a broken serializer for EdDSAPublicKey to make test more robust
* Split serializer registration into default and non-default registrations. Run registrations at the right time to preserve Cordas own custom serializers.
* Check for duplicate custom checkpoint serializers
* Add doc comments
* Add doc comments to CustomSerializerCheckpointAdaptor
* Add test to check duplicate serializers are logged
* Do not log the duplicate serializer warning when the duplicate is the same class
* Update doc comment for CheckpointCustomSerializer
* Sort serializers by classname so we are not registering in an unknown or random order
* Add test to serialize a class that references itself
* Store custom serializer type in the Kryo stream so we can spot when a different serializer is being used to deserialize
* Testing has shown that registering custom serializers as default is more robust when adding new cordapps
* Remove new line character
* Remove unused imports
* Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt
* Remove comment
* Update comment on exception
* Make CustomSerializerCheckpointAdaptor internal
* Revert "Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt"
This reverts commit b835de79bd.
* Restore "Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt""
This reverts commit 718873a4e9.
* Pass the class loader instead of the context
* Do less work in test setup
* Make the serialization context unique for CustomCheckpointSerializerTest so we get a new Kryo pool for the test
* Rebuild the Kryo pool for the given context when we change custom serializers
* Rebuild all Kryo pools on serializer change to keep serializer list consistent
* Move the custom serializer list into CheckpointSerializationContext to reduce scope from global to a serialization context
* Remove unused imports
* Make the new checkpointCustomSerializers property default to the empty list
* Delegate implementation using kotlin language feature
* NexusIQ every build related to Corda X.Y (GA, RC, HC, patch or
snapshot) uses the same NexusIQ application
* NexusIQ application application has to exist before a build starts
* Fixed repository name for publishing, use OS instead of Ent one
* every build related to Corda X.Y (GA, RC, HC, patch or snapshot) uses
the same NexusIQ application
* NexusIQ application application *has* to exist before a build starts
* Update to Jackson 2.9.8 to address multiple security issues, and update warning note about
updates to clarify that it refers to 2.10+. When the note was added 2.9.7 as the highest
available version in the 2.9.x series.
* Add PR code checks Jenkinsfile
* Use a virtual repo (corda-remotes) containing all Corda repositories with dependencies
* activated when CORDA_USE_CACHE environment variable is set
* Update Jenkins configuration to use new functionality
* it does *not* affect local builds as long as environment variable is not set!
* CORDA-3696: Temporary update to enable JDK11 build and test. Will eventually be switchable.
* CORDA-3696: Filter out the Nashorn warning.
* CORDA-3696: Add JDK11 classifier.
* CORDA-3696: Updated match string to cope with JDK11.
* CORDA-3696: Filtering out SPHINCS256_SHA256 where failing due to JDK11.
* CORDA-3696: Now remove SPHINCS256_SHA256 only if JDK11.
* CORDA-3696: Fix test failure - switch to regex matching.
* CORDA-3696: Hide the illegal access warnings.
* CORDA-3696: Check for Java11 when disabling Java11 warnings.
* CORDA-3696: Fix unneccessary non null check.
* CORDA-3696: Reverting build env to JDK8
* CORDA-3696: Revert hiding of illegal access warnings via Unsafe class.
* CORDA-3696: Remove internal access warnings and new JDK11 version checker.
* CORDA-3696: Updated build file for OS
* CORDA-3696: Removed typo
* CORDA-3696: Fixed space typo.
* CORDA-3696: Open modules to remove the illegal access warnings.
Co-authored-by: Adel El-Beik <adelel-beik@19LDN-MAC108.local>
* Remove unused dependencies from test-common
* Explicit imports and formatting
* Add core-test-utils project
* Add dependency
* Move Kryo serialization context to node-api (not serialization as we do not want to pull kryo into the serialization lib)
* Move AMQP server serialization scheme to node api
* Move serialization tests to node-api
* Move internal test helpers without further dependencies.
* Move out some types from RPCClientProxyHandler to node-api in preparation for moving the AMQP scheme
* Move client AMQP context to node-api so we can move the test serialization rule out.
* Move InternalSerializationTestHelpers to core-test-utils
* Moved testing.core to core-test-utils
* Make detekt happy
* Add api-scanner to core-test-utils
* Remove inlined package names introduced by IntelliJ refactoring
* Update api-current.txt to account for reordering.
* Add core-test-utils to list of published artifacts.
* Add missing import
* Location of things in api text has moved again (publish name of artefact?)
* Revert all additions to the API, leaving just the reordering
* Code review: fix up core-test-utils build.gradle and introduce kryo version constant.
* Remove OpenSsl flag from ssl config stub (can't be used from node-api)
* Suppress detekt warning
* Move core test util tests to the right module
* Expose kotlin test as a transient dependency - projects have come to rely on that.
* Fix typo in package name
* TM-204 attempting to fix regression builds
* TM-204 attempting to fix regression builds
* TM-204 reverting spring boot version and only removing missing dependency
* TM-204 reverting to original build.gradle
* TM-204 re applying dependency
* TM-204 consolidating dependencies
* TM-204 setting spring boot dependency
* TM-204 reverting and upgrading dependency management plugin version in parent gradle file
* TM-204 apply dependency plugin differently
* TM-204 new gradle files
* TM-204 try and fix corda rpc evaluation
* TM-204 try and fix corda rpc evaluation
* TM-204 separate slow integration and smoke test
* TM-204 modifying regression builds to separate slow integration tests and smoke tests as they sometimes result in a bad gradle dependency graph
* TM-204 separating slow integration tests from the rest
* TM-204 change to allow ci03 to run this jenkinsfile
* TM-204 switching to aks label to allow testing on ci03
* TM-204 now that test was successful, switching back to correct prod label
* TM-204 attempting to fix regression builds
* TM-204 attempting to fix regression builds
* TM-204 reverting spring boot version and only removing missing dependency
* TM-204 reverting to original build.gradle
* TM-204 re applying dependency
* TM-204 consolidating dependencies
* TM-204 setting spring boot dependency
* TM-204 reverting and upgrading dependency management plugin version in parent gradle file
* TM-204 apply dependency plugin differently
* TM-204 new gradle files
* TM-204 try and fix corda rpc evaluation
* TM-204 try and fix corda rpc evaluation
* TM-204 separate slow integration and smoke test
* TM-197 Setting bouncy castle provider in order for the test to pass
* TM-197 setting timeout for all builds at 3 hours
* TM-197 ignoring unstable tests
* TM-197 switching 4.3 to use local k8s instances and also make the maximum duration of builds 3 hours, fix 1 test and ignore 2 flaky ones
* update to use local-k8s version of the plugin
Co-authored-by: Stefano Franz <roastario@gmail.com>
* switch to new version of plugin
* change plugin version to force refresh
* try using pre-populated cache
* do not preallocate
* no daemon
* reduce cores per fork to increase parallelism
* even more aggresive cpu allocation
Deprecate FlowAsyncOperation and reimplement public versions FlowExternalOperation and FlowExternalAsyncOperation.
await added to FlowLogic to allow easy calling from both Java and Kotlin. There are two overrides of await (one for FlowExternalOperation and FlowExternalAsyncOperation).
Implementations of FlowExternalOperation return a result (written as blocking code) from their execute function. This operation will then be executed using a thread provided by the externalOperationExecutor.
Implementations of FlowExternalAsyncOperation return a future from their execute function. This operation must be executed on a newly spawned thread or one provided by a thread pool. It is up to developers to handle threading in this scenario.
The default thread pool (externalOperationExecutor) can be configured through the flowExternalOperationThreadPoolSize node config.
The current implementation leaves FlowAsyncOperation alone, meaning that any developers that have used it (even though it is internal) won't need to change their apps. If this was not concern I would delete it completely and replumb the state machine code. Instead, it has been marked with @DoNotImplement and executeAsync is annotated with @Deprecated
* CORDA-2942: Port minimal set of changes to make lifecycle events work
... and make codebase compile.
* CORDA-2942: Undo some changes which are not strictly speaking necessary
* CORDA-2942: Make `NodeServicesContext` leaner and delete `extensions-api` module
* CORDA-2942: Reduce even more number of files affected
* CORDA-2942: Integration test fix
* CORDA-2942: Make events `AfterStart` and `BeforeStop` generic w.r.t. `NodeServicesContext`
* CORDA-2942: `NodeLifecycleObserverService` and a set of integration tests.
Public API violations are expected as well as integration tests failing.
* CORDA-2942: Re-work to introduce `ServiceLifecycleObserver`
* CORDA-2942: Explicitly mention a type of exception that may be thrown for some events.
* CORDA-2942: Register `ServiceLifecycleObserver` through `AppServiceHub`
* CORDA-2942: Fix integration test + KDocs update
* CORDA-2942: Detekt and `api-current` update
* CORDA-2942: Improvement to `CordaServiceLifecycleFatalTests`
... or else it has side effects on other tests.
* CORDA-2942: Add an integration test for new API use in Java
Driver test is written in Kotlin, but services definition is written in Java.
Also KDocs improvements.
* CORDA-2942: Documentation and release notes update
* CORDA-2942: First set of changes following review by @mnesbit
* CORDA-2942: Second set of changes following review by @mnesbit
* CORDA-2942: Added multi-threaded test
* CORDA-2942: Fixes
* CORDA-2942: Undo changes to `api-current.txt`
* CORDA-2942: Bare mimimum change to `api-current.txt` for CI gate to pass.
* CORDA-2942: Address review feedback from @rick-r3
* CORDA-2942: Detekt update
* CORDA-2942: Delete `ServiceLifecycleObserverPriority` and replace it with `Int` after discussion with @mnesbit
* CORDA-2942: Introduce more `NodeLifecycleEvent` and switch services to listen for those events
* CORDA-2942: Few more changes after input from @rick-r3
* First stub on integration test
Unfinished - hang on issue and pay
* CORDA-2942: Switch to use out-of-process nodes for the inetgration test
Currently Alice and Notary stuck waiting to hear from each other.
* CORDA-2942: Extra log lines during event distribution
* CORDA-2942: Asynchronously distribute lifecycle events
* CORDA-2942: Await for complete P2P client start-up
Next step: Add vault query to integration test
* CORDA-2942: Asynchronously distribute lifecycle events
Next step: Improve integration test
* CORDA-2942: Fix test broken by recent changes and improve logging
* CORDA-2942: Improvement of the test to be able to monitor actions performed by @CordaService in the remote process
* CORDA-2942: Add node re-start step to the integration test
* CORDA-2942: Remove `CORDAPP_STOPPED` event for now
* CORDA-2942: s/CORDAPP_STARTED/STATE_MACHINE_STARTED/
* CORDA-2942: Inverse the meaning of `priority` as requested by @rick-r3
* CORDA-2942: Register `AppServiceHubImpl` for lifecycle events and put a warning when SMM is not ready.
* modify CollectSignaturesFlow.kt to allow multiple collections from wellKnown party initiated sessions
* detekt fixes
* review comments
* move require lambdas back outside of the function definition of requires
* address review comments
* fix detekt
* fix api scanner
* fix merge issue with sidecar image work
* add logic to refresh token in GKE
* set streaming to true
* set streaming to true
* set streaming to false, and increment number of workers for regression test as approaching 1h limit
* set PR build to run unit and integration tests
* try method distribution to see if faster
* rename test stage to reflect the fact it runs unit tests
* remove commented out parallel unit test code
* * CORDA-2876: Migrate DJVM serialization modules into Corda.
* Pre-generate Corda classes for DJVM sandbox when node boots in production mode.
* Ensure that all DJVM test contract CorDapps are signed.
* Test examining attachments within DJVM sandbox.
* Test Contract.verify() using cryptographic verify function.
* Add test cases for more non-determinism in Contract.verify().
* Update node-driver to support testing nodes with DJVM support.
* Modify Node to allow alternative DJVM configurations for testing.
* Refactor DeterministicVerifierFactoryService for default use-case.
* Small whitespace and code-style refactors.
* Create and activate a DJVM execution profile for the Node.
* Revert making Verifier implement AutoCloseable.
* Allow the node to cache sandboxed Corda byte-code for reuse.
* Use updated Quasar agent that knows not to touch DJVM classloaders.
* Fix Quasar's package exclusions globs for DJVM.
* Deserialise LedgerTransaction into the sandbox for Contract.verify().
* Add the DJVM's serialisation modules to the Corda node.
* Update the node for the latest DJVM API, and preserve the ConstructorForDeserialization annotation on user contract classes.
* Add corda-dev to repositories while DJVM is SNAPSHOT.
* Migrate DJVM specialisation into AbstractNode's ServiceHubInternalImpl.
* Exclude sandbox.** and shaded djvm.** classes from Quasar agent.
* Add the corda-dev repository to :node for the deterministic runtime.
* Turn Verifier into an abstract base class that is specialised by BasicVerifier and DeterministicVerifier.
* Add the Corda deterministic libraries to the Node, and split the DJVM sandbox across two SandboxClassLoader instances.
* Add DJVM to contract verification path inside Corda Node.
* Minor lambda simplifications and removing unused import.
* CORDA-2871: Remove @CordaSerializable from LedgerTransaction.
* CORDA-2871: Add a callback to ServicesForResolution to allow the Node to modify a LedgerTransaction object.
* CORDA-2871: Refactor the contract verification code into a separate class,
and allow LedgerTransaction to choose different Verifier objects.
* Update DJVM to use Corda 4.4-SNAPSHOT. (#95)
* CORDA-3330: Allow DJVM to preload / pregenerate classes from selected jars. (#92)
* Add support for SourceClassLoader.getResources() to DJVM.
* Allow a SandboxConfiguration to preload sandbox byte-code for all classes inside jars containing META-INF/DJVM-preload.
* CORDA-3309: Remove explicit try-catch in favour of UncaughtExceptionHandler. (#91)
* CORDA-3309: Install UncaughtExceptionHandler for DJVM tasks. (#88)
* Fix tests broken by Windows line endings. (#82)
* CORDA-3292: Reimplement ExecutionProfile as a data class. (#80)
* CORDA-2877: Refactor how we create child SandboxConfiguration objects. (#76)
* CORDA-2877: Load bytecode from a persistent cache to prevent repeated rewriting. (#75)
* Refactor byte-code cache to SandboxConfiguration instead of AnalysisConfiguration. We cannot "mix and match" byte-code generated by different sets of rules.
* CORDA-3137: Enhance annotation handling so that we can allow some annotations to be mapped into the sandbox without also needing to be stitched. (#72)
* CORDA-2871: Minor cosmetic fixes. (#69)
* CORDA-3218: Align DJVM with internal Corda Serialisation API. (#68)
* Ensure we get the latest SNAPSHOT of the serialisation code.
* CORDA-2871: Refactor SourceClassLoader to define source classes. (#66)
* Rewrite SourceClassLoader to support parent/child relationships.
* Revert catching TypNotPresebtException - it was a symptom of a bigger problem.
* Remove AutoCloseable from AnalysisConfiguration and SourceClassLoader.
* SourceClassLoader.getResource() must delegate to its parent first.
* CORDA-2871: Ensure ClassLoader.loadClass() throws ClassNotFoundException for all cases where the class cannot be found. (#64)
* CORDA-2871: Modify sandbox tasks to implement both java.Function and sandbox.Function (#62)
* Make TaskExecutors implement BiFunction to make them composable.
* Create ImportTask to wrap a java.Function inside a sandbox.Function.
* Add createExecutor() and createRawExecutor() APIs to SandboxClassLoader.
* Update serialization to use SandboxClassLoader.toSandboxClass().
* Remove a layer of lambdas from the serialisation code.
* Update SandboxExecutor and SandboxRawExecutor.
* Rename Executor to TaskFactory.
* Rename dangling executor -> taskFactory.
* CORDA-2871: Sanity fixes! (#63)
* Improve message for SandboxClassLoadingException.
* Fix serialisation API for using sandboxed environment.
* CORDA-3174: Extend serialisation to include InputStream and OpaqueBytesSubSequence. (#60)
* Update DJVM Example project for serialisation.
* Add serializers for InputStream and OpaqueBytesSubSequence.
* Support ZIP Inflater and CRC32 inside the sandbox.
* Allow the DJVM to wrap java.io.InputStream as sandbox.java.io.InputStream.
* Configure tests also to preserve @DeprecatedConstructorForDeserialization.
* CORDA-3174: Implement Corda serialization modules. (#59)
* Create DJVM serialization modules.
* Create test cases for Array<T>, List<T> and List<Array<T>>.
* Refactor SandboxPrimiveSerializer for all primitive types.
* Implement SandboxCollectionSerializer to support Collection types.
* Implement SandboxMapSerializer to support Map types.
* Attempt to fix infinite loop when computing Collection and Map fingerprints.
* Apply special handling when deserialising sandbox.java.lang.Character.
* Remap Java primitive types to sandbox Java object types to deter evolution.
* Use Class.getPackage().getName() to determine sandbox package name.
* Implement SandboxEnumSerializer to support Enum types.
* Implement SandboxPublicKeySerializer to support Java security keys.
* Add serialization projects to the composite example project.
* Implement serializers for BigInteger, BigDecimal, Currency and StringBuffer.
* Test that deserialising does not instantiate the untrusted user classes.
* Implement serializers for java.time.* types.
* Add serialiser for BitSet - currently disabled until BitSet itself is supported.
* Add serialisers for EnumSet and Class.
* Include support for EnumMap in the SandboxMapSerializer.
* Ensure the DJVM Example project's tests preserve @CordaSerializable.
* Add support for UUID as a primitive type.
* Use common abortReadOnly() method for declaring serialization as unsupported.
* Streamline the API for deserialising into the sandbox.
* Add preliminary support for deserialising X.509 certificates.
* Implement serializer for java.util.Optional.
* Refactor configuration of the sandbox serialization scheme.
* Add tests for deserialising arrays of basic types.
* Include method annotations in annotation stitching. This ensures that `@ConstructorForDeserialization` is not dropped.
* Enable test for SandboxBitSetSerializer.
* Enable tests for X.509 serializers.
* Implement serializers for ProtonJ primitive types.
* Serialize java.util.Date as a primitive type.
* Add the bintray Gradle plugin to the serialisation modules.
* Do not publish serialisation modules - they will become part of Corda itself.
* CORDA-2876: Only apply DJVM sources to Node Driver when devMode=true.
* Resolve DeteKT warnings.
* Require Node's JVM to set -Dnet.corda.djvm=true in order to enable DJVM.
* Enable DJVM for DemoBench nodes.
* Disable Quasar instrumentation verification for DemoBench nodes.
* Upgrade to DJVM 1.0-RC01.
* Try to modify DriverParameters in a more "ABI friendly" way.
* Refactor and simplify sandbox deserialisation of primitive objects.
* Review fixes.
* Update EvolutionSerializerFactory to handle sandboxed primitive boxed types.
* CORDA-3350: Increase size of constraints column (#5639)
* Detekt
* Update api file with new threshold
* Add check in transaction builder
* Revert "Add check in transaction builder"
This reverts commit ca3128f44c.
* Add check for max number of keys
* Update api file
* Address Tudor's comments
* Remove check for pre-5 and add test for EC keys
* fix typo and rename liquibase script
* updated docs with measurement numbers for composite keys
* Make detekt happy again
* TM-68 reenabling persistent volume claims using azure files
* TM-68 jenkins stackstracee
* TM-68 removing duplicate volume
* TM-68 pushing storage class yaml file
* TM-68 writing all results to the new persistent volume
* TM-68 fix wrong directory
* TM-68 fix wrong directory
* reapply lost merge commit
* investigate missing POD from test results
* more investigations around pods not executing their tests
* make Pod command line more strict with regards to sub command failure
* make logs an artifact within jenkins
* tidy up command line
* use zulu for jdk
add some parallel groups
* port kubesTest to Java
remove asterix from tests listed by ListTests, instead add after allocation
* attempt to setup unit test builds with correct github integrations
# Conflicts:
# .ci/dev/unit/Jenkinsfile
* fix issue with github context
* add credentials block
* start pre-allocating pods for builds
* test
* add blocks for reporting build stages
* add logic to preallocate pods during image building
* tidy up Jenkinsfile for unit tests
* add magic command line flag to enable preallocation of pods
* make docker tag deterministic
* fix issue concatenating docker tag inputs
* add build type specific Jenkinsfile
* try new preallocation approach
* make pre-allocation prefix group specific
* force deAllocator to wait for pods to be actually deleted
* revert jenkinsfiles in .ci
* use smarter waiting logic to address review comments
* add --stacktrace to builds to help debugging
* fix issue with closed stream
* add some logging around preallocation
* tidy up by refactoring (de)allocate task generation into method
* change default from 20 pods to 5 pods
* fix issue where docker tag was unstable between building and running tests
* more documentation
* add some infrastructure around setting the log level for a given build
* change preallocation pod duration to 5min
* see if fast enough if using combined unit and integration tests
* disable unit tests
* print out test summaries
* try and make the kubes client a per-use object, rather than a long lived object. This is step one of making GKE use possible
* add log line about what command is executed in the pod
* add ability to index by class OR method
* disable unit tests again
* pending commit
* stream output of build for debugging
* remove git from the base image
* re-enable NodeRPCTests
* add ability to distribute tests based on TeamCity CSV output for test times
* try enabling unit tests again after test distribution has been fixed
* refactor BucketingAllocator to be a class, which bucketingAllocatorTask delegates to.
* finishing touches for improved test distribution
* create new pipelines for builds
Party and AnonymousParty have been retrofitted to implement this interface and are currently the only supported types. A new FlowLogic.initiateFlow(Destination) method overload to easily support the addition of new destination types in future versions.
* Tests for custom registry restrictions
* ENT-3121 restrict custom serialisation
* Remove redundant code
* Only count declared annotations
* Check annotation on superclasses, remove annotation from ByteArray
* Forbid custom serialization of primitive types
* Remove @CordaSerializable from another class that is always handled by custom serialisation
* Add log warnings to aid diagnosis of custom serialization issues
* Remove another annotation
* Remove another annotation
* Remove another annotation
* Remove another annotation
* Fixup api-current
* Fixup api-current
* KDocs on exceptions
* CORDA-2669 - pendingFlowsCount not in public API
Reintroduce `pendingFlowsCount` to public API (as deprecated). Advise
to use the `gracefulShutdown` command in the shell instead.
* CORDA-2669 - Add pendingFlowsCount to api-current.txt
CORDA-2595 - Fix test and api.
CORDA-2595 add test
CORDA-2595 fix tests
CORDA-2595 fix test and address code review comments
CORDA-2595 address code review comments
* First pass at fixing 2563.
* In memory KMS now maps keys to IDs.
* CreateDatabaseAndMockServices now creates a persistent key management service and a can take a persistent identity service, so now the external id mapping works for mock services.
* * Created a helper for mock services which allows the creation of a mock services with persistent identity management service key management service and vault.
* MockNode now uses persistent key management service - not sure why it didn't do before?
* * MockNode now uses BasicHSMKeyManagementService
* Updated api-current file
* Little fix required after rebase to master.
* Fixed broken test.
* Added informative error messages to UnsupportedOperationExceptions thrown by E2ETestKeyManagementService.
* Removed redundant private constructor for mock services from api-current.txt.
* Addressed Rick's comments.
With (Contract JARs) rolling upgrades the downgrade rule cannot be effectively check as the platform can't tell the difference between a transaction that's downgrading because of an attack, vs a transaction that's downgrading because Alice has upgraded but Bob hasn't yet. During a rolling upgrade we would expect state versions to fluctuate up and down as data gets read/written by a mix of nodes. With the feature as implemented Alice will upgrade and start trading with Bob. Bob will be able to read and process the states Alice sent him, but the moment he tries to consume such a state he will fail. This will result in cascading flow deaths and a hung business network the moment an upgrade starts.
Previous implementation was in LedgerTransaction and focused only on contract classes,
but every package matters.
Also fixes some exception types and does misc refactorings.
The ResponderFlowFactory parameter is not used and so removed. Also, instead of returning a Future it returns an Observable of responder flows, to support multiple invocations. And finally renamed to registerInitiatedFlow to stick with the existing naming strategy.
* Working version
* api-current.txt change to remove diff due to inheritance hierarchy change.
* api-current.txt change to remove warnings of abstract methods in new interfaces.
* Fix indent
Contract class version non-downgrade rule is check by LedgerTransaction.verify().
TransactionBuilder.toWireTransaction(services: ServicesForResolution) selects attachments for the transaction which obey non downgrade rule.
New ServiceHub method loadAttachmentConstraint(stateRef: StateRef, forContractClassName: ContractClassName? = null) retrieves the attachment contract related to transaction output states of given contract class name.
It exposed the internal SerializationEnvironment class. It previously documented that the SerializationEnvironmentRule JUnit rule should be used instead.
TransactionBuilder loads attachment using attachment storage instead of CordappLoader,
contract class version is now Integer (format and stored in db as Integer).
* Hash to signature constraints migration #1
* After rebase from Attachments Classloader commit.
* Simplified implementation without CZ whitelisting and relaxing the no-overlap rule slightly.
* Further simplification.
* Detailed implementation.
* Use fully loaded Cash contract jar for hash to signature constraints migration test.
Additional debug logging.
* Minor cleanup.
* Address PR review feedback.
* Minor fix.
* Fixes following rebase from master.
* Implemented `calculateEntriesHashes` to improve classloader validation performance.
* Address minor PR review comments.
* Added integration tests and some minor fixes.
* Minor fixes following rebase from master.
* Updates and fixes following integration testing.
* Added changelog entry.
* Fix broken unit tests.
* Fix compilation errors in DriverDSL tests after rebase from master.
* Minor fix to test cordapp jar signing using explicit keystore.
* Run hash-to-signature constraints integration test out of process using a non-validating notary.
* Address PR review feedback: contract version from database + other minor changes.
* Address final PR review feedback: remove signed attachment field from attachmentWithContext
* Resolve conflicts following rebase from master.
* Fix failing junit test.
* Fix Kryo serialization error (forgot to write new `version` identifier field)
* Removed redundant query carried over from previous commit.
* Added documentation.
* Fix test case where explicit Hash Constraint input and Signature Constraint output explicitly configured.
* Addressing PR review comments from SA.
* AttachmentQueryCriteria API: added wither methods and Java Unit tests.
* Fixed compilation error caused by Unit tests being in wrong module.
* Added @CordaInternal to canBeTransitionedFrom function.
* Minimized AttachmentClassloader overlap duplicates checking.
* Moved JarSignatureTestUtils and ContractJarTestUtils to internal pending clean-up and documentation before public release.
* Minor fix following rebase from master.
* Removed redundant checkNotNull(networkParameters) checks now that these are always passed into the main (non-deprecated) constructor.
* Remove capitalization.
* CORDA-2115: Notary whitelist verification changes
- For regular and contract upgrade transactions: check that the notary is in the network parameter whitelist
- For notary change transactions: check the the new notary is in the network parameter whitelist. This enabled support for network merging: the old notary doesn't have to be in the current network's notary whitelist for re-pointing old states to another notary.
These checks are done during transaction construction/verification and also by the non-validating notary.
* Address comments
* Remove stale todo
* Use notary whitelist of current network parameters for platform versoin 3
* Cleanup test
* Move `getHistoricNotary` to `HistoricNetworkParameterStorage` in `core.internal`
* Require `newNotary` to be notary on the network map during notary change
The httpPort parameter is fundamentally wrong as it gives the same monitoring port to each node!
Added JmxPolicy.defaultEnabled() to workaround the (incorrect) default false value of startJmxHttpServer.
* First pass
Update test.
Address review comments.
Added docs and kdocs.
Clean-up.
* Addressed review comments.
Changes to docsite.
* First pass at account service.
Added new hibernate schemas and liquibase scripts.
Added indexes to new tables.
Removed mock network.
Removed fresh key for external id from key management service.
Removed some redundant changes.
Rebase to master.
* Clean up.
* Added try/catch block as recommended by Andras.
* Removed accounts test to another branch.
Removed element collections from fungible states and linear states table.
Added a new state_parties table which stores x500 names and public key hashes.
Added a view which can be used to query by external ID.
* Removed try catch block. It's not required as the checkpoint serialiser deals with this.
Re-used existing DB session instead of creating a new session.
Entity manager auto flushes.
* Added java friendly api.
* This is a combination of 10 commits.
This is the 1st commit message:
Shortened table name.
This is the commit message #2:
Minor changes.
This is the commit message #3:
Common criteria parser now returns a predicate set which is concatenated to the predicate sets of sub-class criteria.
This is the commit message #4:
Fixed api compatibility issue.
Reverted some changes to reduce size of PR.
This is the commit message #5:
Multiple states can now be mapped to the same externalId.
Multiple externalIds can now be mapped to the same state.
This is the commit message #6:
Relaxed upper bound type constraint in some of the vault types.
This is the commit message #7:
Added comment to test.
This is the commit message #8:
Changed name of external id to public key join table.
Removed some comments/TODOs.
This is the commit message #9:
Added docs.
General clean up.
This is the commit message #10:
Fixed participants query bug and updated unit test.
* Removed unused code.
* [CORDA-2219] Show message if CorDapp already exists
* Update definition of `net.corda.core.node.services.Vault$StateMetadata` in `api-current.txt` or else diff tool gets very confused.
FinalityHandler is insecure in that it is open to receive any transaction from any party.
Any CorDapp targeting platform version 4 or above is required use the new c'tors which take in FlowSession objects to the counterpart flow. This flow must subcall ReceiveFinalityFlow to receive and record the finalised transaction.
Old CorDapps (with target platform version < 4) will continue to work as previously. However if there are no old CorDapps loaded then the node will disable FinalityHandler.
* Introduce public subset of config to tweak config via mock net work without exposing internal node config.
* Removal of functions exposing (internal) NodeConfiguration from the public test API
* Code review fixes
* Blank lines removed
* Documented mock network API change in upgrade notes.
* Updated documentation and API doc.
* More documentation/API doc
The entry point to the API has been simplified to just requireing a list of packages to scan, with sensible defaults provided for the metadata. Because of the wither methods, having parameters for the metadata (with default values) seems unnecessary. Also the ability to scan just individual classes has been made internal, as it seems unlikely app developers would need that level of control when testing their apps.
TestCordappImpl is a data class and thus acts as a natural key for the Jar caching, where previously the key was the package names. This fixes an issue where it was not possible to create two CorDapp Jars of the same package but different metadata.
* Fix to enable gradle build caching of test runs.
* Configure gradle build caching to be enabled.
* Generate dependency report and graph:
1) ./gradlew htmlDependencyReport
2) ./gradlew generateDependencyGraphCorda
* Strip out all Jacoco references to prevent Gradle Build Cache error:
"Caching disabled for task ':<module>:test': 'JaCoCo agent configured with `append = true`' satisfied"
* Revert jacoco back into jarFilter gradle build file.
Disable building deterministic modules (including jarFilter).
* Added Gradle build scan plugin.
* Set file encoding to prevent incorrect Gradle build cache keys across different machines.
https://guides.gradle.org/using-build-cache/#system_file_encoding
* Apply gradle build cache settings to buildSrc.
* Added targetted gradle build tasks to leverage cache.
* Updated URL's of several different test Gradle Cache instances.
* Updated CI batch build scripts.
* Updated script perms to be executable.
* Added CI smoke tests batch script.
* Use TestDev Labs gradle cache.
* Echo URL of Gradle Build Repository.
Echo exit status of each Gradle build command.
* Use environment variables to define Gradle Build Cache usage and URL.
* Customisation through parameters.
* Remove dependency graph generation plugin.
Align build cache settings across project and buildSrc gradle files.
* Remove buildSrc gradle build cache config.
* Revert definition of gradle build cache variables back to settings.gradle.
* Fix incorrect path.
* Aligned gradle build cache configuration across buildSrc and project.
* Minor updates to test scope.
* Minor updates to test scope.
* Update scripts to use GRADLE_HOME
* Exit on unset GRADLE_HOME
* Remove duplication following rebase from master.
* Remove fine-grained build task scripts.
* Added back Jacoco reporting.
* Revert jdk8u-deterministic module.
* Incorporating changes from PR review feedback.
* Workaround Jacoco issue associated with Gradle Build Cache test task.
* Update init script.
* Remove redundant build-scan declarations.
* Updates from PR review feedback.
* Remove GRADLE_HOME as no longer needed - everything is driven via gradle wrapper.
* Use CORDA prefix in system environment variable.
* Separate out Checkpoint serialization
* Update kdocs
* Rename checkpoint serialization extension methods
* Fix bungled rename
* Limit API changes
* Simplify CheckpointSerializationFactory
* Add CheckpointSerializationScheme to API checker
* CheckpointSerializationScheme should not be implemented
* Move checkpoint serialisation to internal package
* Remove CheckpointSerializationScheme from api-current
* Quarantine internal classes
* Remove checkpoint context from public API
* Remove checkpoint context from public API
* Fix test failures
* Completely decouple SerializationTestHelpers and CheckpointSerializationTestHelpers
* Remove CHECKPOINT use case
* Remove stray reference to checkpoint use case
* Fix broken test
* Add registerFlowFactory method to public test API
* Return CordaFuture rather than a plain Future
* Rename method
* Improve KDoc
* Hide internal interface with public wrapper
* Modify API current to include ResponderFlowFactory interface
* Take API definition change from generated file
* Note API change in changelog
* Remove all notion of message level retry.
* Introduce randomness into de-duplication IDs based on the session rather than the flow, in support of idempotent flows.
* CORDA-1494: Re-enable notarisation retries in the new state machine manager.
The original message-based retry approach does not work well with the new
flow state machine due to the way sessions are handled. We decided to move
the retry logic to flow-level: introduce RetryableFlow that won't have
checkpoints persisted and will be restarted after a configurable timeout
if it does not complete in time.
The RetryableFlow functionality will be internal for now, as it's mainly
tailored for the notary client flow, and there are many subtle ways it can
fail when used with arbitrary flows.
JPA/Hibernate entities need to impose the correct NULL/NOT NULL constraints on the database - whatever these correct values actually are.
API change: net.corda.core.schemas.PersistentStateRef fields (index and txId) are now non-nullable. Rationale: The fields were always effectively non-nullable - values were set from non-nullable fields of other objects. The class is used in context of database table Primary Key of for other entities and a database already imposes those columns as non-nullable (even if JPA annotation nullable=false was absent).
* Update api current from v3
* Update Cordapp Context as per CORDA-556
* Update Exception base classes as per CORDA-1334: port enterprise statemachine (#2964)
* Move notary service related classes and interfaces in core to internal (#2827)
* CordaRPCClient changes from CORDA-1099: Orchestrated clean shutdown from Shell (#2831)
* [CORDA-1264]: Ensure correct serialisation and masking for throwables raised by a node and propagated through RPC. (#2892)
* CORDA-973 Refactoring for serialization compression support (#2466)
* [CORDA-941]: Add NetworkParameters contract implementation whitelist. (#2580)
* Cleanup and improvements to the serialisation format of JacksonSupport (needed for CORDA-1238) (#3102)
* Existing internal exposures which need removing - see CORDA-1489
* revert to gradle plugins version of api-scanner
* Move notary service related classes and interfaces in core to internal, since we won't be able to stabilise the APIs for writing custom notary services any time soon (the docs already mention it). I left out the wire protocol related classes so we don't accidentally break it.
* ENT-1540: Make sure transactions with "expired" time windows get re-notarised correctly.
Currently the time window is checked before states are being passed to a uniqueness provider. If the time window is
invalid, the transaction will be rejected even if it has already been notarised, which violated idempotency.
For this reason the time window verification was moved alongside state conflict checks.
* Update API - this only affects custom notary interfaces
* Making Corda's JPA classes non-final and Serializable.
* Making Corda's JPA classes non-final and Serializable.
* Making Corda's JPA classes non-final and Serializable.
* Making Corda's JPA classes non-final and Serializable.
* CORDA-1208: Notary service should persist the notarisation request signature along with the committed input states.
This required modifying the uniqueness provider interface to accept the signature in addition to input states.
Until now the committed state log used to be stored as a map of (state reference -> (tranasction id, consuming party)).
Adding the serialized signature would mean inflating each state entry by around 700 bytes, which would be grossly inefficient.
Instead, two tables are now used: one for storing (state referece -> transaction id) map, and another for storing the notarisation
request details (transaction id, consuming party, date, signature).
* Update api - all of these changes are only related to custom notaries
The problem with the previous implementation is that the transaction would be deserialized with the schema specified
in the serialized form, but the calculation of the id would involve re-serializing properties using a local serialization context
which might produce a different result.
* Update from v3
* Remove CordaInternal attribute as this is removed by v4 api-scanner plugin
* Make SerializationContext @DoNotImplement
* Addition of CordappConfig to contstructor of CordappContext
* Alteration of getOffset/getSize properties in ByteSequence, OpaqueBytes & OpaqueBytesSubSequence
* Make network parameters on MockServices final
* Fix another non-serializable exception, add docs, fix a possible security issue.
* Update API definition to reflect methods added to make more exceptions serializable
* CORDA-986 and CORDA-985 CompositeKey and Signature verification performance fixes (#2467)
* CORDA-696: Create separate transaction types for contract upgrade transactions.
Add rationale around upgrade transactions
Move contract upgrade transaction resolution logic into internal until it's stabilised.
Throw a better exception when contract attachment not found
Default legacy contract constraint to always accepting - needs to be changed to whitelist constraint before merging
Introduce a new upgraded contract interface that allows specifying the legacy constraint.
Remove StateLoader, make all tx resolution functions take in ServicesForResolution
Contract upgrade transactions can handle whitelist by zone constraints
When creating a contract upgrade transaction, make sure the attachment of the old cordapp gets attached when using hash constraints.
Attachment lookup for a given contract class name only scans currently loaded cordapps, and we don't load old versions of cordapps.
CORDA-696: Update upgrade docs
CORDA-1010: Notary flow - clients now send a signature over a notarisation
request in addition to the transaction. This will be logged by the notary
to be able to prove that a particular party has requested the consumption
of a particular state.
* Update check api changes to look for internals
* Update several more uses of internal
* Make check-api-changes script filter out internal class usages
* Make CordaClock part of API
* Update api-current.txt
* Remove exclusion of nodeapi.internal
* Remove access to CordaPersistence from public api
* Don't expose DB Connection from StartedMockNode and remove unnecessary transaction from CustomVaultQueryTest
* Make internal tests that use need db access use InternalMockNetwork
* Make test certificates internal
* Address further review comments
* Revert some accidental changes to api-current.txt
* Address Shams' review comments
* Update Api Scanner to filter out CordaInternal attribute
* Update api-current.txt
* Remove superfluous brackets
* Add transaction to StartedMockNode
* More leaky transaction fixes
