* wip
* wip
* wip (need to review IEE comments)
* wip
* wip
* Small refactoring, fixed network-verifier's TestNotaryFlow
* Added command line option to explicitly enable hash agility support
* wip-do-not-push
* wip
* wip
* wip
* aligned merkletree/transaction hash algorithms
* wip
* Added mixed algorithm support for nodes vs leaves and corrected mixed algorithm tests
* moved global computeNonce and componentHash to DigestService
* added comment for failing test to fix
* wip
* Minor cleanups, added deprecated componentHash/computeNonce
* restored exploratory changes to failing SignedTransaction test
* cleaned up and minor rafactoring
* Fixed some tests with hardcoded hash algorithm
* some changes and cleanups following code review
* WIP commit before large change
* WIP Fixed 3 tests
* WIP removed direct references to randomSHA256() and sha256()
* Updated/added liquibase migrations to support larger hash algorithms
* Reviewed, cleanups, comments, fixes
* removing direct references to sha256()
* WIP verifying obligations test errors
* reviewing obligation/attachment issues with sha3_256
* Full review before PR - intermediate commits
* Reviewed and cleaned up
* Futher cleanup
* Fixed partial tree backward compatible json and cleanups
* all tests passing
* Removed couple of unused imports
* Reworked global componentHash function to avoid deprecated warnings
* replaced SHA3s with some alternate SHA2s
* Removed SHA3-256 and SHA3-512 references
* fixed some tests using non ubiquitous hash algorithms
* Fixed ABI compatibility (not for TransactionBuilder)
* Fixed ABI compatibility to TransactionBuilder
* couple of fixes
* fixed DigestService's randomHash
* Removed constructor with loosely typed args for private constructor of LedgerTransaction class (API removal)
* re-introduced LedgerTransaction deprecated ctor for deserialization
* Add possibility to load CustomMessageDigest bypassing JCA (#6798)
* Change api-current for DigestAlgorithm
* disable flaky tests
* addressed liquibase migration script versions
* Removed TODOs and cleanups
* relaxed privacy salt validation
* Fixed privacy salt test to comply with relaxed validation
* detekt and privacySalt validation
* diff cleanup
* diff cleanup
* removed unused import
* removed PrivacySalt's validateFor method and references
* removed invalid character
Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
* wip
* wip
* wip (need to review IEE comments)
* wip
* wip
* Small refactoring, fixed network-verifier's TestNotaryFlow
* Added command line option to explicitly enable hash agility support
* wip-do-not-push
* wip
* wip
* wip
* aligned merkletree/transaction hash algorithms
* wip
* Added mixed algorithm support for nodes vs leaves and corrected mixed algorithm tests
* moved global computeNonce and componentHash to DigestService
* added comment for failing test to fix
* wip
* Minor cleanups, added deprecated componentHash/computeNonce
* restored exploratory changes to failing SignedTransaction test
* cleaned up and minor rafactoring
* Fixed some tests with hardcoded hash algorithm
* some changes and cleanups following code review
* WIP commit before large change
* WIP Fixed 3 tests
* WIP removed direct references to randomSHA256() and sha256()
* Updated/added liquibase migrations to support larger hash algorithms
* Reviewed, cleanups, comments, fixes
* removing direct references to sha256()
* WIP verifying obligations test errors
* reviewing obligation/attachment issues with sha3_256
* Full review before PR - intermediate commits
* Reviewed and cleaned up
* Futher cleanup
* Fixed partial tree backward compatible json and cleanups
* all tests passing
* Removed couple of unused imports
* Reworked global componentHash function to avoid deprecated warnings
* replaced SHA3s with some alternate SHA2s
* Removed SHA3-256 and SHA3-512 references
* fixed some tests using non ubiquitous hash algorithms
* Fixed ABI compatibility (not for TransactionBuilder)
* Fixed ABI compatibility to TransactionBuilder
* couple of fixes
* fixed DigestService's randomHash
* Removed constructor with loosely typed args for private constructor of LedgerTransaction class (API removal)
* re-introduced LedgerTransaction deprecated ctor for deserialization
* Add possibility to load CustomMessageDigest bypassing JCA (#6798)
* Change api-current for DigestAlgorithm
* disable flaky tests
Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
* Do not use Security.addProvider(BouncyCastleProvider()) in tests, to avoid disruptions of other tests.
* Forcibly register security providers before starting Jimfs, to resolve a sequencing problem where Jimfs triggers loading of the SFTP filesystem provider, which in turn registers the standard BouncyCastle provider rather than the patched version Corda needs.
* INFRA-683 Move Corda OS release branch builds to serial (#6703)
Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
* INFRA-424 linux1 jenkinsfile
* INFRA-424 full run
* INFRA-424 bigger heap size
* Upgraded DJVM to handle BC - latest version of BC is a multirelease JAR.
When reading JKS keystore if a BC EdDSAPrivateKey is returned then swap for a net.i2p EdDSA private key.
* Temporary downgrade of BC
* Removed the BC EdDSA conversion
* INFRA-424 bigger heap size
* Upgrading Quasar to handle openJ9 different fields.
* INFRA-424: Handle lack of SUPPRESSED_SENTINEL in openj9.
* INFRA-424: If BCEdDSA public or private key is generated convert to net.i2p EdDSA form.
* INFRA-424 bigger heap size
* INFRA-424: On openJ9 only getting upto milli resolution.
* INFRA-424: Handle keystore returning a BCEdDSAPrivateKey.
* INFRA-424: Disable test on JDK11, as it requires the custom cordapp to generate JDK8 contract code, which we now check for.
* INFRA-424: Truncated time test to resolution of millis for openj9.
* INFRA-424 disabling log intensive tests until a fix is developed
* INFRA-424 one more test disabled
* INFRA-424: Disabled a couple of tests failing on openj9.
* INFRA-424: Disabling failing openj9 tests.
* INFRA-424: Disabling test failing on openj9.
* INFRA-424: Ignoring another flaky sleep test on openj9.
* INFRA-424 run integrationTests
* INFRA-424 set timeout to 4 hours
* INFRA-424: Cope with exception message from openj9.
* INFRA-424: Handle the coloured text characters openj9 adds.
* INFRA-424: Disabling test as it is generating JDK11 contract code under JDK11. Currently on JDK8 contract code allowed.
* INFRA-424: Commenting test out for openj9. Output of the processs thats read by the test is sometimes garbled.
* INFRA-424 switching to smoke tests
* INFRA-424 switching to slow integration tests
* INFRA-424 full run
* INFRA-424 moving jenkinsfile
* INFRA-424 removing references
* INFRA-424: Created common IS_OPENJ9 func for ignoring tests.
Co-authored-by: Schife <razvan.codreanu@r3.com>
Integrate `DBFlowException` with the rest of the checkpoint schema, so now
we are saving the flow's exception result in the database.
Making statemachine not remove `FAILED` flows' checkpoints from the
database if they are started with a clientId.
Retrieve the DBFlowException from the database to construct a
`FlowStateMachineHandle` future and complete exceptionally the flow's result
future for requests (`startFlowDynamicWithClientId`) that pick FAILED flows ,
started with client id, of status Removed.
On killing a flow the client id mapping of the flow gets removed.
The storage serialiser is used for serialising exceptions. Note, that if an
exception cannot be serialised, it will not fail and will instead be stored
as a `CordaRuntimeException`. This could be improved in future
changes.
* CORDA-3663 MockServices crashes when two of the provided packages to scan are deemed empty in 4.4 RC05
this happends when a given package is not found on the classpath. Now it is handled and an exception is thrown
* replace dummy package names in tests with valid ones
* allow empty package list for CustomCordapps and exclude those from the created jars
* detekt fix
* always true logic fix
* fix to check for empty packages instead of empty classes
* fix for classes and fixups
* logic refactor because of detekt stupidity
* PR related minor refactors
Introducing a new flow start method (`startFlowDynamicWithClientId`) passing in a `clientId`.
Once `startFlowDynamicWithClientId` gets called, the `clientId` gets injected into `InvocationContext` and also pushed to the logging context.
If a new flow starts with this method, then a < `clientId` to flow > pair is kept on node side, even after the flow's lifetime. If `startFlowDynamicWithClientId` is called again with the same `clientId` then the node identifies that this `clientId` refers to an existing < `clientId` to flow > pair and returns back to the rpc client a `FlowStateMachineHandle` future, created out of that pair.
`FlowStateMachineHandle` interface was introduced as a thinner `FlowStateMachine`. All `FlowStateMachine` properties used by call sites are moved into this new interface along with `clientId` and then `FlowStateMachine` extends it.
Introducing an acknowledgement method (`removeClientId`). Calling this method removes the < `clientId` to flow > pair on the node side and frees resources.
* CORDA-3769: Switched attachments class loader cache to use caffeine with original implementation used by determinstic core.
* CORDA-3769: Removed default ctor arguments.
* CORDA-3769: Switched mapping function to Function type to avoid synthetic method being generated.
* CORDA-3769: Now using a cache created from NamedCacheFactory for the attachments class loader cache.
* CORDA-3769: Making detekt happy.
* CORDA-3769: The finality tests now check for UntrustedAttachmentsException which will actually happen in reality.
* CORDA-3769: Refactored after review comments.
* CORDA-3769: Removed the AttachmentsClassLoaderSimpleCacheImpl as DJVM does not need it. Also updated due to review comments.
* CORDA-3769: Removed the generic parameters from AttachmentsClassLoader.
* CORDA-3769: Removed unused imports.
* CORDA-3769: Updates from review comments.
* CORDA-3769: Updated following review comments. MigrationServicesForResolution now uses cache factory. Ctor updated for AttachmentsClassLoaderSimpleCacheImpl.
* CORDA-3769: Reduced max class loader cache size
* CORDA-3769: Fixed the attachments class loader cache size to a fixed default
* CORDA-3769: Switched attachments class loader size to be reduced by fixed value.
* CORDA-3755: Switched attachments map to a WeakHashMap (#6214)
* CORDA-3772: Now specify source and target of 8 when compiling contract classes.
* CORDA-3651: addManifest now uses separate files for reading and writing. (#6026)
* CORDA-3651: addManifest now uses separate files for reading and writing.
* CORDA-3651: The jar scanning loader now closes itsself.
Co-authored-by: Adel El-Beik <adelel-beik@19LDN-MAC108.local>
Co-authored-by: Adel El-Beik <adelel-beik@19LDN-MAC108.local>
Removing the ability to initialise schema from the node config, and add a new sub-command to initialise the schema (that does not do anything else and exits afterwards).
Also adding a command line flag that allow app schema to be maintained by hibernate for legacy cordapps, tests or rapid development.
Patching up mock net and driver test frameworks so they create the required schemas for tests to work, defaulting schema migration and hibernate schema management to true to match pre-existing behaviour.
Modified network bootstrapper to run an initial schema set-up so it can register nodes.
* Bump OS release version 4.6
* CORDA-3755: Switched attachments map to a WeakHashMap
* CORDA-3755: Added explicit strong references to map key.
* CORDA-3755: Keeping detekt happy.
* CORDA-3755: Test a gc in verify.
* CORDA-3755: Making detekt happy.
* CORDA-3755: Suppress warnings for weak reference test.
* CORDA-3755: Fixing build failure with attachments.
* CORDA-3755: Rewrite based on Ricks input - now handles attachment already existing in map!
* CORDA-3755: Refactor WeakReference behaviour into AttachmentsHolderImpl and provide alternate version of this class for core-deterministic.
* CORDA-3755: Added more tests for WeakHashMap.
* CORDA-3755: Ignore the tests using System.gc keep for local testing only
* CORDA-3755: Adding comment to explain the ignored tests.
* Make AttachmentsHolderImpl package-private inside core-deterministic, just like it is inside core.
* CORDA-3755: Update assertions following review comments.
* CORDA-3755: Removing import
* CORDA-3755: Removed unused var.
* CORDA-3755: Reverting files that somehow got changed in rebase.
Co-authored-by: nargas-ritu <ritu.gupta@r3.com>
Co-authored-by: Chris Rankin <chris.rankin@r3.com>
* CORDA-3715: When loading cordapps now check that contract classes have class version between 49 and 52
* CORDA-3715: Now check class version when contract verification takes place.
* CORDA-3715: Making detekt happy with number of levels in func
* CORDA-3715: Make use of new ClassGraph release which provides class file major version number.
* CORDA-3715: Changed package name in test jars
* CORDA-3715: Use ClassGraph when loading attachments.
* CORDA-3715: Reverted file to 4.5 version
* CORDA-3715: Updating method to match non deterministic version.
* CORDA-3715: Added in default param.
* CORDA-3715: Adjusted min JDK version to 1.1
* CORDA-3715: Switching check to JDK 1.2
* CORDA-3715: Now version check SerializationWhitelist classes.
* CORDA-3715: Switched default to null for range.
* CORDA-3291 `isKilled` flag and session errors for killed flows
## Summary
Two major improvements have been worked on:
- A new flag named `isKilled` has been added to `FlowLogic` to allow
developers to break out of loops without suspension points.
- Killed flows now send session errors to their counter parties allowing
their flows to also terminate without further coordination.
Achieving these changes required a __fundamental__ change to how flows are
killed as well as how they sleep.
## `isKilled` flag
The addition of `FlowLogic.isKilled` allows flows to check if the
current flow has been killed. They can then throw an exception to lead
to the flow's termination (following the standard error pathway). They
can also perform some extra logic or not throw an exception if they
really wanted to.
No matter what, once the flag is set, the flow will terminate. Due to
timing, a killed flow might successfully process its next suspension
event, but it will then process a killed transition and terminate.
## Send session errors when killing a flow
A flow will now send session errors to all of its counter parties. They
are transferred as `UnexpectedFlowEndException`s. This allows initiated
flows to handle these errors as they see fit, although they should
probably just terminate.
## How flows are killed
### Before
Originally we were relying on Quasar to interrupt a flow's fiber, we
could then handle the resulting `InterruptedException`. The problem with
this solution is that it only worked when a flow was already suspended
or when a flow moved into suspension. Flows stuck in loops did not work.
### After
We now *do not* use Quasar to interrupt a flow's fiber. Instead, we
switch `FlowStateMachine.isKilled` to true and schedule a new event.
Any event that is processed after switching this flag will now cause a
`KilledFlowTransition`. This transition follows similar logic to how
error propagation works. Note, the extra event allows a suspended flow
to be killed without waiting for the event that it was _really_ waiting
for.
This allows a lot of the tidy up code in `StateMachineManager.killFlow`
to be removed as tidy up is executed as part of removing a flow.
Deleting a flow's checkpoint and releasing related soft locks is still
handled manually in case of infinite loops but also triggered as part
of the actions executed in a transition.
This required flow sleeping to be changed as we no longer rely on
quasar.
## How flows now sleep
The reliance on Quasar to make a flow sleep has been removed.
Instead, when a flow sleeps we create a `ScheduledFuture` that is
delayed for the requested sleep duration. When the future executes it
schedules a `WakeUpFromSleep` event that wakes up the flow... Duh.
`FlowSleepScheduler` handles the future logic. It also uses the same
scheduled thread pool that timed flows uses.
A future field was added to `StateMachineState`. This removes the
need for concurrency control around flow sleeps as the code path does
not need to touch any concurrent data structures.
To achieve this:
- `StateMachineState.future` added as a `var`
- When the `ScheduledFuture` is created to wake up the flow the passed
in `StateMachineState` has its `future` value changed
- When resumed `future` and `isWaitingForFuture` are set to `null` and
`false` respectively
- When cancelling a sleeping flow, the `future` is cancelled and nulled
out. `isWaitingForFuture` is not changed since the flow is ending anyway
so really the value of the field is not important.
* Stop capturing 'FlowLogic' references in flowAsyncOperation;
Creating concrete classes removes the implicit reference to FlowLogic (as this) being included in the anonymous object
* Modify test code so that lambdas no longer get implicit references to their enclosing 'FlowLogic'
* Stop capturing 'FlowLogic' references in flowAsyncOperation;
Creating concrete classes removes the implicit reference to FlowLogic (as this) being included in the anonymous object
* Modify test code so that lambdas no longer get implicit references to their enclosing 'FlowLogic'
* CORDA-3669 Do not execute `ExecuteAsyncOperation` multiple times
When a `FlowExternalOperation` or `FlowExternalAsyncOperation` executes
and completes a flag (`isFlowResumed`) is switched to true.
This flag was used inside of `DoRemainingWorkTransition` to decide
whether to skip over the execution of an event.
Since this flag was being switched to true when the external operation's
future completed, it was possible for _unexpected_ events to be placed
in the fiber's queue that would retrigger the
`FlowIORequest.ExecuteAsyncOperation`, that is held as the checkpoint's
next `FlowIORequest`to process.
By using the existing `StateMachineState.isTransactionTracked` (and
renaming it to `isWaitingForFuture`) we can decide to not process the
`FlowIORequest.ExecuteAsyncOperation` if it has already been called
before. This moves this code path in line with
`FlowIORequest.WaitForLedgerCommit`.
Random `DoRemainingWork` events can now be pushed to the fiber's queue
without causing the `FlowIORequest.ExecuteAsyncOperation` to execute
again.
* CORDA-3668: Prevent AttachmentURLConnection from assigning ALL_PERMISSIONS to classes inside an attachment.
* Strengthen the comment warning about AttachmentURLConnection.getPermission.
* NOTICK Increase timeouts in external operation tests
It seems these tests are timing out when they shouldn't be when running
on the CI servers.
To resolve this, the timeouts of `getOrThrow` calls have been increased
to a minute.
Reliance on timeouts has also been reduced through using locks. This
should speed up tests that don't need to wait for the timeout anymore to
progress.
* NOTICK Remove hospital checking code
* NOTICK Change locking code
