Commit Graph

3345 Commits

Author SHA1 Message Date
Dan Newton
6b5e76f87a Merge branch 'release/os/4.6' into dan/os-4.6-to-4.7-merge-2020-10-30 2020-10-30 16:08:15 +00:00
Dan Newton
72c359b165
ENT-5886 Parameterised queries in DBCheckpointStorage (#6768)
Direct SQL was causing issues with the saving of instants not
maintaining their timezones properly.

Changing to parameterised queries fixed this issue.

Also changed other queries to do the same, since it is good practice to
query in this way.
2020-10-22 13:38:39 +01:00
Denis Rekalov
c9056f171b
CORDA-4062: Bump platform version to 9 for safe identity key rotation (#6777) 2020-10-20 12:31:47 +03:00
Denis Rekalov
4193adf6fd
CORDA-3979: Support for multiple trust roots (#6772) 2020-10-20 12:18:00 +03:00
Denis Rekalov
401d8b8856
ENT-5888: Resurrect node_hash_to_key (#6776) 2020-10-19 12:56:12 +03:00
Denis Rekalov
551b3f0811
CORDA-4054: combine different identities of the same notary after its key rotation (#6734) 2020-10-16 13:53:04 +01:00
Adel El-Beik
e7df5818e4
INFRA-424 CORDA-4058: Disable flaky tests on openj9 (#6774) 2020-10-16 13:03:25 +01:00
Ross Nicoll
1ccd84bbcb Merge remote-tracking branch 'origin/release/os/4.6' into rni/notick/merge-46-47 2020-10-14 15:52:56 +01:00
Ross Nicoll
64dc73c7b1
ENT-5679 Correct instability of linear builds (#6722) (#6770)
Disable test which triggers OOM
2020-10-14 13:51:35 +01:00
Ross Nicoll
eb65cdc08a
CORDA-4045 Adjust test to remove scope for race conditions (#6769)
Change assertion in `Restart does not set senderUUID` to verify a single message has a sender UUID set, rather than the last to be recorded as sent has no sender UUID.
2020-10-14 13:44:41 +01:00
Ben McMahon
12f1603822
INFRA-504 Change to use an out of process node (#6538)
* Change to use an out of process node
* Remove exception swallowing and add package for test constants to test process
2020-10-12 17:00:39 +01:00
Denis Rekalov
cdd725e79c
CORDA-3972: Support for node identity rotation in IdentityService (#6752) 2020-10-12 18:01:32 +03:00
Ramzi El-Yafi
e2efbaea35
AG-341 Notary backpressure mechanism improvements (#6762)
Notary backpressure mechanism improvements
2020-10-08 09:12:45 +01:00
Ben McMahon
2790e945c6
INFRA-687 Change to use an out of process node (#6696)
* Change to use an out of process node
* Remove exception swallowing and add package for test constants to test process
2020-10-05 18:12:31 +01:00
Denis Rekalov
e837bb0ca8 Merge branch 'release/os/4.5' into denis/merge_os_4.5_to_4.6 2020-10-02 21:43:57 +01:00
Dan Newton
dcc65b1d65
NOTICK Change class byteman references (#6749)
Scripts now reference the class `ProxyConnection` which is an implementation of `Connection`, to address JDK 11 compatibility issue.
2020-09-25 15:42:14 +01:00
Ross Nicoll
5c1ca20dae Detekt 2020-09-25 10:38:46 +01:00
Ross Nicoll
5a93f848b0 Merge remote-tracking branch 'origin/release/os/4.4' into rni/notick-merge-44-45 2020-09-25 09:57:10 +01:00
Ross Nicoll
8048cb4966
CORDA-3845: Update BC, log4j, slf4j (#6699) (#6742)
* CORDA-3845: Update BC, log4j, slf4j (#6699)

* CORDA-3845: Update BC to 1.64
* CORDA-3845: Upgraded log4j to 2.12.1
* We can remove the use of Manifests from the logging package so that when _it_ logs it doesn't error on the fact the stream was already closed by the default Java logger.
* Remove the logging package as a plugin
* latest BC version
* Remove old test
* Fix some rebased changes to log file handling
* Update slf4j too

Co-authored-by: Ryan Fowler <fowlerrr@users.noreply.github.com>
Co-authored-by: Adel El-Beik <adel.el-beik@r3.com>
2020-09-25 09:54:59 +01:00
Jerome Gerakis
25ab7667a5
backport of bugfix ENT-5752 into OS 4.3 (#6724) (#6744) 2020-09-24 14:30:35 +01:00
Kyriakos Tharrouniatis
60309114ec NOTICK Properly merge the two RPCPermissionResolver classes as they diverged 2020-09-22 20:57:49 +01:00
Kyriakos Tharrouniatis
5ff95efbce NOTICK Remove unused imports 2020-09-22 19:14:12 +01:00
Kyriakos Tharrouniatis
a274655e7b NOTICK Delete duplicate class after merge 2020-09-22 18:58:56 +01:00
Kyriakos Tharrouniatis
98bf8a7996 NOTICK Remove eliminated code that was re-introduced from merge 2020-09-22 18:53:25 +01:00
Kyriakos Tharrouniatis
2b0d58c7aa NOTICK Fix callsite after merge 2020-09-22 18:42:31 +01:00
Kyriakos Tharrouniatis
88386f38a9 NOTICK Remove leftover var after merge 2020-09-22 18:38:33 +01:00
Ryan Fowler
ed9f6f0ce1 Merge branch 'release/os/4.6' into rfowler-os-4.6-ent-4.6-20200922 2020-09-22 11:42:00 +01:00
Ross Nicoll
f97bb594b2 Merge remote-tracking branch 'origin/release/os/4.3' into rni/notick/merge-43-44 2020-09-18 16:42:26 +01:00
Ross Nicoll
2064a572ba
CORDA-4034 ENT-5679 CORDA-3845 Correct instability of linear builds (#6722)
* CORDA-4034 Reduce forkEvery to 15 to attempt to mitigate memory leak.
* ENT-5679 Disable test which triggers OOM
* Run tests on two Jenkins agents
* Fixed processing JUnit test results by Allure
* Add timeouts to VaultObserverExceptionTest
* Revert "CORDA-3845: Update BC, log4j, slf4j (#6699)" to eliminate introduced memory leaks

Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
2020-09-18 00:23:16 +01:00
Adel El-Beik
01cd8d6415 Merge remote-tracking branch 'origin/release/os/4.5' into adel/merge-from-4.5-17-Sept 2020-09-17 10:34:01 +01:00
Denis Rekalov
5c6acb0909
CORDA-3968: Prepare keystore handling for certificate rotation [OS] (#6698)
* CORDA-3968: Move keystore logic to KeyStoreHandler.

* CORDA-3967: Stop generating node legal identity in runtime.

* CORDA-3969: Remove Node CA access at node normal runtime.

* CORDA-3968: Fix handling of absent keystore file and wrong password.
2020-09-17 10:15:46 +01:00
Denis Rekalov
acb82f77b4
CORDA-3974: NetworkMapCache should link entries with different public keys by X.500 name (#6711) 2020-09-17 09:25:42 +01:00
bpaunescu
2498bb3ae1
[ENT-5752]-Fall back on NM cache if NM service is unreachable during node start-up (#6716) 2020-09-16 18:13:44 +01:00
Ross Nicoll
cc14c8e53a
CORDA-4034 Rebuild AuthDBTests so H2 shuts down correctly (#6718)
* CORDA-4034 Rebuild AuthDBTests so H2 shuts down correctly
2020-09-16 16:44:31 +01:00
Ryan Fowler
afd3876faf
CORDA-4036: Put the identity failures on the same log so they aren't missed. (#6717) 2020-09-16 16:37:16 +01:00
Dan Newton
bd7b96e816
ENT-5768 startFlowWithClientId permissions (#6708)
Do not let a user reattach to a flow started by another user.

Reattaching to a flow using startFlowWithClientId for a flow not
started by the current user throws a PermissionException

Reattaching to a flow using reattachFlowWithClientId for a flow not
started by the current user returns null.

finishedFlowsWithClientIds does not return flows started by other
users.

Normal rpc permissions around startFlowWithClientId and
startFlowDynamicWithClientId has also been added.

To allow admins to remove client ids as well as be able to see all the
client ids on the node, admin versions have been added that bypass the
user restrictions. These can be permitted via rpc to only provide
their usage to admins.
2020-09-16 16:11:06 +01:00
Ross Nicoll
0226f7ccb9
CORDA-4034 Shut down node in AuthDBTests (#6714) 2020-09-16 13:10:54 +01:00
Christian Sailer
b746d5cb22
ENT-5759 Give the failing test CorDapp a different package name so it does not get added automatically to all tests. (#6712) 2020-09-15 16:52:08 +01:00
Waldemar Zurowski
ebd36c22e7 Removed unused import 2020-09-15 15:42:17 +02:00
Kyriakos Tharrouniatis
342041c737
NOTICK - Remove unused code (#6694) 2020-09-15 13:53:57 +01:00
Waldemar Zurowski
99841dcaeb Merge branch 'release/os/4.4' into INFRA-683-os-45-linear-builds 2020-09-15 14:27:34 +02:00
Ross Nicoll
bcce0e9519
INFRA-683 Move OS 4.4 release branch builds to linear (#6705)
* Do not use Security.addProvider(BouncyCastleProvider()) in tests, to avoid disruptions of other tests.
* Forcibly register security providers before starting Jimfs, to resolve a sequencing problem where Jimfs triggers loading of the SFTP filesystem provider, which in turn registers the standard BouncyCastle provider rather than the patched version Corda needs.
* INFRA-683 Move Corda OS release branch builds to serial (#6703)

Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
2020-09-15 13:18:46 +01:00
Ross Nicoll
c0d354e97a
CORDA-3845: Update BC, log4j, slf4j (#6699)
* CORDA-3845: Update BC to 1.64
* CORDA-3845: Upgraded log4j to 2.12.1
* We can remove the use of Manifests from the logging package so that when _it_ logs it doesn't error on the fact the stream was already closed by the default Java logger.
* Remove the logging package as a plugin
* latest BC version
* Remove old test
* Fix some rebased changes to log file handling
* Update slf4j too

Co-authored-by: Ryan Fowler <fowlerrr@users.noreply.github.com>
Co-authored-by: Adel El-Beik <adel.el-beik@r3.com>
2020-09-15 11:59:46 +01:00
Kyriakos Tharrouniatis
6829be7ae9
ENT-5754 - RPC startFlow cannot reattach to existing client id flows when flows draining mode is enabled (#6707)
* Make existing client id flows re-attachable via rpc 'startFlow' when flows draining mode is enabled

* Fix detekt issue

* Remove unneeded/ unreached waiting on flow's return future
2020-09-15 09:04:17 +01:00
Christian Sailer
879e93ed47
ENT-5760 Log db configuration issues (#6706)
ENT-5760 Return config error instead of throwing an exception so that the config error logging mechanism works.
2020-09-14 15:55:38 +01:00
Ross Nicoll
b3d9d1291f
INFRA-683 Move Corda OS release branch builds to serial (#6703)
Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
2020-09-14 11:29:42 +01:00
Christian Sailer
ea423215c2
ENT-5759 check for app schema migration (#6701)
* Fix behaviour for missing schema outside devMode plus test
2020-09-14 11:25:18 +01:00
Ross Nicoll
ba802c6619
CORDA-4028 Forcibly register security providers (#6702)
Forcibly register security providers before starting Jimfs, to resolve a sequencing problem where Jimfs triggers loading of the SFTP filesystem provider, which in turn registers the standard BouncyCastle provider rather than the patched version Corda needs.
2020-09-12 23:52:52 +01:00
Ross Nicoll
df7b1bccd6
NOTICK Do not use Security.addProvider(BouncyCastleProvider()) in tests (#6250) (#6700)
Do not use Security.addProvider(BouncyCastleProvider()) in tests, to avoid disruptions of other tests.

Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
2020-09-11 16:50:12 +01:00
Dan Newton
7ec59da318
ENT-5750 Keep killed flows that were started with client ids (#6697)
If a flow is started with a client id, do not delete it from the
database. Instead, set the status to `KILLED` and store a
`KilledFlowException` in the database.
Keeping it around allows it to be reattached to using the same
mechanisms as active, completed or failed flows. Furthermore, without
this change it could be possible for a flow to be triggered again if
killed while the reconnecting rpc client is being used.
If there is no client id related to the flow, then kill flow keeps its
original behaviour of deleting all traces of the flow.
Flows cannot be killed if they are `COMPLETED`, `FAILED` or `KILLED`
already.
Logs have been added if requests to kill flows in these statuses are
made.
Do not update the status + persist the exception if the flow was killed
during flow initialisation (before persisting its first checkpoint).
Remove the client id mapping if the flow was killed and did not persist
its original checkpoint.
2020-09-10 11:58:55 +01:00
Ryan Fowler
6f2cac146c
ENT-5492: Don't do reconnect logic on illegal argument for attachments (#6693)
* ENT-5492: Don't do reconnect logic on illegal argument for attachments

* Use a dedicated exception for missing attachments.
2020-09-09 12:20:16 +01:00
Rick Parker
fd341c7e48
INFRA-670 Stop shutting down the database before some services that use it (#6690)
* INFRA-670 Stop shutting down the database before some services that use it.

* Test fixes
2020-09-08 08:34:19 +01:00
Dan Newton
45fe3f3aef
CORDA-4026 Make early end session test not flakey (#6691) 2020-09-07 16:23:37 +01:00
Yash Nabar
d1735b8c42
EG-3456: Make network parameters path configurable (#6531)
* Update docker image

* EG-3456: Make network parameters path configurable

* EG-3456: Remove filename from network parameters path property

* EG-3456: Add tests for network parameters and node configuration

* Revert docker image update

* EG-3456: Resolve network parameters path against base directory

* EG-3456: Add tests for network parameters path, increase test timeout limits

* EG-3456: Replace baseDirectory argument with networkParametersPath in NetworkParametersReader

* EG-3456: Refactor tests to reuse node configuration object

* EG-3456: Pass networkParametersPath as argument instead of baseDirectory
2020-09-07 09:49:37 +01:00
Dan Newton
64dc94bd18
ENT-5666 Extract shutdown events when retrying a flow (#6684)
* ENT-5666 Extract shutdown events when retrying a flow

When a flow is retrying, only a select set of events are transferred
over to the new fiber. Shutdown events were not included in this set.

This meant that if a flow retries when an `Event.SoftShutdown` is in its
queue, it will never process it. This causes the node to hang, as the
node awaits `liveFibers`, which never reaches 0 because the shutdown
event is never processed.

To resolve this, `Event.SoftShutdown` is added to the set of events to
extract and reschedule.

* ENT-5666 Don't schedule extra shutdown event

When a flow is stopped by an `Event.SoftShutdown` it will eventually
reschedule another shutdown even when it reaches `SMM.removeFlow`. It
won't actually be processed because the flow returns an abort
continuation. But, it does look odd.

Therefore, it now does nothing instead since that is what it was
implicitly doing.
2020-09-04 15:44:14 +01:00
Ross Nicoll
0c16957622 Merge remote-tracking branch 'origin/release/os/4.6' into rni/notick/46-47-merge-2020-09-04 2020-09-04 14:15:47 +01:00
Rick Parker
e937889b64
Merge pull request #6678 from corda/jzd/merge-os4.6-to-os4.7-2020-09-01
NOTICK: Merge OS 4.6 into OS 4.7
2020-09-04 14:08:25 +01:00
Jonathan Locke
e562c5828b
INFRA-604: Switch tests to use database snapshots (#6671)
Created a database snapshot of a clean Corda OS 4.5.1 database, which can now be used for testing by both the node driver and mock network.

The MockNetwork was changed from using an in memory database to using an on disk database, and makes use of the snapshot to speed up setup times.

The Node Driver was changed from defaulting to an in-memory database to defaulting to an on-disk database. Tests that do not specify the type of database to use will thus use an on-disk database. Tests that opt in for an in-memory database will continue to use an in-memory database as before.

The database snapshots are copied to the node directory inside the build folder, therefore, they should be cleaned up after a build.

Co-authored-by: Ross Nicoll <ross.nicoll@r3.com>
2020-09-03 14:43:55 +01:00
conalsmith-r3
3ce78b813d
EG-3458 - Missing onError implementation message logged in the node l… (#6602)
* EG-3458 - Missing onError implementation message logged in the node log file with ERROR level - the changes made on top of 4.6 branch

* EG-3458 - Reducing the number of logs by only logging on first consecutive error. Retry without completing the observable

* EG-3458 - Refactor the overly complex method to smaller functions

* EG-3458 - Reducing the number of functions in the class
2020-09-02 15:21:48 +01:00
Adel El-Beik
9962c9085d
INFRA-424: Merge openj9 updates into 4.6 (#6683)
* INFRA-424 linux1 jenkinsfile

* INFRA-424 full run

* INFRA-424 bigger heap size

* Upgraded DJVM to handle BC - latest version of BC is a multirelease JAR.
When reading JKS keystore if a BC EdDSAPrivateKey is returned then swap for a net.i2p EdDSA private key.

* Temporary downgrade of BC

* Removed the BC EdDSA conversion

* INFRA-424 bigger heap size

* Upgrading Quasar to handle openJ9 different fields.

* INFRA-424: Handle lack of SUPPRESSED_SENTINEL in openj9.

* INFRA-424: If BCEdDSA public or private key is generated convert to net.i2p EdDSA form.

* INFRA-424 bigger heap size

* INFRA-424: On openJ9 only getting upto milli resolution.

* INFRA-424: Handle keystore returning a BCEdDSAPrivateKey.

* INFRA-424: Disable test on JDK11, as it requires the custom cordapp to generate JDK8 contract code, which we now check for.

* INFRA-424: Truncated time test to resolution of millis for openj9.

* INFRA-424 disabling log intensive tests until a fix is developed

* INFRA-424 one more test disabled

* INFRA-424: Disabled a couple of tests failing on openj9.

* INFRA-424: Disabling failing openj9 tests.

* INFRA-424: Disabling test failing on openj9.

* INFRA-424: Ignoring another flaky sleep test on openj9.

* INFRA-424 run integrationTests

* INFRA-424 set timeout to 4 hours

* INFRA-424: Cope with exception message from openj9.

* INFRA-424: Handle the coloured text characters openj9 adds.

* INFRA-424: Disabling test as it is generating JDK11 contract code under JDK11. Currently on JDK8 contract code allowed.

* INFRA-424: Commenting test out for openj9. Output of the processs thats read by the test is sometimes garbled.

* INFRA-424 switching to smoke tests

* INFRA-424 switching to slow integration tests

* INFRA-424 full run

* INFRA-424 moving jenkinsfile

* INFRA-424 removing references

* INFRA-424: Created common IS_OPENJ9 func for ignoring tests.

Co-authored-by: Schife <razvan.codreanu@r3.com>
2020-09-02 14:35:30 +01:00
Nikolett Nagy
6113cbbd39
verify with NETWORK_PARAMETERS role and test (#6628) 2020-09-02 09:48:01 +01:00
Ross Nicoll
d928c29019 Merge remote-tracking branch 'origin/release/os/4.5' into rni/notick/os-45-46-docs-db 2020-09-01 17:00:00 +01:00
Ross Nicoll
aa1b74d510 Merge remote-tracking branch 'origin/release/os/4.4' into rni/notick/os-44-45-db 2020-09-01 16:06:03 +01:00
Ross Nicoll
6945426ef1
NOTICK Robustness improvements for DB race condition test (#6556) (#6677)
Switch away from log monitoring to verify that DB race conditions are reported correctly, in an attempt to resolve a test instability issue.
2020-09-01 16:04:12 +01:00
Joseph Zuniga-Daly
eecc294820 Merge OS 4.6 OS 4.7
Conflicts:
- .ci/api-current.txt - Kept existing version on 4.7 as I'm about to regenerate the file
2020-09-01 15:28:37 +01:00
Adel El-Beik
14e23430c0
CORDA-4003: Now support + in CorDapp filenames (#6673)
* CORDA-4003: Now cope with file: prefix not being in class path element.

* CORDA-4003: Switched to new URL type filter.

* CORDA-4003: Switched to a URL comparison. In the string comparison the scheme was removed in latest version of classgraph.

* CORDA-4003: Moved to latest version of classgraph that has support for + in filenames.

* CORDA-4003: Switched to accept version of the deprecated classgraph methods.
2020-09-01 10:30:49 +01:00
Dries Samyn
d08b62da39
Infra-656 - NoSuchFileException in NodeInfoWatcher fix (#6672)
* Filter out tmp files

* Ignore .tmp files

* Ignore .tmp files

* Remove unused import
2020-08-27 16:41:25 +01:00
David Golding
9a018e7bee
Updated CLI usage error (#6661) 2020-08-27 12:15:21 +01:00
Dries Samyn
0cfe6b3084
Filter out tmp files (#6668) 2020-08-27 08:48:08 +01:00
Joseph Zuniga-Daly
8fd4fd1bf8 Merge OS 4.6 into OS 4.7 2020-08-26 11:09:02 +01:00
Dan Newton
99f835bb4a
CORDA-3995 Redeliver external events if number of suspends differs (#6646)
* CORDA-3995 Redeliver external events in number of suspends differs

When retrying a flow, only redeliver external events held in a flow's
pending deduplication handlers if there is a difference in the
`numberOfSuspends` on the `currentState`'s checkpoint or the checkpoint
in the database.

If the checkpoint committed, but the flow retried, then the external
events would have been persisted to the database as part of the same
transaction. Therefore there is no need to replay them, as they have
already been processed as saved as part of the checkpoint.

This change is only relevant when the checkpoint persists, but the flow
still needs to retry after this occurs (within the same
transition/event).

* CORDA-3995 Redeliver external events in number of commits differs

When retrying a flow, only redeliver external events held in a flow's
pending deduplication handlers if there is a difference in the
`numberOfCommits` on the `currentState`'s checkpoint or the checkpoint
in the database.

If the checkpoint committed, but the flow retried, then the external
events would have been persisted to the database as part of the same
transaction. Therefore there is no need to replay them, as they have
already been processed as saved as part of the checkpoint.

This change is only relevant when the checkpoint persists, but the flow
still needs to retry after this occurs (within the same
transition/event).

* CORDA-3995 Redeliver external events if number of commits differs

When retrying a flow, only redeliver external events held in a flow's
pending deduplication handlers if there is a difference in the
`currentState`'s `numberOfCommits` or the `numberOfCommits`
the checkpoint has recorded in the database.

If the checkpoint committed, but the flow retried, then the external
events would have been persisted to the database as part of the same
transaction. Therefore there is no need to replay them, as they have
already been processed as saved as part of the checkpoint.

This change is only relevant when the checkpoint persists, but the flow
still needs to retry after this occurs (within the same
transition/event).

* Add @Suspendable to a test flow.

I am surprised this worked at all.

* Fix a few minor things based on review.

Co-authored-by: Will Vigor <william.vigor@r3.com>
2020-08-25 11:54:55 +01:00
Walter Oggioni
49f598308b
added checkpoints debug shell command (#6574) 2020-08-25 10:10:25 +01:00
Matthew Nesbit
57f4858a29
Merge pull request #6638 from corda/nnagy-os-4.6-os-4.7-20200814
NOTICK - OS 4.6 to OS 4.7 merge 20200814
2020-08-24 15:30:39 +01:00
William Vigor
133e6fe39a
CORDA-4001 Verify Paused Checkpoints on Node Startup (#6655)
We should check that PAUSED Checkpoints can be deserialised on node
startup as we do for RUNNABLE checkpoints. Otherwise a user might
get into trouble if they update the CorDapp.
2020-08-24 12:59:51 +01:00
Dan Newton
39491e8835
NOTICK - Fix flow framework tests (#6650)
Clear hooks between tests.
2020-08-20 11:20:35 +01:00
Nikolett Nagy
ab21833390
Re-add lost fix (#6652) 2020-08-20 09:23:03 +01:00
Kyriakos Tharrouniatis
f428836f33 Clear hospital hooks after tests; should be fixing tests hanging 2020-08-20 09:22:56 +01:00
nikinagy
8b900c7c08 Re-add lost fix 2020-08-19 18:57:39 +01:00
Kyriakos Tharrouniatis
5d24b70227
CORDA-3998 - Commit db transaction before starting flows at node start (#6647)
Delay the firing of future/callback chain in 'AbstractNode.start' to after db transaction commit
2020-08-19 15:56:12 +01:00
Kyriakos Tharrouniatis
771fade972 Clear flow hospital hooks - commented out for now 2020-08-19 14:24:54 +01:00
Dan Newton
742312b85a
NOTICK Do not replace stacktrace for local errors (#6635)
We should not overwrite the stack trace of local errors thrown by
`FlowContinuation.Throw` as it hides the real cause of the error.

Exceptions received from peer nodes are still overwritten.
2020-08-18 12:05:05 +01:00
Viktor Kolomeyko
9fc896beb3
CORDA-3960: Port MultiRPCClient to OS (#6644)
* CORDA-3960: Port MultiRPCClient to OS

* CORDA-3960: Carefully restore serialisation environment in `MultiRpcClientTest` to eliminate side effects on other tests

* CORDA-3960: Move ThreadDumpUtils.kt to `core-utils`
2020-08-18 10:35:19 +01:00
Kyriakos Tharrouniatis
abfe83626f
CORDA-3809 - Remove tests applicable only in ENT (#6645) 2020-08-17 16:41:45 +01:00
Dan Newton
949489a117
CORDA-3994 Retry errors in flow init started with client ids (#6643)
Flows that were started with a client id would hang because it would
retrieve the existing flow's future and wait for it to finish. But,
because the flow has failed its flow init and not saved its initial
checkpoint, it is relying on `startFlow` to start the flow again (by
redelivering the start flow external event).

`FlowWithClientIdStatus` now holds the flow id that it is related to.
This is then checked in `startFlow`. If a matching client id is found
for a flow start, it then checks the flow id as well. If the flow id
matches, then it lets the `startFlow` call continue, allowing it to
actually start the flow again (how a flow without a client id would
retry in this situation).
2020-08-17 10:35:22 +01:00
Dan Newton
854e6638ff
CORDA-3881 Get all finished flows with client ids (#6580)
Return map of `clientId` -> success/fail
2020-08-17 10:27:32 +01:00
Ross Nicoll
f1b7bc9dcb
CORDA-3993 Correct mock network handling (#6642)
Correct mock network handling in VaultQueryJoinTest so it does not block other mock networks.
2020-08-15 12:28:03 +01:00
Dan Newton
8534aad3b1
ENT-5672 Know if paused flow is hospitalized (#6641)
Missing change from the original commit.
2020-08-15 10:08:37 +01:00
William Vigor
be6b76ff89
ENT-5684 Reconnect flow's progress tracker when unpausing (#6640)
Previously we were just throwing this away when pausing, meaning
updates would not be passed back to the user.

The progress tracker is now maintained in the `NonResidentFlow`
allowing it to be reused in the flow when it is retried.
2020-08-14 21:11:48 +01:00
William Vigor
32cb085a53
ENT-5672 Know if a paused flow is hospitalized (#6639)
* ENT-5672 Update database query to get paused flows which have previously been hospitalised

* NOTICK Remove unneeded check if a database exception was removed when switching a flow to RUNNABLE since we were to remove it anyway
2020-08-14 20:07:00 +01:00
Dan Newton
c4027e23bf
ENT-5649 Always load from db when flow retries (#6637)
Always attempt to load a checkpoint from the database when a flow
retries.

This is to prevent transient errors where the checkpoint is committed to
the database but throws an error back to the node. When the node tries
to retry in this scenario, `isAnyCheckpointPersisted` is false, meaning
that it will try to insert when it tries to save its initial checkpoint
again.

By loading from the existing checkpoint, even though it doesn't
really use it because it is `Unstarted`, the flag gets put into the
right state and will update rather than insert later on.
2020-08-14 17:42:19 +01:00
nikinagy
c9b2fa11cd Merge branch 'release/os/4.6' into nnagy-os-4.6-os-4.7-20200814
# Conflicts:
#	client/rpc/src/main/kotlin/net/corda/client/rpc/internal/ReconnectingCordaRPCOps.kt
2020-08-14 17:26:58 +01:00
Matthew Nesbit
55133b02b9
Merge pull request #6626 from corda/bugfix/ENT-5654-run-migration-scripts-completes-with-unexpected-error
ENT-5654: Fixed migration error message, improved success message
2020-08-14 17:17:16 +01:00
Matthew Nesbit
2fb21373a4
Merge pull request #6632 from corda/nnagy-os-4.5-os-4.6-20200813
NOTICK - OS 4.5 to OS 4.6 merge 20200813
2020-08-14 16:16:40 +01:00
Yiftach Kaplan
da065a6215
CORDA-3981: Fix bad keys are ignored and warned for (#6636)
The issue with the test was that the environment variable are kept as a static member so it passed if it was the first one to run, but failed if another test runs the config beforehand.
2020-08-14 15:36:46 +01:00
Dan Newton
845ef8d3d1
CORDA-3989 Terminate sessions instantly (#6634)
Terminate sessions that need to be removed instantly in whatever transition is currently executing, rather than scheduling another event and doing so at a later time.

To do this, update the transition being created in `TopLevelTransition` to remove the sessions and append the `RemoveSessionBindings` action to it.

This achieves the same outcome as the original code but does so with 1 less transition. Doing this also removes the race condition that can occur where another external event is added to the flow's event queue before the terminate event could be added.
2020-08-14 11:13:42 +01:00
Dan Newton
1cbfb74022
CORDA-3986 Increase sleep in FlowSessionCloseTest (#6629)
* CORDA-3986 Increase sleep in `FlowSessionCloseTest`

A sleep duration needed to be increased to ensure that an end session
message has time to be processed by the other node.

Locks do not fully fix this because some internal processing needs to be
completed that can't be waited for using a lock. Therefore the sleep
time was increased generously.
2020-08-14 10:57:24 +01:00
Stefano Franz
205ce84033
[EG-3461] removed dependency from tools.jar (#6631)
* removed dependency from tools.jar

I removed the log line in /node/src/main/kotlin/net/corda/node/internal/NodeStartup.kt because I felt it was not so important
and I modified the checkpoint agent detection simply using a static field (I tested both with and without the checkpoint agent running and detection works correctly)

* move method to node-api to address review comments

Co-authored-by: Walter Oggioni <walter.oggioni@r3.com>
2020-08-14 10:56:37 +01:00
Ross Nicoll
bf53e47f0d
ENT-5669 Improve robustness of FlowReloadAfterCheckpointTest (#6627)
Improve robustness of `FlowReloadAfterCheckpointTest` by adding a countdown latch to observe for when the reloads should have finished.
2020-08-13 15:04:52 +01:00
Viktor Kolomeyko
b81eb1559d
CORDA-3959: Eliminate InternalCordaRPCOps (#6600)
* CORDA-3959: Make `ThreadContextAdjustingRpcOpsProxy` flexible for `RPCOps` it uses

* CORDA-3959: More changes towards supporting multiple `RPCOps` implementations

* CORDA-3959: Detekt baseline update

* CORDA-3959: Integration test compilation fix

* CORDA-3959: Introduce `CheckpointRPCOpsImpl` and wire it on

* CORDA-3959: Use multiple RPCOps interfaces in the shell commands

* CORDA-3959: Detekt baseline update

* CORDA-3959: Update RPCPermissionsTests

* CORDA-3959: Update RPCSecurityManagerTest

* CORDA-3959: Remove deprecated marker and rename the property

* CORDA-3959: Detekt baseline

* CORDA-3959: Introduce AttachmentTrustInfoRPCOpsImpl and wire it on

* CORDA-3959: Delete `InternalCordaRPCOps`

* CORDA-3959: Detekt baseline update

* CORDA-3959: Rename `CheckpointRPCOps` to `FlowManagerRPCOps`
2020-08-13 14:41:52 +01:00
nikinagy
1a0e445a49 Merge branch 'release/os/4.5' into nnagy-os-4.5-os-4.6-20200813
# Conflicts:
#	node/src/main/kotlin/net/corda/node/services/vault/HibernateQueryCriteriaParser.kt
2020-08-13 14:25:58 +01:00
Edoardo Ierina
a7ea8df9a7
CORDA-3954: run database migration scripts during initial node registration (#6624)
* CORDA-3954: Added step to run database migration scripts during initial node registration with -s / --skip-schema-creation option (default to false) to prevent migration.

* CORDA-3954: Applied code convention to if statement

* CORDA-3954: Marked NodeCmdLineOptions' -s/--skip-schema-creation as deprecated and hidden in line with --initial-registration
2020-08-13 14:19:16 +01:00
Anurag Garg
294c2aa514
Merge pull request #6584 from filipesoliveira/filipeoliveira/corda-3931
CORDA-3931 - Fixed a bug which was preventing the custom JVM arguments from  being picked up when the command line "-f" flag was used
2020-08-13 14:13:09 +01:00
Ross Nicoll
4aea7b876a
ENT-5666 Disable test due to Gradle process death (#6617)
Disable `SignatureConstraintMigrationFromHashConstraintsTests.HashConstraint cannot be migrated to SignatureConstraint if a HashConstraint is specified for one state and another uses an AutomaticPlaceholderConstraint()` as it frequently appears in reports about Gradle process failures, to try isolating the actual cause.
2020-08-13 13:17:49 +01:00
Edoardo Ierina
07fe1b0960 ENT-5654: Fixed migration error message, improved success message 2020-08-13 11:32:56 +02:00
Nikolett Nagy
a6b2a3159d
CORDA-3879 - query with OR combinator returns too many results (#6456)
* fix suggestion and tests

* detekt suppress

* making sure the forced join works with IndirectStatePersistable and removing unnecessary joinPredicates from parse with sorting

* remove joinPredicates and add tests

* rename sorting

* revert deleting joinPredicates and modify the force join to use `OR` instead of `AND`

* add system property switch
2020-08-13 10:04:53 +01:00
Ross Nicoll
748480c33e
CORDA-3981 Disable unstable test (#6625) 2020-08-12 18:39:36 +01:00
Alexey Kadyrov
518026c6c4
Changed the counter parties in the query to be CordaX509Name (#6622) 2020-08-12 14:53:03 +01:00
Yiftach Kaplan
28b440d1f7
CORDA-3981: Change test to avoid timeout (#6612) 2020-08-12 10:31:58 +01:00
Alexey Kadyrov
0b6b69bbda
CORDA-3657/5459 inspect waiting flows (#6540)
* CORDA-3657 Extract information from state machine

`FlowReadOperations` interface provides functions that extract
information about flows from the state machine manager.

`FlowOperator` implements this interface (along with another currenly
empty interface).

* CORDA-3657 Rename function and use set

* initial test is passing

* wip

* done tests

* additional tests to cover more FlowIORequest variations

* completed tests

* The quasar.jar should nat have been changed

* Fixed issues reported by detekt

* got rid of sync objects, instead relying on nodes being offline

* Added extra grouping test and minor simplification

* Hospital test must use online node which fails on otherside

* Added additional information required for the ENT

* Added tests to cover SEND FlowIORequests

* using node name constants from the core testing module

* Changed flow operator to the query pattern

* made query fields mutable to simply building query

* fixed detekt issue

* Fixed test which had dependency on the order int the result (failed for windows)

* Fixed recommendations in PR

* Moved WrappedFlowExternalOperation and  WrappedFlowExternalAsyncOperation to FlowExternalOperation.kt as per PR comment

* Moved extension to FlowAsyncOperation

* removed unnecessarily brackets

Co-authored-by: LankyDan <danknewton@hotmail.com>
2020-08-12 10:14:05 +01:00
Ross Nicoll
9965af180e
CORDA-5985 Simplify network map test (#6618)
Remove parameterization from NetworkMapTest as it doesn't actually significantly improve coverage, and costs 6 and a half minutes on every build.
2020-08-12 09:01:58 +01:00
Ross Nicoll
12e7fa1d93
CORDA-3981 Disable unstable config helper test (#6610) 2020-08-10 20:35:15 +01:00
Ross Nicoll
29e87a586a
CORDA-3973 Fix memory leak due to DB not shutting down (#6605)
Fix memory leak due to DB not shutting down in FlowFrameworkPersistenceTests.flow restarted just after receiving payload.

Also reduces number of class-wide variables to reduce scope for references being accidentally held between runs.
2020-08-10 20:04:51 +01:00
Dan Newton
c191960cb8
CORDA-3948 Make KillFlowTest less flakey (#6606)
`KillFlowTest` is failing quite often. This is probably due to issues in ordering when taking and releasing locks. By using `CountDownLatch` in places instead of `Semaphore`s should reduce the likelihood of tests failing.
2020-08-10 16:59:28 +01:00
Yiftach Kaplan
8aafb1db4a
INFRA-545: Convert a few tests to unit tests; rename another test (#6562) 2020-08-10 15:25:28 +01:00
Dan Newton
66406ba0fb
ENT-5450 Resume flow when transition creation errors (#6604)
If an error occurs when creating a transition (a.k.a anything inside of
`TopLevelTransition`) then resume the flow with the error that occurred.

This is needed, because the current code is swallowing all errors thrown
at this point and causing the flow to hang.

This change will allow better debugging of errors since the real error
will be thrown back to the flow and will get handled and logged by the
normal error code path.

Extra logging has been added to `processEventsUntilFlowIsResumed`, just
in case an exception gets thrown out of the normal code path. We do not
want this exception to be swallowed as it can make it impossible to
debug the original error.
2020-08-10 13:09:43 +01:00
Dan Newton
0d5ee8b0fa
NOTICK Save exception for hospitalized session init errors (#6587)
Save the exception for flows that fail during session init when they are
kept for observation.

Change the exception tidy up logic to only update the flow's status if
the exception was removed.
2020-08-06 22:35:05 +01:00
Yiftach Kaplan
849d51c8cd
INFRA-505: Move integration tests to unit tests (#6530) 2020-08-06 15:16:27 +01:00
Kyriakos Tharrouniatis
0005317fec Align code with merging changes 2020-08-06 13:35:46 +01:00
Kyriakos Tharrouniatis
2afedeabb4 Merge branch 'release/os/4.6' into os_4.6-feature_pass_in_client_id_when_starting_a_flow 2020-08-06 13:21:45 +01:00
Dan Newton
3f31aeaa5f
CORDA-3822 Add CordaRPCOps.reattachFlowWithClientId (#6579)
Add `CordaRPCOps.reattachFlowWithClientId` to allow clients to reattach
to an existing flow by only providing a client id. This behaviour is the
same as calling `startFlowDynamicWithClientId` for an existing
`clientId`. Where it differs is `reattachFlowWithClientId` will return
`null` if there is no flow running or finished on the node with the same
client id.

Return `null` if record deleted from race-condition
2020-08-06 11:42:02 +01:00
filipesoliveira
8eaf237a27 Refactored the previous bug fix to minimize duplication by reusing an existing function.
Bear in mind that the condition in the previous code was (config == null) and now is (config == null || config.equals("")).
2020-08-06 11:15:19 +01:00
Will Vigor
7acc510534 CORDA-3602 Set a Checkpoint as incompatible if it can't be deserialised (#3653)
Update the compatible flag in the DB if the flowstate cannot be deserialised.

The most common cause of this problem is if a CorDapp has been upgraded
without draining flows from the node.

`RUNNABLE` and `HOSPITALISED` flows are restored on node startup so
the flag is set for these then. The flag can also be set when a flow
retries for some reason (see retryFlowFromSafePoint) in this case the
problem has been caused by another reason.
2020-08-06 11:00:02 +01:00
filipesoliveira
80d279a70e Fixed a bug which would make the "-f" flag to be ignored. 2020-08-06 10:51:23 +01:00
Will Vigor
4a828fcb99 ENT-5397 Pause individual running and hospitalised flows (#3564)
Added a newpause event to the statemachine which returns an Abort
continuation and causes the flow to be moved into the Paused flow Map.

Flows can receive session messages whilst paused.
2020-08-06 10:40:09 +01:00
Will Vigor
bbf5a93761 ENT-5396 Allow Retrying a Hospitalised Flow from the Statemachine (#3499)
Added functionality to the statemachine to enable retrying a Hospitalised flow without a node restart.
2020-08-06 10:11:15 +01:00
Dan Newton
a73dad00e2
CORDA-3850 Add a per flow lock (#6437)
Add a lock to `StateMachineState`, allowing every flow to lock
themselves when performing a transition or when an external thread (such
as `killFlow`) tries to interact with a flow from occurring at the same
time.

Doing this prevents race-conditions where the external threads mutate
the database or the flow's state causing an in-flight transition to
fail.

A `Semaphore` is used to acquire and release the lock. A `ReentrantLock`
is not used as it is possible for a flow to suspend while locked, and
resume on a different thread. This causes a `ReentrantLock` to fail when
releasing the lock because the thread doing so is not the thread holding
the lock. `Semaphore`s can be used across threads, therefore bypassing
this issue.

The lock is copied across when a flow is retried. This is to prevent
another thread from interacting with a flow just after it has been
retried. Without copying the lock, the external thread would acquire the
old lock and execute, while the fiber thread acquires the new lock and
also executes.
2020-08-06 09:51:42 +01:00
Kyriakos Tharrouniatis
5ba8477733
CORDA-3491 - Do not keep flow state in memory after a flow has finished (#6573)
Do not retain in database `checkpoint.flowState` for flows that have completed 
or failed and have started with a client id, after their lifetime.
2020-08-06 09:46:04 +01:00
Ross Nicoll
fd374bfc6d
CORDA-3943 Remove sleeps from flow reload tests (#6567)
* Remove use of Thread.sleep() FROM FlowReloadAfterCheckpointTest, instead relying on CountdownLatch to wait until the target number has been hit or a timeout occurs, so the thread can continue as soon as the target is hit.
* Replace use of hashmaps to a concurrent queue, to mitigate risk of complex threading issues.
2020-08-05 21:10:11 +01:00
Ross Nicoll
b779fb354c
CORDA-3948 Disable flaky test (#6572) 2020-08-05 14:16:15 +01:00
Christian Sailer
6f2ca8978d
Merge pull request #6534 from corda/feature/ENT-5273-unified-database-management
ENT-5273 unified database management
2020-08-05 14:09:07 +01:00
Dan Newton
f216853c3f
CORDA-3946 Make RetryFlowMockTest less flakey (#6570) 2020-08-05 13:10:35 +01:00
Christian Sailer
45618f346d
Code review fixes (#6571) 2020-08-05 12:48:17 +01:00
Kyriakos Tharrouniatis
5192a9a2dd
CORDA-3612 - Delete flow results/ exceptions once received by rpc client (#6566)
Enhance rpc acknowledgement method (`removeClientId`) to remove checkpoint 
from all checkpoint database tables.

Optimize `CheckpointStorage.removeCheckpoint` to not delete from all checkpoint
tables if not needed. This includes excluding the results (`DBFlowResult`) and 
exceptions (`DBFlowException`) tables.
2020-08-05 12:40:35 +01:00
Viktor Kolomeyko
9fd8107922
INFRA-570: Disable new SSL tests when running on JDK11 (#6560) 2020-08-05 10:02:23 +01:00
Ross Nicoll
ff5bdcb559
NOTICK Close node handles to reduce memory usage (#6557)
Note in theory this shouldn't make a difference, however given the pass rate of this PR I figure it's worth trying.
2020-08-05 08:54:28 +01:00
Kyriakos Tharrouniatis
5d42b8847c
CORDA-3681 - Store serialised exception in database for failed and hospitalized flows (#6539)
Integrate `DBFlowException` with the rest of the checkpoint schema, so now 
we are saving the flow's exception result in the database.

Making statemachine not remove `FAILED` flows' checkpoints from the 
database if they are started with a clientId.

Retrieve the DBFlowException from the database to construct a 
`FlowStateMachineHandle` future and complete exceptionally the flow's result 
future for requests (`startFlowDynamicWithClientId`) that pick FAILED flows , 
started with client id, of status Removed.

On killing a flow the client id mapping of the flow gets removed.

The storage serialiser is used for serialising exceptions. Note, that if an
exception cannot be serialised, it will not fail and will instead be stored
as a `CordaRuntimeException`. This could be improved in future
changes.
2020-08-04 16:33:44 +01:00
Christian Sailer
e6dfa4da39 Remove unused cmd line option 2020-08-04 14:20:41 +01:00
Christian Sailer
2888aa6b1d Merge remote-tracking branch 'origin/release/os/4.6' into christians/ENT-5273-updateandfixes 2020-08-04 14:19:46 +01:00
Ross Nicoll
25d1d61685
NOTICK Robustness improvements for DB race condition test (#6556)
Switch away from log monitoring to verify that DB race conditions are reported correctly, in an attempt to resolve a test instability issue.
2020-08-04 11:45:57 +01:00
Ross Nicoll
743dc507cc Merge remote-tracking branch 'origin/release/os/4.5' into rni/notick/4.5-4.6 2020-08-04 10:29:27 +01:00
Ross Nicoll
1ca9b998de
NOTICK Replace example URL with an unresolvable version (#6555)
Replace example URL with an unresolvable version to avoid issues with network failures causing
the test to fail, i.e. https://ci02.dev.r3.com/job/Corda-Enterprise/job/Corda-ENT-Release-Branch-Tests/job/enterprise/job/release%252Fent%252F4.6/127/testReport/net.corda.node.services.config/NodeConfigurationImplTest/compatibilityZoneURL_populates_NetworkServices/
2020-08-03 22:11:26 +01:00
Tamas Veingartner
0b7fdbb35d
CORDA-3663 Remove dummy package names (#6553)
Dummy package names cause build failure as they are not found on the classpath when trying to import them. Now that empty package name list is allowed, the dummy names are removed.
2020-08-03 19:09:30 +01:00
Christian Sailer
71a6081ec8 Fix new integration tests to not use hibernate schema modification. 2020-08-03 18:50:36 +01:00
Christian Sailer
89759510ae Merge remote-tracking branch 'origin/release/os/4.6' into christians/ENT-5273-update-from-os-4.6-20200803 2020-08-03 16:28:20 +01:00
Tamas Veingartner
85be50779b
CORDA-3663 MockServices crashes when two of the provided packages to … (#6472)
* CORDA-3663 MockServices crashes when two of the provided packages to scan are deemed empty in 4.4 RC05

this happends when a given package is not found on the classpath. Now it is handled and an exception is thrown

* replace dummy package names in tests with valid ones

* allow empty package list for CustomCordapps and exclude those from the created jars

* detekt fix

* always true logic fix

* fix to check for empty packages instead of empty classes

* fix for classes and fixups

* logic refactor because of detekt stupidity

* PR related minor refactors
2020-08-03 09:19:48 +01:00
Dan Newton
c288073e7c
Merge pull request #6524 from corda/ENT-5532-retrying-flow-with-sessions-to-close
ENT-5532 Terminate sessions after original io request
NOTICK Resume flow when wrong message received
2020-07-31 17:08:43 +01:00
LankyDan
82bcde573b NOTICK Resume flow when wrong message received
When an incorrect message is received, the flow should resume to allow
it to throw the error back to user code and possibly cause the flow to
fail.

For now, if an `EndSessionMessage` is received instead of a
`DataSessionMessage`, then an `UnexpectedFlowEndException` is thrown
back to user code. Allowing it to correctly re-enter normal flow error
handling.

Without this change, the flow will hang due to it failing while creating
a transition which exists outside of the general state machine error
handling code path.
2020-07-31 14:28:57 +01:00
LankyDan
39dbe22c9d ENT-5532 Terminate sessions after original io request
Sessions are now terminated after performing the original
`FlowIORequest` passed into `StartedFlowTransition`, instead of before.
This is done by scheduling an `Event.TerminateSessions` if there are
sessions to terminate when performing a suspending event.

Originally this was done by hijacking a transition that is trying to
perform a `StartedFlowTransition`, terminating the sessions and then
scheduling another `Event.DoRemainingWork` to perform the original
transition. This introduced a bug where, another event (from a external
message) could be placed onto the queue before the
`Event.DoRemainingWork` could be added. In most scenarios, that should
be ok. But, if a flow is retrying (while in an uninitiated state) and
this occurs the flow could fail due to being in an unexpected state.

Terminating the sessions after performing the original transition
removes this possibility. Meaning that a restarting flow will always
perform the transition they supposed to do (based on the called
suspending event).
2020-07-31 12:37:44 +01:00
James Higgs
250ed8a21a
Merge pull request #6533 from corda/eg-177merge
EG-177   Network Parameter Hotloading
2020-07-31 09:45:23 +01:00
Viktor Kolomeyko
c498c5bf7c
CORDA-3871: New integration test for handshake timeout in AMQPClient (#6519)
* CORDA-3871: Import external code

Compiles, but does not work for various reasons

* CORDA-3871: More improvements to imported code

Currently fails due to keystores not being found

* CORDA-3871: Initialise keystores for the server

Currently fails due to keystores for client not being found

* CORDA-3871: Configure certificates to client

The program started to run

* CORDA-3871: Improve debug output

* CORDA-3871: Few more minor changes

* CORDA-3871: Add AMQClient test

Currently fails due to `localCert` not being set

* CORDA-3871: Configure server to demand client to present its certificate

* CORDA-3871: Changes to the test to make it pass

ACK status is not delivered as server is not talking AMQP

* CORDA-3871: Add delayed handshake scenario

* CORDA-3871: Tidy-up imported classes

* CORDA-3871: Hide thread creation inside `ServerThread`

* CORDA-3871: Test description

* CORDA-3871: Detekt baseline update

* CORDA-3871: Trigger repeated execution of new tests

To make sure they are not flaky

* CORDA-3871: Improve robustness of the newly introduced tests

* CORDA-3871: Improve robustness of the newly introduced tests

* CORDA-3871: New tests proven to be stable - reduce number of iterations to 1

* CORDA-3871: Adding Alex Karnezis to the list of contributors
2020-07-31 09:26:32 +01:00
Ross Nicoll
68feb1c35f
CORDA-3932 Correct race condition in FlowVersioningTest (#6536)
Correct race condition in FlowVersioningTest where the last message is read (and the session close can be triggered)
before one side has finished reading metadata from the session.
2020-07-31 08:32:20 +01:00
Christian Sailer
81d68abe7e Merge remote-tracking branch 'origin/release/os/4.6' into christians/ENT-5273-update-fb-from-os.4.6 2020-07-30 18:39:04 +01:00
Peter Nemeth
777be6c11a Merge branch 'feature/EG-177' of https://github.com/corda/corda into feature/EG-177 2020-07-30 16:20:01 +01:00
pnemeth
2a27f3ac37
EG-2055 Implement network parameters hotloading (#6517)
* Implement network parameters hotloading

* fixed failing unit test

* PR comments

* PR comments

* added integr tests+ renamed updater to hotloader

* moved exiting logic out of hotloader

* extra tests

* latest PR comments

* refactor

* address detekt/suppress if more significant refactoring needed

* extra tests

* addressed PR comments

* detekt

* formatting
2020-07-30 16:02:33 +01:00
Ross Nicoll
fd5472b053
NOTICK Remove memory leak endurance test (#6514)
Remove memory leak endurance test as it spends 8 minutes testing a single failure case that's not end user visible,
and ultimately manifests elsewhere in test failures (which is where this came from in the beginning). It was a good
idea to confirm the change fixed the issue, but this isn't critical enough to retain.
2020-07-30 15:39:28 +01:00
Kyriakos Tharrouniatis
f2bcc85f30 Fix compilation error 2020-07-30 10:54:24 +01:00
Kyriakos Tharrouniatis
8a37108a58 Merge branch 'release/os/4.6' into os_4.6-feature_pass_in_client_id_when_starting_a_flow-merge 2020-07-30 10:34:03 +01:00
Kyriakos Tharrouniatis
ad61d7b3f4
CORDA-3692 - Store result information in checkpoint results table (#6473)
Making statemachine not remove COMPLETED flows' checkpoints from the database 
if they are started with a clientId, instead they are getting persisted and retained within 
the database along with their result (`DBFlowResult`).

On flow start with a client id (`startFlowDynamicWithClientId`), if the client id maps to
a flow that was previously started with the same client id and the flow is now finished, 
then fetch the `DBFlowResult` from the database to construct a 
`FlowStateMachineHandle` done future and return it back to the client.

Object stored as results must abide by the storage serializer rules. If they fail to do so
the result will not be stored and an exception is thrown to the client to indicate this.
2020-07-29 15:57:00 +01:00
Yiftach Kaplan
0bedbd8c75
INFRA-530: Start notary node in process (#6521) 2020-07-29 15:47:45 +01:00
Dan Newton
c2fd8253ea
CORDA-3777 Reload after every checkpoint (#6494)
Enable reloading of a flow after every checkpoint is saved. This
includes reloading the checkpoint from the database and recreating the
fiber.

When a flow and its `StateMachineState` is created it checks the node's
config to see if the `reloadCheckpointAfterSuspend` is set to true. If it is
it initialises `StateMachineState.reloadCheckpointAfterSuspendCount`
with the value 0. Otherwise, it remains `null`.

This count represents how many times the flow has reloaded from its
checkpoint (not the same as retrying). It is incremented every time the
flow is reloaded.

When a flow suspends, it processes the suspend event like usual, but
it will now also check if `reloadCheckpointAfterSuspendCount` is not
`null` (that it is activated) and process a 
`ReloadFlowFromCheckpointAfterSuspend`event, if and only if 
`reloadCheckpointAfterSuspendCount` is greater than
`CheckpointState.numberOfSuspends`.

This means idempotent flows can reload from the start and not reload
again until reaching a new suspension point.

Flows that skip checkpoints can reload from a previously saved
checkpoint (or from the initial checkpoint) and will continue reloading
on reaching the next new suspension point (not the suspension point that
it skipped saving).

If the flow fails to deserialize the checkpoint from the database upon
reloading a `ReloadFlowFromCheckpointException` is throw. This causes
the flow to be kept for observation.
2020-07-28 16:27:51 +01:00
Stefan Iliev
52cbe04b8c
EG-2375 - batching notary open sourcing. (#6507) 2020-07-28 15:50:19 +01:00
Dimitris Gounaris
1e6be340eb
CORDA-3844: bulk node infos request (#6411)
* CORDA-3844: Add new functions to network map client

* CORDA-3844: Apply new fetch logic to nm updater

* CORDA-3844: Fix base url and warnings

* CORDA-3844: Change response object and response validation

In order to make sure that the returned node infos are not maliciously modified, either a signed list response
or a signed reference object would need to be provided. As providing a signed list requires a lot of effort from NM and Signer services,
the signed network map is provided instead, allowing nodes to validate that the list provided conforms to the entries of the signed network map.

* CORDA-3844: Add clarifications and comments

* CORDA-3844: Add error handling for bulk request

* CORDA-3844: Enhance testing

* CORDA-3844: Fix detekt issues

* EG-3844: Apply pr suggestions
2020-07-28 15:02:53 +01:00
Dimos Raptis
f2336f397d
CORDA-3506 - Add test for session close API (#6512) 2020-07-28 10:20:24 +01:00
Christian Sailer
19e11619b4 Remove unused import 2020-07-23 17:59:54 +01:00
Ryan Fowler
2ca10464b7
CORDA-3845: Update BC, log4j, slf4j (#6464)
* CORDA-3845: Update BC to 1.64

* CORDA-3845: Upgraded log4j to 2.13.3

* We can remove the use of Manifests from the logging package so that when _it_ logs it doesn't error on the fact the stream was already closed by the default Java logger.

* Some more tidy up

* Remove the logging package as a plugin

* latest BC version

* Remove old test

* fix up

* Fix some rebased changes to log file handling

* Fix some rebased changes to log file handling

* Update slf4j too

Co-authored-by: Adel El-Beik <adel.el-beik@r3.com>
2020-07-23 16:37:29 +01:00
Yiftach Kaplan
4acf41ea3d
INFRA-477: Start nodes in parallel when possible (#6460)
Co-authored-by: Ross Nicoll <ross.nicoll@r3.com>
2020-07-23 16:35:34 +01:00
Christian Sailer
db94f65d8a Merge remote-tracking branch 'origin/release/os/4.6' into christians/ENT-5273-update-from-os-4.6 2020-07-23 14:02:18 +01:00
Joseph Zuniga-Daly
c33720c73d
CORDA-3717: Apply custom serializers to checkpoints (#6392)
* CORDA-3717: Apply custom serializers to checkpoints

* Remove try/catch to fix TooGenericExceptionCaught detekt rule

* Rename exception

* Extract method

* Put calls to the userSerializer on their own lines to improve readability

* Remove unused constructors from exception

* Remove unused proxyType field

* Give field a descriptive name

* Explain why we are looking for two type parameters when we only use one

* Tidy up the fetching of types

* Use 0 seconds when forcing a flow checkpoint inside test

* Add test to check references are restored correctly

* Add CheckpointCustomSerializer interface

* Wire up the new CheckpointCustomSerializer interface

* Use kryo default for abstract classes

* Remove unused imports

* Remove need for external library in tests

* Make file match original to remove from diff

* Remove maySkipCheckpoint from calls to sleep

* Add newline to end of file

* Test custom serializers mapped to interfaces

* Test serializer configured with abstract class

* Move test into its own package

* Rename test

* Move flows and serializers into their own source file

* Move broken map into its own source file

* Delete comment now source file is simpler

* Rename class to have a shorter name

* Add tests that run the checkpoint serializer directly

* Check serialization of final classes

* Register as default unless the target class is final

* Test PublicKey serializer has not been overridden

* Add a broken serializer for EdDSAPublicKey to make test more robust

* Split serializer registration into default and non-default registrations. Run registrations at the right time to preserve Cordas own custom serializers.

* Check for duplicate custom checkpoint serializers

* Add doc comments

* Add doc comments to CustomSerializerCheckpointAdaptor

* Add test to check duplicate serializers are logged

* Do not log the duplicate serializer warning when the duplicate is the same class

* Update doc comment for CheckpointCustomSerializer

* Sort serializers by classname so we are not registering in an unknown or random order

* Add test to serialize a class that references itself

* Store custom serializer type in the Kryo stream so we can spot when a different serializer is being used to deserialize

* Testing has shown that registering custom serializers as default is more robust when adding new cordapps

* Remove new line character

* Remove unused imports

* Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt

* Remove comment

* Update comment on exception

* Make CustomSerializerCheckpointAdaptor internal

* Revert "Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt"

This reverts commit b835de79bd.

* Restore "Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt""

This reverts commit 718873a4e9.

* Pass the class loader instead of the context

* Do less work in test setup

* Make the serialization context unique for CustomCheckpointSerializerTest so we get a new Kryo pool for the test

* Rebuild the Kryo pool for the given context when we change custom serializers

* Rebuild all Kryo pools on serializer change to keep serializer list consistent

* Move the custom serializer list into CheckpointSerializationContext to reduce scope from global to a serialization context

* Remove unused imports

* Make the new checkpointCustomSerializers property default to the empty list

* Delegate implementation using kotlin language feature
2020-07-22 17:31:59 +01:00
Dan Newton
a41152edf6
CORDA-3899 Refactor flow's transient fields (#6441)
Refactor `FlowStateMachineImpl.transientValues` and
`FlowStateMachineImpl.transientState` to stop the fields from exposing
the fact that they are nullable.

This is done by having private backing fields `transientValuesReference`
and `transientStateReference` that can be null. The nullability is still
needed due to serialisation and deserialisation of flow fibers. The
fields are transient and therefore will be null when reloaded from the
database.

Getters and setters hide the private field, allowing a non-null field to
returned.

There is no point other than in `FlowCreator` where the transient fields
can be null. Therefore the non null checks that are being made are
valid.

Add custom kryo serialisation and deserialisation to `TransientValues`
and `StateMachineState` to ensure that neither of the objects are ever
touched by kryo.
2020-07-22 16:19:20 +01:00
Yiftach Kaplan
a03fb8c6fd
INFRA-438: Close session factory before closing the locator (#6477) 2020-07-21 18:10:09 +01:00
Christian Sailer
377c3f9d78 Merge remote-tracking branch 'origin/release/os/4.6' into christians/ENT-5273-update-from-os-4.6 2020-07-21 14:17:27 +01:00
Dimos Raptis
7261fa690f
CORDA-3506 - Implement session close operations (#6357) 2020-07-21 13:26:11 +01:00
Viktor Kolomeyko
234ac2c246 Merge remote-tracking branch 'origin/release/os/4.5' into vkolomeyko/os-4.5-os-4.6-merge 2020-07-21 10:29:07 +01:00
Viktor Kolomeyko
1c48418904
NOTICK: Remove relaxedThoroughness variable (#6483)
It is:
a. No longer works in Jenkins
b. May result spawning 100s of process that kill Windows build. Evidence: https://ci02.dev.r3.com/blue/organizations/jenkins/Corda-Enterprise%2FCorda-ENT-MS-Win-Compatibility%2Fenterprise/detail/PR-3553/25/pipeline/
2020-07-21 09:58:29 +01:00
Kyriakos Tharrouniatis
5fef0726a2
Add missing 'Suspendable' annotations to suspending methods (#6480) 2020-07-20 15:09:26 +01:00
Ryan Fowler
d4444e520a
ENT-5140: Tighten network parameters checks (#6390)
* ENT-5140: Tighten network parameters checks
2020-07-20 11:26:27 +01:00
jakubbielawa
3721b7c701
Decrease the ring buffer size (#6470) 2020-07-20 11:14:23 +01:00
Dimos Raptis
e5a8351dc3
ENT-5437 - Add test for sendAll with multiple sessions from the same party (#6476) 2020-07-20 11:13:49 +01:00
Oliver Knowles
a500084d38
CORDA-3201 - Enforce separate key for notary identity (#6308) 2020-07-20 10:59:08 +01:00
Yiftach Kaplan
e6d5842a23
INFRA-482: Correct exception caught testing for node death (#6471) 2020-07-17 13:57:40 +01:00
Ross Nicoll
5d7060ec3a
CORDA-3901 Eliminate random reconnect test (#6446)
Remove a legacy test for RPC reconnection, which takes 5 minutes to run a random set of tests. This is expensive and low value.
2020-07-17 12:26:51 +01:00
Kyriakos Tharrouniatis
22d92d5ef0
CORDA-3809 Expose client side unique RPC ID for flow starts (#6307)
Introducing a new flow start method (`startFlowDynamicWithClientId`) passing in a `clientId`.

Once `startFlowDynamicWithClientId` gets called, the `clientId` gets injected into `InvocationContext` and also pushed to the logging context.

If a new flow starts with this method, then a < `clientId` to flow > pair is kept on node side, even after the flow's lifetime. If `startFlowDynamicWithClientId` is called again with the same `clientId` then the node identifies that this `clientId` refers to an existing < `clientId` to flow > pair and returns back to the rpc client a `FlowStateMachineHandle` future, created out of that pair.

`FlowStateMachineHandle` interface was introduced as a thinner `FlowStateMachine`. All `FlowStateMachine` properties used by call sites are moved into this new interface along with `clientId` and then `FlowStateMachine` extends it.

Introducing an acknowledgement method (`removeClientId`). Calling this method removes the < `clientId` to flow > pair on the node side and frees resources.
2020-07-16 10:52:08 +01:00
Adel El-Beik
2fa6b5a208
CORDA-3769: Switched attachments class loader cache to use caffeine (#6326)
* CORDA-3769: Switched attachments class loader cache to use caffeine with original implementation used by determinstic core.

* CORDA-3769: Removed default ctor arguments.

* CORDA-3769: Switched mapping function to Function type to avoid synthetic method being generated.

* CORDA-3769: Now using a cache created from NamedCacheFactory for the attachments class loader cache.

* CORDA-3769: Making detekt happy.

* CORDA-3769: The finality tests now check for UntrustedAttachmentsException which will actually happen in reality.

* CORDA-3769: Refactored after review comments.

* CORDA-3769: Removed the AttachmentsClassLoaderSimpleCacheImpl as DJVM does not need it. Also updated due to review comments.

* CORDA-3769: Removed the generic parameters from AttachmentsClassLoader.

* CORDA-3769: Removed unused imports.

* CORDA-3769: Updates from review comments.

* CORDA-3769: Updated following review comments. MigrationServicesForResolution now uses cache factory. Ctor updated for AttachmentsClassLoaderSimpleCacheImpl.

* CORDA-3769: Reduced max class loader cache size

* CORDA-3769: Fixed the attachments class loader cache size to a fixed default

* CORDA-3769: Switched attachments class loader size to be reduced by fixed value.
2020-07-16 09:58:36 +01:00
Dan Newton
ac4907a429
CORDA-3721 Finishing + retrying a flow cancels its future (#6454)
Cancel the future being run by a flow when finishing or retrying it. The
cancellation of the future no longer cares about what type of future it
is.

`StateMachineState` has the `future` field, which holds the 3
(currently) possible types of futures:
- sleep
- wait for ledger commit
- async operation / external operation

Move the starting of all futures triggered by actions into
`ActionFutureExecutor`.
2020-07-14 08:04:52 +01:00
Chris Rankin
e684cfd1e6
NOTICK: Avoid configuring more tasks in the deterministic modules. (#6452) 2020-07-13 10:33:23 +01:00
Christian Sailer
057a8d8ae9
NOTICK fix smoke tests and slow integration tests (#6422)
* Add schema migration to smoke tests
* Fix driver to work correctly for out-of-proc node with persistent database.
Co-authored-by: Ross Nicoll <ross.nicoll@r3.com>
2020-07-09 15:13:20 +01:00
Dan Newton
52240dc3d1
Merge pull request #6451 from corda/dan/os-4.5-to-4.6-merge-2020-07-08
NOTICK OS 4.5 to 4.6 merge 2020-07-08
2020-07-09 14:02:22 +01:00
LankyDan
8b231dc936 NOTICK Suppress detekt warning on StaffedFlowHospital 2020-07-08 16:18:16 +01:00
LankyDan
b6d649634f ENT-5196 handle errors during flow initialisation (#6378)
Update flow error handling tests after merging from earlier releases
2020-07-08 16:02:17 +01:00
LankyDan
b05c0f0cc1 ENT-5196 handle errors during flow initialisation (#6378)
Changes to `StaffedFlowHospital` after merging from earlier releases.
2020-07-08 14:43:24 +01:00
LankyDan
2204f44332 ENT-5196 handle errors during flow initialisation (#6378)
Changes to `TopLevelTransition` after merging from earlier releases.

When a flow is kept for observation and its checkpoint is saved as
HOSPITALIZED in the database, we must acknowledge the session init and
flow start events so that they are not replayed on node startup.

Otherwise the same flow will be ran twice when the node is restarted,
one from the checkpoint and one from artemis.
2020-07-08 14:41:29 +01:00
Chris Rankin
72938fdf38
CORDA-3892: Upgrade to Gradle 5.6.4 again, with build fixes. (#6448) 2020-07-08 11:34:25 +01:00
LankyDan
fdae04fc28 Merge branch 'release/os/4.5' into dan/os-4.5-to-4.6-merge-2020-07-08 2020-07-08 10:44:47 +01:00
Chris Rankin
16eb9dfc08
NOTICK: Fix some Gradle technical debt. (#6430) 2020-07-07 09:08:58 +01:00
Kyriakos Tharrouniatis
b619356bff
Reduce exception_message and stack_trace lengths in table node_flow_exceptions (#6432)
Reduce exception_message and stack_trace lengths in table node_flow_exceptions from 4000 to 2000 to fix Oracle failing with: 'ORA-00910: specified length too long for its datatype'
2020-07-07 09:07:01 +01:00
Dan Newton
9a8ae0fd32
CORDA-3841 Update session init flow error handling tests (#6431)
These tests were removed after doing a merge from 4.4. They needed
updating after the changes from 4.4 anyway. These have been included in
this change.

Also fix kill flow tests and send initial tests.
2020-07-07 08:47:09 +01:00
Dan Newton
a1e1bf4e6d
CORDA-3848 Uncaught exception hospitalises flow (#6377)
When an uncaught exception propagates all the way to the flow exception
handler, the flow will be forced into observation/hospitalised.

The updating of the checkpoints status is done on a separate thread as
the fiber cannot be relied on anymore. The new thread is needed to allow
database transaction to be created and committed. Failures to the status
update will be rescheduled to ensure that this information is eventually
reflected in the database.
2020-07-06 22:43:48 +01:00
Ryan Fowler
0d5bed5243
ENT-5131: Avoid NPE by throwing a catchable exception when openAttachment fails. (#6408) 2020-07-06 11:42:36 +01:00
Ross Nicoll
6aa19723e6
INFRA-417 Improve driver DSL test stability (#6415)
* Move log messages that are not useful in typical usage from info to debug level to reduce log spam.
* Add node startup check before attempting to connect.
2020-07-03 20:42:29 +01:00
Dan Newton
6bc2c79e23
NOTICK NodeBasedTest take in cordapps (#6424)
In enterprise, `AuthDBTests` picked up a schema from a unit test and
included it in the cordapp it builds. This schema does not have a
migration and therefore fails the integration tests.

`NodeBasedTest` now lets cordapps to be defined and passed in to avoid
this issue. It defaults to making a cordapp from the tests base
directory if none are provided.
2020-07-02 16:14:51 +01:00
Denis Rekalov
adc0879e8e
CORDA-3867: Add tests for AMQ_VALIDATED_USER (#6418)
* CORDA-3867: Add tests for AMQ_VALIDATED_USER

* CORDA-3867: detekt
2020-07-02 09:20:23 +01:00
Ross Nicoll
9f12e6bbc5
INFRA-433 Rebuild node config parsing tests (#6423)
Replace node configuration parsing tests with lighter weight equivalents which just parse the configuration rather than starting a full node.
2020-07-01 22:47:09 +01:00
Dan Newton
5f4401d16a NOTICK Non-database error handling in withEntityManager (#6239)
When a non-database exception is thrown out of a `withEntityManager`
block, always check if the session needs to be rolled back.

This means if a database error is caught and a new non-database error is
thrown out of the `withEntityManager` block, the transaction is still
rolled back. The flow can then continue progressing as normal.
2020-07-01 18:18:21 +01:00
LankyDan
c06830d851 Merge branch 'release/os/4.4' into dan/os-4.4-to-4.5-merge-2020-07-01
# Conflicts:
#	core/src/main/kotlin/net/corda/core/node/ServiceHub.kt
#	node/src/integration-test-slow/kotlin/net/corda/node/services/statemachine/StatemachineGeneralErrorHandlingTest.kt
#	node/src/integration-test-slow/kotlin/net/corda/node/services/statemachine/StatemachineKillFlowErrorHandlingTest.kt
#	node/src/integration-test/kotlin/net/corda/node/flows/FlowEntityManagerTest.kt
#	node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt
#	node/src/main/kotlin/net/corda/node/services/statemachine/TransitionExecutorImpl.kt
#	node/src/main/kotlin/net/corda/node/services/statemachine/interceptors/HospitalisingInterceptor.kt
2020-07-01 17:32:36 +01:00
Kyriakos Tharrouniatis
5499e2c050
ENT-5384 Rename MAX_SQL_IN_CLAUSE_SET (#6414)
Rename constant to `DEFAULT_SOFT_LOCKING_SQL_IN_CLAUSE_SIZE`
2020-07-01 09:31:12 +01:00
Dan Newton
efd633c7b9 CORDA-3722 withEntityManager can rollback its session (#6187)
* CORDA-3722 withEntityManager can rollback its session

Improve the handling of database transactions when using
`withEntityManager` inside a flow.

Extra changes have been included to improve the safety and
correctness of Corda around handling database transactions.

This focuses on allowing flows to catch errors that occur inside an
entity manager and handle them accordingly.

Errors can be caught in two places:

- Inside `withEntityManager`
- Outside `withEntityManager`

Further changes have been included to ensure that transactions are
rolled back correctly.

Errors caught inside `withEntityManager` require the flow to manually
`flush` the current session (the entity manager's individual session).
By manually flushing the session, a `try-catch` block can be placed
around the `flush` call, allowing possible exceptions to be caught.

Once an error is thrown from a call to `flush`, it is no longer possible
to use the same entity manager to trigger any database operations. The
only possible option is to rollback the changes from that session.
The flow can continue executing updates within the same session but they
will never be committed. What happens in this situation should be handled
by the flow. Explicitly restricting the scenario requires a lot of effort
and code. Instead, we should rely on the developer to control complex
workflows.

To continue updating the database after an error like this occurs, a new
`withEntityManager` block should be used (after catching the previous
error).

Exceptions can be caught around `withEntityManager` blocks. This allows
errors to be handled in the same way as stated above, except the need to
manually `flush` the session is removed. `withEntityManager` will
automatically `flush` a session if it has not been marked for rollback
due to an earlier error.

A `try-catch` can then be placed around the whole of the
`withEntityManager` block, allowing the error to be caught while not
committing any changes to the underlying database transaction.

To make `withEntityManager` blocks work like mini database transactions,
save points have been utilised. A new savepoint is created when opening
a `withEntityManager` block (along with a new session). It is then used
as a reference point to rollback to if the session errors and needs to
roll back. The savepoint is then released (independently from
completing successfully or failing).

Using save points means, that either all the statements inside the
entity manager are executed, or none of them are.

- A new session is created every time an entity manager is requested,
but this does not replace the flow's main underlying database session.
- `CordaPersistence.transaction` can now determine whether it needs
to execute its extra error handling code. This is needed to allow errors
escape `withEntityManager` blocks while allowing some of our exception
handling around subscribers (in `NodeVaultService`) to continue to work.
2020-06-30 11:54:16 +01:00
Dan Newton
f6b5737277
ENT-5196 handle errors during flow initialisation (#6378)
## Summary

This change deals with multiple issues:

* Errors that occur during flow initialisation.

* Errors that occur when handling the outcome of an existing flow error.

* Failures to rollback and close a database transaction when an error
occurs in `TransitionExecutorImpl`.

* Removal of create and commit transaction actions around retrying a flow.

## Errors that occur during flow initialisation

Flow initialisation has been moved into the try/catch that exists inside
`FlowStateMachineImpl.run`. This means if an error is thrown all the way
out of `initialiseFlow` (which should rarely happen) it will be caught 
and move into a flow's standard error handling path. The flow should 
then properly terminate.

`Event.Error` was changed to make the choice to rollback be optional. 
Errors during flow initialisation cause the flow to not have a open 
database transaction. Therefore there is no need to rollback.

## Errors that occur when handling the outcome of an existing flow error

When an error occurs a flow goes to the flow hospital and is given an 
outcome event to address the original error. If the transition that was 
processing the error outcome event (`StartErrorPropagation` and
`RetryFlowFromSafePoint`) has an error then the flow aborts and
nothing happens. This means that the flow is left in a runnable state.

To resolve this, we now retry the original error outcome event whenever 
another error occurs doing so.

This is done by adding a new staff member that looks for 
`ErrorStateTransitionException` thrown in the error code path of 
`TransitionExecutorImpl`. It then takes the last outcome for that flow 
and schedules it to run again. This scheduling runs with a backoff.

This means that a flow will continually retry the original error outcome
event until it completes it successfully.

## Failures to rollback and close a database transaction when an error occurs in `TransitionExecutorImpl`
   
Rolling back and closing the database transaction inside of 
`TransitionExecutorImpl` is now done inside individual try/catch blocks
as this should not prevent the flow from continuing.

## Removal of create and commit transaction actions around retrying a flow

The database commit that occurs after retrying a flow can fail which 
required some custom code just for that event to prevent inconsistent 
behaviour. The transaction was only needed for reading checkpoints from 
the database, therefore the transaction was moved into 
`retryFlowFromSafePoint` instead and the commit removed.

If we need to commit data inside of `retryFlowFromSafePoint` in the 
future, a commit should be added directly to `retryFlowFromSafePoint`. 
The commit should occur before the flow is started on a new fiber.
2020-06-30 11:50:42 +01:00
Tamas Veingartner
18cd24c2b4
NOTICK Ignore pagination large volume test as its completion is environment dependent and might cause issues in slower environments (#6420) 2020-06-30 10:12:31 +01:00
Denis Rekalov
e75dc9e415 Merge branch 'release/os/4.5' into denis/CORDA-3856-amqp-header-os-4.6 2020-06-29 13:13:48 +01:00
Denis Rekalov
7686ca2944 Merge branch 'release/os/4.4' into denis/CORDA-3856-amqp-header-os-4.5 2020-06-29 12:35:05 +01:00
pnemeth
6fc9f2eacf
EG-2854 Failing Test: net.corda.node.internal.NodeStartupCliTest.--nodeconf using absolute path will not be changed (#6393)
* EG-2854

Failing Test: net.corda.node.internal.NodeStartupCliTest.--nodeconf using absolute path will not be changed

* fix bug EG-2854

* PR comment
2020-06-29 09:40:12 +01:00
Denis Rekalov
3f03de6fbd
CORDA-3856: Add Artemis plugin for validating AMQP message header and type (#6407) 2020-06-29 09:23:29 +01:00
Dan Newton
516a4bf3a1
Merge pull request #6403 from corda/NOTICK-rfowler-merge-OS4.5-OS4.6-20200626
NOTICK rfowler merge os4.5 os4.6 20200626
2020-06-29 08:40:48 +01:00
Dan Newton
796e92b512
CORDA-3720 Extract locking of InnerState out of SMM (#6289)
The state machines state is held within `InnerState` which lived inside
the SMM. `InnerState` has been extracted out of the SMM to allow the SMM
to be refactored in the future. Smaller classes can now be made that
focus on a single goal as the locking of the state can be accessed from
external classes. To achieve this, pass the `InnerState` into the class
and request a lock if needed.

The locking of `InnerState` has been made a property of the `InnerState`
itself. It has a `lock` field that allows locks to be taken out when
needed.

An inline `withLock` function has been added to tidy up the code and not
harm performance.

Some classes have been made internal to prevent invalid usage of purely
node internal classes.

As part of this change, flow timeouts have been extracted out into
`FlowTimeoutScheduler`.
2020-06-26 12:48:15 +01:00
Ryan Fowler
31a1ee8803 Merge branch 'release/os/4.5' into NOTICK-rfowler-merge-OS4.5-OS4.6-20200626 2020-06-26 12:14:56 +01:00
Ryan Fowler
881d8d687c Merge branch 'release/os/4.4' into NOTICK-rfowler-merge-OS4.4-OS4.5-20200626 2020-06-26 09:52:16 +01:00
Ryan Fowler
ef582900cf
NOTICK Expand the regex to match what we already do in ENT (#6400) 2020-06-25 17:26:55 +01:00
Kyriakos Tharrouniatis
6ec2910f15
NOTICK Revert node_flow_exceptions type length back to 256 (#6395) 2020-06-25 09:40:58 +01:00
Chris Rankin
6485a025c7
ENT-5430: Increase test coverage when serializing Optional fields. (#6387) 2020-06-22 16:51:40 +01:00
Waldemar Zurowski
1a4efbac7f Merge branch 'release/os/4.5' into merge-45-to-46 2020-06-19 19:48:08 +01:00
pnemeth
d6cab0e131
EG-1557 - Configuration data from "include" section ignored while command line contains the path to config file without leading ./ (#6354)
Configuration data from "include" section ignored while command line contains the path to config file without leading ./
2020-06-19 10:32:55 +01:00
alicer3
e77f7a7546
center console message for registration (#6191) 2020-06-19 09:49:07 +01:00
LankyDan
56d0bbc036 CORDA-3841 Check isAnyCheckpointPersisted in startFlowInternal (#6351)
Only hit the database if `StateMachineState.isAnyCheckpointPersisted`
returns true. Otherwise, there will be no checkpoint to retrieve from the
database anyway. This can prevent errors due to a transient loss of
connection to the database.

Update tests after merging to 4.6
2020-06-18 16:15:15 +01:00
LankyDan
e8b17ff7b9 Merge branch 'release/os/4.5' into dan/os-4.5-to-4.6-merge-2020-06-18
# Conflicts:
#	node/src/main/kotlin/net/corda/node/services/persistence/DBCheckpointStorage.kt
#	node/src/main/kotlin/net/corda/node/services/statemachine/SingleThreadedStateMachineManager.kt
2020-06-18 15:50:46 +01:00
Christian Sailer
4091fdc8b1
ENT-5264 Synchronise schema on the command line (#6353)
* Decouple DatabaseConfig and CordaPersistence etc.
* Add schema sync to schema migration + test
* Add command line parameters for synchronising schema
2020-06-18 11:38:46 +01:00
Chris Rankin
1ef62870bb Merge commit 'fe617818895edab334d80c5e8de2b38f39e67af6' into chrisr3-os44-merge 2020-06-17 18:54:54 +01:00
Chris Rankin
d0c0a1d9ba
ENT-5430: Fix deserialisation of commands containing generic types. (#6359) 2020-06-17 17:28:26 +01:00
James Higgs
24b0240d82
EG-2654 - Ensure stack traces are printed to the logs in error reporting (#6345)
* EG-2654 Ensure stack trace is printed to the logs in error reporting

* EG-2654 - Add a test case for exception logging
2020-06-17 14:32:12 +01:00
Dan Newton
7ab6a8f600
CORDA-3841 Check isAnyCheckpointPersisted in startFlowInternal (#6351)
Only hit the database if `StateMachineState.isAnyCheckpointPersisted`
returns true. Otherwise, there will be no checkpoint to retrieve from the
database anyway. This can prevent errors due to a transient loss of
connection to the database.
2020-06-16 09:22:26 +01:00
Tamas Veingartner
26d4bfb89f
CORDA-3578 add corda prefix to conf file names as original issue was … (#6322)
* CORDA-3578 add corda prefix to conf file names as original issue was having non-corda reference.conf files on classpath causes DriverDSLImp failure

it is better to have this naming convention and avoid further conflicts of conf files.

* fixed weird duplicates

* revert renaming changes for web-reference.conf and loadtest-reference.conf
2020-06-16 09:15:51 +01:00
Christian Sailer
836dd559e8
ENT-5316 split schema migration
* ENT-5273 Split schema migration into separate core and app schema migration, with separate command line flags
2020-06-15 15:52:31 +01:00
Christian Sailer
2c26f4db5d Merge remote-tracking branch 'origin/release/os/4.6' into christians/update-fb-2020-06-12 2020-06-15 09:02:55 +01:00
Christian Sailer
f1126226a8 Fix config tests (remove tx isolation level from config files) 2020-06-12 20:54:36 +01:00
Christian Sailer
35c661b9f6
Merge pull request #6341 from corda/chrisr3-45-merge
NOTICK: Merge from OS 4.5 up to ef00fa1.
2020-06-12 16:42:51 +01:00
Stefano Franz
64f0011a62
Make Checkpoint classes data classes (#6342)
* Make Checkpoint classes data classes

* tidy up null-checks for array equality
2020-06-12 16:35:32 +01:00
Christian Sailer
d00dc42b18 Merge remote-tracking branch 'origin/release/os/4.6' into christians/update-fb-2020-06-12 2020-06-12 14:51:43 +01:00
Chris Rankin
3f67e314c0 Merge commit 'ef00fa1388db37e155ab8cfed3763c14801f8aa9' into chrisr3-45-merge 2020-06-12 13:14:44 +01:00
James Higgs
6e349f298e
NOTICK - Ignore a potentially dodgy test (#6336) 2020-06-11 16:47:48 +01:00
James Higgs
ab023d0b07 Merge branch 'release/os/4.5' into jamesh/os-4.5-4.6-merge-11062020 2020-06-11 09:40:39 +01:00
James Higgs
58af87c988
EG-2225 - Create log directory in correct place with verbose flag set (#6321)
* Ensure logs directory is written to correct location
* Remove a superfluous set of log path property
* Add a unit test to catch bad log paths
* Address detekt issues
2020-06-10 10:46:57 +01:00
James Higgs
8b7275eb97
EG-2564 - Move printed error to logger (#6323) 2020-06-10 10:45:50 +01:00
Schife
fb184839f4 Merge branch 'release/os/4.5' of https://github.com/corda/corda into razvan-os-4.5-to-4.6-merge 2020-06-05 07:55:48 +01:00
nikinagy
caf5482244
ENT-4064, ENT-4608 - checking for unsigned cordapps in prod mode and checking the PV (#6291)
* checking for unsigned cordapps in prod mode and shutting down if it the contracts are not signed

* inheriting from CordaRuntimeException instead of Exception

* had the same tests twice, removed the shutdown for minimumplatformversion, modified some of the tests

* shut down when we get invalid platformversion and modified the test according to this

* making the message more accurate
2020-06-04 16:24:49 +01:00
Denis Rekalov
45614cf29e
Merge pull request #6266 from corda/denis/CORDA-3805-custom-migration-scripts
CORDA-3805: cut dependency from PersistentIdentityService for custom migration scripts
2020-06-04 14:20:26 +01:00
Dan Newton
f0d2c9fe71
NOTICK Correct StatemachineGeneralErrorHandlingtest (#6306) 2020-06-04 08:04:41 +01:00
Christian Sailer
98f62f60f1 Merge branch 'release/os/4.6' into christians/update-feat-20200502 2020-06-02 18:15:32 +01:00
Christian Sailer
2cb89897b4 Merge branch 'release/os/4.6' into christians/update-feat-20200502 2020-06-02 17:55:14 +01:00
Kyriakos Tharrouniatis
b8b462f68e
NOTICK Fix schema validation error for flow parameters (#6304)
Adding 'corda-blob' type, fixed 'Schema-validation: wrong column type'
2020-06-02 17:52:50 +01:00
williamvigorr3
0554c98d18
NOTICK Restrict to shell commands (#6303)
Remove shell command for pausing flows
2020-06-02 16:04:10 +01:00
James Higgs
04ddb267fd
[EG-2225] Prevent extra directories being created when relative base directories are specified (#6282)
Don't create an extra directory if a relative base path specified
2020-06-02 15:26:40 +01:00
Tamas Veingartner
9c4a76d367
CORDA-3176 Inefficient query generated on vault queries with custom p… (#6241)
* CORDA-3176 Inefficient query generated on vault queries with custom paging

a check added to avoid self joins
local postgres db tests were executed on large volume test data (50k states) to ensure that the DB optimizes the suspicious query and so it does not cuase performance issues.
A test added to check that a pagination query on large volume of sorted data is completed in a reasonable time

* foreach detekt fix
2020-06-02 10:33:59 +01:00
Christian Sailer
0ed6307577 Merge remote-tracking branch 'origin/release/os/4.6' into christians/update-feat-20200502 2020-06-02 09:03:11 +01:00
Denis Rekalov
98af7f10f9 CORDA-3805: Make custom migration scripts independent on the latest PersistentIdentityService schema 2020-06-01 16:22:21 +01:00
Rick Parker
9f2bd1dcae
Merge pull request #6295 from corda/feature/checkpoint_table_improvements
CORDA-3432 Feature/checkpoint table improvements
2020-06-01 11:31:13 +01:00
Kyriakos Tharrouniatis
4507b55857
CORDA-3725 Fix SQL deadlocks coming from soft locking states (#6287)
Adding to the query -explicitly- the flow's locked states resolved the SQL Deadlocks in SQL server. The SQL Deadlocks would come up  from `softLockRelease` when only the `lockId` was passed in as argument. In that case the query optimizer would use `lock_id_idx(lock_id, state_status)` to search and update entries in `VAULT_STATES` table. 

However, all rest of the queries would follow the opposite direction meaning they would use PK's `index(output_index, transaction_id)` but they would also update the `lock_id` column and therefore the `lock_id_idx` as well, because `lock_id` is a part of it. That was causing a circular locking among the different transactions (SQL processes) within the database.

To resolve this, whenever a flow attempts to reserve soft locks using their flow id (remember the flow id is always the flow id for the very first flow in the flow stack), we then save these states to the fiber. Then, upon releasing soft locks the fiber passes that set to the release soft locks query. That way the database query optimizer will use the primary key index of VAULT_STATES table, instead of lock_id_idx in order to search rows to update. That way the query will be aligned with the rest of the queries that are following that route as well (i.e. making use of the primary key), and therefore its locking order of resources within the database will be aligned with the rest queries' locking orders (solving SQL deadlocks).

* Fixed SQL deadlocks caused from softLockRelease, by saving locked states per fiber; NodeVaultService.softLockRelease query then uses VAULT_STATES PK index instead of lock_id_idx

Speed up SQL server by breaking down queries with > 16 elements in their IN clause, into sub queries with 16 elements max in their IN clauses

* Allow softLockedStates to remove states

* Add to softLockedStates only states soft locked under our flow id

* Fix softLockRelease not to take into account flowStateMachineImpl.softLockedStates when using lockId != ourFlowId

* Moved CriteriaBuilder.executeUpdate at the bottom of the file
2020-05-29 12:35:05 +01:00
Denis Rekalov
499c09e77b
Merge pull request #6285 from corda/denis/CORDA-3818-sync-identity-service
CORDA-3818: Synchronize OS implementation of PublicKeyToOwningIdentityCache with CE
2020-05-28 14:21:04 +01:00
Kyriakos Tharrouniatis
598228634f
CORDA-3608 Performance tune the new checkpoint schema (#6277)
Performance tuning of the new checkpoint schema;

- Checkpoint tables are now using `flowId` as join keys. 
- Indexes consist of a PK's index on `node_checkpoints(flow_id)` and then unique indexes on `node_checkpoint_blobs(flow_id)` and `node_flow_metadata(flow_id)`.
- Serialization of `checkpointState` is being done with `CHECKPOINT_CONTEXT` so that we can have compression. This is needed when messages get passed into `checkpointState.sessions` therefore `checkpointState` grows in size upon serialized and 
saved into the database.

* Deserialize checkpointState with CHECKPOINT_CONTEXT

* Align tests with schema update; We cannot add and update a checkpoint in the same session now, ends up with hibernate complaining: two different objects with same identifier

* Fix indentation and format

* Ignore tests that assert DBFlowResult or DBFlowException

* Set DBFlowCheckpoint.blob to null whenever the flow errors or hospitalizes; this way we save an extra SELECT in such cases;

* Fix test; cleared Hibernate session, it would fail at checkpoint with 'org.hibernate.NonUniqueObjectException'

* Changing VARCHAR to NVARCHAR

* Rename v17 liquibase scripts to v19 to resolve collision with ENT v17 scripts
2020-05-27 16:46:56 +01:00
Chris Rankin
6e156bc0ea Merge commit '6ebc6e9b16575a7afbb781b0d93a8f04b3affba5' into chrisr3-45-merge 2020-05-27 10:40:36 +01:00
Denis Rekalov
5afdb63c94 CORDA-3818: Synchronize OS implementation of PublicKeyToOwningIdentityCache with CE 2020-05-26 16:10:14 +01:00
Chris Rankin
6ebc6e9b16
CORDA-3750: Reimplement Corda's Crypto object for use inside the sandbox. (#6193)
* CORDA-3750: Use hand-written sandbox Crypto object that delegates to the node.

* CORDA-3750: Add integration test for deterministic CashIssueAndPayment flow.

* Tidy up generics for Array instances.

* Upgrade to DJVM 1.1-RC04.
2020-05-26 15:46:29 +01:00
Dan Newton
330a95cb68
ENT-4627 Log transaction context missing exception (#6278)
When a flow is missing its database transaction the _real_ stacktrace
of the exception is missing in the logs. This is due to the normal error
handling path way does not work due the transaction being missing. When
it goes to process the next error event (due to the original transaction
context missing error) it will fail for the same error as it needs the
context to be able to process the error event.

The extra logging should help diagnose future errors.
2020-05-26 11:32:42 +01:00
Christian Sailer
70f1ea0a9d
ENT-5258 db schema set-up only via command line flag (#6280)
Removing the ability to initialise schema from the node config, and add a new sub-command to initialise the schema (that does not do anything else and exits afterwards).
Also adding a command line flag that allow app schema to be maintained by hibernate for legacy cordapps, tests or rapid development.
Patching up mock net and driver test frameworks so they create the required schemas for tests to work, defaulting schema migration and hibernate schema management to true to match pre-existing behaviour.
Modified network bootstrapper to run an initial schema set-up so it can register nodes.
2020-05-22 16:27:10 +01:00
Kyriakos Tharrouniatis
57b1a5e0fc
ENT-5339 Failing tests against Oracle in VaultObserverExceptionTest (#6275)
* Fix erroneous sql statement for oracle; It was failing tests with 'ORA-00933: SQL command not properly ended'

* Fixed flaky test; it didn't wait for counter party flow to get hospitalized as the test implied
2020-05-22 10:15:51 +01:00
Rick Parker
67298eb780
Merge pull request #6270 from corda/nnagy-os-4.5-os-4.6-20200520
Nnagy os 4.5 os 4.6 20200520
2020-05-21 13:19:58 +01:00
Viktor Kolomeyko
31612901cf
CORDA-3800: Flaky ProtonWrapperTests test investigation (#6258) 2020-05-21 12:33:56 +01:00
nikinagy
301fc9f928 Merge branch 'release/os/4.5' into nnagy-os-4.5-os-4.6-20200520 2020-05-20 10:38:45 +01:00
nikinagy
938d038c97
ENT-5278 - add missed checks for empty lists (#6261)
* add missed checks for empty lists

* adding missing brackets

* adding missing timeout
2020-05-19 16:55:09 +01:00
Dan Newton
36a11e868a
NOTICK Fix timing issue in FlowIsKilledTest (#6264)
Also remove `Configurator` that seems to fail on some agents.
2020-05-19 16:54:34 +01:00
williamvigorr3
eb52de1b40
CORDA-3490 Add option to start node without starting checkpointed flows (#6136)
Added command-line option: `--pause-all-flows` to the Node to control this.
This mode causes all checkpoints to be set to status PAUSED when the
state machine starts up (in StartMode.Safe mode). 

Changed the state machine so that PAUSED checkpoints are loaded into
memory (the checkpoint is deserialised but the flow state is left serialised)
but not started.

Messages from peers are queued whilst the flow is paused and processed
once the flow is resumed.
2020-05-19 16:27:41 +01:00
nikinagy
cc8ce3ca99
empty list checks (#6262) 2020-05-19 14:04:19 +01:00
Adel El-Beik
b9b0ca58e3 Merge remote-tracking branch 'origin/release/os/4.5' into adel/merge-from-4.5 2020-05-15 10:23:17 +01:00
Ryan Fowler
6dd72ac079
ENT-5275: Replace hardcoded platform versions with descriptive values (#6236)
* ENT-5275: Replace hardcoded platform versions with descriptive values

* PR updates

* missed versions

* missed versions

* fix

* add FIRST_VERSION
2020-05-14 14:43:52 +01:00
James Higgs
06467d04b1
EG-1980 - Remove stale docs links from OS (#6251)
Remove hardcoded docs references, push docs link to gradle
2020-05-14 12:57:59 +01:00
Denis Rekalov
8387e99535
[NOTICK]: Do not use Security.addProvider(BouncyCastleProvider()) in tests to avoid disruptions of other tests (#6250) 2020-05-14 10:53:54 +01:00
Ivan Schasny
bd657fad65
Merge pull request #6245 from corda/EG-1991-ConfigHelper-update
EG-1991 Ignore warning for CORDA environment variables in all caps
2020-05-13 15:37:42 +01:00
Dan Newton
c2ac22a41f
NOTICK Non-database error handling in withEntityManager (#6239)
When a non-database exception is thrown out of a `withEntityManager`
block, always check if the session needs to be rolled back.

This means if a database error is caught and a new non-database error is
thrown out of the `withEntityManager` block, the transaction is still
rolled back. The flow can then continue progressing as normal.
2020-05-13 15:17:51 +01:00
john.buckle@r3.com
febb5a44b7 Added comment 2020-05-13 21:55:04 +08:00
Ryan Fowler
9caf6538ce
ENT-5298: Increase timeout by two minutes because the test occasionally takes longer than five minutes (#6246) 2020-05-13 13:15:05 +01:00
john.buckle@r3.com
c3ea2c1470 Ignore warning for CORDA environment variables in all caps 2020-05-13 09:51:35 +08:00
nikinagy
e0b5319515 Merge branch 'release/os/4.4' into nnagy-os-4.4-os-4.5-20200511 2020-05-11 16:32:53 +01:00
nikinagy
350066d386
fix for handling empty lists in vault query (#6231) 2020-05-11 15:41:50 +01:00
Ryan Fowler
5f8cceae45 Merge branch 'release/os/4.4' into rfowler-os-4.4-os-4.5-20200511 2020-05-11 11:40:09 +01:00
LankyDan
8125b9bdb6 NOTICK Change platform version assert in FlowMetadataRecordingTest 2020-05-06 10:03:05 +01:00
LankyDan
565afc5fdb NOTICK Fix kill flow tests due to storing failed flows
Failed flows are stored after the checkpoint table rework. This meant
that some of the asserts in `FlowIsKilledTest` and `KillFlowTest` were
wrong.
2020-05-06 10:02:40 +01:00
Chris Rankin
9a5be9e47d
CORDA-3738: Upgrade to DJVM 1.1-RC03. (#6219) 2020-05-05 17:32:39 +01:00
LankyDan
217fd906b3 NOTICK Remove unused imports 2020-05-05 17:22:42 +01:00
LankyDan
7fe2845272 NOTICK Fix KilledFlowTransition after merge 2020-05-05 17:15:54 +01:00
LankyDan
4ccd0fd3df Merge branch 'release/os/4.6' into dan/4.6-into-checkpoint-feature-branch-2020-05-05
# Conflicts:
#	node/src/main/kotlin/net/corda/node/services/statemachine/Event.kt
#	node/src/main/kotlin/net/corda/node/services/statemachine/FlowStateMachineImpl.kt
#	node/src/main/kotlin/net/corda/node/services/statemachine/SingleThreadedStateMachineManager.kt
#	node/src/main/kotlin/net/corda/node/services/statemachine/transitions/TopLevelTransition.kt
2020-05-05 17:05:23 +01:00
Ryan Fowler
b6dc1d8c4a
NOTICK - Reduce flow count for test to improve run time (#6213) 2020-05-05 09:38:55 +01:00
nargas-ritu
54e4503866
Merge pull request #6212 from corda/dan/update-byteman-dependency
NOTICK Update byteman dependencies
2020-05-04 16:29:29 +01:00
Ryan Fowler
55797612b2 ENT-5237: Remove DISTINCT (backport) 2020-05-04 15:02:33 +01:00
LankyDan
96c1120194 NOTICK Update byteman dependencies
Tests that were using byteman when running on Java 11 were failing.
Updating the byteman version fixed the issue while still working on Java
8.
2020-05-04 14:08:38 +01:00
Denis Rekalov
32407b2c46
CORDA-3753: Increase Artemis security-invalidation-interval to avoid frequent CRL checks (#6207) 2020-05-04 13:00:22 +01:00
Ryan Fowler
ebdd40049c CORDA-3662: Use an INNER JOIN for network map cache queries, (#6062)
- rename add or update function for clarity
- put removal of old nodes after retrieval of new ones to avoid gaps in the map
- plus add a test
2020-05-04 11:10:21 +01:00
Chris Rankin
107819f5b5
CORDA-3745: Modify DJVM serializers to support Enum Evolution. (#6189) 2020-04-30 14:59:10 +01:00
Joseph Zuniga-Daly
2ce76e407d
ENT-5237: Remove DISTINCT because Oracle cannot apply DISTINCT to BLOB fields (#6203) 2020-04-30 14:27:45 +01:00
Katelyn Baker
99fd08909d
EG-433 - Error Reporting Framework (Merge)
Error Reporting Framework - Phase 1
2020-04-30 13:12:54 +01:00
Chris Rankin
83dd9a96da
CORDA-3738: Upgrade to DJVM 1.1-RC02. (#6195)
* CORDA-3738: Upgrade to DJVM 1.1-RC02.

* Update comment for DJVM 1.1's new requirements.
2020-04-30 11:58:03 +01:00
Adel El-Beik
9bcb9b2f54
CORDA-3191: When querying for attachment ids now only retrieve the attachment id columm not whole object. (#6199) 2020-04-30 11:52:47 +01:00
Dan Newton
cb84fd86ee
CORDA-3291 Fix broken tests (#6197) 2020-04-30 10:27:36 +01:00
Adel El-Beik
3259b595d7
CORDA-3715: Check contract classes hav… (#6155)
* CORDA-3715: When loading cordapps now check that contract classes have class version between 49 and 52

* CORDA-3715: Now check class version when contract verification takes place.

* CORDA-3715: Making detekt happy with number of levels in func

* CORDA-3715: Make use of new ClassGraph release which provides class file major version number.

* CORDA-3715: Changed package name in test jars

* CORDA-3715: Use ClassGraph when loading attachments.

* CORDA-3715: Reverted file to 4.5 version

* CORDA-3715: Updating method to match non deterministic version.

* CORDA-3715: Added in default param.

* CORDA-3715: Adjusted min JDK version to 1.1

* CORDA-3715: Switching check to JDK 1.2

* CORDA-3715: Now version check SerializationWhitelist classes.

* CORDA-3715: Switched default to null for range.
2020-04-30 08:57:37 +01:00
James Higgs
1956bdb5d4 [MERGE] Fix detekt issues 2020-04-29 11:57:37 +01:00
James Higgs
adbe030a2c Merge branch 'release/os/4.5' into jamesh/error-reporting-sync-29-04-20
# Conflicts:
#	node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt
2020-04-29 11:47:14 +01:00
James Higgs
ab95aa57a2
[EG-440] Add some error codes and the error resource generation tool (#6192)
* [EG-438] First commit of error code interface

* [EG-438] Implement error reporter and a few error codes

* [EG-438] Add unit tests and default properties files

* [EG-438] Add the error table builder

* [EG-438] Update initial properties files

* [EG-438] Add some Irish tests and the build.gradle

* [EG-438] Fall back for aliases and use different resource strategy

* [EG-438] Define the URL using a project-specific context

* [EG-438] Tidy up initialization code

* [EG-438] Add testing to generator and tidy up

* [EG-438] Remove direct dependency on core and add own logging config

* [EG-438] Fix compiler warnings and tidy up logging

* [EG-438] Fix detekt warnings

* [EG-438] Improve error messages

* [EG-438] Address first set of review comments

* [EG-438] Use enums and a builder for the reporter

* [EG-438] Address first set of review comments

* [EG-438] Use enums and a builder for the reporter

* [EG-438] Add kdocs for error resource static methods

* [EG-440] Add error code for duplicate CorDapp loading

* [EG-438] Handle enums defined with underscores

* [EG-440] Add errors for some CorDapp loading scenarios

* [EG-440] Finish adding errors for CorDapp loading

* [EG-440] Fix up errors in properties files

* [EG-440] Start change to error code definition

* [EG-440] Update error code definition and add resource generation tool

* [EG-440] Tidy up error resource generation tool frontend

* [EG-440] Small refactorings and add kdocs

* [EG-440] Generate all missing resources

* [EG-440] Some refactoring and start writing a test

* [EG-440] Update unit test for resource generator

* [EG-440] Renaming of various parts of the error tool

* [EG-440] Add testing for errors and fix an issue in resource generation

* [EG-440] Add a kdoc for context provider API

* [EG-440] Remove old code from repository

* [EG-440] Address some review comments
2020-04-29 11:21:50 +01:00
Dan Newton
297e504740
CORDA-3291 isKilled flag and session errors for killed flows (#6170)
* CORDA-3291 `isKilled` flag and session errors for killed flows

## Summary

Two major improvements have been worked on:

- A new flag named `isKilled` has been added to `FlowLogic` to allow
developers to break out of loops without suspension points.
- Killed flows now send session errors to their counter parties allowing
their flows to also terminate without further coordination.

Achieving these changes required a __fundamental__ change to how flows are
killed as well as how they sleep.

## `isKilled` flag

The addition of `FlowLogic.isKilled` allows flows to check if the
current flow has been killed. They can then throw an exception to lead
to the flow's termination (following the standard error pathway). They
can also perform some extra logic or not throw an exception if they
really wanted to.

No matter what, once the flag is set, the flow will terminate. Due to
timing, a killed flow might successfully process its next suspension
event, but it will then process a killed transition and terminate.

## Send session errors when killing a flow

A flow will now send session errors to all of its counter parties. They
are transferred as `UnexpectedFlowEndException`s. This allows initiated
flows to handle these errors as they see fit, although they should
probably just terminate.

## How flows are killed

### Before

Originally we were relying on Quasar to interrupt a flow's fiber, we
could then handle the resulting `InterruptedException`. The problem with
this solution is that it only worked when a flow was already suspended
or when a flow moved into suspension. Flows stuck in loops did not work.

### After

We now *do not* use Quasar to interrupt a flow's fiber. Instead, we
switch `FlowStateMachine.isKilled` to true and schedule a new event.
Any event that is processed after switching this flag will now cause a
`KilledFlowTransition`. This transition follows similar logic to how
error propagation works. Note, the extra event allows a suspended flow
to be killed without waiting for the event that it was _really_ waiting
for.

This allows a lot of the tidy up code in `StateMachineManager.killFlow`
to be removed as tidy up is executed as part of removing a flow.
Deleting a flow's checkpoint and releasing related soft locks is still
handled manually in case of infinite loops but also triggered as part
of the actions executed in a transition.

This required flow sleeping to be changed as we no longer rely on
quasar.

## How flows now sleep

The reliance on Quasar to make a flow sleep has been removed.

Instead, when a flow sleeps we create a `ScheduledFuture` that is
delayed for the requested sleep duration. When the future executes it
schedules a `WakeUpFromSleep` event that wakes up the flow... Duh.

`FlowSleepScheduler` handles the future logic. It also uses the same
scheduled thread pool that timed flows uses.

A future field was added to `StateMachineState`. This removes the 
need for concurrency control around flow sleeps as the code path does
not need to touch any concurrent data structures.

To achieve this:

- `StateMachineState.future` added as a `var`
- When the `ScheduledFuture` is created to wake up the flow the passed
in `StateMachineState` has its `future` value changed
- When resumed `future` and `isWaitingForFuture` are set to `null` and
`false` respectively
- When cancelling a sleeping flow, the `future` is cancelled and nulled
out. `isWaitingForFuture` is not changed since the flow is ending anyway
so really the value of the field is not important.
2020-04-28 15:53:44 +01:00
James Higgs
ab43238420
[EG-438] Error Reporting Framework (#6125)
* [EG-438] First commit of error code interface

* [EG-438] Implement error reporter and a few error codes

* [EG-438] Add unit tests and default properties files

* [EG-438] Add the error table builder

* [EG-438] Update initial properties files

* [EG-438] Add some Irish tests and the build.gradle

* [EG-438] Fall back for aliases and use different resource strategy

* [EG-438] Define the URL using a project-specific context

* [EG-438] Tidy up initialization code

* [EG-438] Add testing to generator and tidy up

* [EG-438] Remove direct dependency on core and add own logging config

* [EG-438] Fix compiler warnings and tidy up logging

* [EG-438] Fix detekt warnings

* [EG-438] Improve error messages

* [EG-438] Address first set of review comments

* [EG-438] Use enums and a builder for the reporter

* [EG-438] Add kdocs for error resource static methods

* [EG-438] Handle enums defined with underscores

* [EG-438] Slight refactoring of startup code

* [EG-438] Port changes to error reporting code from future branch

* [EG-438] Also port test changes

* [EG-438] Suppress a deliberately unused parameter
2020-04-28 14:07:50 +01:00
Dan Newton
9a2ae8ae19
CORDA-3722 withEntityManager can rollback its session (#6187)
* CORDA-3722 withEntityManager can rollback its session

## Summary

Improve the handling of database transactions when using
`withEntityManager` inside a flow.

Extra changes have been included to improve the safety and
correctness of Corda around handling database transactions.

This focuses on allowing flows to catch errors that occur inside an
entity manager and handle them accordingly.

Errors can be caught in two places:

- Inside `withEntityManager`
- Outside `withEntityManager`

Further changes have been included to ensure that transactions are
rolled back correctly.

## Catching errors inside `withEntityManager`

Errors caught inside `withEntityManager` require the flow to manually
`flush` the current session (the entity manager's individual session).
By manually flushing the session, a `try-catch` block can be placed
around the `flush` call, allowing possible exceptions to be caught.

Once an error is thrown from a call to `flush`, it is no longer possible
to use the same entity manager to trigger any database operations. The
only possible option is to rollback the changes from that session.
The flow can continue executing updates within the same session but they
will never be committed. What happens in this situation should be handled
by the flow. Explicitly restricting the scenario requires a lot of effort
and code. Instead, we should rely on the developer to control complex
workflows.

To continue updating the database after an error like this occurs, a new
`withEntityManager` block should be used (after catching the previous
error).

## Catching errors outside `withEntityManager`

Exceptions can be caught around `withEntityManager` blocks. This allows
errors to be handled in the same way as stated above, except the need to
manually `flush` the session is removed. `withEntityManager` will
automatically `flush` a session if it has not been marked for rollback
due to an earlier error.

A `try-catch` can then be placed around the whole of the
`withEntityManager` block, allowing the error to be caught while not
committing any changes to the underlying database transaction.

## Savepoints / Transactionality

To make `withEntityManager` blocks work like mini database transactions,
save points have been utilised. A new savepoint is created when opening
a `withEntityManager` block (along with a new session). It is then used
as a reference point to rollback to if the session errors and needs to
roll back. The savepoint is then released (independently from
completing successfully or failing).

Using save points means, that either all the statements inside the
entity manager are executed, or none of them are.

## Some implementation details

- A new session is created every time an entity manager is requested,
but this does not replace the flow's main underlying database session.
- `CordaPersistence.transaction` can now determine whether it needs
to execute its extra error handling code. This is needed to allow errors
escape `withEntityManager` blocks while allowing some of our exception
handling around subscribers (in `NodeVaultService`) to continue to work.
2020-04-28 11:20:00 +01:00
nikinagy
ec96a844bd
CORDA-3659 - adding kdoc for RestrictedEntityManager and RestrictedConnection (#6179)
* adding kdoc for RestrictedEntityManager and RestrictedConnection

* adding kdoc for RestrictedEntityManager and RestrictedConnection
2020-04-28 09:35:47 +01:00
Matthijs van den Bos
a298a94960
Load CordaServices before NotaryService (#6173)
On node start, load CordaServices before starting the NotaryService,
so that the NotaryService can check that the services it requires are
available when starting.

Resolves #6172.
2020-04-24 09:15:38 +01:00
nikinagy
335372cfc1 Merge branch 'release/os/4.3' into nnagy-os-4.3-os-4.4-20200423 2020-04-23 16:03:17 +01:00
Joseph Zuniga-Daly
69a4f80cd2
ENT-5141: Fix ConcurrentModificationException in FetchDataFlow (#6176)
* ENT-5141: Fix ConcurrentModificationException in FetchDataFlow

* Make detekt happy

* Make CheckpointSerializationEnvironmentRule inheritable
2020-04-23 13:11:23 +01:00
Viktor Kolomeyko
257026f606
CORDA-3726: Remove memory leaks when many InProcess nodes started (#6169)
* CORDA-3762: Integration test exposing the problem reported

* CORDA-3726: Additional logging

* CORDA-3726: Prevent thread leaks

* CORDA-3726: New `journalBufferTimeout` parameter

* CORDA-3726: Override `journalBufferTimeout` parameter

* CORDA-3726: Making Detekt happier

* CORDA-3276: Account for extra thread user in MockNetwork

For real node this does not matter as `shutdown` can safely be called multiple times, which is not true for server thread provided by MockNetwork

* CORDA-3276: Do not make SMM shutdown "executor" as it belongs to AbstractNode

* CORDA-3276: Address input from @rick-r3

* CORDA-3276: Fix test after rebase
2020-04-23 08:53:51 +01:00
Denis Rekalov
0d441c3760
ENT-4912: Enable CRL checking with embedded Artemis (#6154) 2020-04-22 10:35:58 +01:00
nikinagy
2bcaa2ac80
CORDA-3569 - Add RestrictedConnection and more blocked methods to RestrictedEntityManager (#6129)
* adding blocked functions ro RestrictedEntityManager and creating RestrictedConnection class

* adding flow tests and fixing issues regarding the review

* adding quasar util to gradle

* updating flow tests

* adding space before } at .isThrownBy()

* adding spaces
2020-04-21 14:39:41 +01:00
Dimos Raptis
8faf72f7b5
[ENT-5210] - Whitelist SNAPPY encoding (#6163)
* [ENT-5210] - Whitelist SNAPPY encoding

* Remove unused imports
2020-04-20 08:09:38 +01:00
Ramzi El-Yafi
45b43f116d
[EG-503] Test cases for exposed notaryService (#6161) 2020-04-17 16:51:27 +01:00
Ramzi El-Yafi
6a07284324
[EG-503] Spent state audit tool (#6107)
* [EG-503] Spent state audit tool

Fixes

* Refinements to notary query interfaces. Feature complete.

* EG-503: Introduce optional `notaryService` in `ServiceHubCoreInternal`

* Remove redundant logic following change to use extensions API

Co-authored-by: Viktor Kolomeyko <viktor.kolomeyko@r3.com>
2020-04-16 16:05:21 +01:00
Adel El-Beik
27ea570fbb
CORDA-3696: JDK 11 Testing branch (#6131)
* CORDA-3696: Temporary update to enable JDK11 build and test. Will eventually be switchable.

* CORDA-3696: Filter out the Nashorn warning.

* CORDA-3696: Add JDK11 classifier.

* CORDA-3696: Updated match string to cope with JDK11.

* CORDA-3696: Filtering out SPHINCS256_SHA256 where failing due to JDK11.

* CORDA-3696: Now remove SPHINCS256_SHA256 only if JDK11.

* CORDA-3696: Fix test failure - switch to regex matching.

* CORDA-3696: Hide the illegal access warnings.

* CORDA-3696: Check for Java11 when disabling Java11 warnings.

* CORDA-3696: Fix unneccessary non null check.

* CORDA-3696: Reverting build env to JDK8

* CORDA-3696: Revert hiding of illegal access warnings via Unsafe class.

* CORDA-3696: Remove internal access warnings and new JDK11 version checker.

* CORDA-3696: Updated build file for OS

* CORDA-3696: Removed typo

* CORDA-3696: Fixed space typo.

* CORDA-3696: Open modules to remove the illegal access warnings.

Co-authored-by: Adel El-Beik <adelel-beik@19LDN-MAC108.local>
2020-04-16 10:20:30 +01:00
Dan Newton
896b0ab246
CORDA-3691 delete checkpoint record when complete (#6134)
* CORDA-3691 Delete checkpoint when flow finishes

The checkpoint and its related records in joined tables should be deleted
when a flow finishes.

Keeping these flows around will be completed in the future.

* CORDA-3691 Ignore some flow metadata tests

Ignore tests around recording the finish time of flow metadata records
since we are not currently keeping COMPLETED flows in the database.
2020-04-09 16:57:03 +01:00
Kyriakos Tharrouniatis
501b766e71
CORDA-3604 Store failed and hospitalised errors along with corresponding statuses (#6061)
Flows that are kept for overnight observation:

- Save their Checkpoint.status as 'HOSPITALIZED' in the database
- Save the error that caused the hospitalization in the database

A new Event was added for this reason. Whenever the hospital determines
a flow for hospitalization, it adds this Event in the flow's fiber queue.
When processed it creates a new DB transaction, stores the checkpoint status along with
the error, and it adds a 'FlowContinuation.ProcessEvents' continuation so that the fiber keeps
processing events (effectively since there are no more events in the fiber's channel, the fiber will suspend).

Flows that error:

- Their checkpoints are kept in the database
- Save their Checkpoint.status as 'FAILED'
- Save the error that caused the error in the database

Upon erroring, the flow's Checkpoint.status gets updated('FAILED') and the checkpoint is stored
in the database instead of getting removed. The flow then propagates the error to counterparties,
sets its future with the error and gets removed from memory.
2020-04-07 17:58:00 +01:00
Chris Rankin
39cc5e3403
CORDA-3698: Require no classifier for Open Core and DJVM-related modules. (#6132)
* ENT-4967: Require no classifier for corda-node-djvm, corda-deserializers-djvm.

* Also remove classifiers from core, serialization and finance-contracts.

* Compile corda-serialization-djvm for Java 8 and remove its classifier.
2020-04-06 11:00:40 +01:00
Viktor Kolomeyko
1d7c13276c
ENT-5089: Forcibly free associated with transport resources if it is already closed (#6130) 2020-04-03 17:30:29 +01:00
Stefan Iliev
1718af3a79
Notary logs warning on startup when it's not in the whitelist (#5857) 2020-04-02 10:50:03 +01:00
Chris Rankin
a7358ffa1a
CORDA-3643: Initialise MockNetwork's serialisation environment with all custom cordapps. (#6115) 2020-04-01 17:42:29 +01:00
Chris Rankin
0771a22d5b
ENT-4967: Allow quasar_classifier to be either null or an empty string. (#6119) 2020-03-31 15:32:36 +01:00
williamvigorr3
024d63147d
CORDA-3491 Remove the flow state when a flow finishes (#6083)
Added a new field Completed to the in-memory object FlowState.

FlowState.Completed is corresponds to flow_state=Null in the DB.

This change will save disk space.
2020-03-30 16:56:03 +01:00
Chris Rankin
f1ebaa761b
CORDA-3680: Add CorDapp custom serialisers to Driver's in-process nodes. (#6098)
* Run serialisation tests with both in-process and out-of-process nodes.

* Add custom serialisers and whitelists to Driver's AMQPServerSerializationScheme.
2020-03-27 10:01:30 +00:00
Dan Newton
678fb6eb94
Merge pull request #6108 from corda/dan/os-4.4-to-4.5-merge-2020-03-26
OS 4.4 to 4.5 merge 2020-03-26
2020-03-27 09:39:41 +00:00
Chris Rankin
cccbbe1c80
CORDA-3680: Add CorDapp custom serialisers to Driver's in-process nodes. (#6102)
* Run serialisation tests with both in-process and out-of-process nodes.

* Add custom serialisers and whitelists to Driver's AMQPServerSerializationScheme.
2020-03-27 09:16:31 +00:00
Dimos Raptis
963de40902
[NOTICK] - Enable check in detekt for unused imports (#6106)
* [NOTICK] - Enable check in detekt for unused imports

* Put back accidental removal of used import

* Some more accidental removals
2020-03-26 15:46:33 +00:00
LankyDan
d9c1907c88 Merge branch 'release/os/4.4' into dan/os-4.4-to-4.5-merge-2020-03-26 2020-03-26 14:26:41 +00:00
Kyriakos Tharrouniatis
6baa775e23 Merge branch 'release/os/4.5' into os_4.5-feature_checkpoint_table_improvements-merge 2020-03-26 11:37:00 +00:00
Joseph Zuniga-Daly
b766eff284 Merge remote-tracking branch 'origin/release/os/4.3' into jzd/merge-os4.3-into-os4.4-2020-03-26 2020-03-26 11:01:39 +00:00
Dan Newton
79b36aea8f
CORDA-3601 Record a flow's finish time (#6079)
* CORDA-3601 Record a flow's finish time

Record a flow's finish time by updating its metadata record. It is set
in `updateCheckpoint` by checking the status of the checkpoint. If it is
 `COMPLETED` it will set the `finishInstant` on the metadata object and
 update it.

* CORDA-3601 Record flow finish time for all finished statuses

Update the flow finish time for the following statuses:

- COMPLETED
- KILLED
- FAILED

* CORDA-3601 Use platform clock in `DBCheckpointStorage`
2020-03-25 13:47:00 +00:00
Denis Rekalov
b1180b467b ENT-4659: Remove CryptoServiceFactory from OS 2020-03-25 12:56:33 +00:00
Joseph Zuniga-Daly
2dbf90cafe
ENT-4857: Fix race condition in trackTransaction (#6096)
- Fix issue
- Emit warning if we are inside a DB transaction
- Include a path that does not emit warning
- Add unit tests
2020-03-25 11:53:06 +00:00
Dimos Raptis
b73a498062
[ENT-4754] - Move subflow preparation logic in FlowStateMachine 2020-03-25 09:02:14 +00:00
Ramzi El-Yafi
c697b5850b
[EG-896] Additional uniqueness provider test cases (#6085) 2020-03-24 11:10:27 +00:00
LankyDan
82e068be94 Merge branch 'release/os/4.4' into dan/os-4.4-to-4.5-merge-2020-03-23 2020-03-23 11:46:24 +00:00
Dan Newton
668748b054
CORDA-3669 Do not execute ExecuteAsyncOperation multiple times (#6087)
* CORDA-3669 Do not execute `ExecuteAsyncOperation` multiple times

When a `FlowExternalOperation` or `FlowExternalAsyncOperation` executes
and completes a flag (`isFlowResumed`) is switched to true.

This flag was used inside of `DoRemainingWorkTransition` to decide
whether to skip over the execution of an event.

Since this flag was being switched to true when the external operation's
 future completed, it was possible for _unexpected_ events to be placed
in the fiber's queue that would retrigger the
`FlowIORequest.ExecuteAsyncOperation`, that is held as the checkpoint's
next `FlowIORequest`to process.

By using the existing `StateMachineState.isTransactionTracked` (and
renaming it to `isWaitingForFuture`) we can decide to not process the
`FlowIORequest.ExecuteAsyncOperation` if it has already been called
before. This moves this code path in line with
`FlowIORequest.WaitForLedgerCommit`.

Random `DoRemainingWork` events can now be pushed to the fiber's queue
without causing the `FlowIORequest.ExecuteAsyncOperation` to execute
again.
2020-03-20 19:02:34 +00:00
Kyriakos Tharrouniatis
da320514a5 Merge branch 'release/os/4.5' into os_4.5-feature_checkpoint_table_improvements-merge
# Conflicts:
#	node/src/main/kotlin/net/corda/node/services/statemachine/transitions/StartedFlowTransition.kt
2020-03-19 21:49:55 +00:00
Dan Newton
1c7126afb7
NOTICK Rename rpc_user column to started_by in metadata table (#6081) 2020-03-19 16:07:00 +00:00
Christian Sailer
82d9995717
ENT-5109 Harmonize config-common, make everything compile again and harmonize NetworkParameterOverridesSpec. (#6082) 2020-03-19 15:43:08 +00:00
Dimos Raptis
56067acd20
[CORDA-3628] - Remove overloads for sendAll (#6078) 2020-03-18 14:37:04 +00:00
Chris Rankin
29a36c6b4f
Prevent Quasar from instrumenting classes that belong to AttachmentsClassLoader. (#6077) 2020-03-18 13:59:10 +00:00
Dan Newton
ca23612fe1
CORDA-3596 Record flow metadata (#6067)
* CORDA-3596 Record flow metadata

Record flow metadata during the zero'th checkpoint that occurs before
calling the flow's `call` function.

This required adding an RPC call's arguments to the `InvocationContext`
that gets created. These arguments are then accessible within the
statemachine and from the `Checkpoint` class. The arguments are then
extracted when recording a flow's metadata inside of
`DBCheckpointStorage`.

Updated the size of the started by column to 128 since it was not long
enough to hold the fully qualified class of a service that started a
flow.

* CORDA-3596 Remove arguments from in-memory checkpoint

When executing a flows first real suspend (from flow code) the arguments
 contained in the `InvocationContext` are removed. This saves holding
 these arguments for the whole lifecyle of a flow.

* CORDA-3596 Increase `cordapp_name` column to 128

* CORDA-3596 Join metadata by `flow_id`

Due to changes in where metadata is recorded, there is no need for
having `invocation_id` as the metadata table's primary key. The
`flow_id` is now the primary key of the table and is used to join to the
 main checkpoints table.

The `invocation_id` has been removed from the checkpoints table since it
 is not needed for the join anymore.

* CORDA-3596 Remove `received_time` from metadata table

* CORDA-3596 Remove unused `StartReason` enum

* CORDA-3596 Simple `DBCheckpointStorageTests` for metadata

* CORDA-3596 Truncate really long flow names
2020-03-17 17:28:32 +00:00
Ryan Fowler
349bd5a511
CORDA-3662: Use an INNER JOIN for network map cache queries, (#6062)
- rename add or update function for clarity
- put removal of old nodes after retrieval of new ones to avoid gaps in the map
- plus add a test
2020-03-17 17:02:08 +00:00
Dimos Raptis
eba113621c
[CORDA-3628] - Avoid sending actions to the state machine if no messages are to be sent (#6074) 2020-03-17 16:51:07 +00:00
LankyDan
944fbce740 Merge branch 'feature/checkpoint_table_improvements' into dan/merge-4.5-into-feature-branch 2020-03-16 15:38:14 +00:00
Kyriakos Tharrouniatis
121c789c59
CORDA-3655 Do not run COMPLETED, FAILED or KILLED flows on node startup (#6070)
* CheckpointStorage.getAllCheckpoints will not fetch COMPLETED, FAILED and KILLED flows by default

* Rename getAllCheckpoints to getAllRunnableCheckpoints for clarity

* Fix Detekt issue

* Rename getAllRunnableCheckpoints to getRunnableCheckpoints

* Minor kdoc update

* Bring back in CheckpointStorage.getAllCheckpoints to co-exist with getRunnableCheckpoints
2020-03-16 15:25:36 +00:00