Commit Graph

8995 Commits

Author SHA1 Message Date
Waldemar Żurowski
922dbe8544
Merge pull request #6592 from corda/wz/merge-os-45-46-2020-08-07
NOTICK: Merging forward updates from OS 4.5 to OS 4.6
2020-08-07 13:23:49 +01:00
Waldemar Zurowski
affd9ebad6 Merge branch 'release/os/4.5' into wz/merge-os-45-46-2020-08-07 2020-08-07 12:24:13 +02:00
Waldemar Żurowski
e155cf8d96
Merge pull request #6590 from corda/wz/merge-os-44-45-2020-08-07
NOTICK: Merging forward updates from OS 4.4 to OS 4.5
2020-08-07 11:22:33 +01:00
Waldemar Zurowski
8e368a2b76 Synched JDK11 Azul builds configuration with regression build
* carries over INFRA-541 and INFRA-542
2020-08-07 11:44:14 +02:00
Tamas Veingartner
e6af60edda
NOTICK Migrate recent RPC related changes to OS, as these were initially imp… (#6532)
* Migrate recent RPC related changes to OS, as these were initially implemented in ENT only

* tests cleanup

* cleanup imports
2020-08-07 09:18:09 +01:00
Waldemar Zurowski
805e9e9f9f NOTICK: Merging forward updates from OS 4.4 to OS 4.5 2020-08-07 06:16:53 +02:00
Waldemar Zurowski
56b574c66e Fixed missing elements 2020-08-07 06:07:29 +02:00
Dan Newton
0d5ee8b0fa
NOTICK Save exception for hospitalized session init errors (#6587)
Save the exception for flows that fail during session init when they are
kept for observation.

Change the exception tidy up logic to only update the flow's status if
the exception was removed.
2020-08-06 22:35:05 +01:00
Chris Rankin
ef27dbfdbb
INFRA-563: Upgrade to distributed-testing-plugin 1.3.0. (#6588) 2020-08-06 22:13:57 +01:00
Waldemar Żurowski
e524f41738
INFRA-542: Allow change of the default Nexus IQ stage parameter (#6586)
* adds a parameter to a build
* default value of the parameter is the same as original stage parameter
2020-08-06 20:20:19 +01:00
Waldemar Żurowski
d19c6dddaf
INFRA-541: Remove obsolete jenkins prefix from Nexus AppID (#6585) 2020-08-06 20:17:15 +01:00
Yiftach Kaplan
849d51c8cd
INFRA-505: Move integration tests to unit tests (#6530) 2020-08-06 15:16:27 +01:00
Rick Parker
02b71845bb
Merge pull request #6583 from corda/feature_pass_in_client_id_when_starting_a_flow-os_4.6
ENT-4565 - It is always clear whether a flow has been successfully started via RPC
2020-08-06 15:15:13 +01:00
Dan Newton
b4235c6842
Merge pull request #6581 from corda/os_4.6-feature_pass_in_client_id_when_starting_a_flow
NOTICK - OS 4.6 to feature/pass_in_client_id_when_starting_a_flow
2020-08-06 14:31:44 +01:00
Kyriakos Tharrouniatis
0005317fec Align code with merging changes 2020-08-06 13:35:46 +01:00
Kyriakos Tharrouniatis
2afedeabb4 Merge branch 'release/os/4.6' into os_4.6-feature_pass_in_client_id_when_starting_a_flow 2020-08-06 13:21:45 +01:00
Rick Parker
f280ec9fd9
Merge pull request #6578 from corda/WillV/ENT-5395-Pause-and-Resume-Flows
ENT-5395 Pause and Resume Flows
2020-08-06 11:47:38 +01:00
Dan Newton
3f31aeaa5f
CORDA-3822 Add CordaRPCOps.reattachFlowWithClientId (#6579)
Add `CordaRPCOps.reattachFlowWithClientId` to allow clients to reattach
to an existing flow by only providing a client id. This behaviour is the
same as calling `startFlowDynamicWithClientId` for an existing
`clientId`. Where it differs is `reattachFlowWithClientId` will return
`null` if there is no flow running or finished on the node with the same
client id.

Return `null` if record deleted from race-condition
2020-08-06 11:42:02 +01:00
Will Vigor
7acc510534 CORDA-3602 Set a Checkpoint as incompatible if it can't be deserialised (#3653)
Update the compatible flag in the DB if the flowstate cannot be deserialised.

The most common cause of this problem is if a CorDapp has been upgraded
without draining flows from the node.

`RUNNABLE` and `HOSPITALISED` flows are restored on node startup so
the flag is set for these then. The flag can also be set when a flow
retries for some reason (see retryFlowFromSafePoint) in this case the
problem has been caused by another reason.
2020-08-06 11:00:02 +01:00
Will Vigor
4a828fcb99 ENT-5397 Pause individual running and hospitalised flows (#3564)
Added a newpause event to the statemachine which returns an Abort
continuation and causes the flow to be moved into the Paused flow Map.

Flows can receive session messages whilst paused.
2020-08-06 10:40:09 +01:00
Will Vigor
bbf5a93761 ENT-5396 Allow Retrying a Hospitalised Flow from the Statemachine (#3499)
Added functionality to the statemachine to enable retrying a Hospitalised flow without a node restart.
2020-08-06 10:11:15 +01:00
Dan Newton
a73dad00e2
CORDA-3850 Add a per flow lock (#6437)
Add a lock to `StateMachineState`, allowing every flow to lock
themselves when performing a transition or when an external thread (such
as `killFlow`) tries to interact with a flow from occurring at the same
time.

Doing this prevents race-conditions where the external threads mutate
the database or the flow's state causing an in-flight transition to
fail.

A `Semaphore` is used to acquire and release the lock. A `ReentrantLock`
is not used as it is possible for a flow to suspend while locked, and
resume on a different thread. This causes a `ReentrantLock` to fail when
releasing the lock because the thread doing so is not the thread holding
the lock. `Semaphore`s can be used across threads, therefore bypassing
this issue.

The lock is copied across when a flow is retried. This is to prevent
another thread from interacting with a flow just after it has been
retried. Without copying the lock, the external thread would acquire the
old lock and execute, while the fiber thread acquires the new lock and
also executes.
2020-08-06 09:51:42 +01:00
Kyriakos Tharrouniatis
5ba8477733
CORDA-3491 - Do not keep flow state in memory after a flow has finished (#6573)
Do not retain in database `checkpoint.flowState` for flows that have completed 
or failed and have started with a client id, after their lifetime.
2020-08-06 09:46:04 +01:00
Ross Nicoll
fd374bfc6d
CORDA-3943 Remove sleeps from flow reload tests (#6567)
* Remove use of Thread.sleep() FROM FlowReloadAfterCheckpointTest, instead relying on CountdownLatch to wait until the target number has been hit or a timeout occurs, so the thread can continue as soon as the target is hit.
* Replace use of hashmaps to a concurrent queue, to mitigate risk of complex threading issues.
2020-08-05 21:10:11 +01:00
Anurag Garg
01451d57b9
Merge pull request #6577 from corda/adel/CORDA-3950
CORDA-3950: Now disable SPHINCS for JDK8 as well as JDK11. Broken in …
2020-08-05 19:11:52 +01:00
Adel El-Beik
4a5f928ac3 CORDA-3950: Doh. Fix the unused import. 2020-08-05 16:37:36 +01:00
Adel El-Beik
f952021b58 CORDA-3950: Now disable SPHINCS for JDK8 as well as JDK11. Broken in JDK1.8_252 2020-08-05 16:30:38 +01:00
Jonathan Locke
badfcf7f07
CORDA-3942: Disable IRSDemoTest (#6564)
Temporarily disable IRSDemoTest as it is breaking many OS builds.
2020-08-05 15:49:12 +01:00
Ross Nicoll
b779fb354c
CORDA-3948 Disable flaky test (#6572) 2020-08-05 14:16:15 +01:00
Christian Sailer
6f2ca8978d
Merge pull request #6534 from corda/feature/ENT-5273-unified-database-management
ENT-5273 unified database management
2020-08-05 14:09:07 +01:00
Dan Newton
f216853c3f
CORDA-3946 Make RetryFlowMockTest less flakey (#6570) 2020-08-05 13:10:35 +01:00
Chris Rankin
d60feb1138
INFRA-563 Update Corda to test distributed-testing-plugin 1.3-SNAPSHOT. (#6551)
Integrate YAML profile support, and the eagle-eyed will notice that the plugin no longer needs to be applied at the very bottom of the build.gradle file!
 
Other features include:
* Implicit upgrade to docker-remote-api plugin v5.3.0
* Fixing a ClassGraph-related memory leak by closing the ScanResult objects after use.
* More logging of any exceptions from Kubenetese.
* The gradlecache volume is now created with a hostPath of "/gradle/$podName/$podIdx-$taskForExecuteName", which should allow having multiple pods on a single node.
2020-08-05 12:48:29 +01:00
Christian Sailer
45618f346d
Code review fixes (#6571) 2020-08-05 12:48:17 +01:00
Kyriakos Tharrouniatis
5192a9a2dd
CORDA-3612 - Delete flow results/ exceptions once received by rpc client (#6566)
Enhance rpc acknowledgement method (`removeClientId`) to remove checkpoint 
from all checkpoint database tables.

Optimize `CheckpointStorage.removeCheckpoint` to not delete from all checkpoint
tables if not needed. This includes excluding the results (`DBFlowResult`) and 
exceptions (`DBFlowException`) tables.
2020-08-05 12:40:35 +01:00
Viktor Kolomeyko
9fd8107922
INFRA-570: Disable new SSL tests when running on JDK11 (#6560) 2020-08-05 10:02:23 +01:00
Ross Nicoll
b3a98763ea
CORDA-3944 Clean up ExceptionsErrorCodeFunctionsTest (#6568)
* Increase timeout to provide more of an error margin, after seeing a test failure in Jenkins.
* Move shared strings to constants.
* Extract chain building code into recursive function.
2020-08-05 08:56:41 +01:00
Ross Nicoll
ff5bdcb559
NOTICK Close node handles to reduce memory usage (#6557)
Note in theory this shouldn't make a difference, however given the pass rate of this PR I figure it's worth trying.
2020-08-05 08:54:28 +01:00
Ross Nicoll
de5568854c
INFRA-438 Handle observers not having error handling (#6434)
* INFRA-438 Handle observers not having error handling

When the RPC client connection is closed, it notifies observers using onError(), which may not be the correct approach (TBD) but changing this is a much more invasive change. Where observers do not subscribe to error notifications, this is reflected to the calling client by an exception thrown.

This change catches that exception and lots it as debug rather an error level.
2020-08-05 00:05:33 +01:00
Christian Sailer
0df7c905c3
Merge pull request #6565 from corda/christians/ENT-5273-updateandfixes
ENT-5273 update and fixes
2020-08-04 17:48:59 +01:00
Joseph Zuniga-Daly
dbfea9015d
CORDA-3908: Update corda-4.4 .ci/api-current.txt to match the previous release (corda-4.3) (#6502)
* CORDA-3908: Update .ci/api-current.txt to match the previous release (corda-4.3)

* Reapply changes from PR 5633
https://github.com/corda/corda/pull/5633

* Reapply changes from PR 5711
https://github.com/corda/corda/pull/5711

* Reapply changes from PR 5800
https://github.com/corda/corda/pull/5800

* Reapply changes from PR 5846
https://github.com/corda/corda/pull/5846

* Reapply changes from PR 5833
https://github.com/corda/corda/pull/5833

* Swap order of implemented interfaces - No API change

* Add missing change from PR 5711

* A new parameter was added to MaxRpcRetryException in PR 5869

* LedgerTransaction gained 5 new parameters in PR 5633
2020-08-04 17:09:05 +01:00
Kyriakos Tharrouniatis
5d42b8847c
CORDA-3681 - Store serialised exception in database for failed and hospitalized flows (#6539)
Integrate `DBFlowException` with the rest of the checkpoint schema, so now 
we are saving the flow's exception result in the database.

Making statemachine not remove `FAILED` flows' checkpoints from the 
database if they are started with a clientId.

Retrieve the DBFlowException from the database to construct a 
`FlowStateMachineHandle` future and complete exceptionally the flow's result 
future for requests (`startFlowDynamicWithClientId`) that pick FAILED flows , 
started with client id, of status Removed.

On killing a flow the client id mapping of the flow gets removed.

The storage serialiser is used for serialising exceptions. Note, that if an
exception cannot be serialised, it will not fail and will instead be stored
as a `CordaRuntimeException`. This could be improved in future
changes.
2020-08-04 16:33:44 +01:00
Christian Sailer
4f38f32f17 Change default tx isolation level to match docs and values used in perf cluster. 2020-08-04 15:47:30 +01:00
Christian Sailer
e6dfa4da39 Remove unused cmd line option 2020-08-04 14:20:41 +01:00
Christian Sailer
2888aa6b1d Merge remote-tracking branch 'origin/release/os/4.6' into christians/ENT-5273-updateandfixes 2020-08-04 14:19:46 +01:00
Ross Nicoll
25d1d61685
NOTICK Robustness improvements for DB race condition test (#6556)
Switch away from log monitoring to verify that DB race conditions are reported correctly, in an attempt to resolve a test instability issue.
2020-08-04 11:45:57 +01:00
Ross Nicoll
12c813de18
Merge pull request #6559 from corda/rni/notick/4.5-4.6
NOTICK Merge OS 4.5 -> 4.6
2020-08-04 10:35:17 +01:00
Ross Nicoll
743dc507cc Merge remote-tracking branch 'origin/release/os/4.5' into rni/notick/4.5-4.6 2020-08-04 10:29:27 +01:00
Christian Sailer
5968df349b
Merge pull request #6552 from corda/christians/ENT-5273-update-from-os-4.6-20200803
ENT-5273 update from os 4.6
2020-08-04 09:48:29 +01:00
Ross Nicoll
1ca9b998de
NOTICK Replace example URL with an unresolvable version (#6555)
Replace example URL with an unresolvable version to avoid issues with network failures causing
the test to fail, i.e. https://ci02.dev.r3.com/job/Corda-Enterprise/job/Corda-ENT-Release-Branch-Tests/job/enterprise/job/release%252Fent%252F4.6/127/testReport/net.corda.node.services.config/NodeConfigurationImplTest/compatibilityZoneURL_populates_NetworkServices/
2020-08-03 22:11:26 +01:00
Ross Nicoll
6bcd3a9f9f
CORDA-3938 Restrict Gradle dependency locations (#6554)
* Restrict Gradle dependency locations so Gradle does not check Corda/Kotlin specific repositories for other dependencies, to save on wasted requests.
* Remove dependx plugin as it appears to be abandoned
2020-08-03 21:28:26 +01:00