Commit Graph

9066 Commits

Author SHA1 Message Date
Chris Rankin
d60feb1138
INFRA-563 Update Corda to test distributed-testing-plugin 1.3-SNAPSHOT. (#6551)
Integrate YAML profile support, and the eagle-eyed will notice that the plugin no longer needs to be applied at the very bottom of the build.gradle file!
 
Other features include:
* Implicit upgrade to docker-remote-api plugin v5.3.0
* Fixing a ClassGraph-related memory leak by closing the ScanResult objects after use.
* More logging of any exceptions from Kubenetese.
* The gradlecache volume is now created with a hostPath of "/gradle/$podName/$podIdx-$taskForExecuteName", which should allow having multiple pods on a single node.
2020-08-05 12:48:29 +01:00
Christian Sailer
45618f346d
Code review fixes (#6571) 2020-08-05 12:48:17 +01:00
Kyriakos Tharrouniatis
5192a9a2dd
CORDA-3612 - Delete flow results/ exceptions once received by rpc client (#6566)
Enhance rpc acknowledgement method (`removeClientId`) to remove checkpoint 
from all checkpoint database tables.

Optimize `CheckpointStorage.removeCheckpoint` to not delete from all checkpoint
tables if not needed. This includes excluding the results (`DBFlowResult`) and 
exceptions (`DBFlowException`) tables.
2020-08-05 12:40:35 +01:00
Viktor Kolomeyko
9fd8107922
INFRA-570: Disable new SSL tests when running on JDK11 (#6560) 2020-08-05 10:02:23 +01:00
Ross Nicoll
b3a98763ea
CORDA-3944 Clean up ExceptionsErrorCodeFunctionsTest (#6568)
* Increase timeout to provide more of an error margin, after seeing a test failure in Jenkins.
* Move shared strings to constants.
* Extract chain building code into recursive function.
2020-08-05 08:56:41 +01:00
Ross Nicoll
ff5bdcb559
NOTICK Close node handles to reduce memory usage (#6557)
Note in theory this shouldn't make a difference, however given the pass rate of this PR I figure it's worth trying.
2020-08-05 08:54:28 +01:00
Ross Nicoll
de5568854c
INFRA-438 Handle observers not having error handling (#6434)
* INFRA-438 Handle observers not having error handling

When the RPC client connection is closed, it notifies observers using onError(), which may not be the correct approach (TBD) but changing this is a much more invasive change. Where observers do not subscribe to error notifications, this is reflected to the calling client by an exception thrown.

This change catches that exception and lots it as debug rather an error level.
2020-08-05 00:05:33 +01:00
Christian Sailer
0df7c905c3
Merge pull request #6565 from corda/christians/ENT-5273-updateandfixes
ENT-5273 update and fixes
2020-08-04 17:48:59 +01:00
Joseph Zuniga-Daly
dbfea9015d
CORDA-3908: Update corda-4.4 .ci/api-current.txt to match the previous release (corda-4.3) (#6502)
* CORDA-3908: Update .ci/api-current.txt to match the previous release (corda-4.3)

* Reapply changes from PR 5633
https://github.com/corda/corda/pull/5633

* Reapply changes from PR 5711
https://github.com/corda/corda/pull/5711

* Reapply changes from PR 5800
https://github.com/corda/corda/pull/5800

* Reapply changes from PR 5846
https://github.com/corda/corda/pull/5846

* Reapply changes from PR 5833
https://github.com/corda/corda/pull/5833

* Swap order of implemented interfaces - No API change

* Add missing change from PR 5711

* A new parameter was added to MaxRpcRetryException in PR 5869

* LedgerTransaction gained 5 new parameters in PR 5633
2020-08-04 17:09:05 +01:00
Kyriakos Tharrouniatis
5d42b8847c
CORDA-3681 - Store serialised exception in database for failed and hospitalized flows (#6539)
Integrate `DBFlowException` with the rest of the checkpoint schema, so now 
we are saving the flow's exception result in the database.

Making statemachine not remove `FAILED` flows' checkpoints from the 
database if they are started with a clientId.

Retrieve the DBFlowException from the database to construct a 
`FlowStateMachineHandle` future and complete exceptionally the flow's result 
future for requests (`startFlowDynamicWithClientId`) that pick FAILED flows , 
started with client id, of status Removed.

On killing a flow the client id mapping of the flow gets removed.

The storage serialiser is used for serialising exceptions. Note, that if an
exception cannot be serialised, it will not fail and will instead be stored
as a `CordaRuntimeException`. This could be improved in future
changes.
2020-08-04 16:33:44 +01:00
Christian Sailer
4f38f32f17 Change default tx isolation level to match docs and values used in perf cluster. 2020-08-04 15:47:30 +01:00
Christian Sailer
e6dfa4da39 Remove unused cmd line option 2020-08-04 14:20:41 +01:00
Christian Sailer
2888aa6b1d Merge remote-tracking branch 'origin/release/os/4.6' into christians/ENT-5273-updateandfixes 2020-08-04 14:19:46 +01:00
Ross Nicoll
25d1d61685
NOTICK Robustness improvements for DB race condition test (#6556)
Switch away from log monitoring to verify that DB race conditions are reported correctly, in an attempt to resolve a test instability issue.
2020-08-04 11:45:57 +01:00
Ross Nicoll
12c813de18
Merge pull request #6559 from corda/rni/notick/4.5-4.6
NOTICK Merge OS 4.5 -> 4.6
2020-08-04 10:35:17 +01:00
Ross Nicoll
743dc507cc Merge remote-tracking branch 'origin/release/os/4.5' into rni/notick/4.5-4.6 2020-08-04 10:29:27 +01:00
Christian Sailer
5968df349b
Merge pull request #6552 from corda/christians/ENT-5273-update-from-os-4.6-20200803
ENT-5273 update from os 4.6
2020-08-04 09:48:29 +01:00
Ross Nicoll
1ca9b998de
NOTICK Replace example URL with an unresolvable version (#6555)
Replace example URL with an unresolvable version to avoid issues with network failures causing
the test to fail, i.e. https://ci02.dev.r3.com/job/Corda-Enterprise/job/Corda-ENT-Release-Branch-Tests/job/enterprise/job/release%252Fent%252F4.6/127/testReport/net.corda.node.services.config/NodeConfigurationImplTest/compatibilityZoneURL_populates_NetworkServices/
2020-08-03 22:11:26 +01:00
Ross Nicoll
6bcd3a9f9f
CORDA-3938 Restrict Gradle dependency locations (#6554)
* Restrict Gradle dependency locations so Gradle does not check Corda/Kotlin specific repositories for other dependencies, to save on wasted requests.
* Remove dependx plugin as it appears to be abandoned
2020-08-03 21:28:26 +01:00
Tamas Veingartner
0b7fdbb35d
CORDA-3663 Remove dummy package names (#6553)
Dummy package names cause build failure as they are not found on the classpath when trying to import them. Now that empty package name list is allowed, the dummy names are removed.
2020-08-03 19:09:30 +01:00
Christian Sailer
71a6081ec8 Fix new integration tests to not use hibernate schema modification. 2020-08-03 18:50:36 +01:00
Christian Sailer
09b5e21d97 Fix wrong name in test names. 2020-08-03 16:43:40 +01:00
Christian Sailer
89759510ae Merge remote-tracking branch 'origin/release/os/4.6' into christians/ENT-5273-update-from-os-4.6-20200803 2020-08-03 16:28:20 +01:00
Christian Sailer
9f2bd94f1b
ENT-5624 Remove code to migrate from pre-V4 versions of Corda - these will have to go via Corda 4.5 (#6548) 2020-08-03 10:30:14 +01:00
Tamas Veingartner
85be50779b
CORDA-3663 MockServices crashes when two of the provided packages to … (#6472)
* CORDA-3663 MockServices crashes when two of the provided packages to scan are deemed empty in 4.4 RC05

this happends when a given package is not found on the classpath. Now it is handled and an exception is thrown

* replace dummy package names in tests with valid ones

* allow empty package list for CustomCordapps and exclude those from the created jars

* detekt fix

* always true logic fix

* fix to check for empty packages instead of empty classes

* fix for classes and fixups

* logic refactor because of detekt stupidity

* PR related minor refactors
2020-08-03 09:19:48 +01:00
Dan Newton
c288073e7c
Merge pull request #6524 from corda/ENT-5532-retrying-flow-with-sessions-to-close
ENT-5532 Terminate sessions after original io request
NOTICK Resume flow when wrong message received
2020-07-31 17:08:43 +01:00
Ross Nicoll
9aa745ef14
INFRA-553 Move HashLookupCommandTest to unit test (#6537)
Move HashLookupCommandTest to unit test and add a test covering that a nonsense hash ID is correctly rejected.
2020-07-31 15:54:53 +01:00
LankyDan
82bcde573b NOTICK Resume flow when wrong message received
When an incorrect message is received, the flow should resume to allow
it to throw the error back to user code and possibly cause the flow to
fail.

For now, if an `EndSessionMessage` is received instead of a
`DataSessionMessage`, then an `UnexpectedFlowEndException` is thrown
back to user code. Allowing it to correctly re-enter normal flow error
handling.

Without this change, the flow will hang due to it failing while creating
a transition which exists outside of the general state machine error
handling code path.
2020-07-31 14:28:57 +01:00
LankyDan
39dbe22c9d ENT-5532 Terminate sessions after original io request
Sessions are now terminated after performing the original
`FlowIORequest` passed into `StartedFlowTransition`, instead of before.
This is done by scheduling an `Event.TerminateSessions` if there are
sessions to terminate when performing a suspending event.

Originally this was done by hijacking a transition that is trying to
perform a `StartedFlowTransition`, terminating the sessions and then
scheduling another `Event.DoRemainingWork` to perform the original
transition. This introduced a bug where, another event (from a external
message) could be placed onto the queue before the
`Event.DoRemainingWork` could be added. In most scenarios, that should
be ok. But, if a flow is retrying (while in an uninitiated state) and
this occurs the flow could fail due to being in an unexpected state.

Terminating the sessions after performing the original transition
removes this possibility. Meaning that a restarting flow will always
perform the transition they supposed to do (based on the called
suspending event).
2020-07-31 12:37:44 +01:00
Christian Sailer
4097d54a31
Merge pull request #6535 from corda/christians/ENT-5273-update-fb-from-os.4.6
ENT-5273 update feature branch from os.4.6
2020-07-31 11:51:55 +01:00
James Higgs
250ed8a21a
Merge pull request #6533 from corda/eg-177merge
EG-177   Network Parameter Hotloading
2020-07-31 09:45:23 +01:00
Viktor Kolomeyko
c498c5bf7c
CORDA-3871: New integration test for handshake timeout in AMQPClient (#6519)
* CORDA-3871: Import external code

Compiles, but does not work for various reasons

* CORDA-3871: More improvements to imported code

Currently fails due to keystores not being found

* CORDA-3871: Initialise keystores for the server

Currently fails due to keystores for client not being found

* CORDA-3871: Configure certificates to client

The program started to run

* CORDA-3871: Improve debug output

* CORDA-3871: Few more minor changes

* CORDA-3871: Add AMQClient test

Currently fails due to `localCert` not being set

* CORDA-3871: Configure server to demand client to present its certificate

* CORDA-3871: Changes to the test to make it pass

ACK status is not delivered as server is not talking AMQP

* CORDA-3871: Add delayed handshake scenario

* CORDA-3871: Tidy-up imported classes

* CORDA-3871: Hide thread creation inside `ServerThread`

* CORDA-3871: Test description

* CORDA-3871: Detekt baseline update

* CORDA-3871: Trigger repeated execution of new tests

To make sure they are not flaky

* CORDA-3871: Improve robustness of the newly introduced tests

* CORDA-3871: Improve robustness of the newly introduced tests

* CORDA-3871: New tests proven to be stable - reduce number of iterations to 1

* CORDA-3871: Adding Alex Karnezis to the list of contributors
2020-07-31 09:26:32 +01:00
Ross Nicoll
68feb1c35f
CORDA-3932 Correct race condition in FlowVersioningTest (#6536)
Correct race condition in FlowVersioningTest where the last message is read (and the session close can be triggered)
before one side has finished reading metadata from the session.
2020-07-31 08:32:20 +01:00
Christian Sailer
81d68abe7e Merge remote-tracking branch 'origin/release/os/4.6' into christians/ENT-5273-update-fb-from-os.4.6 2020-07-30 18:39:04 +01:00
Paul Hatcher
87faf35ecb
CORDA-3929 : quasar 0.7.12_r3 -> quasar 0.7.13_r3 (#6522) 2020-07-30 17:08:00 +01:00
Peter Nemeth
777be6c11a Merge branch 'feature/EG-177' of https://github.com/corda/corda into feature/EG-177 2020-07-30 16:20:01 +01:00
pnemeth
2a27f3ac37
EG-2055 Implement network parameters hotloading (#6517)
* Implement network parameters hotloading

* fixed failing unit test

* PR comments

* PR comments

* added integr tests+ renamed updater to hotloader

* moved exiting logic out of hotloader

* extra tests

* latest PR comments

* refactor

* address detekt/suppress if more significant refactoring needed

* extra tests

* addressed PR comments

* detekt

* formatting
2020-07-30 16:02:33 +01:00
Ross Nicoll
fd5472b053
NOTICK Remove memory leak endurance test (#6514)
Remove memory leak endurance test as it spends 8 minutes testing a single failure case that's not end user visible,
and ultimately manifests elsewhere in test failures (which is where this came from in the beginning). It was a good
idea to confirm the change fixed the issue, but this isn't critical enough to retain.
2020-07-30 15:39:28 +01:00
Dan Newton
35bfa6945f
Merge pull request #6529 from corda/os_4.6-feature_pass_in_client_id_when_starting_a_flow-merge
NOTICK - OS 4.6 to feature/pass_in_client_id_when_starting_a_flow merge
2020-07-30 12:30:21 +01:00
Jonathan Locke
530c5eadf8
INFRA-549: Commented out Sonatype checks during regression builds for now. (#6528)
Commented out Sonatype checks during regression builds for now.
2020-07-30 11:27:45 +01:00
Kyriakos Tharrouniatis
f2bcc85f30 Fix compilation error 2020-07-30 10:54:24 +01:00
Kyriakos Tharrouniatis
8a37108a58 Merge branch 'release/os/4.6' into os_4.6-feature_pass_in_client_id_when_starting_a_flow-merge 2020-07-30 10:34:03 +01:00
Waldemar Żurowski
5f8d70f5e3
Merge pull request #6527 from corda/wz/merge-os-45-46-2020-07-30
NOTICK: Merging forward updates from OS 4.5 to OS 4.6
2020-07-30 07:01:29 +01:00
Waldemar Zurowski
02990b9dc0 Merge branch 'release/os/4.5' into wz/merge-os-45-46-2020-07-30 2020-07-30 05:37:46 +01:00
Waldemar Żurowski
a81e871356
NOTICK: Reverted use of Artifactory cache mechanism (#6526) 2020-07-30 05:25:01 +01:00
Kyriakos Tharrouniatis
ad61d7b3f4
CORDA-3692 - Store result information in checkpoint results table (#6473)
Making statemachine not remove COMPLETED flows' checkpoints from the database 
if they are started with a clientId, instead they are getting persisted and retained within 
the database along with their result (`DBFlowResult`).

On flow start with a client id (`startFlowDynamicWithClientId`), if the client id maps to
a flow that was previously started with the same client id and the flow is now finished, 
then fetch the `DBFlowResult` from the database to construct a 
`FlowStateMachineHandle` done future and return it back to the client.

Object stored as results must abide by the storage serializer rules. If they fail to do so
the result will not be stored and an exception is thrown to the client to indicate this.
2020-07-29 15:57:00 +01:00
Yiftach Kaplan
0bedbd8c75
INFRA-530: Start notary node in process (#6521) 2020-07-29 15:47:45 +01:00
Ross Nicoll
c91dba83d3
Merge pull request #6523 from corda/wz/merge-os-45-46-2020-07-28
NOTICK: Merging forward updates from OS 4.5 to OS 4.6
2020-07-28 16:50:00 +01:00
Dan Newton
c2fd8253ea
CORDA-3777 Reload after every checkpoint (#6494)
Enable reloading of a flow after every checkpoint is saved. This
includes reloading the checkpoint from the database and recreating the
fiber.

When a flow and its `StateMachineState` is created it checks the node's
config to see if the `reloadCheckpointAfterSuspend` is set to true. If it is
it initialises `StateMachineState.reloadCheckpointAfterSuspendCount`
with the value 0. Otherwise, it remains `null`.

This count represents how many times the flow has reloaded from its
checkpoint (not the same as retrying). It is incremented every time the
flow is reloaded.

When a flow suspends, it processes the suspend event like usual, but
it will now also check if `reloadCheckpointAfterSuspendCount` is not
`null` (that it is activated) and process a 
`ReloadFlowFromCheckpointAfterSuspend`event, if and only if 
`reloadCheckpointAfterSuspendCount` is greater than
`CheckpointState.numberOfSuspends`.

This means idempotent flows can reload from the start and not reload
again until reaching a new suspension point.

Flows that skip checkpoints can reload from a previously saved
checkpoint (or from the initial checkpoint) and will continue reloading
on reaching the next new suspension point (not the suspension point that
it skipped saving).

If the flow fails to deserialize the checkpoint from the database upon
reloading a `ReloadFlowFromCheckpointException` is throw. This causes
the flow to be kept for observation.
2020-07-28 16:27:51 +01:00
Stefan Iliev
52cbe04b8c
EG-2375 - batching notary open sourcing. (#6507) 2020-07-28 15:50:19 +01:00