Commit Graph

9105 Commits

Author SHA1 Message Date
Waldemar Żurowski
a5db6b8ac0
Merge pull request from corda/INFRA-683-os-46-linear-builds-parallel-stages
INFRA-683: os 46 linear builds parallel stages
2020-09-18 13:21:33 +02:00
Waldemar Zurowski
f27f6e307f Merge branch 'release/os/4.5' into INFRA-683-os-46-linear-builds-parallel-stages 2020-09-18 11:07:10 +02:00
Waldemar Żurowski
87f7323f0e
Merge pull request from corda/INFRA-683-os-45-linear-builds-parallel-stages
INFRA-683: Refactoring of regression Jenkins configuration
2020-09-18 10:59:49 +02:00
Waldemar Zurowski
a56a93553b Refactoring of regression Jenkins configuration
* removed archiving of Gradle JUnit HTML reports
* collection of JUnit tests moved to stage in parallel branch
2020-09-18 10:57:54 +02:00
Denis Rekalov
0b20d8b928
Merge pull request from corda/adel/merge-from-4.5-17-Sept
NOTICK: Merge OS 4.5 to OS 4.6
2020-09-17 12:32:47 +01:00
Adel El-Beik
01cd8d6415 Merge remote-tracking branch 'origin/release/os/4.5' into adel/merge-from-4.5-17-Sept 2020-09-17 10:34:01 +01:00
bpaunescu
2498bb3ae1
[ENT-5752]-Fall back on NM cache if NM service is unreachable during node start-up () 2020-09-16 18:13:44 +01:00
Ryan Fowler
83ec8289b6
ENT-5492 Let the MissingAttachmentException through the RPC ()
* CORDA-4036: Put the identity failures on the same log so they aren't missed.

* ENT-5492: Let the MissingAttachmentException propagate out
2020-09-16 16:41:11 +01:00
Ryan Fowler
afd3876faf
CORDA-4036: Put the identity failures on the same log so they aren't missed. () 2020-09-16 16:37:16 +01:00
Dan Newton
bd7b96e816
ENT-5768 startFlowWithClientId permissions ()
Do not let a user reattach to a flow started by another user.

Reattaching to a flow using startFlowWithClientId for a flow not
started by the current user throws a PermissionException

Reattaching to a flow using reattachFlowWithClientId for a flow not
started by the current user returns null.

finishedFlowsWithClientIds does not return flows started by other
users.

Normal rpc permissions around startFlowWithClientId and
startFlowDynamicWithClientId has also been added.

To allow admins to remove client ids as well as be able to see all the
client ids on the node, admin versions have been added that bypass the
user restrictions. These can be permitted via rpc to only provide
their usage to admins.
2020-09-16 16:11:06 +01:00
Adel El-Beik
64b3000c27
CORDA-4035: Fixed issue with backslash on windows when scanning. () 2020-09-16 13:12:54 +01:00
Ross Nicoll
1ac88c28d5
Merge pull request from corda/INFRA-683-os-45-linear-builds
INFRA-683: OS 45 linear builds
2020-09-16 13:07:03 +01:00
Waldemar Zurowski
c387c2cb95 Synchronised changes with OS 4.4 branch 2020-09-16 10:23:11 +02:00
Christian Sailer
b746d5cb22
ENT-5759 Give the failing test CorDapp a different package name so it does not get added automatically to all tests. () 2020-09-15 16:52:08 +01:00
Waldemar Zurowski
ebd36c22e7 Removed unused import 2020-09-15 15:42:17 +02:00
Kyriakos Tharrouniatis
342041c737
NOTICK - Remove unused code () 2020-09-15 13:53:57 +01:00
Waldemar Zurowski
99841dcaeb Merge branch 'release/os/4.4' into INFRA-683-os-45-linear-builds 2020-09-15 14:27:34 +02:00
Ross Nicoll
bcce0e9519
INFRA-683 Move OS 4.4 release branch builds to linear ()
* Do not use Security.addProvider(BouncyCastleProvider()) in tests, to avoid disruptions of other tests.
* Forcibly register security providers before starting Jimfs, to resolve a sequencing problem where Jimfs triggers loading of the SFTP filesystem provider, which in turn registers the standard BouncyCastle provider rather than the patched version Corda needs.
* INFRA-683 Move Corda OS release branch builds to serial ()

Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
Co-authored-by: Denis Rekalov <denis.rekalov@r3.com>
Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
2020-09-15 13:18:46 +01:00
Kyriakos Tharrouniatis
6829be7ae9
ENT-5754 - RPC startFlow cannot reattach to existing client id flows when flows draining mode is enabled ()
* Make existing client id flows re-attachable via rpc 'startFlow' when flows draining mode is enabled

* Fix detekt issue

* Remove unneeded/ unreached waiting on flow's return future
2020-09-15 09:04:17 +01:00
Christian Sailer
879e93ed47
ENT-5760 Log db configuration issues ()
ENT-5760 Return config error instead of throwing an exception so that the config error logging mechanism works.
2020-09-14 15:55:38 +01:00
Christian Sailer
ea423215c2
ENT-5759 check for app schema migration ()
* Fix behaviour for missing schema outside devMode plus test
2020-09-14 11:25:18 +01:00
Dan Newton
7ec59da318
ENT-5750 Keep killed flows that were started with client ids ()
If a flow is started with a client id, do not delete it from the
database. Instead, set the status to `KILLED` and store a
`KilledFlowException` in the database.
Keeping it around allows it to be reattached to using the same
mechanisms as active, completed or failed flows. Furthermore, without
this change it could be possible for a flow to be triggered again if
killed while the reconnecting rpc client is being used.
If there is no client id related to the flow, then kill flow keeps its
original behaviour of deleting all traces of the flow.
Flows cannot be killed if they are `COMPLETED`, `FAILED` or `KILLED`
already.
Logs have been added if requests to kill flows in these statuses are
made.
Do not update the status + persist the exception if the flow was killed
during flow initialisation (before persisting its first checkpoint).
Remove the client id mapping if the flow was killed and did not persist
its original checkpoint.
2020-09-10 11:58:55 +01:00
Ryan Fowler
18a9545e07
CORDA-3924: Add a missing exit code for graceful shutdowns () 2020-09-09 14:08:42 +01:00
Ryan Fowler
6f2cac146c
ENT-5492: Don't do reconnect logic on illegal argument for attachments ()
* ENT-5492: Don't do reconnect logic on illegal argument for attachments

* Use a dedicated exception for missing attachments.
2020-09-09 12:20:16 +01:00
Rick Parker
fd341c7e48
INFRA-670 Stop shutting down the database before some services that use it ()
* INFRA-670 Stop shutting down the database before some services that use it.

* Test fixes
2020-09-08 08:34:19 +01:00
Dan Newton
45fe3f3aef
CORDA-4026 Make early end session test not flakey () 2020-09-07 16:23:37 +01:00
Dan Newton
64dc94bd18
ENT-5666 Extract shutdown events when retrying a flow ()
* ENT-5666 Extract shutdown events when retrying a flow

When a flow is retrying, only a select set of events are transferred
over to the new fiber. Shutdown events were not included in this set.

This meant that if a flow retries when an `Event.SoftShutdown` is in its
queue, it will never process it. This causes the node to hang, as the
node awaits `liveFibers`, which never reaches 0 because the shutdown
event is never processed.

To resolve this, `Event.SoftShutdown` is added to the set of events to
extract and reschedule.

* ENT-5666 Don't schedule extra shutdown event

When a flow is stopped by an `Event.SoftShutdown` it will eventually
reschedule another shutdown even when it reaches `SMM.removeFlow`. It
won't actually be processed because the flow returns an abort
continuation. But, it does look odd.

Therefore, it now does nothing instead since that is what it was
implicitly doing.
2020-09-04 15:44:14 +01:00
Jonathan Locke
e562c5828b
INFRA-604: Switch tests to use database snapshots ()
Created a database snapshot of a clean Corda OS 4.5.1 database, which can now be used for testing by both the node driver and mock network.

The MockNetwork was changed from using an in memory database to using an on disk database, and makes use of the snapshot to speed up setup times.

The Node Driver was changed from defaulting to an in-memory database to defaulting to an on-disk database. Tests that do not specify the type of database to use will thus use an on-disk database. Tests that opt in for an in-memory database will continue to use an in-memory database as before.

The database snapshots are copied to the node directory inside the build folder, therefore, they should be cleaned up after a build.

Co-authored-by: Ross Nicoll <ross.nicoll@r3.com>
2020-09-03 14:43:55 +01:00
Adel El-Beik
9962c9085d
INFRA-424: Merge openj9 updates into 4.6 ()
* INFRA-424 linux1 jenkinsfile

* INFRA-424 full run

* INFRA-424 bigger heap size

* Upgraded DJVM to handle BC - latest version of BC is a multirelease JAR.
When reading JKS keystore if a BC EdDSAPrivateKey is returned then swap for a net.i2p EdDSA private key.

* Temporary downgrade of BC

* Removed the BC EdDSA conversion

* INFRA-424 bigger heap size

* Upgrading Quasar to handle openJ9 different fields.

* INFRA-424: Handle lack of SUPPRESSED_SENTINEL in openj9.

* INFRA-424: If BCEdDSA public or private key is generated convert to net.i2p EdDSA form.

* INFRA-424 bigger heap size

* INFRA-424: On openJ9 only getting upto milli resolution.

* INFRA-424: Handle keystore returning a BCEdDSAPrivateKey.

* INFRA-424: Disable test on JDK11, as it requires the custom cordapp to generate JDK8 contract code, which we now check for.

* INFRA-424: Truncated time test to resolution of millis for openj9.

* INFRA-424 disabling log intensive tests until a fix is developed

* INFRA-424 one more test disabled

* INFRA-424: Disabled a couple of tests failing on openj9.

* INFRA-424: Disabling failing openj9 tests.

* INFRA-424: Disabling test failing on openj9.

* INFRA-424: Ignoring another flaky sleep test on openj9.

* INFRA-424 run integrationTests

* INFRA-424 set timeout to 4 hours

* INFRA-424: Cope with exception message from openj9.

* INFRA-424: Handle the coloured text characters openj9 adds.

* INFRA-424: Disabling test as it is generating JDK11 contract code under JDK11. Currently on JDK8 contract code allowed.

* INFRA-424: Commenting test out for openj9. Output of the processs thats read by the test is sometimes garbled.

* INFRA-424 switching to smoke tests

* INFRA-424 switching to slow integration tests

* INFRA-424 full run

* INFRA-424 moving jenkinsfile

* INFRA-424 removing references

* INFRA-424: Created common IS_OPENJ9 func for ignoring tests.

Co-authored-by: Schife <razvan.codreanu@r3.com>
2020-09-02 14:35:30 +01:00
Nikolett Nagy
6113cbbd39
verify with NETWORK_PARAMETERS role and test () 2020-09-02 09:48:01 +01:00
Ross Nicoll
b1e2d6a48c
NOTICK Reformat DriverDSLImpl.kt ()
Reformat DriverDSLImpl to standard IntelliJ style
2020-09-01 19:40:39 +01:00
Ross Nicoll
b895a976a7
Merge pull request from corda/rni/notick/os-45-46-docs-db
NOTICK Merge OS 4.5 to 4.6
2020-09-01 17:47:54 +01:00
Ross Nicoll
d928c29019 Merge remote-tracking branch 'origin/release/os/4.5' into rni/notick/os-45-46-docs-db 2020-09-01 17:00:00 +01:00
Ross Nicoll
aaff9ce650
Merge pull request from corda/rni/notick/os-44-45-db
NOTICK Merge OS 4.4 to 4.5
2020-09-01 16:53:48 +01:00
Ross Nicoll
aa1b74d510 Merge remote-tracking branch 'origin/release/os/4.4' into rni/notick/os-44-45-db 2020-09-01 16:06:03 +01:00
Ross Nicoll
6945426ef1
NOTICK Robustness improvements for DB race condition test () ()
Switch away from log monitoring to verify that DB race conditions are reported correctly, in an attempt to resolve a test instability issue.
2020-09-01 16:04:12 +01:00
Ross Nicoll
0133ada0ac
NOTICK Robustness improvements for DB race condition test () ()
Switch away from log monitoring to verify that DB race conditions are reported correctly, in an attempt to resolve a test instability issue.
2020-09-01 14:05:36 +01:00
Adel El-Beik
14e23430c0
CORDA-4003: Now support + in CorDapp filenames ()
* CORDA-4003: Now cope with file: prefix not being in class path element.

* CORDA-4003: Switched to new URL type filter.

* CORDA-4003: Switched to a URL comparison. In the string comparison the scheme was removed in latest version of classgraph.

* CORDA-4003: Moved to latest version of classgraph that has support for + in filenames.

* CORDA-4003: Switched to accept version of the deprecated classgraph methods.
2020-09-01 10:30:49 +01:00
Kyriakos Tharrouniatis
90ab495798
NOTICK - Fix concurrency issue in flaky test ()
* Fix concurrency issue in test; should be fixing flaky test
* Replace 'incrementAndGet' occurences with 'getAndIncrement'
2020-08-28 12:48:20 +01:00
Dries Samyn
d08b62da39
Infra-656 - NoSuchFileException in NodeInfoWatcher fix ()
* Filter out tmp files

* Ignore .tmp files

* Ignore .tmp files

* Remove unused import
2020-08-27 16:41:25 +01:00
David Golding
9a018e7bee
Updated CLI usage error () 2020-08-27 12:15:21 +01:00
Dries Samyn
0cfe6b3084
Filter out tmp files () 2020-08-27 08:48:08 +01:00
Joseph Zuniga-Daly
3890064095
CORDA-3908: Update corda-4.6 .ci/api-current.txt to match the previous release (corda-4.5) ()
* CORDA-3908: Update corda-4.6 .ci/api-current.txt to match the previous release (corda-4.5)

* Reapply changes from PR 6392

* Apply change introduced by PR 

* Apply change introduced by PR 
2020-08-26 17:41:18 +01:00
Adel El-Beik
9629ff6ba2
CORDA-4009: Upgrade DJVM to 1.1.1 to support the new multi release BC JAR. () 2020-08-26 13:59:17 +01:00
Rick Parker
0a290a73f4
Merge pull request from corda/jzd/merge-os4.5-to-os4.6-2020-08-25
NOTICK: Merge OS 4.5 into OS 4.6
2020-08-26 09:59:55 +01:00
Joseph Zuniga-Daly
797ecf3b7c Don't change api-current because we are about to regenerate it 2020-08-25 12:00:37 +01:00
Joseph Zuniga-Daly
2a09dd066d Merge OS 4.5 into OS 4.6 2020-08-25 11:57:54 +01:00
Dan Newton
99f835bb4a
CORDA-3995 Redeliver external events if number of suspends differs ()
* CORDA-3995 Redeliver external events in number of suspends differs

When retrying a flow, only redeliver external events held in a flow's
pending deduplication handlers if there is a difference in the
`numberOfSuspends` on the `currentState`'s checkpoint or the checkpoint
in the database.

If the checkpoint committed, but the flow retried, then the external
events would have been persisted to the database as part of the same
transaction. Therefore there is no need to replay them, as they have
already been processed as saved as part of the checkpoint.

This change is only relevant when the checkpoint persists, but the flow
still needs to retry after this occurs (within the same
transition/event).

* CORDA-3995 Redeliver external events in number of commits differs

When retrying a flow, only redeliver external events held in a flow's
pending deduplication handlers if there is a difference in the
`numberOfCommits` on the `currentState`'s checkpoint or the checkpoint
in the database.

If the checkpoint committed, but the flow retried, then the external
events would have been persisted to the database as part of the same
transaction. Therefore there is no need to replay them, as they have
already been processed as saved as part of the checkpoint.

This change is only relevant when the checkpoint persists, but the flow
still needs to retry after this occurs (within the same
transition/event).

* CORDA-3995 Redeliver external events if number of commits differs

When retrying a flow, only redeliver external events held in a flow's
pending deduplication handlers if there is a difference in the
`currentState`'s `numberOfCommits` or the `numberOfCommits`
the checkpoint has recorded in the database.

If the checkpoint committed, but the flow retried, then the external
events would have been persisted to the database as part of the same
transaction. Therefore there is no need to replay them, as they have
already been processed as saved as part of the checkpoint.

This change is only relevant when the checkpoint persists, but the flow
still needs to retry after this occurs (within the same
transition/event).

* Add @Suspendable to a test flow.

I am surprised this worked at all.

* Fix a few minor things based on review.

Co-authored-by: Will Vigor <william.vigor@r3.com>
2020-08-25 11:54:55 +01:00
Rick Parker
7a7289c2f1
Merge pull request from corda/jzd/merge-os4.4-to-os4.5-2020-08-24
Merge OS 4.4 into OS 4.5
2020-08-25 10:59:27 +01:00
Joseph Zuniga-Daly
df7e3ca184
CORDA-3908: Update corda-4.5 .ci/api-current.txt to match the previous release (corda-4.4) ()
* CORDA-3908: Update corda-4.5 .ci/api-current.txt to match the previous release (corda-4.4)

* Reapply changes from PR 5998
https://github.com/corda/corda/pull/5998
2020-08-24 14:32:10 +01:00