Commit Graph

8903 Commits

Author SHA1 Message Date
Kyriakos Tharrouniatis
8a37108a58 Merge branch 'release/os/4.6' into os_4.6-feature_pass_in_client_id_when_starting_a_flow-merge 2020-07-30 10:34:03 +01:00
Waldemar Żurowski
5f8d70f5e3
Merge pull request #6527 from corda/wz/merge-os-45-46-2020-07-30
NOTICK: Merging forward updates from OS 4.5 to OS 4.6
2020-07-30 07:01:29 +01:00
Waldemar Zurowski
02990b9dc0 Merge branch 'release/os/4.5' into wz/merge-os-45-46-2020-07-30 2020-07-30 05:37:46 +01:00
Waldemar Żurowski
a81e871356
NOTICK: Reverted use of Artifactory cache mechanism (#6526) 2020-07-30 05:25:01 +01:00
Kyriakos Tharrouniatis
ad61d7b3f4
CORDA-3692 - Store result information in checkpoint results table (#6473)
Making statemachine not remove COMPLETED flows' checkpoints from the database 
if they are started with a clientId, instead they are getting persisted and retained within 
the database along with their result (`DBFlowResult`).

On flow start with a client id (`startFlowDynamicWithClientId`), if the client id maps to
a flow that was previously started with the same client id and the flow is now finished, 
then fetch the `DBFlowResult` from the database to construct a 
`FlowStateMachineHandle` done future and return it back to the client.

Object stored as results must abide by the storage serializer rules. If they fail to do so
the result will not be stored and an exception is thrown to the client to indicate this.
2020-07-29 15:57:00 +01:00
Yiftach Kaplan
0bedbd8c75
INFRA-530: Start notary node in process (#6521) 2020-07-29 15:47:45 +01:00
Ross Nicoll
c91dba83d3
Merge pull request #6523 from corda/wz/merge-os-45-46-2020-07-28
NOTICK: Merging forward updates from OS 4.5 to OS 4.6
2020-07-28 16:50:00 +01:00
Dan Newton
c2fd8253ea
CORDA-3777 Reload after every checkpoint (#6494)
Enable reloading of a flow after every checkpoint is saved. This
includes reloading the checkpoint from the database and recreating the
fiber.

When a flow and its `StateMachineState` is created it checks the node's
config to see if the `reloadCheckpointAfterSuspend` is set to true. If it is
it initialises `StateMachineState.reloadCheckpointAfterSuspendCount`
with the value 0. Otherwise, it remains `null`.

This count represents how many times the flow has reloaded from its
checkpoint (not the same as retrying). It is incremented every time the
flow is reloaded.

When a flow suspends, it processes the suspend event like usual, but
it will now also check if `reloadCheckpointAfterSuspendCount` is not
`null` (that it is activated) and process a 
`ReloadFlowFromCheckpointAfterSuspend`event, if and only if 
`reloadCheckpointAfterSuspendCount` is greater than
`CheckpointState.numberOfSuspends`.

This means idempotent flows can reload from the start and not reload
again until reaching a new suspension point.

Flows that skip checkpoints can reload from a previously saved
checkpoint (or from the initial checkpoint) and will continue reloading
on reaching the next new suspension point (not the suspension point that
it skipped saving).

If the flow fails to deserialize the checkpoint from the database upon
reloading a `ReloadFlowFromCheckpointException` is throw. This causes
the flow to be kept for observation.
2020-07-28 16:27:51 +01:00
Stefan Iliev
52cbe04b8c
EG-2375 - batching notary open sourcing. (#6507) 2020-07-28 15:50:19 +01:00
Dimitris Gounaris
1e6be340eb
CORDA-3844: bulk node infos request (#6411)
* CORDA-3844: Add new functions to network map client

* CORDA-3844: Apply new fetch logic to nm updater

* CORDA-3844: Fix base url and warnings

* CORDA-3844: Change response object and response validation

In order to make sure that the returned node infos are not maliciously modified, either a signed list response
or a signed reference object would need to be provided. As providing a signed list requires a lot of effort from NM and Signer services,
the signed network map is provided instead, allowing nodes to validate that the list provided conforms to the entries of the signed network map.

* CORDA-3844: Add clarifications and comments

* CORDA-3844: Add error handling for bulk request

* CORDA-3844: Enhance testing

* CORDA-3844: Fix detekt issues

* EG-3844: Apply pr suggestions
2020-07-28 15:02:53 +01:00
Waldemar Zurowski
e95ff388da Merge branch 'release/os/4.5' into wz/merge-os-45-46-2020-07-28 2020-07-28 14:14:43 +01:00
Waldemar Żurowski
ebdcc97dda
Merge pull request #6520 from corda/INFRA-427-Store-14-days-of-builds
Added missing buildDiscarder to docs API publisher Jenkins configuration
2020-07-28 11:31:52 +01:00
Dimos Raptis
f2336f397d
CORDA-3506 - Add test for session close API (#6512) 2020-07-28 10:20:24 +01:00
Waldemar Zurowski
7f94f6cb39 Added missing buildDiscarder to docs API publisher Jenkins configuration
* updated regex for recognised tag to match JenkinsUI configuration
2020-07-28 09:36:11 +01:00
Ryan Fowler
d4189c4f37
CORDA-3918: Port of ENT-5417: Allow exceptions to propagate when shutdown commands are called (#6516) 2020-07-27 14:09:26 +01:00
Ross Nicoll
310fa5e082
Merge pull request #6513 from corda/rni/notick/merge-4.4-4.5
NOTICK Merge 4.4 to 4.5
2020-07-23 18:33:08 +01:00
Ross Nicoll
9527c553ec Merge remote-tracking branch 'origin/release/os/4.4' into rni/notick/merge-4.4-4.5 2020-07-23 17:24:36 +01:00
Ross Nicoll
13073c300f
NOTICK: OS 4.3 to OS 4.4 merge (#6506)
* CORDA-3917 Update to Jackson 2.9.8 (#6493)

* Update to Jackson 2.9.8 to address multiple security issues, and update warning note about
updates to clarify that it refers to 2.10+. When the note was added 2.9.7 as the highest
available version in the 2.9.x series.
* Add PR code checks Jenkinsfile

* CORDA-3916 Update to BouncyCastle 1.61 (#6492)

Update to BouncyCastle 1.61. Updating one version at a time to mitigate risk of a complex breaking change being introduced.

* Added missing collection of JUnit tests and logs

Co-authored-by: Waldemar Zurowski <waldemar.zurowski@r3.com>
2020-07-23 16:59:42 +01:00
Ryan Fowler
2ca10464b7
CORDA-3845: Update BC, log4j, slf4j (#6464)
* CORDA-3845: Update BC to 1.64

* CORDA-3845: Upgraded log4j to 2.13.3

* We can remove the use of Manifests from the logging package so that when _it_ logs it doesn't error on the fact the stream was already closed by the default Java logger.

* Some more tidy up

* Remove the logging package as a plugin

* latest BC version

* Remove old test

* fix up

* Fix some rebased changes to log file handling

* Fix some rebased changes to log file handling

* Update slf4j too

Co-authored-by: Adel El-Beik <adel.el-beik@r3.com>
2020-07-23 16:37:29 +01:00
Yiftach Kaplan
4acf41ea3d
INFRA-477: Start nodes in parallel when possible (#6460)
Co-authored-by: Ross Nicoll <ross.nicoll@r3.com>
2020-07-23 16:35:34 +01:00
Ross Nicoll
05c2428f2f
NOTICK Add more detail on signature build failures (#6510)
Add details of the signature provider and key algorithm if `InvalidKeyException` is thrown when constructing a `ContentSigner`, in order to be able to usefully diagnose incorrect signature providers or similar errors.
2020-07-23 15:17:59 +01:00
Waldemar Zurowski
0b1ccb48d0 Merge branch 'release/os/4.5' into wz/merge-os-45-46-2020-07-23 2020-07-23 11:31:43 +01:00
Waldemar Żurowski
6a51a3fe1f
Merge pull request #6509 from corda/wz/merge-os-44-45-2020-07-23
NOTICK: Merging forward Jenkins configuration updates from OS 4.4 to OS 4.5
2020-07-23 11:29:31 +01:00
Waldemar Zurowski
2e6bd97fe9 Updates
* INFRA-508: NexusIQ stage for GA releases updated to `release`
* do not truncate stdio when collecting JUnit tests
2020-07-23 10:46:32 +01:00
Waldemar Zurowski
49c86af354 Merge branch 'release/os/4.4' into wz/merge-os-44-45-2020-07-23 2020-07-23 10:39:12 +01:00
Waldemar Żurowski
14279a30cc
INFRA-508: NexusIQ stage for GA releases updated to release (#6508) 2020-07-23 10:36:48 +01:00
Waldemar Zurowski
4e0ebefbcc Merge branch 'release/os/4.5' into release/os/4.6 2020-07-22 19:56:07 +01:00
Waldemar Zurowski
136743ae61 Merge branch 'release/os/4.4' into release/os/4.5 2020-07-22 19:55:12 +01:00
Waldemar Zurowski
f60f06a85f INFRA-508: Change appID in sonatype stage for Corda and Corda Enterprise 2020-07-22 19:54:41 +01:00
Waldemar Zurowski
8a4e522227 Merge branch 'release/os/4.5' into release/os/4.6 2020-07-22 19:45:50 +01:00
Waldemar Zurowski
4d03bcb7ed Merge branch 'release/os/4.4' into release/os/4.5 2020-07-22 19:44:21 +01:00
Waldemar Zurowski
0842ea26b7 NOTICK: Backport Jenkins configuration for nightly builds to release 4.4 2020-07-22 19:43:08 +01:00
Waldemar Zurowski
b1571cc241 Merge branch 'release/os/4.5' into wz/merge-os-45-46-2020-07-22 2020-07-22 19:35:49 +01:00
Joseph Zuniga-Daly
c33720c73d
CORDA-3717: Apply custom serializers to checkpoints (#6392)
* CORDA-3717: Apply custom serializers to checkpoints

* Remove try/catch to fix TooGenericExceptionCaught detekt rule

* Rename exception

* Extract method

* Put calls to the userSerializer on their own lines to improve readability

* Remove unused constructors from exception

* Remove unused proxyType field

* Give field a descriptive name

* Explain why we are looking for two type parameters when we only use one

* Tidy up the fetching of types

* Use 0 seconds when forcing a flow checkpoint inside test

* Add test to check references are restored correctly

* Add CheckpointCustomSerializer interface

* Wire up the new CheckpointCustomSerializer interface

* Use kryo default for abstract classes

* Remove unused imports

* Remove need for external library in tests

* Make file match original to remove from diff

* Remove maySkipCheckpoint from calls to sleep

* Add newline to end of file

* Test custom serializers mapped to interfaces

* Test serializer configured with abstract class

* Move test into its own package

* Rename test

* Move flows and serializers into their own source file

* Move broken map into its own source file

* Delete comment now source file is simpler

* Rename class to have a shorter name

* Add tests that run the checkpoint serializer directly

* Check serialization of final classes

* Register as default unless the target class is final

* Test PublicKey serializer has not been overridden

* Add a broken serializer for EdDSAPublicKey to make test more robust

* Split serializer registration into default and non-default registrations. Run registrations at the right time to preserve Cordas own custom serializers.

* Check for duplicate custom checkpoint serializers

* Add doc comments

* Add doc comments to CustomSerializerCheckpointAdaptor

* Add test to check duplicate serializers are logged

* Do not log the duplicate serializer warning when the duplicate is the same class

* Update doc comment for CheckpointCustomSerializer

* Sort serializers by classname so we are not registering in an unknown or random order

* Add test to serialize a class that references itself

* Store custom serializer type in the Kryo stream so we can spot when a different serializer is being used to deserialize

* Testing has shown that registering custom serializers as default is more robust when adding new cordapps

* Remove new line character

* Remove unused imports

* Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt

* Remove comment

* Update comment on exception

* Make CustomSerializerCheckpointAdaptor internal

* Revert "Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt"

This reverts commit b835de79bd.

* Restore "Add interface net.corda.core.serialization.CheckpointCustomSerializer to api-current.txt""

This reverts commit 718873a4e9.

* Pass the class loader instead of the context

* Do less work in test setup

* Make the serialization context unique for CustomCheckpointSerializerTest so we get a new Kryo pool for the test

* Rebuild the Kryo pool for the given context when we change custom serializers

* Rebuild all Kryo pools on serializer change to keep serializer list consistent

* Move the custom serializer list into CheckpointSerializationContext to reduce scope from global to a serialization context

* Remove unused imports

* Make the new checkpointCustomSerializers property default to the empty list

* Delegate implementation using kotlin language feature
2020-07-22 17:31:59 +01:00
Dan Newton
a41152edf6
CORDA-3899 Refactor flow's transient fields (#6441)
Refactor `FlowStateMachineImpl.transientValues` and
`FlowStateMachineImpl.transientState` to stop the fields from exposing
the fact that they are nullable.

This is done by having private backing fields `transientValuesReference`
and `transientStateReference` that can be null. The nullability is still
needed due to serialisation and deserialisation of flow fibers. The
fields are transient and therefore will be null when reloaded from the
database.

Getters and setters hide the private field, allowing a non-null field to
returned.

There is no point other than in `FlowCreator` where the transient fields
can be null. Therefore the non null checks that are being made are
valid.

Add custom kryo serialisation and deserialisation to `TransientValues`
and `StateMachineState` to ensure that neither of the objects are ever
touched by kryo.
2020-07-22 16:19:20 +01:00
Waldemar Żurowski
d810067ab6
NOTICK: Fixed for JDK11 builds (#6501)
* NexusIQ every build related to Corda X.Y (GA, RC, HC, patch or
  snapshot) uses the same NexusIQ application
* NexusIQ application application has to exist before a build starts
* Fixed repository name for publishing, use OS instead of Ent one
2020-07-22 14:43:14 +01:00
Waldemar Zurowski
8ee070953a Merge branch 'release/os/4.5' into wz/merge-os-45-46-2020-07-22 2020-07-22 13:00:42 +01:00
Waldemar Zurowski
6413873f68 Merge branch 'release/os/4.4' into wz/merge-os-44-45-2020-07-22 2020-07-22 12:57:45 +01:00
Waldemar Żurowski
05532c0419
NexusIQ updates (#6499)
* every build related to Corda X.Y (GA, RC, HC, patch or snapshot) uses
  the same NexusIQ application
* NexusIQ application application *has* to exist before a build starts
2020-07-22 12:41:22 +01:00
Waldemar Żurowski
ca37b9b737
PR Code Checks use standard Jenkins agent (#6496) 2020-07-22 09:26:39 +01:00
Yiftach Kaplan
a03fb8c6fd
INFRA-438: Close session factory before closing the locator (#6477) 2020-07-21 18:10:09 +01:00
Waldemar Żurowski
1924e5a7d1
Merge pull request #6491 from corda/wz/merge-45-46-2020-07-21
NOTICK: Empty JUnit results are not allowed
2020-07-21 16:05:22 +01:00
Waldemar Zurowski
a90197ed81 Merge branch 'release/os/4.5' into wz/merge-45-46-2020-07-21 2020-07-21 15:59:07 +01:00
Waldemar Żurowski
e7f4e4f717
Merge pull request #6490 from corda/wz/merge-44-45-2020-07-21
NOTICK: Empty JUnit results are not allowed
2020-07-21 15:58:09 +01:00
Waldemar Zurowski
5ee2626530 Empty JUnit results are not allowed 2020-07-21 15:55:56 +01:00
Waldemar Zurowski
35d0d5c510 Merge branch 'release/os/4.4' into wz/merge-44-45-2020-07-21 2020-07-21 15:53:59 +01:00
Waldemar Żurowski
b52b242837
Merge pull request #6489 from corda/wz/merge-43-44-2020-07-21
NOTICK: Empty JUnit results are not allowed
2020-07-21 15:52:53 +01:00
Waldemar Zurowski
7437630d56 Empty JUnit results are not allowed 2020-07-21 15:50:21 +01:00
Waldemar Zurowski
969ee34b0a Merge branch 'release/os/4.3' into wz/merge-43-44-2020-07-21 2020-07-21 15:47:01 +01:00
Waldemar Żurowski
50c51d3e6f
Empty JUnit results are not allowed (#6488) 2020-07-21 15:42:57 +01:00