Commit Graph

9358 Commits

Author SHA1 Message Date
Adel El-Beik
9cd02dc62d Merge remote-tracking branch 'origin/release/os/4.4' into adel/merge-from-4.4-19-Feb-21 2021-02-19 13:55:04 +00:00
Denis Rekalov
23fab7ae0c
ENT-5891 Disable flaky test (#6879) 2021-02-19 13:54:34 +00:00
Adel El-Beik
360b3f8d0c
CORDA-4125: Backport AttachmentClassLoader updates to 4.4 to enable ENT-6152 backport (#6878)
* CORDA-3755: Switched attachments map to a WeakHashMap (#6214)

* Bump OS release version 4.6

* CORDA-3755: Switched attachments map to a WeakHashMap

* CORDA-3755: Added explicit strong references to map key.

* CORDA-3755: Keeping detekt happy.

* CORDA-3755: Test a gc in verify.

* CORDA-3755: Making detekt happy.

* CORDA-3755: Suppress warnings for weak reference test.

* CORDA-3755: Fixing build failure with attachments.

* CORDA-3755: Rewrite based on Ricks input - now handles attachment already existing in map!

* CORDA-3755: Refactor WeakReference behaviour into AttachmentsHolderImpl and provide alternate version of this class for core-deterministic.

* CORDA-3755: Added more tests for WeakHashMap.

* CORDA-3755: Ignore the tests using System.gc keep for local testing only

* CORDA-3755: Adding comment to explain the ignored tests.

* Make AttachmentsHolderImpl package-private inside core-deterministic, just like it is inside core.

* CORDA-3755: Update assertions following review comments.

* CORDA-3755: Removing import

* CORDA-3755: Removed unused var.

* CORDA-3755: Reverting files that somehow got changed in rebase.

Co-authored-by: nargas-ritu <ritu.gupta@r3.com>
Co-authored-by: Chris Rankin <chris.rankin@r3.com>

* CORDA-3769: Switched attachments class loader cache to use caffeine (#6326)

* CORDA-3769: Switched attachments class loader cache to use caffeine with original implementation used by determinstic core.

* CORDA-3769: Removed default ctor arguments.

* CORDA-3769: Switched mapping function to Function type to avoid synthetic method being generated.

* CORDA-3769: Now using a cache created from NamedCacheFactory for the attachments class loader cache.

* CORDA-3769: Making detekt happy.

* CORDA-3769: The finality tests now check for UntrustedAttachmentsException which will actually happen in reality.

* CORDA-3769: Refactored after review comments.

* CORDA-3769: Removed the AttachmentsClassLoaderSimpleCacheImpl as DJVM does not need it. Also updated due to review comments.

* CORDA-3769: Removed the generic parameters from AttachmentsClassLoader.

* CORDA-3769: Removed unused imports.

* CORDA-3769: Updates from review comments.

* CORDA-3769: Updated following review comments. MigrationServicesForResolution now uses cache factory. Ctor updated for AttachmentsClassLoaderSimpleCacheImpl.

* CORDA-3769: Reduced max class loader cache size

* CORDA-3769: Fixed the attachments class loader cache size to a fixed default

* CORDA-3769: Switched attachments class loader size to be reduced by fixed value.

* CORDA-4125: Parameter has been added to a private ctor.

Co-authored-by: nargas-ritu <ritu.gupta@r3.com>
Co-authored-by: Chris Rankin <chris.rankin@r3.com>
2021-02-19 13:38:22 +00:00
Dan Newton
41b4f9b3aa
CORDA-4110 startFlowDynamicWithClientId restrict permissions (#6875)
Restrict permissions for `startFlowDynamicWithClientId` to not work with
 `InvokeRpc.startFlow` and instead require `InvokeRpc
 .startFlowWithClientId`.
2021-02-18 14:46:09 +00:00
Nick Dunstone
42bb25462d
CORDA-4098 assumption test (#6871) (#6873) (#6874)
* Upgrade of Bouncy Castle to resolve security issue

* Changed default signature scheme

* Reverted default change in draft

* Key conversion for BC with ed25519

* Initializing BC provider to use X509EdDSAEngine for Signature.Ed25519

* removed unsude imports

Co-authored-by: Nick Dunstone <nick.a.dunstone@gmail.com>

Co-authored-by: Alexey Kadyrov <67952405+alexey-kadyrov-r3@users.noreply.github.com>

Co-authored-by: Alexey Kadyrov <67952405+alexey-kadyrov-r3@users.noreply.github.com>
2021-02-17 17:14:05 +00:00
Nick Dunstone
d41f608e4e
CORDA-4098 assumption test (#6871) (#6873)
* Upgrade of Bouncy Castle to resolve security issue

* Changed default signature scheme

* Reverted default change in draft

* Key conversion for BC with ed25519

* Initializing BC provider to use X509EdDSAEngine for Signature.Ed25519

* removed unsude imports

Co-authored-by: Nick Dunstone <nick.a.dunstone@gmail.com>

Co-authored-by: Alexey Kadyrov <67952405+alexey-kadyrov-r3@users.noreply.github.com>
2021-02-17 12:53:30 +00:00
Alexey Kadyrov
ae2bb9992f
CORDA-4098 assumption test (#6871)
* Upgrade of Bouncy Castle to resolve security issue

* Changed default signature scheme

* Reverted default change in draft

* Key conversion for BC with ed25519

* Initializing BC provider to use X509EdDSAEngine for Signature.Ed25519

* removed unsude imports

Co-authored-by: Nick Dunstone <nick.a.dunstone@gmail.com>
2021-02-17 10:27:45 +00:00
William Vigor
7261442c98
CORDA-4123 Change MockNetworkParameterStorage to not rehash NetworkParameters (#6872)
* CORDA-4123 Change Mock Network Parameter Storage to not rehash the Network Parameters

* Fix Detekt
2021-02-16 16:25:38 +00:00
Adel El-Beik
1a7401472f
CORDA-4120: Disable the URLConnection cache to prevent file handle leaks in ServiceLoader. (#6869) 2021-02-12 17:24:00 +00:00
William Vigor
20dbbf008d
CORDA-4103 Feature Branch: Serialization injection for transaction building (#6867)
* CORDA-4105 Add public API to allow custom serialization schemes (#6848)

* CORDA-4105 Add public API to allow custom serialization schemes

* Fix Detekt

* Suppress warning

* Fix usused import

* Improve API to use generics

This does not break Java support (only Intelij gets confused).

* Add more detailed documentation to public interfaces

* Change internal variable name after rename public API

* Update Public API to use ByteSquence instead of SerializedBytes

* Remove unused import

* Fix whitespace.

* Add added public API to .ci/api-current.txt

* Improve public interfaces

Rename CustomSchemeContext to SerializationSchemeContext to improve
clarity and move to it's own file. Improve kdoc to make things less
confusing.

* Update API current with changed API

* CORDA-4104 Implement custom serialization scheme discovery mechanism (#6854)

* CORDA-4104 Implement CustomSerializationScheme Discovery Mechanism

Discovers a single CustomSerializationScheme implementation inside
the drivers dir using a system property.

* Started MockNetwork test

* Add driver test of Custom Serialization Scheme

* Fix detekt and minor style error

* Respond to review comments

Allow non-single arg constructors (there must be one no args
constructor), move code from SerializationEnviroment into its
own file, improve exceptions to be more user friendly.

* Fix minor bug in Scheme finding code  + improve error messages

* CORDA-4104 Improve test coverage of custom serialization scheme discovery (#6855)

* CORDA-4104 Add test of classloader scanning for CustomSerializationSchemes

* Fix Detekt

* NOTICK Clarify KDOC on SerializationSchemeContext (#6865)

* CORDA-4111 Change Component Group Serialization to use contex when the lazy map is constructed (#6856)

Currently the component group will recheck the thread local (global)
serialization context when component groups are serialized lazily.
Instead store the serialization context when the lazy map is constructed
and use that latter when doing serialization lazily.

* CORDA-4106 Test wire transaction can still be written to the ledger (#6860)

* Add test that writes transaction to the Database

* Improve test check serialization scheme in test body

* CORDA-4119 Minor changes to serialisation injection for transaction building (#6868)

* CORDA-4119 Minor changes to serialisation injection for transaction building

Scan the CorDapp classloader instead of the drivers classloader.
Add properties map to CustomSerialiaztionContext (copied from SerializationContext).
Change API to let a user pass in the serialization context in TransactionBuilder.toLedgerTransaction

* Improve KDOC + fix shawdowing issue in CordaUtils

* Pass only the properties map into theTransactionBuilder.toWireTransaction

Not the entire serializationContext

* Revert change to CordaUtils

* Improve KDOC explain pitfalls of setting properties
2021-02-11 15:27:03 +00:00
Dan Newton
4f336a1a67
Merge pull request #6866 from corda/dan/merge-os-4.7-to-4.8-2021-02-08
NOTICK OS 4.7 to 4.8 2021-02-08
2021-02-08 15:31:43 +00:00
Dan Newton
cef5ea40ca Merge branch 'release/os/4.7' into dan/merge-os-4.7-to-4.8-2021-02-08 2021-02-08 14:38:17 +00:00
Dan Newton
0659932be0
Merge pull request #6862 from corda/dan/os-4.6-to-4.7-merge-2021-02-05
NOTICK OS 4.6 to 4.7 merge 2021-02-08
2021-02-08 14:36:13 +00:00
Dan Newton
26d6ccdc25 CORDA-4110 startFlowDynamicWithClientId permissions 2021-02-08 11:34:25 +00:00
Dan Newton
d807082cee Merge branch 'release/os/4.6' into dan/os-4.6-to-4.7-merge-2021-02-05
# Conflicts:
#	node/src/main/kotlin/net/corda/node/internal/rpc/proxies/AuthenticatedRpcOpsProxy.kt
2021-02-05 16:13:28 +00:00
Dan Newton
4e437ace2a
CORDA-4110 startFlowDynamicWithClientId permissions (#6857) 2021-02-05 16:05:55 +00:00
Edoardo Ierina
d7c9f20019
CORDA-4100: Updates to DigestAlgorithm interface (#6849)
Updated DigestAlgorithm interface to separate componentHash and computeNonce hashing functions.
2021-02-01 10:52:47 +00:00
Rick Parker
ade302be73
Merge pull request #6852 from corda/dan/ENT-6092-port-to-os
ENT-6092 Add flush to `MockServices.withEntityManager` (#4180)
2021-01-29 17:06:12 +00:00
Dan Newton
682de39d4c ENT-6092 Add flush to MockServices.withEntityManager (#4180) 2021-01-29 17:01:55 +00:00
Dan Newton
d214f5ecbf
Merge pull request #6851 from corda/dan/ENT-6142-port-to-os
ENT-6142 port to OS
2021-01-29 13:29:25 +00:00
Dan Newton
88172b630d ENT-6142 Use ArrayList for SessionState structures (#4169)
Prevent some serialization errors that occur due to serialization
and deserialization of `ArrayList$SubList` found inside the
`SessionState` data structures.

To prevent this, an explicit `ArrayList` is used rather than a `List`.

Overload the `List` operator functions so that `+` returns an
`ArrayList` instead of a `List`.

Create `toArrayList` for a few conversions.
2021-01-29 11:15:28 +00:00
Dan Newton
c79ad972d0 ENT-6142 Flows become dead due to uncaught exceptions (#4158)
If a flow fails outside of its normal error processing code path it will
end up in `FlowDefaultUncaughtExceptionHandler`.

This handler will put the flow into overnight observation if possible.
This is done in-memory and the database.

Even with this being done, the fiber itself has blown up and therefore
does not manage to get to `SMM.removeFlow` which is where
`SMM.decrementLiveFibers` is called. For example, a flow that errored
will hit this code eventually. This code is also hit when a flow is
suspended and a shutdown event is sent to it.

The `liveFibers` latch blocks the SMM from shutting down until all flows
have finished or processed shutdown events.

The changes described below resolve this problem.

Any flow that goes to the `FlowDefaultUncaughtExceptionHandler` will be
put marked as dead (`StateMachineState.isDead`). Highlighting that the
flow cannot continue to process events normally as it has broken out
of its event loop

Retrying and shutdown are done manually rather than injecting events
into the flow fiber's queue, because it can't execute its event loop.

Killing a dead flow executes an altered version of
`retryFlowFromSafePoint`. It does this so it can delete the checkpoint
and then continue using the checkpoint it just deleted to run the
kill flow transition on a new fiber.

If a killed flow reaches the `FlowDefaultUncaughtExceptionHandler` it
will be forcibly killed via `killFlowForcibly` which deletes the
checkpoint/or updates it to KILLED and then calls `removeFlow` to bypass
any event processing. This means that a flow that was dead and was killed
will be terminated manually if it reaches the handler again. The same is
true for flows that were not dead before but reached the handler after
being killed.

Also, `FlowCreator.createFlowFromCheckpoint` now retains the `isKilled`
state of the previous fiber's state.
2021-01-29 11:15:28 +00:00
Alexey Koren
56df286410
Add ability to set custom initiating flows for NotaryService (#6832) 2021-01-26 13:56:42 +00:00
Adel El-Beik
284fd48918
CORDA-4101: Remove attachmentPresence cache (#6845)
* CORDA-4101: Remove the attachmentCache and just rely on attachmentContentCache

* CORDA-4101: Remove the attachmentCacheBound variable.
2021-01-13 16:28:56 +00:00
Denis Rekalov
af87d1d50d
Merge pull request #6842 from corda/nandor/4.7-4.8-merge-os
NOTICK OS 4.7 -> 4.8 merge
2021-01-08 15:13:45 +00:00
nkovacsx
be6587a602 Merge branch 'release/os/4.7' into nandor/4.7-4.8-merge-os 2021-01-08 14:25:33 +00:00
nkovacsx
7a079a166b
CORDA-4097 Return stream instead of List in getAllCertificates (#6840) 2021-01-08 09:41:31 +00:00
Ramzi El-Yafi
ec92801904
Port of notary query API from ENT (#6837) 2020-12-18 09:59:12 +00:00
Denis Rekalov
47baba3650
Merge pull request #6834 from corda/denis/merge-4.7-to-4.8
NOTICK: Merge OS 4.7 to 4.8
2020-12-17 18:58:23 +00:00
Ross Nicoll
bab99e2fb4
NOTICK Move docs external links to HTTPS (#6835) 2020-12-15 15:27:52 +00:00
Denis Rekalov
f5d9d97573 Merge branch 'release/os/4.7' into denis/merge-4.7-to-4.8 2020-12-14 10:56:23 +00:00
Ross Nicoll
62fe4bc65a
NOTICK Publish to correct Docker repository (#6831) 2020-12-03 21:42:57 +00:00
Ross Nicoll
534bccc88d
INFRA-1038 Publish RC/HC builds to internal repo (#6830)
Publish RC/HC builds to internal repo instead of DockerHub.
2020-12-02 16:26:15 +00:00
Ross Nicoll
dce0e581e7
INFRA-1033 Publish RCs to DockerHub (#6829) 2020-11-30 22:30:29 +00:00
Ramzi El-Yafi
b3c46429cb
NOTICK Update notary code owners (#6828) 2020-11-30 20:17:13 +00:00
Stefan Iliev
2c25a5c768
AG-323 Flow startup queue metric (#6811) 2020-11-30 16:49:08 +00:00
Edoardo Ierina
17b0edffd1
restored cash changelog v1 to its original state (#6822) 2020-11-27 12:16:50 +00:00
Ross Nicoll
d9e3220577
NOTICK Switch published Docker image to official (#6824) 2020-11-26 13:47:35 +00:00
Ross Nicoll
86aa0c4252
INFRA-1009 Remove undefined COMMON_GRADLE_PARAMS (#6819) 2020-11-26 10:31:50 +00:00
Denis Rekalov
503644ca29
CORDA-4083: Fix Postresql migration, synch migration scripts from ENT (#6817) 2020-11-24 20:03:36 +00:00
Ross Nicoll
a8ae7fdd6a
INFRA-1009 Update Docker publication tasks (#6818) 2020-11-24 17:36:53 +00:00
nargas-ritu
5a13e48ea8 NOTICK: Changes done for creating new branch for 4.8 2020-11-23 14:37:14 +00:00
Walter Oggioni
734d35b719
INFRA-803 Rebuild Docker image tasks (#6804) 2020-11-20 10:21:53 +00:00
Edoardo Ierina
02018b75e9
ENT-6018: renamed hash agility system properties (#6808) 2020-11-17 12:50:31 +00:00
Waldemar Żurowski
700cb51091
INFRA-965: forward merge from 4.6 2020 11 15 (#6807)
* CORDA-4045 Disable flaky test (#6792)

Disable `Restart does not set senderUUID` as it is unstable.

* INFRA-965: forward merge from 4.5 (2020-11-15) (#6806)

* EG-4168 Updating contributors.md list for OS 4.5 release branch (#6784)

* INFRA-965: forward merge from 4.4 (2020-11-14) (#6805)

* EG-4168 Updating contributors.md list for OS 4.4 release branch (#6785)

* INFRA-965: forward merge from 4.3 (2020-11-13) (#6803)

* INFRA-965: Jenkins/NexusIQ integration should target patches (#6802)

as well as major/minor releases

* Updated nightly builds to match changes for release branch builds

* Updated JDK11 builds to match changes for release branch builds

Co-authored-by: ivanterziev <61829352+ivanterziev-r3@users.noreply.github.com>

* Backporting updates for check-pr-title workflow

Co-authored-by: ivanterziev <61829352+ivanterziev-r3@users.noreply.github.com>

Co-authored-by: Ross Nicoll <ross.nicoll@r3.com>
Co-authored-by: ivanterziev <61829352+ivanterziev-r3@users.noreply.github.com>
2020-11-16 07:18:22 +00:00
Waldemar Żurowski
b421f0daa0
INFRA-965: forward merge from 4.5 (2020-11-15) (#6806)
* EG-4168 Updating contributors.md list for OS 4.5 release branch (#6784)

* INFRA-965: forward merge from 4.4 (2020-11-14) (#6805)

* EG-4168 Updating contributors.md list for OS 4.4 release branch (#6785)

* INFRA-965: forward merge from 4.3 (2020-11-13) (#6803)

* INFRA-965: Jenkins/NexusIQ integration should target patches (#6802)

as well as major/minor releases

* Updated nightly builds to match changes for release branch builds

* Updated JDK11 builds to match changes for release branch builds

Co-authored-by: ivanterziev <61829352+ivanterziev-r3@users.noreply.github.com>

* Backporting updates for check-pr-title workflow

Co-authored-by: ivanterziev <61829352+ivanterziev-r3@users.noreply.github.com>
2020-11-15 20:36:25 +00:00
Waldemar Żurowski
bb8fcfe6be
INFRA-965: forward merge from 4.4 (2020-11-14) (#6805)
* EG-4168 Updating contributors.md list for OS 4.4 release branch (#6785)

* INFRA-965: forward merge from 4.3 (2020-11-13) (#6803)

* INFRA-965: Jenkins/NexusIQ integration should target patches (#6802)

as well as major/minor releases

* Updated nightly builds to match changes for release branch builds

* Updated JDK11 builds to match changes for release branch builds

Co-authored-by: ivanterziev <61829352+ivanterziev-r3@users.noreply.github.com>
2020-11-14 20:56:32 +00:00
Waldemar Żurowski
90238f22a0
INFRA-965: forward merge from 4.3 (2020-11-13) (#6803)
* INFRA-965: Jenkins/NexusIQ integration should target patches (#6802)

as well as major/minor releases

* Updated nightly builds to match changes for release branch builds
2020-11-14 10:18:51 +00:00
Waldemar Żurowski
19274751d9
INFRA-965: Jenkins/NexusIQ integration should target patches (#6802)
as well as major/minor releases
2020-11-13 17:03:33 +00:00
Waldemar Żurowski
d6154ab59e
Updated pr-lint-action to the latest version
v1.1.1 doesn't work any longer
2020-11-13 15:47:43 +00:00