Commit Graph

2759 Commits

Author SHA1 Message Date
Denis Rekalov
bc96bea24a ENT-4628: Harmonize CryptoService and BCCryptoService between OS and ENT (#5822) 2019-12-18 16:54:39 +00:00
Denis Rekalov
8d5781db43 ENT-4628: Harmonize net.corda.nodeapi.internal.crypto between OS and ENT (#5820)
* ENT-4628: Harmonize net.corda.nodeapi.internal.crypto between OS and ENT

* ENT-4628: Fix detekt
2019-12-18 13:59:30 +00:00
Chris Rankin
545a463a5b NOTICK: Remove buildscript block from Node's build.gradle. (#5809) 2019-12-16 16:24:47 +00:00
Viktor Kolomeyko
91cdffd429 ENT-4382: Core module changes to support multi-RPC work (#5814)
* ENT-4382: Move `InvocationHandlerTemplate` into `core`

This is an internal helper which is general enough and does not have any Node specific code.

* ENT-4382: Make @CordaInternal applicable to classes

And apply it on `AttachmentTrustCalculator` which is `core/internal` interface anyway.
2019-12-16 16:03:48 +00:00
Christian Sailer
4a35b99283 CORDA-3479: Added timestamp to the node_transactions table. (#2660) (#5812)
* ENT-4237: Added timestamp to the node_transactions table.

* ENT-4237: Clock for timestamp now retrieved from ServiceHub. And now record verification time as well.

* ENT-4237: Fixed tests. Also enabled stream output in allParallelIntegrationTest.

* ENT-4237: Changed timestamp to a val.

* ENT-4237: Changed streamOutput to false for allParallelIntegrationTest

* ENT-4237: Unit tests added for new timestamp column. Also now passing a clock into DBTransactionStorage.

* ENT-4237: Added more unit tests to check timestamp

* ENT-4237: Fix test to actually change clock time when testing transaction time does not change.
2019-12-13 15:14:21 +00:00
Kyriakos Tharrouniatis
b0903efa50 CORDA-3202 Add a specific exception for flows to hospitalise themselves (#5767)
* Introducing a new type of exception and a new hospital staff member to pause flows by immediately hospitalising them.

* Renaming exception to "HospitalizeFlowException".

* Making HospitalizeFlowException an open class.

* Overloading constructors of HospitalizeFlowException to be available in Java.

* Using Throwable#mentionsThrowable.

* Moving HospitalizeFlowException in its own file.

* Update kdocs for HospitalizeFlowException and StaffedFlowHospital#SedationNurse.

* Added tests, testing various HospitalizeFlowException types thrown.

* Fix Detekt issues.

* Imports optimizing.

* Add safe casting.

* Update api-flows and node-flow-hospital docs.

* Minor code comment change.

* Add DOCSTART-DOCEND signs in HospitalizeFlowException for makeDocs. It is referenced by api-flows.rst.

* Minor change in note.

* Code formatting.

* Remove comment.

* Remove if statement that makes example worse.

* Remove redundant comment.

* Moving 'Internal Corda errors' at the bottom.

* Changing node-flow-hospital.rst as per review.

* Change HospitalizeFlowException description as per review.

* Adding an example for FlowException.

* Minor indentation fix.

* Update FlowException example label as per review.

* Correcting handling of custom exception.
2019-12-11 10:35:58 +00:00
Christian Sailer
14050826e9 ENT-4595 harmonize core and serialization (#5792)
* Harmonize serialization/core and deterministic counterparts

* Fix test for changed private alias key behaviour

* Detekt errors

* roll back project.xml
2019-12-09 14:17:48 +00:00
Viktor Kolomeyko
43205e1f1a CORDA-3471: Create CordaTransactionSupport and make it accessible through AppServiceHub (#5768)
* CORDA-3471: Create `CordaTransactionSupport` and use wherever possible instead of `CordaPersistence`

* CORDA-3471: Address comments by @mnesbit
- Relocate `CordaTransactionSupport` to `core`
- Create a lighter version of transaction - `VaultTransaction` that gives access to `session` object only.

* CORDA-3471: More changes after discussion with  @mnesbit
- Rename `VaultTransaction` into `SessionScope`.

* CORDA-3471: Revert changes to most of the files after conversation with @mnesbit and @rick-r3

* CORDA-3471: Introduce `CordaTransactionSupportImpl` and make it accessible via `AppServiceHub`.

* CORDA-3471: Minor change (comment).

* CORDA-3471: Address input from @mnesbit

* CORDA-3471: Address input from @rick-r3

* CORDA-3471: Make Detekt happier

* CORDA-3471: Add a new test that proves transactions can be started from client threads

As requested by @mnesbit

* CORDA-3471: Change log and documentation update.

As requested by @mnesbit
2019-12-04 17:18:40 +00:00
Chris Rankin
5a41ec9b82 CORDA-3464: Also scan attachment:// URLs for custom serializers. (#5769)
* CORDA-3464: Also scan attachment:// URLs for custom serializers.

* Only scan the given classloader - ignore this classloader's parents.

* Upgrade to ClassGraph 4.8.58 - for "robustness fixes".

* Register the attachment:// URL scheme using AttachmentsClassLoader.

* Add integration test for custom serializer in contract state.

* Rename Currancy -> Currantsy, just to make the point.
2019-12-03 15:50:21 +00:00
Nick Dunstone
e6f9b46584 https://r3-cev.atlassian.net/browse/ENT-3829 (#5763)
Open Source
2019-12-03 10:28:00 +00:00
Rick Parker
6e467f20b3
CORDA-3470 Fix flow async operations (#5780) 2019-12-02 14:40:07 +00:00
Christian Sailer
06f97cfed5 ENT-4610 Move tx signing to the Uniqueness provider (#5773)
* ENT-4610 Move tx signing to the Uniqueness provider

* Make detekt happy

* Remove unused imports

* Address review comment
2019-11-29 17:30:33 +00:00
Dan Newton
45d6d3ead4 CORDA-3356 Subflow ledger consistency tests + move statemachine tests to slow integration tests (#5722)
* CORDA-3356 Subflow ledger consistency tests + move statemachine tests to slow integration tests

Add tests for subflows that fail during transitions.

Split out `StatemachineErrorHandlingTest` into a series of smaller tests.

Move these tests into the `integration-test-slow` category so they are
not run against every PR.

* CORDA-3356 Fix detekt issue

* CORDA-3356 Tidy test names
2019-11-25 17:11:54 +00:00
Kyriakos Tharrouniatis
21a075b727 CORDA-3197 Fix flow has been waiting message (#5660)
* Added a timestamp property to Checkpoint getting a new Instant.now() value at every Checkpoint instantiation/ copy instantiation. FlowMonitor is now using this new property (Checkpoint#timestamp) and StateMachineState#isFlowResumed to determine which flows are actually suspended. It leaves out flows that are doing work in their FlowLogic#call method.

* Cleaner comment

* Broke FlowMonitor#logFlowsWaitingForParty into logFlowsWaitingForParty and waitingFlowsToDurations. This way waitingFlowsToDurations is modular and can be tested.

Made FlowMonitor constructor get StateMachineManager instead of the retrieveFlows lamda. This way FlowMonitor is more consistent as a service, and entire flow filtering process is now being done in FlowMonitor#waitingFlowsToDurations.

Removed "smm as? StateMachineManagerInternal" in AbstractNode#start as it made no sense.

Updated CheckpointDumper to mention the Checkpoint#timestamp when writing the checkpoint as json.

* Added tests for FlowMonitor service.

* Remove old comment

* 1. FLowMonitor#waitingFlowDurations now returns a Sequence to have an iteration less.
It used to be, one iteration from returning a Set from FLowMonitor#waitingFlowDurations plus one iteration from FlowMonitor#logFlowsWaitingForParty.

2. Code reformattings

* 1. Remove constructor keyword from FlowMonitor

2. Code reformattings

3. Update detekt baseline

* Resolve conflict in Detekt baseline
2019-11-25 16:49:35 +00:00
Ed Prosser
e692ec292a Merge branch 'release/os/4.3' of https://github.com/corda/corda into EdP/CORDA-3446-4.4 2019-11-20 10:36:38 +00:00
Stefan Iliev
926429647d CORDA-3307: Fix for underscore variables (#5682)
* Revert "Revert "CORDA-3307 - add support for environment variables in linux (#5523)" (#5643)"

This reverts commit 03ab258fc2.

* Env variables with underscore are now validated using schema validation and checking for unknown key errors.

* Resolving comments from PR review.

* Fix for deprecated import.

* Reworked logic according to PR review.

* Resolved bad string parsing problems where the json structure could be broken if some symbols were included in the key or value.
2019-11-19 17:51:52 +00:00
Chris Rankin
ce8ce2ba7c CORDA-2876: Upgrade to deterministic-rt 1.0-RC01. (#5738) 2019-11-19 15:02:35 +00:00
Denis Rekalov
a791e38319 CORDA-3443: Do not log warning when appending duplicated network parameters (#5739) 2019-11-19 15:02:07 +00:00
Stefano Franz
221576d94a
move - responding flow from suffix to prefix of test name (#5731) 2019-11-18 15:41:15 +00:00
Chris Rankin
c68f8161fc CORDA-2876: Upgrade DJVM to 1.0-RC03. 2019-11-16 16:51:24 +00:00
Oliver Knowles
4a8c73bf4a CORDA-2744 - Prevent ClassCastException if error during initial registration (#5712)
* Prevent class cast exception if error during initial registration

* Appeasing detekt
2019-11-14 16:13:08 +00:00
Jonathan Locke
c31cddaffc Fixed merge conflicts 2019-11-14 10:40:12 +00:00
Chris Rankin
987415dd83 CORDA-3407: Install external byte-code cache for sandboxed CorDapps. (#5699) 2019-11-14 09:09:57 +00:00
Jonathan Locke
86f1f9e371
CORDA-3393: Lazy load MessageDeduplicationHandler.flowId (#5679)
CORDA-3393: Lazy load `MessageDeduplicationHandler.flowId` (#5679)
2019-11-07 09:44:03 +00:00
Dan Newton
b15db200e3 NOTICK: Quick and dirty change to stop "Unable to start notaries." error message (#5686)
* Quick and dirty change to stop "Unable to start notaries." error message (#5686)

"Unable to start notaries. A required port might be bound already" is
returned whenever a startup error occurs while starting the notary nodes
in driver tests. This hides the real error.

This change prints the actual error to std_err and read from file
at a later point. This means the real error is not lost and will be
shown in failed builds.

* Suppress detekt warnings
2019-11-07 09:43:29 +00:00
stefano
5d6fc34f88 fix merge issue with exception thrown in RpcExceptionHandlingTest 2019-11-06 12:28:05 +00:00
LankyDan
91f35d5967 CORDA-3393 Lazy load MessageDeduplicationHandler.flowId
This is to potentially help with debugging in the future as the
`flowId` could become confusing for received messages where the `flowId`
has nothing to do with the current flow.
2019-11-06 11:26:25 +00:00
stefano
03d8009c7d fix temp folder rules 2019-11-05 17:56:06 +00:00
stefano
d8ed3492b9 fix merge conflicts due to changes in Signed test cordapp creation 2019-11-05 17:42:38 +00:00
stefano
6a3a9eded3 Merge remote-tracking branch 'origin/release/os/4.3' into my_merge_branch
# Conflicts:
#	core/src/main/kotlin/net/corda/core/internal/TransactionVerifierServiceInternal.kt
#	node/src/integration-test/kotlin/net/corda/node/services/rpc/RpcExceptionHandlingTest.kt
#	testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/DriverDSLImpl.kt
2019-11-05 17:19:18 +00:00
Chris Rankin
f226ddc4f2 CORDA-2876: Integrate the DJVM with the Corda Node. (#5633)
* * CORDA-2876: Migrate DJVM serialization modules into Corda.
* Pre-generate Corda classes for DJVM sandbox when node boots in production mode.
* Ensure that all DJVM test contract CorDapps are signed.
* Test examining attachments within DJVM sandbox.
* Test Contract.verify() using cryptographic verify function.
* Add test cases for more non-determinism in Contract.verify().
* Update node-driver to support testing nodes with DJVM support.
* Modify Node to allow alternative DJVM configurations for testing.
* Refactor DeterministicVerifierFactoryService for default use-case.
* Small whitespace and code-style refactors.
* Create and activate a DJVM execution profile for the Node.
* Revert making Verifier implement AutoCloseable.
* Allow the node to cache sandboxed Corda byte-code for reuse.
* Use updated Quasar agent that knows not to touch DJVM classloaders.
* Fix Quasar's package exclusions globs for DJVM.
* Deserialise LedgerTransaction into the sandbox for Contract.verify().
* Add the DJVM's serialisation modules to the Corda node.
* Update the node for the latest DJVM API, and preserve the ConstructorForDeserialization annotation on user contract classes.
* Add corda-dev to repositories while DJVM is SNAPSHOT.
* Migrate DJVM specialisation into AbstractNode's ServiceHubInternalImpl.
* Exclude sandbox.** and shaded djvm.** classes from Quasar agent.
* Add the corda-dev repository to :node for the deterministic runtime.
* Turn Verifier into an abstract base class that is specialised by BasicVerifier and DeterministicVerifier.
* Add the Corda deterministic libraries to the Node, and split the DJVM sandbox across two SandboxClassLoader instances.
* Add DJVM to contract verification path inside Corda Node.
* Minor lambda simplifications and removing unused import.
* CORDA-2871: Remove @CordaSerializable from LedgerTransaction.
* CORDA-2871: Add a callback to ServicesForResolution to allow the Node to modify a LedgerTransaction object.
* CORDA-2871: Refactor the contract verification code into a separate class,
 and allow LedgerTransaction to choose different Verifier objects.
* Update DJVM to use Corda 4.4-SNAPSHOT. (#95)
* CORDA-3330: Allow DJVM to preload / pregenerate classes from selected jars. (#92)
* Add support for SourceClassLoader.getResources() to DJVM.
* Allow a SandboxConfiguration to preload sandbox byte-code for all classes inside jars containing META-INF/DJVM-preload.
* CORDA-3309: Remove explicit try-catch in favour of UncaughtExceptionHandler. (#91)
* CORDA-3309: Install UncaughtExceptionHandler for DJVM tasks. (#88)
* Fix tests broken by Windows line endings. (#82)
* CORDA-3292: Reimplement ExecutionProfile as a data class. (#80)
* CORDA-2877: Refactor how we create child SandboxConfiguration objects. (#76)
* CORDA-2877: Load bytecode from a persistent cache to prevent repeated rewriting. (#75)
* Refactor byte-code cache to SandboxConfiguration instead of AnalysisConfiguration. We cannot "mix and match" byte-code generated by different sets of rules.
* CORDA-3137: Enhance annotation handling so that we can allow some annotations to be mapped into the sandbox without also needing to be stitched. (#72)
* CORDA-2871: Minor cosmetic fixes. (#69)
* CORDA-3218: Align DJVM with internal Corda Serialisation API. (#68)
* Ensure we get the latest SNAPSHOT of the serialisation code.
* CORDA-2871: Refactor SourceClassLoader to define source classes. (#66)
* Rewrite SourceClassLoader to support parent/child relationships.
* Revert catching TypNotPresebtException - it was a symptom of a bigger problem.
* Remove AutoCloseable from AnalysisConfiguration and SourceClassLoader.
* SourceClassLoader.getResource() must delegate to its parent first.
* CORDA-2871: Ensure ClassLoader.loadClass() throws ClassNotFoundException for all cases where the class cannot be found. (#64)
* CORDA-2871: Modify sandbox tasks to implement both java.Function and sandbox.Function (#62)
* Make TaskExecutors implement BiFunction to make them composable.
* Create ImportTask to wrap a java.Function inside a sandbox.Function.
* Add createExecutor() and createRawExecutor() APIs to SandboxClassLoader.
* Update serialization to use SandboxClassLoader.toSandboxClass().
* Remove a layer of lambdas from the serialisation code.
* Update SandboxExecutor and SandboxRawExecutor.
* Rename Executor to TaskFactory.
* Rename dangling executor -> taskFactory.
* CORDA-2871: Sanity fixes! (#63)
* Improve message for SandboxClassLoadingException.
* Fix serialisation API for using sandboxed environment.
* CORDA-3174: Extend serialisation to include InputStream and OpaqueBytesSubSequence. (#60)
* Update DJVM Example project for serialisation.
* Add serializers for InputStream and OpaqueBytesSubSequence.
* Support ZIP Inflater and CRC32 inside the sandbox.
* Allow the DJVM to wrap java.io.InputStream as sandbox.java.io.InputStream.
* Configure tests also to preserve @DeprecatedConstructorForDeserialization.
* CORDA-3174: Implement Corda serialization modules. (#59)
* Create DJVM serialization modules.
* Create test cases for Array<T>, List<T> and List<Array<T>>.
* Refactor SandboxPrimiveSerializer for all primitive types.
* Implement SandboxCollectionSerializer to support Collection types.
* Implement SandboxMapSerializer to support Map types.
* Attempt to fix infinite loop when computing Collection and Map fingerprints.
* Apply special handling when deserialising sandbox.java.lang.Character.
* Remap Java primitive types to sandbox Java object types to deter evolution.
* Use Class.getPackage().getName() to determine sandbox package name.
* Implement SandboxEnumSerializer to support Enum types.
* Implement SandboxPublicKeySerializer to support Java security keys.
* Add serialization projects to the composite example project.
* Implement serializers for BigInteger, BigDecimal, Currency and StringBuffer.
* Test that deserialising does not instantiate the untrusted user classes.
* Implement serializers for java.time.* types.
* Add serialiser for BitSet - currently disabled until BitSet itself is supported.
* Add serialisers for EnumSet and Class.
* Include support for EnumMap in the SandboxMapSerializer.
* Ensure the DJVM Example project's tests preserve @CordaSerializable.
* Add support for UUID as a primitive type.
* Use common abortReadOnly() method for declaring serialization as unsupported.
* Streamline the API for deserialising into the sandbox.
* Add preliminary support for deserialising X.509 certificates.
* Implement serializer for java.util.Optional.
* Refactor configuration of the sandbox serialization scheme.
* Add tests for deserialising arrays of basic types.
* Include method annotations in annotation stitching. This ensures that `@ConstructorForDeserialization` is not dropped.
* Enable test for SandboxBitSetSerializer.
* Enable tests for X.509 serializers.
* Implement serializers for ProtonJ primitive types.
* Serialize java.util.Date as a primitive type.
* Add the bintray Gradle plugin to the serialisation modules.
* Do not publish serialisation modules - they will become part of Corda itself.

* CORDA-2876: Only apply DJVM sources to Node Driver when devMode=true.

* Resolve DeteKT warnings.

* Require Node's JVM to set -Dnet.corda.djvm=true in order to enable DJVM.

* Enable DJVM for DemoBench nodes.

* Disable Quasar instrumentation verification for DemoBench nodes.

* Upgrade to DJVM 1.0-RC01.

* Try to modify DriverParameters in a more "ABI friendly" way.

* Refactor and simplify sandbox deserialisation of primitive objects.

* Review fixes.

* Update EvolutionSerializerFactory to handle sandboxed primitive boxed types.
2019-11-05 13:44:18 +00:00
LankyDan
532e4989d6 @Ignore P2PMessagingTest - communicating with a distributed service which we're part of 2019-11-05 11:52:32 +00:00
Dimos Raptis
485feb2d6c CORDA-3350: Increase size of constraints column (#5639)
* CORDA-3350: Increase size of constraints column (#5639)

* Detekt

* Update api file with new threshold

* Add check in transaction builder

* Revert "Add check in transaction builder"

This reverts commit ca3128f44c.

* Add check for max number of keys

* Update api file

* Address Tudor's comments

* Remove check for pre-5 and add test for EC keys

* fix typo and rename liquibase script

* updated docs with measurement numbers for composite keys

* Make detekt happy again
2019-11-05 11:00:26 +00:00
stefano
5ac1e50135 Merge remote-tracking branch 'origin/release/os/4.3' into my_merge_branch 2019-11-04 17:48:15 +00:00
LankyDan
54394f6747 Release state soft locks when a flow is killed via killFlow 2019-11-04 13:21:06 +00:00
LankyDan
3c0631a26a Fix VaultObserverExceptionTest due to differences between ENT and OS 2019-11-04 09:42:41 +00:00
LankyDan
be2e34b33a Fix error from warning from detekt 2019-11-01 16:00:09 +00:00
LankyDan
3aaddb47ea Attempt to fix detekt issues in DriverDSLImpl again 2019-11-01 14:36:47 +00:00
LankyDan
b05bd76a77 Fix error in VaultObserverExceptionTest due to bad merge 2019-11-01 13:38:42 +00:00
LankyDan
e737b01184 Port ledger integrity work to SingleThreadedStateMachineManager
- Port ledger integrity work to `SingleThreadedStateMachineManager`
- Fix `StatemachineErrorHandlingTest`
- Fix compile errors in `RetryFlowMockTest` + `VaultObserverExceptionTest`
- Add method to `StaffedFlowHospital` that was missed during original merge
2019-11-01 11:50:16 +00:00
Christian Sailer
f670cf774b Suppress/fix more warnings 2019-11-01 11:48:07 +00:00
Christian Sailer
1f82213827 Suppress/Baseline detekt issues that were highlighted freshly because of the merge. 2019-11-01 11:48:07 +00:00
Christian Sailer
5b9c5a6b83 Fix and or suppress detekt warnings 2019-11-01 11:48:07 +00:00
Christian Sailer
119f939ee1 Fix and or suppress detekt warnings 2019-11-01 11:48:07 +00:00
Christian Sailer
48aa2f2faa Update compiler.xml for new projects (and code style settings) 2019-11-01 11:48:07 +00:00
LankyDan
c5124689ca CORDA-3194 Update TODOs with jira tickets 2019-11-01 11:48:07 +00:00
Christian Sailer
8f2923c7a3 CID-843 code review fixes (#2681)
* Layout fix

* Fix test to use proper hook in flow hospital

* Fix comment
2019-11-01 11:48:07 +00:00
LankyDan
44206a0bea CORDA-3194 Update flow hospital docs, tidy comment and make exception public 2019-11-01 11:48:07 +00:00
Dan Newton
f4e9b9d5d2 CORDA-3194 Stop killed flows from re-entering hospital (#2664)
* CORDA-3194 Do not allow killed flows back into the hospital

This change has been made to prevent killed flows from being added back
to the hospital after being forcibly removed by `killFlow`. Not doing so,
could leave references to a flow inside of the hospital, which is not
the correct behaviour.

`killFlow` now sets a flow's `StatemachineState.isRemoved` to true.

This check is then used in `StaffedFlowHospital` and the
`DumpHistoryOnErrorInterceptor`.

* CORDA-3194 Log different message for transition error due to killed flow

When a flow is killed, its checkpoint is deleted. Currently, the
statemachine will still try a process the next event even if it has
been killed. This can lead to an error when trying to update the
deleted checkpoint. The exception thrown from this is logged out.

An if statement has been added to log a different message at debug level
if it is due to an update error for a killed flow. This is done to not
alarm node operators of the exception.
2019-11-01 11:48:07 +00:00
Dan Newton
3b3dbd7352 CORDA-3194 Add extra test cases to StatemachineErrorHandlingTest (#2651)
* CORDA-3194 - Add `ReceiveFinalityFlow` byteman tests

* CORDA-3194 Tidy up `StatemachineErrorHandlingTest`
2019-11-01 11:48:07 +00:00
Dan Newton
a591c8e25b CORDA-3194 Relax duplicate insert flow hospital handling (#2643)
* CORDA-3194 Relax duplicate insert flow hospital handling

Revert a previous change to now make the duplicate insert staff member to
give a diagnosis of discharge or not my speciality (previously gave
terminal).

This is to prevent duplicate insert handling from overriding finality
flow error handling.
2019-11-01 11:48:07 +00:00
kyriathar
877ce5587f CORDA-3196 warning at intervals when flows waiting in flow hospital (#2636)
* added the warning as a TimerTask at StaffedFlowHospital#delayedDischargeTimer

* moved the scheduling of the warning task at StaffedFlowHospital#init block. That way we ensure that the task will be scheduled only once at StaffedFlowHospital initialization.

* Corrected overnight observation warning task's logging message. Changed StaffedFlowHospital#delayedDischargeTimer to the more generic StaffedFlowHospital#hospitalJobTimer since it now schedules delayed discharges tasks as well the overnight observation warning task. Removed this from property reference

* switching to fun timerTask for the instantiation of anonymous TimerTask classes

* Correct condition to log patients who are currently in the hospital, whose last record in their medical records is Outcome.OVERNIGHT_OBSERVATION. Extended logging to include treatableSessionInits staying in the hospital

* Add not empty check for patientsUnderOvernightObservation. Correct strings.
2019-11-01 11:48:07 +00:00
Dan Newton
ef01a99737 CORDA-3194 Failure during flow retry forces the flow into overnight observation (#2640)
When a flow fails to retry, it should be kept in for overnight observation and aborted.

In the future, it might be possible to retry flows again that failed during their retry, but for now keeping for observation and aborting is satisfactory.

* CORDA-3194 Remove hospitalised flows from `HospitalisingInterceptor`

Small refactor to remove some of the hospital logic out of the
`HospitalisingInterceptor` and into the `StaffedFlowHospital`.

Add some comments to help clarify the purpose of the two maps inside
of the hospital.

* CORDA-3194 When a flow fails to retry force it into observation

When a flow fails to retry, it should be kept in for overnight
observation and aborted.

In the future, it might be possible to retry flows again that failed
during their retry, but for now keeping for observation and aborting is
satisfactory.

* CORDA-3194 Test for database commit failure when retrying a flow

Failing during the database commit failure that occurs after the retry
flow action does not stop the flow from actually retrying. This test
confirms this functionality.

The retried flow gets scheduled as part of the retry action. The failure
in the commit action does not prevent this since it has already been
scheduled.
2019-11-01 11:48:07 +00:00
Dan Newton
268d129838 CORDA-3194 Replay start flow events when responding flow fails initial checkpoint commit (#2601)
* CORDA-3194 Replay start flow events when responding flow fails initial checkpoint commit

Logic has already been added to recover from initial checkpoint commit
failures on the initiating flow side but this did not suffice for
the same failure occurring on the responding flow's side.

The same idea has been added to resolve the responding flow's issue.

`ExternalMessageEvent` now has a `flowId` that is maintained on the
event. Messages can then be replayed to start/restart the flow, while
the event provides the flow id to each flow start.

Each `ExternalMessageEvent` implementation generates a random `flowId`
when constructed.

Events are stored in Artemis. This allows the solution to recover across
node restarts as the events will be pulled from artemis again when
restarting.

In the future `flowId`s will probably moved off of the events and
generated purely on the responding flow's node.

* CORDA-3194 Add test to verify that errors removing a responding flow are recoverable
2019-11-01 11:48:07 +00:00
Christian Sailer
1f71b071aa CORDA-3217 and CORDA-3195 Various bits arond SQL exceptions and flow hospital (#2605)
* Unwrap rx.OnErrorNotImplementedException so the hospital can handle the cause appropriately

* Add db failure cordapp

* Renamed folders to avoid ambiguity in gradle

* Add integration test for exception hospitalisation when thrown from an RX observable.

* Make the test slightly cleaner

* Fix the schema to actually match the requirements for my custom state. Thanks a bunch, H2.

* Switch test to use SqlException base class.

* Schedule error event if we detect that a commit or db flush has thrown (forcing the flow to error even if customer code then goes ahead to swallow the exception)

* Revert change to schedule extra error

* Add more tests for edge case with DB exceptions, changed CorDapp to suppor this an hook in the flow hospital

* Warning about unsubscribe
Check state transitioned from clean to error for hospital admission.

* Match the test to our actual expectations

* Revert "Revert change to schedule extra error"

This reverts commit 43d47937

* Prevent suppression of errors arising in `transaction()` and `jdbcConnection()`

* Test for SqlException caught trying to escape from recordTransaction and suppressed outside being intercepted.

* More tests for various error/catch combinations

* Clean up and comments

* Code reformat

* Fix test compilation
2019-11-01 11:48:07 +00:00
Dan Newton
9b169df2b8 CORDA-3194 Wrap state transition exceptions and add flow hospital error handling for them (#2542)
Wrap exceptions that occur in state machine transitions with a custom exception type which is
then handled inside of the flow hospital. As part of this change, a number of side negative side
effects have been addressed.

General summary:

- `StateTransitionException` wraps exceptions caught in `TransitionExecutorImpl`
- `StateTransitionExceptions` are handled in the flow hospital, retried 3 times and then kept in
    for observation if errors persist (assuming conditions below are false)
- Exceptions that occur in `FlowAsyncOperation` events are wrapped in
  `AsyncOperationTransitionException` and ignored by the flow hospital transition staff member
- `InterruptException`s are given a `TERMINAL` diagnosis by the flow hospital transition staff
   member (can occur due to `killFlow`)
- Allow flows which have not persisted their original checkpoint to still retry by replaying their
   start flow messages
- Swallow exceptions in `AcknowledgeMessages` actions

Detailed summary:

* CORDA-3194 Add state machine transition error handling to flow hospital

Wrap exceptions that are caught in `TransitionExecutorImpl` (coming from
new errors) with `StateTransitionException`. This exception is then
handled explicitly by the flow hospital.

Add `TransitionErrorGeneralPractitioner` to `StaffedFlowHospital`. This
staff member handles errors that mention `StateTransitionException`.
Errors are retried and then kept in the hospital if the errors persist.

* CORDA-3194 Remove a fiber from the `hospitalisedFlows` if its previous state was clean

If the fiber's previous state was clean then remove it from
`HospitalisingInterceptor.hospitalisedFlows`. This allows flows that are
being retried to clean themselves. Doing this allows them to re-enter
the flow hospital after executing the fiber's transition (if an error
occurs).

This is important for retrying a flow that has errored during a
transition.

* CORDA-3194 Set `isAnyCheckpointPersisted` to true when retrying a flow

Added to prevent a single flow from creating multiple checkpoints when
a failure occurs during `Action.AcknowledgeMessages`.

More specifically, to `isAnyCheckpointPersisted` is false when retrying
the flow, even though a checkpoint has actually been saved. Due to this
a brand new flow is started with a new flow id (causing duplication).

Setting `isAnyCheckpointPersisted` to true specifically when retrying a
flow resolves this issue.

* CORDA-3194 Add Byteman test to verify transition error handling

Add `StatemachineErrorHandlingTest` to verify transition error handling.

Byteman allows exceptions to be injected at certain points in the code's
execution. Therefore exceptions can be thrown when needed inside of the
state machine.

The current tests check errors in events:
- `InitiateFlow`
- `AcknowledgeMessages`

* CORDA-3194 Swallow all exceptions in `ActionExecutorImpl.executeAcknowledgeMessages`

Swallow the exceptions that occur in the `DeduplicationHandler`s when
inside of `ActionExecutorImpl.executeAcknowledgeMessages`.

The side effects of the failures that can happen in the handlers are
not serious enough to put the transition into a failure state.
Therefore they are now caught. This allows the transition to continue
as normal, even if an error occurs in one any of the handlers.

* CORDA-3194 Wrap unexpected exceptions thrown in async operation transitions

Exceptions thrown inside of `FlowAsyncOperation.execute` implementations
that are not returned as part of the future, are caught, wrapped and
rethrown. This prevents unexpected exceptions thrown by (most likely)
user code from being handled by the hospital by the transition
staff member.

This handling might change moving forward, but it allows the async
operation to continue working as it was before transition error handling
was added.

* CORDA-3194 Verify that errors inside of `AcknowledgeMessages` work as expected

Update `StatemachineErrorHandlingTest` to correctly test errors that
occur when executing the `AcknowledgeMessages` action.

* CORDA-3194 Retry flows that failed to persist their original checkpoint

Allow a flow that failed when creating their original checkpoint (for
example - failing to commit the db transaction) to retry.

The flow will create a brand new checkpoint (as the original did not
saved).

This required adding `flowId` to `ExternalStartFlowEvent` to allow the
event to keep a record of the flow's id. When the flow is retried, the
events are replayed which trigger a flow to be started that has the
id stored in the event.

To allow this change, code was removed from `retryFlowFromSafePoint` to
allow the function to continue, even if no checkpoint matches the passed
in flow id.

* CORDA-3194 Correct `FlowFrameworkTests` test due to error handling

Test assumed that errors in transitions are not retried, this has now
been updated so the test passes with the flow succeeding after an
exception is thrown.

* CORDA-3194 Remove unneeded import

* CORDA-3194 Make the state transition exceptions extend `CordaException`

`StateTransitionException` and `AsyncOperationTransitionException` now
extend `CordaException` instead of `Exception`.

* CORDA-3194 Improve log messages

* CORDA-3194 Remove unneeded code in `HospitalisingInterceptor`

Due to a previous change, a section of code that removes a flow id
from the `hospitalisedFlows` map is no longer required. This code has
been removed.

* CORDA-3194 Constraint violations are given `TERMINAL` diagnosis

Add `Diagnosis.TERMINAL` to `StaffedFlowHospital` to allow an error
to be ignored and left to die a quick and painful death.

`StateTransitionException` changed so it does not cause serialisation
errors when propagated from a flow.

* CORDA-3194 `InterruptedExceptions` are given `TERMINAL` diagnosis
2019-11-01 11:48:07 +00:00
Christian Sailer
938828b52f CORDA-3195 Default behaviour of FlowHospital (WIP) (#2520)
* Add GP to flow hospital, and start working on a list of things the GP knows to be incurable.

* Only hospitalise SQL and Persistence Exceptions (let's see if that is enough?), also rename to DatabaseDentist.

* Disabled hospitalisation of SQL exceptions in flow retry tests

* Fix RPC exception handling test by not using PersistenceException

* Ignore flaky integration test

* Code review: Rename staff member and add testing annotation

* Revert compiler.xml
2019-11-01 11:48:07 +00:00
Andrius Dagys
d033fceeef CORDA-3365: Reintroduce dependency to fix BFT-Smart notary (#5640)
The commons-codec:1.10 library was removed due to a security vulnerability,
but in commons-codec:1.13 it appears to have been fixed.
2019-11-01 09:05:12 +00:00
Dimos Raptis
79c22bd77a Constrain the cordapps loaded from driver tests to the minimum required (#5653) 2019-10-31 11:26:05 +00:00
Stefan Iliev
03ab258fc2 Revert "CORDA-3307 - add support for environment variables in linux (#5523)" (#5643)
This reverts commit c882b221a5.
2019-10-29 17:55:58 +00:00
stefano
8fd3e867f3 Merge remote-tracking branch 'origin/release/os/4.3' into my_merge_branch 2019-10-29 16:24:08 +00:00
Nick Dunstone
4c7fbbc31b SecureRandomHash for internal shell password (#5638) 2019-10-28 15:56:11 +00:00
Ryan Fowler
5da114caa3 CORDA-3281: Drop some errors to warnings and clean up logic around (#5605)
shell "gracefulShutdown" command.
2019-10-22 11:02:04 +01:00
Dimos Raptis
61cdfa5b26 [CORDA-3341] - Add missing liquibase script for hibernate test (#5623) 2019-10-22 09:52:37 +01:00
Anthony Keenan
0a194cfb51 Merge branch 'release/os/4.3' into release/os/4.4
# Conflicts:
#	docs/source/docker-image.rst
2019-10-21 13:44:32 +01:00
Stefan Iliev
c882b221a5 CORDA-3307 - add support for environment variables in linux (#5523)
* Added a new way for environment variables to be loaded, which allows for underscore based separation.

* Moved test to its own kotlin file.

* Added case insensitivity support.

* The corda. prefix is now case insensitive too.

* Removed unused variable.

* Added env variables support for driverDSL. Shadowing corda. properties raises an exception.

* Driver api stability fix.

* Changed type of cordapps param to reflect the real one, rather than what IntelliJ auto completed.

* Some detekt issue fixes. Spread operator removed, baselined api stability constructors and buggy line.

* Fixed misspelled variable.

* Reverted unintentional changes.

* Added suppress instead of changing baseline.

* Reworked logic to handle previously defined CORDA_ starting properties and handle accordingly. Fixed a bug where wrong class was used for reflection walking.

* Fix for detekt issues.

* Changed message to a more understandable one.

* Changelog + doc note, console error grammar.

* Changes according to PR review.
2019-10-21 12:01:14 +01:00
Dimos Raptis
f37638c93d [CORDA-3122] - Cleanup non-finalised, errored flows (#5594)
* [CORDA-3122] - Cleanup non-finalised, errored flows

* detekt
2019-10-16 09:37:28 +01:00
Stefano Franz
6e98adb085
use zulu for jdk in testing image (#5583)
* use zulu for jdk
add some parallel groups

* port kubesTest to Java
remove asterix from tests listed by ListTests, instead add after allocation
2019-10-14 13:35:22 +01:00
Stefano Franz
970f60c625 NOTICK: Fix Initiate Flow with Anonymous party (#5579)
* delete buildSrc block configuring multiple plugins

* remove outer stage block

* fix issues around initiateFlow with anonymous party

* code checks

* disable unit tests

* fix flowframeworktest

* undo some extraneous changes
2019-10-14 11:32:22 +01:00
RogerWillis
c261a8447a Merge branch 'release/os/4.3' into rog-os-merge 2019-10-14 10:37:21 +01:00
Stefan Iliev
6de6702cb4 CORDA-3244, CORDA-3245 - Jolokia docs update (#5486)
* Fixed wrong command line. Added security policy how to.

* Elaborated on security policy to mention when it applies and when not.

* Changes according to PR review.

* Magic to get jolokia version from a single place and forwarded to the docs.
2019-10-14 10:08:01 +01:00
Roger Willis
8a5b6ed52b
Fix for CORDA-3315 (#5580)
* Fix for CORDA-3315. Removed default implementation of partyFromKey and replaced with implementations in IdentityService sub-types.

* Added test.

* Added missing DB transaction to append only persistent map lookup.
2019-10-14 10:01:03 +01:00
Nick Dunstone
7a929f177c CORDA-3043 RPC Invocation fails when calling classes with defaulted constructors O/S (#5569)
* Fix for defaulted constructor arguments

* Code changes from 4.3 applied to O/S

* Detekt and code review
2019-10-11 11:00:40 +01:00
Dimos Raptis
7666ca0d80 [CORDA-3303] - Avoid flushing when inside a cascade (#5575)
* [CORDA-3303] - Avoid flushing when inside a cascade

* Remove listener infrastructure
2019-10-11 10:58:26 +01:00
Stefan Iliev
48fd78d059 CORDA-3236 fix observables not being tagged with notUsed() (#5557)
* Fixed not utilising the observables being returned by stateMachines added response with a notUsed(). Opening a ticket for implementation investigation.

* stateMachinesFeed will unsubscribe on interrupt rather than remaining infinitely subscribed.

* Fixed reported detekt issues on the InteractiveShell.

* Changes according to PR review.
2019-10-10 14:24:32 +01:00
stefano
0488a5ac5d Merge remote-tracking branch 'origin/release/os/4.3' into merge_4.3_to_4.4
# Conflicts:
#	constants.properties
2019-10-08 15:57:14 +01:00
josecoll
efa01410ec CORDA-3263 Improve error handling for registering peer node (#5559)
* Capture and log "nodeInfo" persistence failures, whilst maintaining an optimistic retry mechanism.

* Additional test cases (update and insert)

* Handle both updates and inserts consistently (single transaction for happy path)

* Fix detekt violations and update baseline with false detection.

* Streamline the code a little.

* Update baseline reporting false violation.
2019-10-08 11:38:11 +01:00
josecoll
246ec8766e
CORDA-3224 JDK11 built and published artifacts to include classifier. (#5488)
* JDK11 built and published artifacts to include classifier.

* Added additional non-intuitive gradle dust sprinkled recommended by CR.

* Added additional non-intuitive gradle dust sprinkled recommended by CR.

* Bump corda gradle plugins version.

* Incorporating PR review feedback.

* jdk11Classifier
2019-10-08 10:45:10 +01:00
Dimos Raptis
ec4d20d987 [CORDA-3243] - Improve CorDapp loading logic for duplicates (#5551) 2019-10-07 16:20:55 +01:00
josecoll
42e104ca4f
Publish checkpoint agent jar and allow for inclusion of version id in jar upon run-time execution. (#5553) 2019-10-04 14:28:42 +01:00
Nick Dunstone
dfa6291b2f CORDA-3235: O/S version of fix for slow running in 4.3 (#5531)
* O/S version of fix for slow running in 4.3

* Removal of IdentityServiceInternal from test classes

* Code review comments

* O/S version of fix for slow running in 4.3

* Removal of IdentityServiceInternal from test classes

* Code review comments

* Re-baselined Detekt

* Fixed warning
2019-10-04 11:15:31 +01:00
Dimos Raptis
4f0c87a697 [CORDA-3274] - Enhance backwards compatibility logic to include Inter… (#5549)
* [CORDA-3274] - Enhance backwards compatibility logic to include InternalCordaRPCOps

* Add InternalCordaRPCOps in server-side backwards compatibility logic
2019-10-03 18:17:48 +01:00
josecoll
1a4aaf32f0
CORDA-2050 Prevent node startup failure upon cross-platform execution. (#5527)
* Prevent fat packaging of platform-specific JDK tools.jar (required at compile time for Java 8).

* Catch Throwable.

* Re-instate "tools.jar" in corda.jar as JRE's do not ship with this platform dependency.

* Baseline "TooGenericExceptionCaught".
2019-10-03 15:46:54 +01:00
Stefano Franz
0a4240d58d
Merge pull request #5540 from corda/merge/2019-10-30-os-43-to-44
Merge/2019 10 30 os 43 to 44
2019-10-01 16:42:35 +01:00
Stefano Franz
fa2720783a Revert "OS 4.3 to OS 4.4 merge on 2019/09/30 (#5534)"
This reverts commit 191fb83dec.
2019-10-01 16:37:49 +01:00
josecoll
1ac1c483ff
ENT-4240 Added general exception handler for Virtual Machine errors. (#5530)
* Added general exception handler for capturing and exiting on all types of Virtual Machine errors.

* Minimize additional processing (use constant string for error message).

* Remove redundant DB specific thread termination handler.

* Incorporating PR review feedback.

* Add println before halt.

* Add println before halt ... to stderr
2019-10-01 16:12:55 +01:00
szymonsztuka
191fb83dec OS 4.3 to OS 4.4 merge on 2019/09/30 (#5534)
* CORDA-3273: Replace deprecated use of Class.newInstance() for sake of DJVM. (#5522)

* ENT-4090: move startFlow into try block so exception is caught and managed (#5519)

* CORDA-3091: Move executor thread management into CordaRPCConnection (#5491)

* CORDA-3184 - Vault Query API enhancement: strict participants matching (#5524)

* Implementation of exact participant parsing for common query criteria.

* Added more unit tests + fixed a minor bug.

* Additional clean-up.

* Add documentation and changelog entry.

* Revert debug level SQL logging.

* Fix detekt errors.

* Fix docs formatting in code samples.

* Updating baseline (false positives being reported).

* Fix "Name shadowed: subRoot" code quality check.

* Fix API stability checker failures.

* Update baseline.

* TM-43 updating code style docs to reflect the addition of Detekt (#5521)

* CORDA-3226 Fix dba migration for PostgreSQL following changes in CORDA-3009 and ENT-4192 (#5529)

* Fix dta migration for PostgreSQL following changes for CORDA-3009 Invalid hash function used for PersistentIdentity in PersistentIdentityService
and ENT-4192 Identity service refactor for confidential-identities and accounts.

* Different table definition for PostgreSQL and other dbs in one changeset instead of running generic DDL and the specifically fix table in Postgres (in relation to CORDA-3009 Invalid hash function used for PersistentIdentity in PersistentIdentityService and ENT-4192 Identity service refactor for confidential-identities and accounts).
2019-10-01 13:22:42 +01:00
szymonsztuka
3874e0c178 CORDA-3226 Fix dba migration for PostgreSQL following changes in CORDA-3009 and ENT-4192 (#5529)
* Fix dta migration for PostgreSQL following changes for CORDA-3009 Invalid hash function used for PersistentIdentity in PersistentIdentityService
and ENT-4192 Identity service refactor for confidential-identities and accounts.

* Different table definition for PostgreSQL and other dbs in one changeset instead of running generic DDL and the specifically fix table in Postgres (in relation to CORDA-3009 Invalid hash function used for PersistentIdentity in PersistentIdentityService and ENT-4192 Identity service refactor for confidential-identities and accounts).
2019-09-30 14:20:12 +01:00
josecoll
7dd6ecf96e
CORDA-3184 - Vault Query API enhancement: strict participants matching (#5524)
* Implementation of exact participant parsing for common query criteria.

* Added more unit tests + fixed a minor bug.

* Additional clean-up.

* Add documentation and changelog entry.

* Revert debug level SQL logging.

* Fix detekt errors.

* Fix docs formatting in code samples.

* Updating baseline (false positives being reported).

* Fix "Name shadowed: subRoot" code quality check.

* Fix API stability checker failures.

* Update baseline.
2019-09-27 15:44:04 +01:00
Andrius Dagys
ef57e9909c
CORDA-3124: Ensure transaction is fully verified before sending to notary (#5516)
NotaryFlow.Client flow assumes the passed transaction is verified, which is
the case when it's invoked as a sub-flow in FinalityFlow. However,
NotaryFlow.Client can also be called directly from any custom flow, so
we need to ensure it performs transaction verification to avoid accidentally
sending an invalid transaction to a non-validating notary.
2019-09-27 13:09:36 +01:00
Ryan Fowler
6ca4b589e2 CORDA-3091: Move executor thread management into CordaRPCConnection (#5491) 2019-09-26 17:48:07 +01:00
Viktor Kolomeyko
51330c2e44 CORDA-3232: Support of multiple interfaces for RPC calls (#5495)
* CORDA-3232: Make backward compatible RPC client changes

Such that it will be able to talk to new and old server versions.

* CORDA-3232: Make backward compatible RPC server changes

Such that it will be able to talk to new and old client versions.

* CORDA-3232: Trick Detekt

* CORDA-3232: Integration test for multi-interface communication.

* CORDA-3232: Add legacy mode test.

* CORDA-3232: Making Detekt happier

* CORDA-3232: Fix Detekt baseline after merge with `4.3` branch

* CORDA-3232: Incrementing Platform version

As discussed with @lockathan

* CORDA-3232: Fix legacy test post platform version increment

* CORDA-3232: Use recursive logic to establish complete population of method names

* Revert "CORDA-3232: Incrementing Platform version"

This reverts commit d75f48aa

* CORDA-3232: Remove logic that conditions on PLATFORM_VERSION

* CORDA-3232: Making Detekt happier

* CORDA-3232: Few more changes after conversation with @mnesbit

* CORDA-3232: Make a strict match to `CordaRPCOps` on client side

Or else will fail:
net.corda.tools.shell.InteractiveShellIntegrationTest.dumpCheckpoints creates zip with json file for suspended flow

Flagging that `InternalCordaRPCOps.dumpCheckpoints` cannot be called.

* CORDA-3232: Address PR comments by @rick-r3

* CORDA-3232: Address further review input from @rick-r3

* Change the way how methods stored in the map;
* Extend test to make sure that `CordaRPCOps` can indeed be mixed with other RPC interfaces.
2019-09-26 16:01:14 +01:00
Jonathan Locke
f945e57c17
OS 4.1 -> OS 4.3 merge
OS 4.1 -> OS 4.3 merge
2019-09-26 09:41:58 +01:00
Ed Prosser
defa2565ea Merge branch 'release/os/4.1' of https://github.com/corda/corda into ENT-4104-4.3-EdP 2019-09-25 10:47:06 +01:00
bpaunescu
4672c3a13c
CORDA-3240: optional node.conf property not recognized when overridden (#5505)
* CORDA-3240: optional node.conf property not recognized when overridden

* CORDA-3240: address detekt issues

* [NOTICK]: undo incorrectly commmented out include
2019-09-25 10:34:26 +01:00
josecoll
f4f46af706
CORDA-3209 Fix vault query for participants specified in common criteria (#5508)
* Generalise participant parsing code & additional test cases.

* Use a common predicate to expand the participants query (when specified more than once - eg. in fungible and linear query criteria).

* Introduce some re-usable functions.

* Additional code clean-up and improvements.

* Fix detekt MaxLineLength errors.
2019-09-24 10:32:06 +01:00
Roger Willis
fc45fac84c
These changes should fix the ENT database integration tests. (#5500) 2019-09-20 16:22:08 +01:00
Stefan Iliev
c5dc905705 [CORDA-2368] Added exception handling for missing files that displays appropriate messages rather than defaulting to file names. (#5472) 2019-09-20 09:39:57 +01:00
opticyclic
eaed69f26f Check If Quasar Is Active Using API (#5471)
* Do a proper check for whether the quasar agent is active instead of sniffing JVM args

* Use JavaAgent.isActive instead of SuspendableHelper.isJavaAgentActive for clarity
2019-09-18 17:15:56 +01:00
josecoll
3fafbe551c
CORDA-2050 Upgrade Corda to Java 11 (compatibility mode) (#5356)
Upgrade Corda to run with Java 11 (compatibility mode) - see https://github.com/corda/corda/pull/5356
2019-09-18 10:26:26 +01:00
Ryan Fowler
75e66f9db9 CORDA-3141: Add GracefulReconnect callbacks which allow logic to be performed when RPC disconnects unexpectedly (#5430)
Also removed potential for growing stack trace on reconnects.
2019-09-17 10:00:27 +01:00
Roger Willis
1c2c3d3fed
Identity service refactor for confidential-identities and accounts (#5434)
* Removed IdentityServiceInternal as it is no longer used.

* Removed externalIdForPublicKey API from KMS and added it to IdentityService.
Added a registerKeyToExternalId API on IdentityService.

* Fix remaining compile errors.

* Removed "registerKeyToParty" and in its place added a new registerKey method which takes a PublicKey, Party and optionally a UUID.
Added a cache to the "PersistentIdentityService" to store other node's public keys.
Added the cache and new hibernate entity to all teh places where one needs to add them.
New keys created by teh node now automatically get associated entries in the KEY -> PARTY map and optionally the KEy -> EXT ID map.
Added a test.

* Removed old comments and TODOs.

* Fixed broken test.
Added comments/explanations for what's going on in IdentityService.
Updated kdocs.

* First try at Implementing publicKeysForExternalId.

* Fixed broken test.

* Added migration.
Amended existing persistent identity service migration to handle new migration.
Addressed some review comments.

* Fixed broken test - whoops!

* Implemented mock identity service methods.

* Added back exception when remapping a key to a different party.

* Fixed compile errors.
Fixed broken tests.

* Use set instead of first entry in ourNames.
2019-09-16 13:14:12 +01:00
Shams Asari
39adfbb9ea
CORDA-1836: Checkpoints which cannot be deserialised no longer prevent the node from starting up (#5367)
Instead they are logged with the flow ID. The kill flow RPC has been updated to delete such checkpoints.
2019-09-13 17:27:20 +01:00
Andrius Dagys
c7cd947c32 CORDA-3060: Notary logging improvements
Add more detailed logging to the notary service flow.
2019-09-13 12:13:55 +03:00
Jonathan Locke
16e4650d8d CORDA-3060: Improve Notary logging from an operator/admins point of view
Added logging during the processing steps of the PersistentUniqunessProvider and the RaftUniquenessProvider
Bumped up logging level of existing logging statements that occur while processing from debug to info
Added mention of txId to logging statements to enable a request to be traced through from the time it is added to the queue to the time that it is committed.
2019-09-13 12:13:55 +03:00
willhr3
79c675fb41 CORDA-3200 Fix postgres oid/ bytea column issue (#5465)
* CORDA-3200 Add postgres specific migration to change from @Lob to @Type(type = corda-blob)

* CORDA-3200 Remove unwanted changes

* CORDA-3200 Change column length to 64000 for long cert chains
2019-09-12 11:23:28 +00:00
Stefan Iliev
81a6a9ecba [CORDA-3079] Load drivers directory automatically (#5454) 2019-09-12 10:29:38 +01:00
Zoltan Kiss
f171de7b69 TM-23 Fail build on compiler warnings (#5453)
* java compile respects compilation.allWarningsAsErrors

* suppress or cleanup warnings

* suppress warning

* use non-deprecated kotlin dependency

* rename property

* handle property existence check

* Deal with warnings
2019-09-11 16:34:51 +01:00
Matthew Layton
4e6edd012a CORDA-3133 (Version 2) (#5428)
* CORDA-3133 [v1]
This proposal is a little more flexible by design. It adds a property to the state pointer class, allowing them to be referenced in transactions on a per-state-pointer basis.

* CORDA-3133
 - Updated `resolveAtTransaction` to `isResolved`.
 - Moved `isResolved` out of the constructor into an abstract property.
 - Added deprecation constructor declaration for backwards compatibility.

* CORDA-3133 (version 2)
 - Added required changes to state pointers as per PR comments.
 - Added unit tests to ensure isResolved can be used to configure when state pointers should be resolved to reference inputs.

* CORDA-3133 (version 2)
 - fixed unit tests.
 - added comment to changelog.rst.
 - added helper functions to `StatePointer` to allow easier creation of static and linear pointers.
2019-09-11 08:51:27 +01:00
willhr3
aad2e82d07 CORDA-3200 Split migrations as per https://github.com/corda/enterpris… (#5456)
* CORDA-3200 Split migrations as per https://github.com/corda/enterprise/pull/2514/files

* CORDA-3200 Additional comment to explain the ordering of migration scripts
2019-09-10 15:52:22 +01:00
Tudor Malene
a88c519096 CORDA-2740 - Remove RPC exception obfuscation (#5455) 2019-09-10 15:06:15 +01:00
Dan Newton
4cbe22949d CORDA-3018 Whitelisting attachments by public key - phase two tooling (#5386)
Allow node operators to blacklist signing keys (using blacklistedAttachmentSigningKeys config option). These blacklisted keys prevent attachments that are received over the network from being trusted. The docs have been updated to detail how to generate the key hashes that the config requires.

A new shell command attachments trustRoots has been added to see what attachments exist on the node along with information about their trust and where it comes from.

run dumpCheckpoints has been replaced by checkpoints dump as InternalCordaRPCOps needed to change to prevent a function that is meant to be internal from being visible on the shell.
2019-09-10 12:16:34 +01:00
willhr3
4fb1787f1e CORDA-3200 Use PersistentIdentityMigrationBuilder instead of schema a… (#5449)
* CORDA-3200 Use PersistentIdentityMigrationBuilder instead of schema and correctly name table using node prefix

* CORDA-3200 Remove hacky test migration from PersistentIdentityMigrationNewTable into a unit test to fix postgres failure
2019-09-10 10:51:38 +01:00
tudor.malene@gmail.com
ea73dcfb22 Merge branch 'release/os/4.1' into tudor_merge_4.1_to_4.3
# Conflicts:
#	node/src/test/kotlin/net/corda/node/services/vault/NodeVaultServiceTest.kt
2019-09-09 16:04:39 +03:00
Denis Rekalov
a4650b2bb3 CORDA-3187: Add -XX:+HeapDumpOnOutOfMemoryError -XX:+CrashOnOutOfMemoryError to default JVM args for node (#5432) 2019-09-06 13:49:52 +01:00
Nick Dunstone
9d6922d11e CORDA-3177 Additional Back Chain Resolution performance enhancements (#5415)
* Applied backchain fetch optimisations

* Removed test for bulk fetch

* Removed import

* Removed extraneous import

* Review comment
2019-09-05 14:27:42 +01:00
Dimos Raptis
1ac7715e28 [CORDA-2890] - Close security manager after broker is shut down (#5425) 2019-09-04 14:06:46 +01:00
Stefano Franz
a842740c9e
WIP Kubenetes parallel build (#5396)
* Split integration tests

* add simple example of printing all methods annotated with @Test

* add docker plugin to root project
remove docker plugin from child projects
add Dockerfile for image to use when testing
add task to build testing image to root project

* add comment describing proposed testing workflow

* simple attempt at running tests in docker container

* add my first k8s interaction script

* add fabric8 as dependnency to buildSrc

* before adding classpath

* collect reports from containers and run through testReports

* re-enable kubes backed testing

* for each project
1. add a list tests task
2. use this list tests task to modify the included tests
3. add a parallel version of the test task

* tweak logic for downloading test report XML files

* use output of parallel testing tasks in report tasks to determine build resultCode

* prepare for jenkins test

* prepare for jenkins test

* make docker reg password system property

* add logging to print out docker reg creds

* enable docker build

* fix gradle build file

* gather xml files into root project

* change log level for gradle modification

* stop printing gradle docker push passwd

* tidy up report generation

* fix compilation errors

* split signature constraints test into two

* change Sig constraint tests type hierarchy

* tidy up build.gradle

* try method based test includes

* add unit test for test listing

* fix  bug with test slicing

* stop filtering ignored tests to make the numbers match existing runs

* change log level to ensure print out

* move all plugin logic to buildSrc files

* tidy up test modification
add comments to explain what DistributedTesting plugin does

* move new plugins into properly named packages

* tidy up runConfigs

* fix compile errors due to merge with slow-integration-test work

* add system parameter to enable / disable build modification

* add -Dkubenetise to build command

* address review comments

* type safe declaration of parameters in KubesTest
2019-09-03 15:40:08 +00:00
Roger Willis
99f4e4aac2
CORDA-3182: Added additional property on VaultQueryCriteria for querying by account (#5423)
* Added an additional property on VaultQueryCriteria to specify an externalId/accountId when performing a vault query.
Added logic in hibernate query criteria parser to handle the exernalId join and lookup.
Added a test.

* Fixed error in test.
Fixed backwards incompatible changes.

* Updated changelog.

* Updated docs to remove incorrect instructions for querying by external ID.
2019-09-03 11:55:26 +01:00
Stefano Franz
c2057e0893 CORDA-3180: Added ability to lookup the associated UUID for a public key to KeyManagementService (#5411)
* expose identity cache to KMSinternal

* apply shams comments

* Addressed review comments.
2019-09-01 07:57:46 +01:00
Stefano Franz
cd0d5c7724 Ensure that ServiceHub.WithEntityManager has a database transaction available (#5413)
* tidy up withEntityManager

* address rick review comment

* api break
2019-08-30 15:58:50 +01:00
Roger Willis
6202165957
Fix for CORDA-3178. FilterMyKeys now uses the key store as opposed to the cert store. (#5418) 2019-08-30 10:55:31 +01:00
willhr3
07b96aea18 CORDA-2925 Rebase identity service changes onto 4.3 (#5407)
* CORDA-2925 Rebase identity service changes onto 4.3

* CORDA-2925 Move migration to after v13

* CORDA-2925 Update schema list

* Change corda-version
2019-08-29 11:57:07 +01:00
josecoll
f09bff9c0f
Make concurrent updates to contractStateTypeMappings thread safe. (#5410) 2019-08-29 09:55:21 +01:00
Dimos Raptis
b97062bacc [CORDA-3098] - Close previous connection after reconnection (#5339) 2019-08-28 16:44:57 +01:00
Jonathan Locke
20585266a5
[NOTICK]: Change Corda version
Change Corda version
2019-08-28 15:38:36 +01:00
Stefan Iliev
3a6787437d CORDA-3022 Add wildcard RPC permissions (#5174)
* Added small poc changes to accommodate requested permissions.

* Added node user that can only start flows and has the neccessary permissions.

* Fixed type in the rpc unit tests.

* Finaliase wildcard RPC permissions, remove builtin maintainer/nodeuser roles

* Tidy up
2019-08-28 15:29:06 +01:00
Stefano Franz
e35c0c1df7
CORDA-3009 - Migrate identity service to use to string short (#5217)
* migrate PersistentIdentityService to use key.toShortString()
update definition of PublicKeyToExternalId mapping to allow fast lookup by externalId/publicKey

* fix misspelled table name

* add test of migration script

* add design document for proposal to move IdentityService to using the correct PK.toStringShort() method for hashing a publickey

* add enterprise testing considerations to design

* address review comments

* fix compilation errors

* modify PublicKeyToOwningIdentityCache to use toStringShort() as it's lookup key

* address syzmon's code review comments
2019-08-28 14:16:18 +00:00
Jonathan Locke
5ede8b5761 [NOTICK]: Change Corda version
Removed unnecessary comment which will just get outdated.
2019-08-28 14:35:06 +01:00
Jonathan Locke
c525f77ae9 [NOTICK]: Change Corda version
Changed Corda version in places where it was missed.
2019-08-28 14:14:39 +01:00
Jonathan Locke
35d0170b0a Merge branch 'master' into jlocke/merge_4.1_20190827 2019-08-27 15:21:21 +01:00
Jonathan Locke
e5f314ffbf Merge branch 'release/4' into jlocke/merge_4.1_20190827
Merged release/4 (Corda OS version 4.1) into a branch created from master (Corda OS version 4.3)

Conflicts:
	build.gradle
	client/rpc/src/main/kotlin/net/corda/client/rpc/internal/ReconnectingCordaRPCOps.kt
	common/logging/build.gradle
	common/logging/src/main/kotlin/net/corda/common/logging/CordaVersion.kt
	constants.properties
	core-tests/src/test/kotlin/net/corda/coretests/utilities/ProgressTrackerTest.kt
	core/src/main/kotlin/net/corda/core/transactions/TransactionBuilder.kt
	docs/source/api-contract-constraints.rst
	docs/source/api-stability-guarantees.rst
	docs/source/app-upgrade-notes.rst
	docs/source/changelog.rst
	docs/source/clientrpc.rst
	docs/source/conf.py
	docs/source/corda-network/UAT.md
	docs/source/getting-set-up.rst
	docs/source/network-builder.rst
	docs/source/upgrading-cordapps.rst
	docs/source/versioning.rst
	node/src/integration-test/kotlin/net/corda/node/services/rpc/RpcReconnectTests.kt
	serialization/src/main/kotlin/net/corda/serialization/internal/amqp/PropertyDescriptor.kt
	serialization/src/main/kotlin/net/corda/serialization/internal/model/LocalTypeInformationBuilder.kt
	tools/network-builder/src/main/resources/node-Dockerfile
	tools/network-builder/src/main/resources/notary-Dockerfile
2019-08-27 14:51:22 +01:00
Shams Asari
90284a6143
CORDA-2919: JacksonSupport, for CordaSerializable classes, improved to only uses those properties that are part of Corda serialisation (#5397) 2019-08-27 13:06:28 +01:00
James Higgs
d1b9e95cd5 [ENT-4024] Use string for the status column in the transaction table (#5379) 2019-08-21 10:50:08 +01:00
James Higgs
101d978050 [CORDA-3149] Update cache to check node identity keys in identity table (#5371) 2019-08-19 20:03:51 +01:00
Shams Asari
69d1b4e4e5
CORDA-3138: Removed InMemoryTransactionsResolver as it's not needed and other resolution cleanup (#5370) 2019-08-16 12:34:25 +01:00
Dan Newton
a477d59c15 CORDA-3094 improvements to checkpoint dumper (#5324)
- Handle errors in Jackson and checkpoint deserialisation. A file
  notifying the user that the checkpoint dump failed is created when
  errors occur.
- Handle message deserialisation errors. A string placeholder is used
  if an error occurs.
- Add more information about subflows (include their `FlowLogic`)
- Increase clarity in checkpoint output field names

* CORDA-3904 Add `flowCallStackSummary`

Add `flowCallStackSummary` to the output which contains the same content
as `flowCallStack` minus each subflow's `FlowLogic`. The `FlowLogic`
contains a ton of info which is normally repeated in each subflow.
Adding the summary gives an overview of the steps the flow executed
and which step it is currently on.

The `suspendedOn` field is put underneath the summary and the original
call stack is moved below the suspended info.

This puts the most useful information towards the top of the json file.
2019-08-14 14:08:58 +01:00
James Higgs
72ac722451 [CORDA-3130] Add a cache for looking up external UUIDs from public keys (#5357) 2019-08-14 13:24:56 +01:00
Ryan Fowler
98329444f5 Revert "ENT-3658, ENT-3660: Add timeouts and hospital flow handling to CryptoService (#5364)
This reverts commit 6df142bf

* Clean some code to better merge between versions
2019-08-13 11:42:03 +01:00
Dan Newton
fc265ee472 CORDA-3018 Whitelisting attachments by public key - relax signer restrictions (#5358)
* CORDA-3018 Allow loading an untrusted contract jar if another attachment exists that was signed with the same keys and uploaded by a trusted uploader

`TransactionUtils.isAttachmentTrusted` requirements have been relaxed
to allow an untrusted attachment to be loaded as long as another
attachment exists that is signed by the same keys and was uploaded
by a trusted uploader.

The requirement of containing the same contract classes has been
removed. Therefore the contents of the existing trusted attachment
no longer matters.

* CORDA-3018 Allow a subset/intersection of signers in `isAttachmentTrusted`

Allow a subset/intersection of signers to satisfy the signer
requirements of `isAttachmentTrusted`. This allows an "untrusted"
attachment that is signed by one or more keys to be "trusted" as long
as another trusted attachment already exists that is signed by at least
one of the "untrusted" attachments signers.

A cache of trusted and untrusted public keys is now held (replacing the
previous cache of `List<PublicKey>`.

Tests have been added to `NodeAttachmentServiceTest` to confirm that
an attachment query using an `EQUAL` statement will actually return
attachments that are signed by any of the keys passed into the query.

Confirming this allowed an `EQUAL` query to satisfy the search that
had to be done as part of this change.

`MockAttachmentStorage`'s query criteria was updated to better match
the real `NodeAttachmentService` implementation.

* CORDA-3018 Update cache name and kdoc on `isAttachmentTrusted`

* CORDA-3018 Verify that chains of trust do not occur

* CORDA-3018 Switch keys around to improve chain of trust tests
2019-08-08 09:33:45 +01:00
James Higgs
44428b6048 ENT-3801 Backport to OS (#5355)
* [ENT-3801] Store transactions in the database during transaction resolution (#2305)

* ENT-3801: Store downloaded txns as part of the backchain resolution into the db rather than the checkpoint

It's very inefficient to store the downloaded backchain in the checkpoint as more of it downloaded. Instead, if a threshold is reached (which currently defaults at 0) then the backchain is stored in the transactions table as unverified. A new is_verified column has been added to track this. Initially testing on the OS codebase has been very promising but unfortunately this current code is not quite ready. I had to quickly port it to ENT as this is meant to be an ENT-only optimisation.

To that effect, there is a TransactionResolver abstraction with two implementations: an in-memory one which has the old behaviour, and which will be the behaviour for OS, and a db one.

DBTransactionStorage hasn't been fully updated and I had to comment out the optimistic path for now.

Most of these changes will need to be ported to OS to keep the merge conflicts in check, but obviously not DbTransactionsResolver and the "is_verified" changes in DBTransactionStorage. DBTransactionStorage does have other refactoring which will make sense to port though.

* [ENT-3801] Start work on allowing modifications in AppendOnlyPersistentMap

* [ENT-3801] Add transaction resolver tests

* [ENT-3801] Adjust suspendable annotations

* [ENT-3801] Fix the ResolveTransactionFlow tests

* [ENT-3801] Update ResolveTransactionsFlow tests

* [ENT-3801] Add a liquibase migration script for isVerified

* [ENT-3801] Ensure the migration runs in the correct place

* [ENT-3801] Handle resolution of already present transactions

* [ENT-3801] Fix compile error in performance test app

* [ENT-3801] Logging and comment updates, plus a test case

* [ENT-3801] Add a notary change resolution test

* [ENT-3801] Add a contract upgrade transaction test

* [ENT-3801] Change new column to be a character based status

* [ENT-3801] Migration script type change

* [ENT-3801] Address first round of review comments

* [ENT-3801] Update variable names in AppendOnlyPersistentMap

* [ENT-3801] Another variable name clarification

* [ENT-3801] Fix missing name changes

* [ENT-3801] Make the signature list immutable when constructing cache value

* [ENT-3801] Add a locking strategy for unverified transactions

* [ENT-3801] Address tidying up review comments

* [ENT-3801] First attempt at ensuring locks are released after commit

* [ENT-3801] Remove references to old cache name

* [ENT-3801] Update locking logic

* [ENT-3801] Fix potential deadlock with read/write transaction locks

* [ENT-3801] Remove read locks, and ensure minimal extra suspends

* [ENT-3801] Fix build issues in tests

* [ENT-3801] Use the correct clock when calculating sleep durations

* [ENT-3801] Add a pessimism flag for writing verified transactions

* [ENT-3801] Change logging statement to debug

(cherry picked from commit 8ab6a55e17)

* [NOTICK] Fix up imports for some changed files

* [NOTICK] Fix transaction resolution tests

* [NOTICK] Reinstate the DBTransactionsResolver

* [NOTICK] Add the topological sort back to recordTransactions

* [NOTICK] Adjust test case to remove dependency on query ordering

* [NOTICK] Make test code match that in ENT
2019-08-07 16:49:49 +01:00
Viktor Kolomeyko
b60ab70440 CORDA-2617: Add failover listeners to terminate node process (#5337)
* CORDA-2617: Add failover listeners to terminate node process

This is a backport of changes done in Corda Enterprise.

It will be triggered in case of:
a) Loss of connectivity to in-built Artemis for Bridge Control;
b) Loss of connectivity to in-built Artemis for P2P connectivity.

Note on merge to CE: Disregard these changes and take whatever CE already has.

* CORDA-2617: Update documentation on stability of Corda Node

* CORDA-2617: Documentation update after discussion with @mnesbit
2019-08-06 11:28:16 +01:00
Stefano Franz
fa75711647 CORDA-3055 - Parallel node info download (#5097)
* parallelize download of nodeInfos

* actually call new list based addNodes method

* address review comments
fix NetworkMapUpdaterTest

* ensure threadpools are shutdown after network-map download is completed

* use NamedThreadFactory instead of re-implementing it.

* fix imports after rebase

* address review comments

* remove extra whitespace
2019-08-02 13:54:18 +01:00
Chris Rankin
813ff39eaa CORDA-3127: Move evaluationDependsOn() from core to core-tests. (#5349)
* CORDA-3127: Move evaluationDependsOn() from core to core-tests.

* Increase JVM memory for node unit tests to 3G.
2019-08-02 09:05:19 +01:00
josecoll
99e802c5c5 CORDA-3071 - Checkpoint agent tool (#5295)
* Initial commit based on experimental kryo hook agent.

* WIP

* Added documentation.

* Additional improvements and documentation following more testing.

* Added field level instrumentation + basic type handlers for String, byteArray, charArray, primitive types.

* Working version (without array type handling)

* Missing build.gradle file.

* Handle display of Arrays and String.
Pruning output to avoid repetition (by loop depth, object count).

* Added configurable StackDepth (for display purposes) and filter out ProgressTracker stacks.

* Further array handling (Object arrays, 2D, 3D), improved display and general code cleanup.

* Various fixes and improvements following demo to RP.

* Clean-up

* Further clean-up

* Set checkpoint id before deserialization.

* Update documentation

* Final clean-up.

* Minor documentation fixes.

* Updates following PR review feedback.

* Add changelog entry.
2019-08-01 09:40:52 +01:00
szymonsztuka
eff762a5c3 CORDA-3012 More information in log warning for Cordapps missing advised JAR manifest file entries. (#5325) 2019-07-31 15:30:00 +01:00
Dan Newton
8ac472b933 ENT-3504 Do not throw exception for missing fiber and log instead - OS version
Change to `SingleThreadedStateMachineManager`.

Instead of throwing an exception when a flow's fiber cannot be found,
just log the message at info level.

(cherry picked from commit 70b2a94fda)
2019-07-30 17:27:04 +01:00
LankyDan
9bf26c20e0 CORDA-2920 Hash to Signature Constraint automatic propagation
Allow Hash Constraints to propagate to Signature Constraints. When
propagating, the new jar is added to the transaction instead of the
original contract attachment (jar).

The following requirements must be met to do so:
- System property "net.corda.node.disableHashConstraints" must be set
  to `true`
- The min platform version must be 4 or greater
- There must be an input state with a hash constraint
- There must be no output states with a hash constraint
- The new jar must be signed

If these requirements are not met, the original Hash
Constraint will be kept and the contract attachment related to it is
also used.

This transition is done at the start of `handleContract` as it is not
the normal path a transition would follow. It is considered a backdoor
and should be treated separately from the rest of the attachment and
constraint logic. Furthermore, it will only work in private network
since all nodes must set the special `disableHashConstraints` flag.
2019-07-29 12:58:58 +02:00
Shams Asari
06e709f86b
CORDA-3092: Exception is logged if flow session message can't be deserialised (#5308) 2019-07-18 09:22:29 +01:00
josecoll
b4749eb8f9
CORDA-2991 (Cont): set node info polling interval to 1 second in DriverDSL Node Startup (#5285)
* Fix test execution flakiness on fast hardware: set node info polling interval to 1 second in DriverDSL Node Startup (was only being set to 1 sec in Node Registration). Follow-up to https://github.com/corda/corda/pull/5240

* Diagnostics: add additional logging to NodeInfoFilersCopier.

* Diagnostics: add additional logging to NodeInfoWatcher

* Downgrade logging level severity to debug.

* Make atomic counter unique across instances.
2019-07-17 14:48:46 +01:00
Dan Newton
70b2a94fda ENT-3504 Do not throw exception for missing fiber and log instead - OS version (#5292)
Change to `SingleThreadedStateMachineManager`.

Instead of throwing an exception when a flow's fiber cannot be found,
just log the message at info level.
2019-07-16 11:31:37 +01:00
Dimos Raptis
8962d930d4 [CORDA-2923] - Make the RPC client reconnect with gracefulReconnect param (#5244)
When set to true the RPC client will:

* automatically reconnect when the connection is broken
* simple RPC calls will block until connection is established
* Observables returned from RPC will automatically resubscribe on reconnect so the client continues to receive events. This doesn't guarantee that events will not be lost during the reconnect.
2019-07-16 11:29:21 +01:00
Viktor Kolomeyko
e96dcedfc6 CORDA-3068: Pass base directory when resolving relative paths (#5297) 2019-07-15 10:51:39 +01:00
Shams Asari
dfa3c67919 Removing reference to Enterprise in test (#5293) 2019-07-11 13:36:00 +01:00
Shams Asari
49cda57c81
AppendOnlyPersistentMapBase.allPersisted no longer loads everything into memory at once (#5286)
As a general purpose API, allPersisted should not be loading the entire contents of the database table into memory. Instead now it returns a Stream for processing of elements.
2019-07-10 18:09:02 +01:00
Dan Newton
9bcaecfbe3 Delete unused DuplicateContractClassException (#5284) 2019-07-10 17:01:38 +01:00
gwr3com
22e4b8e962 CORDA-3028: Add Node Diagnostics Info RPC Call - Backport a diff from… (#5287)
* CORDA-3028: Add Node Diagnostics Info RPC Call - Backport a diff from Corda ENT

* CORDA-3028: Add Node Diagnostics Info RPC Call - Rename a value for consistency

* CORDA-3028: Add Node Diagnostics Info RPC Call - One more change
2019-07-10 11:31:44 +01:00
Christian Sailer
687fa81894 ETO-39 Move core tests (#5255)
* Moved core.flows and core.contract tests

* move internal tests

* move node, schemas and serialization tests

* Move transactions tests

* Move test access helper into test package

* Move remaining tests, and create access helpers to test internals.

* Fix test dependencies.

* Delete some cruft that flagged up on the API checker

* Add readme explaing where tests need to go.

* fix typo
2019-07-09 10:09:21 +01:00
gwr3com
933330bc4c CORDA-3028: Add Node Diagnostics Info RPC Call (#5271)
* CORDA-3028: Add Node Diagnostics Info RPC Call - Initial commit

* CORDA-3028: Add Node Diagnostics Info RPC Call - Changes after code review

* CORDA-3028: Add Node Diagnostics Info RPC Call - Additional code review update and documentation

* CORDA-3028: Add Node Diagnostics Info RPC Call - Additional changes to documentation

* CORDA-3028: Add Node Diagnostics Info RPC Call - Another change to documentation

* CORDA-3028: Add Node Diagnostics Info RPC Call - More changes to documentation

* CORDA-3028: Add Node Diagnostics Info RPC Call - Comment change
2019-07-08 14:43:50 +01:00
Dan Newton
6d3a6a3998 ENT-3710 Add TransientConnectionCardiologist to Flow Hospital (#5277) 2019-07-08 11:07:50 +01:00
Rick Parker
306df8e219
CORDA-3053 Validation should pass with systemProperties defined in config (#5276)
* CORDA-3053 Validation should pass with systemProperties defined in config

* CORDA-3053 Fix failing test and add new one
2019-07-05 17:20:57 +01:00
James Higgs
075f68f179 [CORDA-2086] Allow transactions to be re-recorded using StatesToRecord.ALL_VISIBLE (#5184) 2019-07-04 14:00:42 +01:00
Dominic Fox
14a20654bd CORDA-2991 shorten poll intervals for node info file propagation (#5240)
* CORDA-2991 shorten poll intervals for node info file propagation

* Relax minimum poll time constraint

* Relax, not comment out, minimum poll time constraint

* Correct error message
2019-07-03 15:00:55 +01:00
fowlerrr
6df142bf7a ENT-3658, ENT-3660: Add timeouts and hospital flow handling to CryptoServices (#5226) 2019-07-03 12:39:32 +01:00
Manoj
44835bd17c CORDA-2914: Allow certificate directory to be a symlink (#5102) 2019-07-03 10:39:40 +01:00
Stefano Franz
88894bc592
add a shared memory port allocator to allow multiple processes to sha… (#5223)
* add a shared memory port allocator to allow multiple processes to share a single allocation pool

* remove dangerous reset function on port allocator

* set forkCount = 2 in node integration test

* only allow one build of a cordapp at any given time for Driver tests

* make all portallocation requests use same starting point

* globally set forks to 6

* tweak forking parameters to allow parallel builds

* tweak unit test parallelism

* 2 workers for integrationTest

* some more tweaks for parallel builds

* some more tweaks for parallel builds

* seems that 49K is not the start of ephemeral ports on all kernels

* tweak parallel settings

* try fix RPC shutdown test in parallel env

* add some logging for RPC shutdown test

* added some logging around PortAllocation tests - try figure out where they are getting stuck

* added some logging around PortAllocation tests - try figure out where they are getting stuck

* fix api-scanner tests

* minimize api changes

* revert to complying with existing API

* add the AtomicInteger for api compatibility reasons

* make sizing script executable

* address review comments pt1

* address review comments pt2

* fix compile errors after review comments

* return to using home dir as temp dir seemed to interact badly with gradle
2019-07-02 18:38:33 +00:00
Viktor Kolomeyko
0ad5c2176d CORDA-3021: Use CryptoService in Node's ConfigUtilities to minimise merge conflicts with ENT (#5264) 2019-06-28 09:53:41 +01:00
fowlerrr
ffe328c604 CORDA-2720: Refactor NodeConfiguration out of NodeRegistrationHelper (#5181) 2019-06-27 16:56:03 +01:00
Andrius Dagys
c1542f386a CORDA-2996: NotaryLoader - improve exception handling (#5210)
Ignore InvocationTargetException and only propagate the cause to avoid noise.
2019-06-27 13:33:38 +01:00
Viktor Kolomeyko
51f1e442a3 CORDA-3021: Introduce SignOnlyCryptoService and use it whenever possible (#5239)
* CORDA-3021: Introduce `SignOnlyCryptoService` and use it whenever possible

Also modify `CryptoServiceFactory` to show how sign only implementation can be created.

* CORDA-3021: Undo some of my earlier changes

Which after discussion with @dimosr and @fowlerrr proven to be contradictory.
2019-06-27 11:55:59 +01:00
Stefan Iliev
0083931eb7 NOTICK - renamed postgres to postgresql (#5220) 2019-06-26 18:52:45 +01:00
Shams Asari
2e02968c63
ENT-3496: dumpCheckpoints shell command (#5171)
Dumps all the node's checkpoints as JSON into a single zip file in the node's directory. The output contains:

* All the fields for the top-level flow
* The current sub-flow call stack, along with the current progress tracker step for each sub-flow
* The event that suspended the flow, which if it's a send or sendAndReceive will show the payload that was sent
* Low level information on the active sessions with other peers
2019-06-26 15:31:02 +01:00
Stefan Iliev
d1e7850dd1 CORDA-2176 - test that logging is not broken. (#5224)
* Initial commit with a test that can detect a logger that isn't initialised with by lazy that gets called during startup.

* Test improvement. Test fix for api stability.

* Added explanatory comment for the test.

* Changes according to PR review. Added empty file check to make test more robust.
2019-06-25 14:35:39 +01:00
Shams Asari
99bc614b7c Merge remote-tracking branch 'origin/feature/CID-878-non_party_flow_sessions' into shams-merge-cid-878 2019-06-21 16:46:41 +01:00
Shams Asari
76eec9aa8b
CORDA-3033: Introducing Destination interface for initiating flows with (#5242)
Party and AnonymousParty have been retrofitted to implement this interface and are currently the only supported types. A new FlowLogic.initiateFlow(Destination) method overload to easily support the addition of new destination types in future versions.
2019-06-21 16:39:56 +01:00
Christian Sailer
0cd57c81bc Test the scheduler picking up a persisted scheduled state without shutting down/restart the db. (#5236) 2019-06-21 15:16:10 +01:00
Viktor Kolomeyko
3ceb242ee1 CORDA-2935 - Align timeouts for CRL retrieval and TLS handshake (#5126) 2019-06-21 09:34:15 +01:00
James Brown
0101cdbd56 ENT-3542 remove unused commons-codec dep (#5207) 2019-06-21 09:30:40 +01:00
Shams Asari
4e835d5483
CORDA-2333: Updated the majority of the dependencies that were out of date (#5235)
Some where not touched, including the crypto ones, libraries related to messaging and Kotlin.
2019-06-20 16:19:16 +01:00
Shams Asari
843c3a0190
All uses of CheckpointStorage.getAllCheckpoints() close the stream after use (#5230) 2019-06-19 09:45:38 +01:00
Shams Asari
f0f05df9f4 Merge branch 'master' into feature/CID-878-non_party_flow_sessions
# Conflicts:
#	core/src/test/kotlin/net/corda/core/flows/CollectSignaturesFlowTests.kt
2019-06-18 13:14:55 +01:00
Stefano Franz
f9c034aa7c CORDA-3000: Allow AbstractParty to initiate flow (#5219) 2019-06-18 13:13:09 +01:00
LankyDan
e3db3a386e ENT-3496 Address pr comments 2019-06-17 16:43:05 +01:00
LankyDan
afc36615d0 ENT-3496 Address pr comments 2019-06-17 16:04:05 +01:00
Shams Asari
2152961d46
CORDA-2333: Dependency update pass for tests and demos (#5211) 2019-06-13 10:15:13 +01:00
LankyDan
245a14d499 ENT-3496 Add suspendedTimestamp and secondsSpentWaiting to checkpoint dump 2019-06-11 16:22:17 +01:00
Stefan Iliev
3b73e60605 CORDA-2774 Fix for liquibase changelog warnings. (#5199) 2019-06-11 15:33:05 +01:00
LankyDan
143499f6fd ENT-3496 Add the checkpointed flow's simple name to the json file name 2019-06-11 09:38:48 +01:00
LankyDan
330b5a1d3c ENT-3496 Fix compile error in ThreadContextAdjustingRpcOpsProxyTest 2019-06-11 09:06:44 +01:00
LankyDan
31a0c077b8 ENT-3496 Move dumpCheckpoints to the new InternalCordaRPCOps interface
To prevent making `dumpCheckpoints` part of the public API a new
interface, `InternalCordaRPCOps` has been created and the function
has been moved there. `InternalCordaRPCOps` inherits from
`CordaRPCOps`.

`CordaRPCOpsImpl` now implements `InternalCordaRPCOps`.

`RunShellCommand` and `StringToMethodCallParser` required additional
changes due to issues handling inherited functions. This has only been
raised now due to `InternalCordaRPCOps` inheriting from `CordaRPCOps`.

Many classes have had references to `CordaRPCOps` changed to
`InternalCordaRPCOps`.
2019-06-10 16:15:03 +01:00
LankyDan
d8395baf62 ENT-3496 Create log directory to place dumps if it does not already exist 2019-06-10 15:10:34 +01:00
bpaunescu
726798cce8 ENT-3642: move the crypto service builder method to node-api (#5198)
* ENT-3642: move the crypto service builder method to node-api

* ENT-3642: add arg for different crypto services

* ENT-3642: add arg for cryptoservice config

* ENT-3642: add null check for BCC requirement

* ENT-3642: remove double-bang operator

* ENT-3642: cryptoservice factory method moved to its own class, improve API

* ENT-3642: remove import
2019-06-07 09:10:42 +01:00
josecoll
603da3eeff
CORDA-2984 Re-instate CordaCaplet tests and move CordaCaplet code into :node:cap… (#5197)
* Re-instate CordaCaplet tests and move CordaCaplet code into :node:capsule module.

* Clean-up verbose IntelliJ refactoring.

* Updated buildCordaJAR task references to resolve CordaCaplet from :node:capsule project.

* Incorporating feedback from PR review.
2019-06-07 08:55:19 +01:00
LankyDan
684e9cdf58 ENT-3496 Store dump in logs directory and only one dump at a time
Store the flow dump zip in the logs directory.

Only allow one dump to be created at a time. This is protected by
using a `AtomicInteger` lock and checking for the zip's existence.
2019-06-06 15:35:15 +01:00
Rick Parker
0c0101948b
CORDA-2981 Disable slow consumers for RPC since it doesn't work. (#5194)
* CORDA-2981 Disable slow consumers for RPC since it doesn't work.
2019-06-05 11:47:38 +01:00
josecoll
8678bad88d
CORDA-2970 Revert usage of Gradle JUnit 5 Platform Runner (#5173)
* Revert usage of Gradle 5 useJUnitPlatform() which causes significant test execution performance degradation.

* Remove completely top-level Java CordaCaplet unit tests causing consistent TC failure on Azure VMs:

 java.security.NoSuchAlgorithmException: no such algorithm: NONEwithEdDSA for provider BC
2019-06-05 10:17:02 +01:00
szymonsztuka
937e467aac CORDA-2971 - added tests for initialiseSchema configuration option (#5153) 2019-06-04 11:37:32 +00:00
Roger Willis
1fc7405c7f
Fix for CORDA-2972. (#5185) 2019-06-04 11:21:56 +01:00
Tudor Malene
f8d8de7555 CORDA-2975 Remove version uniqueness check - fix tests 2019-06-03 14:41:10 +02:00
Tudor Malene
77ab33d5b0 CORDA-2975 Remove version uniqueness check 2019-06-03 14:41:10 +02:00