Commit Graph

2609 Commits

Author SHA1 Message Date
Denis Rekalov
bc96bea24a ENT-4628: Harmonize CryptoService and BCCryptoService between OS and ENT (#5822) 2019-12-18 16:54:39 +00:00
Denis Rekalov
8d5781db43 ENT-4628: Harmonize net.corda.nodeapi.internal.crypto between OS and ENT (#5820)
* ENT-4628: Harmonize net.corda.nodeapi.internal.crypto between OS and ENT

* ENT-4628: Fix detekt
2019-12-18 13:59:30 +00:00
Chris Rankin
545a463a5b NOTICK: Remove buildscript block from Node's build.gradle. (#5809) 2019-12-16 16:24:47 +00:00
Viktor Kolomeyko
91cdffd429 ENT-4382: Core module changes to support multi-RPC work (#5814)
* ENT-4382: Move `InvocationHandlerTemplate` into `core`

This is an internal helper which is general enough and does not have any Node specific code.

* ENT-4382: Make @CordaInternal applicable to classes

And apply it on `AttachmentTrustCalculator` which is `core/internal` interface anyway.
2019-12-16 16:03:48 +00:00
Christian Sailer
4a35b99283 CORDA-3479: Added timestamp to the node_transactions table. (#2660) (#5812)
* ENT-4237: Added timestamp to the node_transactions table.

* ENT-4237: Clock for timestamp now retrieved from ServiceHub. And now record verification time as well.

* ENT-4237: Fixed tests. Also enabled stream output in allParallelIntegrationTest.

* ENT-4237: Changed timestamp to a val.

* ENT-4237: Changed streamOutput to false for allParallelIntegrationTest

* ENT-4237: Unit tests added for new timestamp column. Also now passing a clock into DBTransactionStorage.

* ENT-4237: Added more unit tests to check timestamp

* ENT-4237: Fix test to actually change clock time when testing transaction time does not change.
2019-12-13 15:14:21 +00:00
Kyriakos Tharrouniatis
b0903efa50 CORDA-3202 Add a specific exception for flows to hospitalise themselves (#5767)
* Introducing a new type of exception and a new hospital staff member to pause flows by immediately hospitalising them.

* Renaming exception to "HospitalizeFlowException".

* Making HospitalizeFlowException an open class.

* Overloading constructors of HospitalizeFlowException to be available in Java.

* Using Throwable#mentionsThrowable.

* Moving HospitalizeFlowException in its own file.

* Update kdocs for HospitalizeFlowException and StaffedFlowHospital#SedationNurse.

* Added tests, testing various HospitalizeFlowException types thrown.

* Fix Detekt issues.

* Imports optimizing.

* Add safe casting.

* Update api-flows and node-flow-hospital docs.

* Minor code comment change.

* Add DOCSTART-DOCEND signs in HospitalizeFlowException for makeDocs. It is referenced by api-flows.rst.

* Minor change in note.

* Code formatting.

* Remove comment.

* Remove if statement that makes example worse.

* Remove redundant comment.

* Moving 'Internal Corda errors' at the bottom.

* Changing node-flow-hospital.rst as per review.

* Change HospitalizeFlowException description as per review.

* Adding an example for FlowException.

* Minor indentation fix.

* Update FlowException example label as per review.

* Correcting handling of custom exception.
2019-12-11 10:35:58 +00:00
Christian Sailer
14050826e9 ENT-4595 harmonize core and serialization (#5792)
* Harmonize serialization/core and deterministic counterparts

* Fix test for changed private alias key behaviour

* Detekt errors

* roll back project.xml
2019-12-09 14:17:48 +00:00
Viktor Kolomeyko
43205e1f1a CORDA-3471: Create CordaTransactionSupport and make it accessible through AppServiceHub (#5768)
* CORDA-3471: Create `CordaTransactionSupport` and use wherever possible instead of `CordaPersistence`

* CORDA-3471: Address comments by @mnesbit
- Relocate `CordaTransactionSupport` to `core`
- Create a lighter version of transaction - `VaultTransaction` that gives access to `session` object only.

* CORDA-3471: More changes after discussion with  @mnesbit
- Rename `VaultTransaction` into `SessionScope`.

* CORDA-3471: Revert changes to most of the files after conversation with @mnesbit and @rick-r3

* CORDA-3471: Introduce `CordaTransactionSupportImpl` and make it accessible via `AppServiceHub`.

* CORDA-3471: Minor change (comment).

* CORDA-3471: Address input from @mnesbit

* CORDA-3471: Address input from @rick-r3

* CORDA-3471: Make Detekt happier

* CORDA-3471: Add a new test that proves transactions can be started from client threads

As requested by @mnesbit

* CORDA-3471: Change log and documentation update.

As requested by @mnesbit
2019-12-04 17:18:40 +00:00
Chris Rankin
5a41ec9b82 CORDA-3464: Also scan attachment:// URLs for custom serializers. (#5769)
* CORDA-3464: Also scan attachment:// URLs for custom serializers.

* Only scan the given classloader - ignore this classloader's parents.

* Upgrade to ClassGraph 4.8.58 - for "robustness fixes".

* Register the attachment:// URL scheme using AttachmentsClassLoader.

* Add integration test for custom serializer in contract state.

* Rename Currancy -> Currantsy, just to make the point.
2019-12-03 15:50:21 +00:00
Nick Dunstone
e6f9b46584 https://r3-cev.atlassian.net/browse/ENT-3829 (#5763)
Open Source
2019-12-03 10:28:00 +00:00
Rick Parker
6e467f20b3
CORDA-3470 Fix flow async operations (#5780) 2019-12-02 14:40:07 +00:00
Christian Sailer
06f97cfed5 ENT-4610 Move tx signing to the Uniqueness provider (#5773)
* ENT-4610 Move tx signing to the Uniqueness provider

* Make detekt happy

* Remove unused imports

* Address review comment
2019-11-29 17:30:33 +00:00
Dan Newton
45d6d3ead4 CORDA-3356 Subflow ledger consistency tests + move statemachine tests to slow integration tests (#5722)
* CORDA-3356 Subflow ledger consistency tests + move statemachine tests to slow integration tests

Add tests for subflows that fail during transitions.

Split out `StatemachineErrorHandlingTest` into a series of smaller tests.

Move these tests into the `integration-test-slow` category so they are
not run against every PR.

* CORDA-3356 Fix detekt issue

* CORDA-3356 Tidy test names
2019-11-25 17:11:54 +00:00
Kyriakos Tharrouniatis
21a075b727 CORDA-3197 Fix flow has been waiting message (#5660)
* Added a timestamp property to Checkpoint getting a new Instant.now() value at every Checkpoint instantiation/ copy instantiation. FlowMonitor is now using this new property (Checkpoint#timestamp) and StateMachineState#isFlowResumed to determine which flows are actually suspended. It leaves out flows that are doing work in their FlowLogic#call method.

* Cleaner comment

* Broke FlowMonitor#logFlowsWaitingForParty into logFlowsWaitingForParty and waitingFlowsToDurations. This way waitingFlowsToDurations is modular and can be tested.

Made FlowMonitor constructor get StateMachineManager instead of the retrieveFlows lamda. This way FlowMonitor is more consistent as a service, and entire flow filtering process is now being done in FlowMonitor#waitingFlowsToDurations.

Removed "smm as? StateMachineManagerInternal" in AbstractNode#start as it made no sense.

Updated CheckpointDumper to mention the Checkpoint#timestamp when writing the checkpoint as json.

* Added tests for FlowMonitor service.

* Remove old comment

* 1. FLowMonitor#waitingFlowDurations now returns a Sequence to have an iteration less.
It used to be, one iteration from returning a Set from FLowMonitor#waitingFlowDurations plus one iteration from FlowMonitor#logFlowsWaitingForParty.

2. Code reformattings

* 1. Remove constructor keyword from FlowMonitor

2. Code reformattings

3. Update detekt baseline

* Resolve conflict in Detekt baseline
2019-11-25 16:49:35 +00:00
Ed Prosser
e692ec292a Merge branch 'release/os/4.3' of https://github.com/corda/corda into EdP/CORDA-3446-4.4 2019-11-20 10:36:38 +00:00
Stefan Iliev
926429647d CORDA-3307: Fix for underscore variables (#5682)
* Revert "Revert "CORDA-3307 - add support for environment variables in linux (#5523)" (#5643)"

This reverts commit 03ab258fc2.

* Env variables with underscore are now validated using schema validation and checking for unknown key errors.

* Resolving comments from PR review.

* Fix for deprecated import.

* Reworked logic according to PR review.

* Resolved bad string parsing problems where the json structure could be broken if some symbols were included in the key or value.
2019-11-19 17:51:52 +00:00
Chris Rankin
ce8ce2ba7c CORDA-2876: Upgrade to deterministic-rt 1.0-RC01. (#5738) 2019-11-19 15:02:35 +00:00
Denis Rekalov
a791e38319 CORDA-3443: Do not log warning when appending duplicated network parameters (#5739) 2019-11-19 15:02:07 +00:00
Stefano Franz
221576d94a
move - responding flow from suffix to prefix of test name (#5731) 2019-11-18 15:41:15 +00:00
Chris Rankin
c68f8161fc CORDA-2876: Upgrade DJVM to 1.0-RC03. 2019-11-16 16:51:24 +00:00
Oliver Knowles
4a8c73bf4a CORDA-2744 - Prevent ClassCastException if error during initial registration (#5712)
* Prevent class cast exception if error during initial registration

* Appeasing detekt
2019-11-14 16:13:08 +00:00
Jonathan Locke
c31cddaffc Fixed merge conflicts 2019-11-14 10:40:12 +00:00
Chris Rankin
987415dd83 CORDA-3407: Install external byte-code cache for sandboxed CorDapps. (#5699) 2019-11-14 09:09:57 +00:00
Jonathan Locke
86f1f9e371
CORDA-3393: Lazy load MessageDeduplicationHandler.flowId (#5679)
CORDA-3393: Lazy load `MessageDeduplicationHandler.flowId` (#5679)
2019-11-07 09:44:03 +00:00
Dan Newton
b15db200e3 NOTICK: Quick and dirty change to stop "Unable to start notaries." error message (#5686)
* Quick and dirty change to stop "Unable to start notaries." error message (#5686)

"Unable to start notaries. A required port might be bound already" is
returned whenever a startup error occurs while starting the notary nodes
in driver tests. This hides the real error.

This change prints the actual error to std_err and read from file
at a later point. This means the real error is not lost and will be
shown in failed builds.

* Suppress detekt warnings
2019-11-07 09:43:29 +00:00
stefano
5d6fc34f88 fix merge issue with exception thrown in RpcExceptionHandlingTest 2019-11-06 12:28:05 +00:00
LankyDan
91f35d5967 CORDA-3393 Lazy load MessageDeduplicationHandler.flowId
This is to potentially help with debugging in the future as the
`flowId` could become confusing for received messages where the `flowId`
has nothing to do with the current flow.
2019-11-06 11:26:25 +00:00
stefano
03d8009c7d fix temp folder rules 2019-11-05 17:56:06 +00:00
stefano
d8ed3492b9 fix merge conflicts due to changes in Signed test cordapp creation 2019-11-05 17:42:38 +00:00
stefano
6a3a9eded3 Merge remote-tracking branch 'origin/release/os/4.3' into my_merge_branch
# Conflicts:
#	core/src/main/kotlin/net/corda/core/internal/TransactionVerifierServiceInternal.kt
#	node/src/integration-test/kotlin/net/corda/node/services/rpc/RpcExceptionHandlingTest.kt
#	testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/DriverDSLImpl.kt
2019-11-05 17:19:18 +00:00
Chris Rankin
f226ddc4f2 CORDA-2876: Integrate the DJVM with the Corda Node. (#5633)
* * CORDA-2876: Migrate DJVM serialization modules into Corda.
* Pre-generate Corda classes for DJVM sandbox when node boots in production mode.
* Ensure that all DJVM test contract CorDapps are signed.
* Test examining attachments within DJVM sandbox.
* Test Contract.verify() using cryptographic verify function.
* Add test cases for more non-determinism in Contract.verify().
* Update node-driver to support testing nodes with DJVM support.
* Modify Node to allow alternative DJVM configurations for testing.
* Refactor DeterministicVerifierFactoryService for default use-case.
* Small whitespace and code-style refactors.
* Create and activate a DJVM execution profile for the Node.
* Revert making Verifier implement AutoCloseable.
* Allow the node to cache sandboxed Corda byte-code for reuse.
* Use updated Quasar agent that knows not to touch DJVM classloaders.
* Fix Quasar's package exclusions globs for DJVM.
* Deserialise LedgerTransaction into the sandbox for Contract.verify().
* Add the DJVM's serialisation modules to the Corda node.
* Update the node for the latest DJVM API, and preserve the ConstructorForDeserialization annotation on user contract classes.
* Add corda-dev to repositories while DJVM is SNAPSHOT.
* Migrate DJVM specialisation into AbstractNode's ServiceHubInternalImpl.
* Exclude sandbox.** and shaded djvm.** classes from Quasar agent.
* Add the corda-dev repository to :node for the deterministic runtime.
* Turn Verifier into an abstract base class that is specialised by BasicVerifier and DeterministicVerifier.
* Add the Corda deterministic libraries to the Node, and split the DJVM sandbox across two SandboxClassLoader instances.
* Add DJVM to contract verification path inside Corda Node.
* Minor lambda simplifications and removing unused import.
* CORDA-2871: Remove @CordaSerializable from LedgerTransaction.
* CORDA-2871: Add a callback to ServicesForResolution to allow the Node to modify a LedgerTransaction object.
* CORDA-2871: Refactor the contract verification code into a separate class,
 and allow LedgerTransaction to choose different Verifier objects.
* Update DJVM to use Corda 4.4-SNAPSHOT. (#95)
* CORDA-3330: Allow DJVM to preload / pregenerate classes from selected jars. (#92)
* Add support for SourceClassLoader.getResources() to DJVM.
* Allow a SandboxConfiguration to preload sandbox byte-code for all classes inside jars containing META-INF/DJVM-preload.
* CORDA-3309: Remove explicit try-catch in favour of UncaughtExceptionHandler. (#91)
* CORDA-3309: Install UncaughtExceptionHandler for DJVM tasks. (#88)
* Fix tests broken by Windows line endings. (#82)
* CORDA-3292: Reimplement ExecutionProfile as a data class. (#80)
* CORDA-2877: Refactor how we create child SandboxConfiguration objects. (#76)
* CORDA-2877: Load bytecode from a persistent cache to prevent repeated rewriting. (#75)
* Refactor byte-code cache to SandboxConfiguration instead of AnalysisConfiguration. We cannot "mix and match" byte-code generated by different sets of rules.
* CORDA-3137: Enhance annotation handling so that we can allow some annotations to be mapped into the sandbox without also needing to be stitched. (#72)
* CORDA-2871: Minor cosmetic fixes. (#69)
* CORDA-3218: Align DJVM with internal Corda Serialisation API. (#68)
* Ensure we get the latest SNAPSHOT of the serialisation code.
* CORDA-2871: Refactor SourceClassLoader to define source classes. (#66)
* Rewrite SourceClassLoader to support parent/child relationships.
* Revert catching TypNotPresebtException - it was a symptom of a bigger problem.
* Remove AutoCloseable from AnalysisConfiguration and SourceClassLoader.
* SourceClassLoader.getResource() must delegate to its parent first.
* CORDA-2871: Ensure ClassLoader.loadClass() throws ClassNotFoundException for all cases where the class cannot be found. (#64)
* CORDA-2871: Modify sandbox tasks to implement both java.Function and sandbox.Function (#62)
* Make TaskExecutors implement BiFunction to make them composable.
* Create ImportTask to wrap a java.Function inside a sandbox.Function.
* Add createExecutor() and createRawExecutor() APIs to SandboxClassLoader.
* Update serialization to use SandboxClassLoader.toSandboxClass().
* Remove a layer of lambdas from the serialisation code.
* Update SandboxExecutor and SandboxRawExecutor.
* Rename Executor to TaskFactory.
* Rename dangling executor -> taskFactory.
* CORDA-2871: Sanity fixes! (#63)
* Improve message for SandboxClassLoadingException.
* Fix serialisation API for using sandboxed environment.
* CORDA-3174: Extend serialisation to include InputStream and OpaqueBytesSubSequence. (#60)
* Update DJVM Example project for serialisation.
* Add serializers for InputStream and OpaqueBytesSubSequence.
* Support ZIP Inflater and CRC32 inside the sandbox.
* Allow the DJVM to wrap java.io.InputStream as sandbox.java.io.InputStream.
* Configure tests also to preserve @DeprecatedConstructorForDeserialization.
* CORDA-3174: Implement Corda serialization modules. (#59)
* Create DJVM serialization modules.
* Create test cases for Array<T>, List<T> and List<Array<T>>.
* Refactor SandboxPrimiveSerializer for all primitive types.
* Implement SandboxCollectionSerializer to support Collection types.
* Implement SandboxMapSerializer to support Map types.
* Attempt to fix infinite loop when computing Collection and Map fingerprints.
* Apply special handling when deserialising sandbox.java.lang.Character.
* Remap Java primitive types to sandbox Java object types to deter evolution.
* Use Class.getPackage().getName() to determine sandbox package name.
* Implement SandboxEnumSerializer to support Enum types.
* Implement SandboxPublicKeySerializer to support Java security keys.
* Add serialization projects to the composite example project.
* Implement serializers for BigInteger, BigDecimal, Currency and StringBuffer.
* Test that deserialising does not instantiate the untrusted user classes.
* Implement serializers for java.time.* types.
* Add serialiser for BitSet - currently disabled until BitSet itself is supported.
* Add serialisers for EnumSet and Class.
* Include support for EnumMap in the SandboxMapSerializer.
* Ensure the DJVM Example project's tests preserve @CordaSerializable.
* Add support for UUID as a primitive type.
* Use common abortReadOnly() method for declaring serialization as unsupported.
* Streamline the API for deserialising into the sandbox.
* Add preliminary support for deserialising X.509 certificates.
* Implement serializer for java.util.Optional.
* Refactor configuration of the sandbox serialization scheme.
* Add tests for deserialising arrays of basic types.
* Include method annotations in annotation stitching. This ensures that `@ConstructorForDeserialization` is not dropped.
* Enable test for SandboxBitSetSerializer.
* Enable tests for X.509 serializers.
* Implement serializers for ProtonJ primitive types.
* Serialize java.util.Date as a primitive type.
* Add the bintray Gradle plugin to the serialisation modules.
* Do not publish serialisation modules - they will become part of Corda itself.

* CORDA-2876: Only apply DJVM sources to Node Driver when devMode=true.

* Resolve DeteKT warnings.

* Require Node's JVM to set -Dnet.corda.djvm=true in order to enable DJVM.

* Enable DJVM for DemoBench nodes.

* Disable Quasar instrumentation verification for DemoBench nodes.

* Upgrade to DJVM 1.0-RC01.

* Try to modify DriverParameters in a more "ABI friendly" way.

* Refactor and simplify sandbox deserialisation of primitive objects.

* Review fixes.

* Update EvolutionSerializerFactory to handle sandboxed primitive boxed types.
2019-11-05 13:44:18 +00:00
LankyDan
532e4989d6 @Ignore P2PMessagingTest - communicating with a distributed service which we're part of 2019-11-05 11:52:32 +00:00
Dimos Raptis
485feb2d6c CORDA-3350: Increase size of constraints column (#5639)
* CORDA-3350: Increase size of constraints column (#5639)

* Detekt

* Update api file with new threshold

* Add check in transaction builder

* Revert "Add check in transaction builder"

This reverts commit ca3128f44c.

* Add check for max number of keys

* Update api file

* Address Tudor's comments

* Remove check for pre-5 and add test for EC keys

* fix typo and rename liquibase script

* updated docs with measurement numbers for composite keys

* Make detekt happy again
2019-11-05 11:00:26 +00:00
stefano
5ac1e50135 Merge remote-tracking branch 'origin/release/os/4.3' into my_merge_branch 2019-11-04 17:48:15 +00:00
LankyDan
54394f6747 Release state soft locks when a flow is killed via killFlow 2019-11-04 13:21:06 +00:00
LankyDan
3c0631a26a Fix VaultObserverExceptionTest due to differences between ENT and OS 2019-11-04 09:42:41 +00:00
LankyDan
be2e34b33a Fix error from warning from detekt 2019-11-01 16:00:09 +00:00
LankyDan
3aaddb47ea Attempt to fix detekt issues in DriverDSLImpl again 2019-11-01 14:36:47 +00:00
LankyDan
b05bd76a77 Fix error in VaultObserverExceptionTest due to bad merge 2019-11-01 13:38:42 +00:00
LankyDan
e737b01184 Port ledger integrity work to SingleThreadedStateMachineManager
- Port ledger integrity work to `SingleThreadedStateMachineManager`
- Fix `StatemachineErrorHandlingTest`
- Fix compile errors in `RetryFlowMockTest` + `VaultObserverExceptionTest`
- Add method to `StaffedFlowHospital` that was missed during original merge
2019-11-01 11:50:16 +00:00
Christian Sailer
f670cf774b Suppress/fix more warnings 2019-11-01 11:48:07 +00:00
Christian Sailer
1f82213827 Suppress/Baseline detekt issues that were highlighted freshly because of the merge. 2019-11-01 11:48:07 +00:00
Christian Sailer
5b9c5a6b83 Fix and or suppress detekt warnings 2019-11-01 11:48:07 +00:00
Christian Sailer
119f939ee1 Fix and or suppress detekt warnings 2019-11-01 11:48:07 +00:00
Christian Sailer
48aa2f2faa Update compiler.xml for new projects (and code style settings) 2019-11-01 11:48:07 +00:00
LankyDan
c5124689ca CORDA-3194 Update TODOs with jira tickets 2019-11-01 11:48:07 +00:00
Christian Sailer
8f2923c7a3 CID-843 code review fixes (#2681)
* Layout fix

* Fix test to use proper hook in flow hospital

* Fix comment
2019-11-01 11:48:07 +00:00
LankyDan
44206a0bea CORDA-3194 Update flow hospital docs, tidy comment and make exception public 2019-11-01 11:48:07 +00:00
Dan Newton
f4e9b9d5d2 CORDA-3194 Stop killed flows from re-entering hospital (#2664)
* CORDA-3194 Do not allow killed flows back into the hospital

This change has been made to prevent killed flows from being added back
to the hospital after being forcibly removed by `killFlow`. Not doing so,
could leave references to a flow inside of the hospital, which is not
the correct behaviour.

`killFlow` now sets a flow's `StatemachineState.isRemoved` to true.

This check is then used in `StaffedFlowHospital` and the
`DumpHistoryOnErrorInterceptor`.

* CORDA-3194 Log different message for transition error due to killed flow

When a flow is killed, its checkpoint is deleted. Currently, the
statemachine will still try a process the next event even if it has
been killed. This can lead to an error when trying to update the
deleted checkpoint. The exception thrown from this is logged out.

An if statement has been added to log a different message at debug level
if it is due to an update error for a killed flow. This is done to not
alarm node operators of the exception.
2019-11-01 11:48:07 +00:00
Dan Newton
3b3dbd7352 CORDA-3194 Add extra test cases to StatemachineErrorHandlingTest (#2651)
* CORDA-3194 - Add `ReceiveFinalityFlow` byteman tests

* CORDA-3194 Tidy up `StatemachineErrorHandlingTest`
2019-11-01 11:48:07 +00:00