Commit Graph

619 Commits

Author SHA1 Message Date
LankyDan
e737b01184 Port ledger integrity work to SingleThreadedStateMachineManager
- Port ledger integrity work to `SingleThreadedStateMachineManager`
- Fix `StatemachineErrorHandlingTest`
- Fix compile errors in `RetryFlowMockTest` + `VaultObserverExceptionTest`
- Add method to `StaffedFlowHospital` that was missed during original merge
2019-11-01 11:50:16 +00:00
Christian Sailer
f670cf774b Suppress/fix more warnings 2019-11-01 11:48:07 +00:00
Christian Sailer
1f82213827 Suppress/Baseline detekt issues that were highlighted freshly because of the merge. 2019-11-01 11:48:07 +00:00
Christian Sailer
119f939ee1 Fix and or suppress detekt warnings 2019-11-01 11:48:07 +00:00
LankyDan
c5124689ca CORDA-3194 Update TODOs with jira tickets 2019-11-01 11:48:07 +00:00
Dan Newton
f4e9b9d5d2 CORDA-3194 Stop killed flows from re-entering hospital (#2664)
* CORDA-3194 Do not allow killed flows back into the hospital

This change has been made to prevent killed flows from being added back
to the hospital after being forcibly removed by `killFlow`. Not doing so,
could leave references to a flow inside of the hospital, which is not
the correct behaviour.

`killFlow` now sets a flow's `StatemachineState.isRemoved` to true.

This check is then used in `StaffedFlowHospital` and the
`DumpHistoryOnErrorInterceptor`.

* CORDA-3194 Log different message for transition error due to killed flow

When a flow is killed, its checkpoint is deleted. Currently, the
statemachine will still try a process the next event even if it has
been killed. This can lead to an error when trying to update the
deleted checkpoint. The exception thrown from this is logged out.

An if statement has been added to log a different message at debug level
if it is due to an update error for a killed flow. This is done to not
alarm node operators of the exception.
2019-11-01 11:48:07 +00:00
Dan Newton
3b3dbd7352 CORDA-3194 Add extra test cases to StatemachineErrorHandlingTest (#2651)
* CORDA-3194 - Add `ReceiveFinalityFlow` byteman tests

* CORDA-3194 Tidy up `StatemachineErrorHandlingTest`
2019-11-01 11:48:07 +00:00
Dan Newton
a591c8e25b CORDA-3194 Relax duplicate insert flow hospital handling (#2643)
* CORDA-3194 Relax duplicate insert flow hospital handling

Revert a previous change to now make the duplicate insert staff member to
give a diagnosis of discharge or not my speciality (previously gave
terminal).

This is to prevent duplicate insert handling from overriding finality
flow error handling.
2019-11-01 11:48:07 +00:00
Dan Newton
ef01a99737 CORDA-3194 Failure during flow retry forces the flow into overnight observation (#2640)
When a flow fails to retry, it should be kept in for overnight observation and aborted.

In the future, it might be possible to retry flows again that failed during their retry, but for now keeping for observation and aborting is satisfactory.

* CORDA-3194 Remove hospitalised flows from `HospitalisingInterceptor`

Small refactor to remove some of the hospital logic out of the
`HospitalisingInterceptor` and into the `StaffedFlowHospital`.

Add some comments to help clarify the purpose of the two maps inside
of the hospital.

* CORDA-3194 When a flow fails to retry force it into observation

When a flow fails to retry, it should be kept in for overnight
observation and aborted.

In the future, it might be possible to retry flows again that failed
during their retry, but for now keeping for observation and aborting is
satisfactory.

* CORDA-3194 Test for database commit failure when retrying a flow

Failing during the database commit failure that occurs after the retry
flow action does not stop the flow from actually retrying. This test
confirms this functionality.

The retried flow gets scheduled as part of the retry action. The failure
in the commit action does not prevent this since it has already been
scheduled.
2019-11-01 11:48:07 +00:00
Dan Newton
268d129838 CORDA-3194 Replay start flow events when responding flow fails initial checkpoint commit (#2601)
* CORDA-3194 Replay start flow events when responding flow fails initial checkpoint commit

Logic has already been added to recover from initial checkpoint commit
failures on the initiating flow side but this did not suffice for
the same failure occurring on the responding flow's side.

The same idea has been added to resolve the responding flow's issue.

`ExternalMessageEvent` now has a `flowId` that is maintained on the
event. Messages can then be replayed to start/restart the flow, while
the event provides the flow id to each flow start.

Each `ExternalMessageEvent` implementation generates a random `flowId`
when constructed.

Events are stored in Artemis. This allows the solution to recover across
node restarts as the events will be pulled from artemis again when
restarting.

In the future `flowId`s will probably moved off of the events and
generated purely on the responding flow's node.

* CORDA-3194 Add test to verify that errors removing a responding flow are recoverable
2019-11-01 11:48:07 +00:00
Christian Sailer
1f71b071aa CORDA-3217 and CORDA-3195 Various bits arond SQL exceptions and flow hospital (#2605)
* Unwrap rx.OnErrorNotImplementedException so the hospital can handle the cause appropriately

* Add db failure cordapp

* Renamed folders to avoid ambiguity in gradle

* Add integration test for exception hospitalisation when thrown from an RX observable.

* Make the test slightly cleaner

* Fix the schema to actually match the requirements for my custom state. Thanks a bunch, H2.

* Switch test to use SqlException base class.

* Schedule error event if we detect that a commit or db flush has thrown (forcing the flow to error even if customer code then goes ahead to swallow the exception)

* Revert change to schedule extra error

* Add more tests for edge case with DB exceptions, changed CorDapp to suppor this an hook in the flow hospital

* Warning about unsubscribe
Check state transitioned from clean to error for hospital admission.

* Match the test to our actual expectations

* Revert "Revert change to schedule extra error"

This reverts commit 43d47937

* Prevent suppression of errors arising in `transaction()` and `jdbcConnection()`

* Test for SqlException caught trying to escape from recordTransaction and suppressed outside being intercepted.

* More tests for various error/catch combinations

* Clean up and comments

* Code reformat

* Fix test compilation
2019-11-01 11:48:07 +00:00
Dan Newton
9b169df2b8 CORDA-3194 Wrap state transition exceptions and add flow hospital error handling for them (#2542)
Wrap exceptions that occur in state machine transitions with a custom exception type which is
then handled inside of the flow hospital. As part of this change, a number of side negative side
effects have been addressed.

General summary:

- `StateTransitionException` wraps exceptions caught in `TransitionExecutorImpl`
- `StateTransitionExceptions` are handled in the flow hospital, retried 3 times and then kept in
    for observation if errors persist (assuming conditions below are false)
- Exceptions that occur in `FlowAsyncOperation` events are wrapped in
  `AsyncOperationTransitionException` and ignored by the flow hospital transition staff member
- `InterruptException`s are given a `TERMINAL` diagnosis by the flow hospital transition staff
   member (can occur due to `killFlow`)
- Allow flows which have not persisted their original checkpoint to still retry by replaying their
   start flow messages
- Swallow exceptions in `AcknowledgeMessages` actions

Detailed summary:

* CORDA-3194 Add state machine transition error handling to flow hospital

Wrap exceptions that are caught in `TransitionExecutorImpl` (coming from
new errors) with `StateTransitionException`. This exception is then
handled explicitly by the flow hospital.

Add `TransitionErrorGeneralPractitioner` to `StaffedFlowHospital`. This
staff member handles errors that mention `StateTransitionException`.
Errors are retried and then kept in the hospital if the errors persist.

* CORDA-3194 Remove a fiber from the `hospitalisedFlows` if its previous state was clean

If the fiber's previous state was clean then remove it from
`HospitalisingInterceptor.hospitalisedFlows`. This allows flows that are
being retried to clean themselves. Doing this allows them to re-enter
the flow hospital after executing the fiber's transition (if an error
occurs).

This is important for retrying a flow that has errored during a
transition.

* CORDA-3194 Set `isAnyCheckpointPersisted` to true when retrying a flow

Added to prevent a single flow from creating multiple checkpoints when
a failure occurs during `Action.AcknowledgeMessages`.

More specifically, to `isAnyCheckpointPersisted` is false when retrying
the flow, even though a checkpoint has actually been saved. Due to this
a brand new flow is started with a new flow id (causing duplication).

Setting `isAnyCheckpointPersisted` to true specifically when retrying a
flow resolves this issue.

* CORDA-3194 Add Byteman test to verify transition error handling

Add `StatemachineErrorHandlingTest` to verify transition error handling.

Byteman allows exceptions to be injected at certain points in the code's
execution. Therefore exceptions can be thrown when needed inside of the
state machine.

The current tests check errors in events:
- `InitiateFlow`
- `AcknowledgeMessages`

* CORDA-3194 Swallow all exceptions in `ActionExecutorImpl.executeAcknowledgeMessages`

Swallow the exceptions that occur in the `DeduplicationHandler`s when
inside of `ActionExecutorImpl.executeAcknowledgeMessages`.

The side effects of the failures that can happen in the handlers are
not serious enough to put the transition into a failure state.
Therefore they are now caught. This allows the transition to continue
as normal, even if an error occurs in one any of the handlers.

* CORDA-3194 Wrap unexpected exceptions thrown in async operation transitions

Exceptions thrown inside of `FlowAsyncOperation.execute` implementations
that are not returned as part of the future, are caught, wrapped and
rethrown. This prevents unexpected exceptions thrown by (most likely)
user code from being handled by the hospital by the transition
staff member.

This handling might change moving forward, but it allows the async
operation to continue working as it was before transition error handling
was added.

* CORDA-3194 Verify that errors inside of `AcknowledgeMessages` work as expected

Update `StatemachineErrorHandlingTest` to correctly test errors that
occur when executing the `AcknowledgeMessages` action.

* CORDA-3194 Retry flows that failed to persist their original checkpoint

Allow a flow that failed when creating their original checkpoint (for
example - failing to commit the db transaction) to retry.

The flow will create a brand new checkpoint (as the original did not
saved).

This required adding `flowId` to `ExternalStartFlowEvent` to allow the
event to keep a record of the flow's id. When the flow is retried, the
events are replayed which trigger a flow to be started that has the
id stored in the event.

To allow this change, code was removed from `retryFlowFromSafePoint` to
allow the function to continue, even if no checkpoint matches the passed
in flow id.

* CORDA-3194 Correct `FlowFrameworkTests` test due to error handling

Test assumed that errors in transitions are not retried, this has now
been updated so the test passes with the flow succeeding after an
exception is thrown.

* CORDA-3194 Remove unneeded import

* CORDA-3194 Make the state transition exceptions extend `CordaException`

`StateTransitionException` and `AsyncOperationTransitionException` now
extend `CordaException` instead of `Exception`.

* CORDA-3194 Improve log messages

* CORDA-3194 Remove unneeded code in `HospitalisingInterceptor`

Due to a previous change, a section of code that removes a flow id
from the `hospitalisedFlows` map is no longer required. This code has
been removed.

* CORDA-3194 Constraint violations are given `TERMINAL` diagnosis

Add `Diagnosis.TERMINAL` to `StaffedFlowHospital` to allow an error
to be ignored and left to die a quick and painful death.

`StateTransitionException` changed so it does not cause serialisation
errors when propagated from a flow.

* CORDA-3194 `InterruptedExceptions` are given `TERMINAL` diagnosis
2019-11-01 11:48:07 +00:00
Christian Sailer
938828b52f CORDA-3195 Default behaviour of FlowHospital (WIP) (#2520)
* Add GP to flow hospital, and start working on a list of things the GP knows to be incurable.

* Only hospitalise SQL and Persistence Exceptions (let's see if that is enough?), also rename to DatabaseDentist.

* Disabled hospitalisation of SQL exceptions in flow retry tests

* Fix RPC exception handling test by not using PersistenceException

* Ignore flaky integration test

* Code review: Rename staff member and add testing annotation

* Revert compiler.xml
2019-11-01 11:48:07 +00:00
Dimos Raptis
79c22bd77a Constrain the cordapps loaded from driver tests to the minimum required (#5653) 2019-10-31 11:26:05 +00:00
Stefan Iliev
03ab258fc2 Revert "CORDA-3307 - add support for environment variables in linux (#5523)" (#5643)
This reverts commit c882b221a5.
2019-10-29 17:55:58 +00:00
Stefan Iliev
c882b221a5 CORDA-3307 - add support for environment variables in linux (#5523)
* Added a new way for environment variables to be loaded, which allows for underscore based separation.

* Moved test to its own kotlin file.

* Added case insensitivity support.

* The corda. prefix is now case insensitive too.

* Removed unused variable.

* Added env variables support for driverDSL. Shadowing corda. properties raises an exception.

* Driver api stability fix.

* Changed type of cordapps param to reflect the real one, rather than what IntelliJ auto completed.

* Some detekt issue fixes. Spread operator removed, baselined api stability constructors and buggy line.

* Fixed misspelled variable.

* Reverted unintentional changes.

* Added suppress instead of changing baseline.

* Reworked logic to handle previously defined CORDA_ starting properties and handle accordingly. Fixed a bug where wrong class was used for reflection walking.

* Fix for detekt issues.

* Changed message to a more understandable one.

* Changelog + doc note, console error grammar.

* Changes according to PR review.
2019-10-21 12:01:14 +01:00
Dimos Raptis
f37638c93d [CORDA-3122] - Cleanup non-finalised, errored flows (#5594)
* [CORDA-3122] - Cleanup non-finalised, errored flows

* detekt
2019-10-16 09:37:28 +01:00
Stefano Franz
6e98adb085
use zulu for jdk in testing image (#5583)
* use zulu for jdk
add some parallel groups

* port kubesTest to Java
remove asterix from tests listed by ListTests, instead add after allocation
2019-10-14 13:35:22 +01:00
josecoll
efa01410ec CORDA-3263 Improve error handling for registering peer node (#5559)
* Capture and log "nodeInfo" persistence failures, whilst maintaining an optimistic retry mechanism.

* Additional test cases (update and insert)

* Handle both updates and inserts consistently (single transaction for happy path)

* Fix detekt violations and update baseline with false detection.

* Streamline the code a little.

* Update baseline reporting false violation.
2019-10-08 11:38:11 +01:00
Dimos Raptis
ec4d20d987 [CORDA-3243] - Improve CorDapp loading logic for duplicates (#5551) 2019-10-07 16:20:55 +01:00
Zoltan Kiss
f171de7b69 TM-23 Fail build on compiler warnings (#5453)
* java compile respects compilation.allWarningsAsErrors

* suppress or cleanup warnings

* suppress warning

* use non-deprecated kotlin dependency

* rename property

* handle property existence check

* Deal with warnings
2019-09-11 16:34:51 +01:00
Tudor Malene
a88c519096 CORDA-2740 - Remove RPC exception obfuscation (#5455) 2019-09-10 15:06:15 +01:00
Stefano Franz
a842740c9e
WIP Kubenetes parallel build (#5396)
* Split integration tests

* add simple example of printing all methods annotated with @Test

* add docker plugin to root project
remove docker plugin from child projects
add Dockerfile for image to use when testing
add task to build testing image to root project

* add comment describing proposed testing workflow

* simple attempt at running tests in docker container

* add my first k8s interaction script

* add fabric8 as dependnency to buildSrc

* before adding classpath

* collect reports from containers and run through testReports

* re-enable kubes backed testing

* for each project
1. add a list tests task
2. use this list tests task to modify the included tests
3. add a parallel version of the test task

* tweak logic for downloading test report XML files

* use output of parallel testing tasks in report tasks to determine build resultCode

* prepare for jenkins test

* prepare for jenkins test

* make docker reg password system property

* add logging to print out docker reg creds

* enable docker build

* fix gradle build file

* gather xml files into root project

* change log level for gradle modification

* stop printing gradle docker push passwd

* tidy up report generation

* fix compilation errors

* split signature constraints test into two

* change Sig constraint tests type hierarchy

* tidy up build.gradle

* try method based test includes

* add unit test for test listing

* fix  bug with test slicing

* stop filtering ignored tests to make the numbers match existing runs

* change log level to ensure print out

* move all plugin logic to buildSrc files

* tidy up test modification
add comments to explain what DistributedTesting plugin does

* move new plugins into properly named packages

* tidy up runConfigs

* fix compile errors due to merge with slow-integration-test work

* add system parameter to enable / disable build modification

* add -Dkubenetise to build command

* address review comments

* type safe declaration of parameters in KubesTest
2019-09-03 15:40:08 +00:00
Dimos Raptis
b97062bacc [CORDA-3098] - Close previous connection after reconnection (#5339) 2019-08-28 16:44:57 +01:00
Jonathan Locke
20585266a5
[NOTICK]: Change Corda version
Change Corda version
2019-08-28 15:38:36 +01:00
Stefano Franz
e35c0c1df7
CORDA-3009 - Migrate identity service to use to string short (#5217)
* migrate PersistentIdentityService to use key.toShortString()
update definition of PublicKeyToExternalId mapping to allow fast lookup by externalId/publicKey

* fix misspelled table name

* add test of migration script

* add design document for proposal to move IdentityService to using the correct PK.toStringShort() method for hashing a publickey

* add enterprise testing considerations to design

* address review comments

* fix compilation errors

* modify PublicKeyToOwningIdentityCache to use toStringShort() as it's lookup key

* address syzmon's code review comments
2019-08-28 14:16:18 +00:00
Jonathan Locke
5ede8b5761 [NOTICK]: Change Corda version
Removed unnecessary comment which will just get outdated.
2019-08-28 14:35:06 +01:00
Jonathan Locke
c525f77ae9 [NOTICK]: Change Corda version
Changed Corda version in places where it was missed.
2019-08-28 14:14:39 +01:00
Jonathan Locke
e5f314ffbf Merge branch 'release/4' into jlocke/merge_4.1_20190827
Merged release/4 (Corda OS version 4.1) into a branch created from master (Corda OS version 4.3)

Conflicts:
	build.gradle
	client/rpc/src/main/kotlin/net/corda/client/rpc/internal/ReconnectingCordaRPCOps.kt
	common/logging/build.gradle
	common/logging/src/main/kotlin/net/corda/common/logging/CordaVersion.kt
	constants.properties
	core-tests/src/test/kotlin/net/corda/coretests/utilities/ProgressTrackerTest.kt
	core/src/main/kotlin/net/corda/core/transactions/TransactionBuilder.kt
	docs/source/api-contract-constraints.rst
	docs/source/api-stability-guarantees.rst
	docs/source/app-upgrade-notes.rst
	docs/source/changelog.rst
	docs/source/clientrpc.rst
	docs/source/conf.py
	docs/source/corda-network/UAT.md
	docs/source/getting-set-up.rst
	docs/source/network-builder.rst
	docs/source/upgrading-cordapps.rst
	docs/source/versioning.rst
	node/src/integration-test/kotlin/net/corda/node/services/rpc/RpcReconnectTests.kt
	serialization/src/main/kotlin/net/corda/serialization/internal/amqp/PropertyDescriptor.kt
	serialization/src/main/kotlin/net/corda/serialization/internal/model/LocalTypeInformationBuilder.kt
	tools/network-builder/src/main/resources/node-Dockerfile
	tools/network-builder/src/main/resources/notary-Dockerfile
2019-08-27 14:51:22 +01:00
LankyDan
9bf26c20e0 CORDA-2920 Hash to Signature Constraint automatic propagation
Allow Hash Constraints to propagate to Signature Constraints. When
propagating, the new jar is added to the transaction instead of the
original contract attachment (jar).

The following requirements must be met to do so:
- System property "net.corda.node.disableHashConstraints" must be set
  to `true`
- The min platform version must be 4 or greater
- There must be an input state with a hash constraint
- There must be no output states with a hash constraint
- The new jar must be signed

If these requirements are not met, the original Hash
Constraint will be kept and the contract attachment related to it is
also used.

This transition is done at the start of `handleContract` as it is not
the normal path a transition would follow. It is considered a backdoor
and should be treated separately from the rest of the attachment and
constraint logic. Furthermore, it will only work in private network
since all nodes must set the special `disableHashConstraints` flag.
2019-07-29 12:58:58 +02:00
Dimos Raptis
8962d930d4 [CORDA-2923] - Make the RPC client reconnect with gracefulReconnect param (#5244)
When set to true the RPC client will:

* automatically reconnect when the connection is broken
* simple RPC calls will block until connection is established
* Observables returned from RPC will automatically resubscribe on reconnect so the client continues to receive events. This doesn't guarantee that events will not be lost during the reconnect.
2019-07-16 11:29:21 +01:00
Shams Asari
dfa3c67919 Removing reference to Enterprise in test (#5293) 2019-07-11 13:36:00 +01:00
gwr3com
22e4b8e962 CORDA-3028: Add Node Diagnostics Info RPC Call - Backport a diff from… (#5287)
* CORDA-3028: Add Node Diagnostics Info RPC Call - Backport a diff from Corda ENT

* CORDA-3028: Add Node Diagnostics Info RPC Call - Rename a value for consistency

* CORDA-3028: Add Node Diagnostics Info RPC Call - One more change
2019-07-10 11:31:44 +01:00
gwr3com
933330bc4c CORDA-3028: Add Node Diagnostics Info RPC Call (#5271)
* CORDA-3028: Add Node Diagnostics Info RPC Call - Initial commit

* CORDA-3028: Add Node Diagnostics Info RPC Call - Changes after code review

* CORDA-3028: Add Node Diagnostics Info RPC Call - Additional code review update and documentation

* CORDA-3028: Add Node Diagnostics Info RPC Call - Additional changes to documentation

* CORDA-3028: Add Node Diagnostics Info RPC Call - Another change to documentation

* CORDA-3028: Add Node Diagnostics Info RPC Call - More changes to documentation

* CORDA-3028: Add Node Diagnostics Info RPC Call - Comment change
2019-07-08 14:43:50 +01:00
Dan Newton
6d3a6a3998 ENT-3710 Add TransientConnectionCardiologist to Flow Hospital (#5277) 2019-07-08 11:07:50 +01:00
Stefano Franz
88894bc592
add a shared memory port allocator to allow multiple processes to sha… (#5223)
* add a shared memory port allocator to allow multiple processes to share a single allocation pool

* remove dangerous reset function on port allocator

* set forkCount = 2 in node integration test

* only allow one build of a cordapp at any given time for Driver tests

* make all portallocation requests use same starting point

* globally set forks to 6

* tweak forking parameters to allow parallel builds

* tweak unit test parallelism

* 2 workers for integrationTest

* some more tweaks for parallel builds

* some more tweaks for parallel builds

* seems that 49K is not the start of ephemeral ports on all kernels

* tweak parallel settings

* try fix RPC shutdown test in parallel env

* add some logging for RPC shutdown test

* added some logging around PortAllocation tests - try figure out where they are getting stuck

* added some logging around PortAllocation tests - try figure out where they are getting stuck

* fix api-scanner tests

* minimize api changes

* revert to complying with existing API

* add the AtomicInteger for api compatibility reasons

* make sizing script executable

* address review comments pt1

* address review comments pt2

* fix compile errors after review comments

* return to using home dir as temp dir seemed to interact badly with gradle
2019-07-02 18:38:33 +00:00
Stefan Iliev
d1e7850dd1 CORDA-2176 - test that logging is not broken. (#5224)
* Initial commit with a test that can detect a logger that isn't initialised with by lazy that gets called during startup.

* Test improvement. Test fix for api stability.

* Added explanatory comment for the test.

* Changes according to PR review. Added empty file check to make test more robust.
2019-06-25 14:35:39 +01:00
Viktor Kolomeyko
3ceb242ee1 CORDA-2935 - Align timeouts for CRL retrieval and TLS handshake (#5126) 2019-06-21 09:34:15 +01:00
Shams Asari
4e835d5483
CORDA-2333: Updated the majority of the dependencies that were out of date (#5235)
Some where not touched, including the crypto ones, libraries related to messaging and Kotlin.
2019-06-20 16:19:16 +01:00
szymonsztuka
937e467aac CORDA-2971 - added tests for initialiseSchema configuration option (#5153) 2019-06-04 11:37:32 +00:00
Stefan Iliev
3b7f0aff92 ENT-3484 - Added periodic log.warn message to remind that the node has been set into draining mode. (#5151)
* Added periodic message logged from a timer which is switched on and off by the node drain events.

* Timer also spawns on start() to ensure that if a node is shutdown in draining mode and started with it, it will continue logging messages.
2019-06-03 13:49:29 +03:00
Viktor Kolomeyko
355604457f CORDA-2935: Align timeouts for CRL retrieval and TLS handshake (#5125) 2019-05-15 09:19:51 +01:00
JamesHR3
d8a790b3ce [CORDA-2866] Adjust RPC test case to prevent failures on non-H2 databases (#5114) 2019-05-10 11:13:12 +01:00
JamesHR3
b4e96778bf [CORDA-2517] Whitelist attachments signed by keys that already sign existing trusted attachments (#5068)
This allows a different signed version of the same CorDapp to be automatically trusted.

This reverts "[CORDA-2575] Allow users to whitelist attachments by public key config (#5035)"
2019-05-07 12:55:27 +01:00
JamesHR3
c533792f3f [CORDA-2866] Prevent node startup if legal identity key is lost but node key isn't (#5090) 2019-05-07 11:49:29 +01:00
Tudor Malene
4c31e8c1fe Increase the wait time for events as it can take longer on some environments (#5085) 2019-05-02 11:03:52 +01:00
Dan Newton
c163365229 CORDA-2280 Move assumeFalse in SignatureConstraintVersioningTests (#5076)
A test was failing due to `assumeFalse` (to stop a test running on windows)
being inside of a `assertThatExceptionOfType` block. The exception it
threw was consumed by the block instead of junit.
2019-04-30 16:08:10 +01:00
Dan Newton
6662d205f8 CORDA-2280 Automatic propagation of whitelisted to Signature Constraints (#5053)
If a single whitelisted constraint is being used by input states and the version of the cordapp changes + is signed, then the constraint will transition to a signature constraint.
2019-04-29 17:37:57 +01:00
Tudor Malene
7b0d177a34 More leniency with auth errors in RpcReconnectTests (#5061) 2019-04-26 15:17:26 +01:00
JamesHR3
7a7c471ebf [CORDA-2575] Allow users to whitelist attachments by public key config (#5035) 2019-04-25 16:55:43 +01:00
Tudor Malene
50649d7b99 CORDA-2743 Add dynamic port allocation (#5042) 2019-04-18 14:22:23 +01:00
Tudor Malene
5821ad5f5c CORDA-2743: Tweak RPC reconnecting test. Adjust the exponential retry factor. (#5026) 2019-04-17 10:56:55 +01:00
Dan Newton
855d1323a4 CORDA-2645 Do not remove exception information in dev mode (#5023)
`ExceptionSerialisingRpcOpsProxy` was removing information about the
original exception in dev mode. This code has been removed. Although
there is no check in `ExceptionSerialisingRpcOpsProxy` for dev mode
(and also due to it being about serialising),
`ExceptionMaskingRpcOpsProxy` will handle the removal of exception data
in non dev mode (production mode).
2019-04-15 13:54:36 +01:00
Tudor Malene
21adcffd31 CORDA-2743 - utilities and test to show rpc operations that support disconnects (#5009) (#5015)
(cherry picked from commit 6771386b4b)
2019-04-15 10:25:55 +01:00
Tudor Malene
6771386b4b CORDA-2743 - utilities and test to show rpc operations that support disconnects (#5009) 2019-04-12 14:03:38 +01:00
Thomas Schroeter
e0f19ca70b CORDA-2647 ROLLBACK (#4964)
* Revert "Address comments"

This reverts commit f884d112c3.

* Revert "CORDA-2647 - Prevent registration when previous state exists"

This reverts commit 4e9d1f1924.
2019-03-29 18:06:12 +00:00
Dimos Raptis
328becb2d3 Introduce global port allocation 2019-03-27 12:30:40 +01:00
Dominic Fox
c2ad64ccde CORDA-2802 use eventually to wait (#4932)
* CORDA-2802 use eventually to wait

* Catch Exception, not Throwable
2019-03-26 16:01:06 +00:00
Chris Rankin
d9a67d7426 BUILD: Use global port allocator for flaky HardRestartTest. (#4921) 2019-03-23 19:32:39 +00:00
Chris Rankin
3333464c39 BUILD: Fix some warnings about @BelongsToContract in tests. (#4902) 2019-03-19 18:16:24 +00:00
Jonathan Locke
dc179d4ea1 ENT-3057: Log hibernate warns and errors in different log (#4889)
* ENT-3057: Log hibernate warns and errors in different log

If a hibernate error occurs (deadlock, for example) that would cause a flow to be sent to the hospital, hibernate logs the warnings and errors before we do. This results in duplication in the logs, and pollutes the log. To solve this, we create a new log appender named diagnostic-{node-name}.log and log any org.hibernate messages of warn and above to that file. This way, messages are not lost, which means that the information can be retrieved if need be.

* Corrected indentation of comment (changed tab to space)

* Updated node-administration document to mention diagnostic logging change

* Fixed integration test. It was breaking because it was fetching the first log file in the folder, assuming there would be only one. This assumption is now invalid because the diagnostic log file that was introduced. Two tests were found that used similar logic to find a log file to examine, hence both were corrected to look for log files beginning with "node"

* Updated documentation as per review comments.
2019-03-18 17:08:13 +00:00
JamesHR3
b3b184c93e [CORDA-2701] Ensure crlCheckSoftFail config option is respected (#4854)
* Plumb through the crlCheckSoftFail configuration option to bridge manager

* Add crlCheckSoftFail test to bridge manager and fix equivalent proton wrapper test

* Update documentation and set the node configuration default to true

* Revert default change and clarify consequences of setting option to false

* Remove NodeConfiguration default to leave only AMQPConfiguration default
2019-03-08 16:23:07 +00:00
josecoll
0bdc8b363a
CORDA-2576 Viral update propagation: select attachments from those loaded by node on startup. (#4745)
* Retrieve contract attachment(s) loaded by node.

* Remove redundant integration test case (downgrade rule checking)

* Address PR review comments.

* Revert changes to classloading dependencies hack.
2019-02-14 13:39:14 +00:00
Mike Hearn
02645f7b9e Rework package namespace ownership check to verify every package of every class file.
Previous implementation was in LedgerTransaction and focused only on contract classes,
but every package matters.

Also fixes some exception types and does misc refactorings.
2019-02-11 17:10:50 +01:00
Shams Asari
c667df9bec
Port over ENT changes (#4706)
Port of commit 9b55ad2680
2019-02-01 18:10:10 +00:00
Shams Asari
88e4b85537
CORDA-2521: Checkpoint verifier no longer cares about the CorDapp jar name (#4669)
The check on the CorDapp hash is sufficient.
2019-01-29 16:32:07 +00:00
Tudor Malene
262a7ad1b7
Backport merge fixes added to ENT (#4618)
Fix test and address code review comments.
2019-01-29 13:45:04 +00:00
Shams Asari
2262d05d5c
Made NetworkMapTest leaner (#4645)
Merged three tests into one to avoid repeating the same node startups. The assertions the original tests made can be done in one test without any contamination.
2019-01-28 17:50:26 +00:00
Shams Asari
e20278fbfd
Made FlowCheckpointVersionNodeStartupCheckTest leaner (#4640)
* Removed `restart node successfully with suspended flow` as it duplicates `TraderDemoTest#Test restart node during flow works properly`
* Removed the need for a notary
2019-01-28 14:08:54 +00:00
Shams Asari
49e23bca82
Removed AsymmetricCorDappsTests (#4642)
This is only testing the internal cordappForClasses method which is used for internal testing. This method is covered by other unit and integration tests.
2019-01-28 13:59:38 +00:00
Shams Asari
3db35d8911
Made NodeRegistrationTest leaner (#4644) 2019-01-28 10:54:30 +00:00
Shams Asari
c6a7d14f4f
Reducing the size of the CorDapp jars used in tests (#4600)
The helper method enclosedCordapp is a replacement to scanning the current package (which pulls in a lot more into the jar than intended).
2019-01-21 13:54:26 +00:00
Tudor Malene
03cff6bd00
Fix flaky test. (#4586)
* Fix flaky test.

* Address code review comments.
2019-01-16 12:02:40 +00:00
Shams Asari
f592025273 Regen isolated jar to fix failing test(s) (#4583)
https://github.com/corda/corda/pull/4541 removed TransactionBuilder c'tor (which was introduced post-V3) which the frozen isolated jar was using
2019-01-15 14:56:57 +00:00
Tudor Malene
44733da8b6
Increase timeout (#4579) 2019-01-15 10:20:56 +00:00
Shams Asari
24a7821dce
CORDA-2434: CordaRPCClient now only has c'tors rather a mix of c'tors and factory methods (#4569) 2019-01-14 16:07:56 +00:00
Shams Asari
3b8347e150
Fixed AttachmentLoadingTests (#4565)
There were two issues:
* The original "sealing violation: can't seal package net.corda.nodeapi" issue was due to the isolated CorDapp containing some code in the net.corda.nodeapi namespace. This has been moved to the isolated namespace.
* The test was not correctly creating the second transaction with the dummy command
2019-01-14 14:32:14 +00:00
Dimos Raptis
5b34020e59 [CORDA-2390] - Add whitelists and custom serializers from cordapps to serialization … (#4551)
* Add whitelists and custom serializers from cordapps to serialization context

* Remove changes in TransactionBuilder, add caching

* Add whitelists and custom serializers from cordapps to serialization context

* Remove changes in TransactionBuilder, add caching

* Address comments

* Increase node memory for SIMM integration test

* Cache only serialization context

* Increase integ test timeout

* Fix API breakage

* Increase max heap size for web server integ test

* Move classloading utils from separate module to core.internal

* Adjust heap size for more integ tests

* Increase time window for IRS demo transactions

* Fix determinator

* Add parameter in core-deterministic

* Stub out class-loading method for DJVM
2019-01-13 20:15:05 +00:00
szymonsztuka
9b8fda0d6d
CORDA-2375 Ensure node has unique attachment contract classname/version from signed JARs (#4535)
Corda Node ensures a given contract class and version can be sourced from only one signed and trusted Attachment (JAR).
An attempt to import a signed JAR as a trusted uploader (or promote to be trusted) with a class and version already present in the other trusted Attachment will raise DuplicateContractClassException.

Minor fixes to Hibernate Attachment Query parser (original query to select attachment without signers would always return no attachments)
2019-01-10 14:13:00 +00:00
Christian Sailer
b5fc095288
Use FlowTimeOut exception rather than SQL deadlock in unit tests to avoid hitting the exponential back-off. (#4527) 2019-01-10 09:59:21 +00:00
josecoll
4530a5e982
CORDA-2395 Add cordapp code signing dev key to production blacklist. (#4519)
* Add cordapp code signing dev key to production blacklist.

* Remove code used to generate the actual PK hash (it is now attached to the JIRA story)

* Remove wiki page reference.

* Remove the non redundant cash transfer.
2019-01-09 15:41:24 +00:00
Matthew Nesbit
93d256649f
Reduce the operation of ip autodetect to nodes specifying localhost (#4435)
* Reduce the operation of ip autodetect to nodes specifying localhost, rather than any node with a 192.168.x.x or 10.x.x.x address. Also, catch the common user error of specifying 0.0.0.0 in p2pAddress.

* Fix test that uses invalid config
2019-01-08 09:55:16 +00:00
Tudor Malene
57a70fa1e8
Attempt to fix master (#4517)
* Attempt to fix master

* Attempt to fix master

* Attempt to fix master

* Attempt to fix master

* Ignore failing test

* Ignore failing test
2019-01-08 08:02:38 +00:00
Tudor Malene
b13e92cb24 Fix tests (#4513) 2019-01-07 09:25:05 +00:00
Tudor Malene
24d91c5f89
CORDA-2384 - Fix driver classpath (#4472)
* Fix driver classpath

* Fix tests

* Fix classloader bug.

* Fix tests

* Fix tests

* Fix api

* Disable failing irs test.

* Address code review comments

* Remove @Ignore from test

* Remove @Ignore from tests

* Address code review comments.

* Attempt to fix simm valuation test

* Attempt to fix simm valuation test

* Comment failing functionality.
2019-01-03 20:06:35 +00:00
Michele Sollecito
5d3b24dcfa
[CORDA-2387]: Seeing warnings for "Double insert in net.corda.node.utilities.AppendOnlyPersistentMap" (fixed) (#4499)
* [CORDA-2387]: Reproduced the issue in a test.

* [CORDA-2387]: Fixed.

* [CORDA-2387]: Fixed.
2019-01-03 17:59:34 +00:00
Shams Asari
b4c3fa1948
Fixed incorrect attachment loading integration test (#4453)
* Fixed incorrect attachment loading integration test

`AttachmentLoadingTests.test that attachments retrieved over the network are not used for code` was a false-positive - it was incorrect on multiple levels. Fixing it required updating the finance:isolated CorDapp, at which point it was given the new MANIFEST metadata for V4, and moved out of the net.corda.finance namespace to avoid package sealing issues.

The new test exposed a bug in the LedgerTransaction verification logic. This was cleaned up as it was too easy to verify on the wrong instance.
2018-12-31 15:02:11 +00:00
Shams Asari
60d215aaa8
Fixed incorrect usages of assertFailsWith (#4458)
The overload that takes in a String does NOT check that the exception thrown has that message, which is what these tests are assuming. Rather it's the assertion message when the test fails.
2018-12-24 09:44:48 +00:00
josecoll
6f9f6fd870 CORDA-2366 - Adjust assertions to cater for any ordering of results. (#4455) 2018-12-23 16:53:01 +00:00
Shams Asari
830959c9f7
CORDA-2345: Simplified TestCordapp to make it inline with the recent CorDapp versioning changes (#4434)
TestCordapp has now two implementations to clearly separate the two use cases it has in the Corda repo:

* TestCordappImpl which implements the revised public API of TestCordapp; namely that a TestCordapp instance references a real CorDapp jar on the classpath. This is either an external dependency jar in which case it’s taken as is and given to the node, or it’s a local gradle project in which case it’s compiled using the gradle “jar” task to generate the CorDapp jar. This approach means the jar has all the original CorDapp versioning information, which is important that it’s correct when testing. To this end, TestCordapp only needs to expose the ability to specify the app’s config. All the remaining properties have moved to CustomCordapp.

* CustomCordapp for creating arbitrary custom CorDapps, including specifying the jar’s MANIFEST values. This is internal API and only used for testing the platform. Technically this shouldn’t implement TestCordapp but does so to reduce the complexity of the driver and mock network.
2018-12-20 09:49:58 +00:00
josecoll
9cdda3bd77
CORDA-2149 CorDapp Contract and Workflow version identifiers (#4363)
* Implementation of Contract and Workflow attribute identifiers.

* Fixes following rebase from master.

* Fix broken JUnit test.

* Fix broken JUnit test.

* Fix broken JUnit test.

* Added missing constants.

* Further clean-up.

* Updated documentation.

* Added changelog entry.

* Updated all samples (using new Gradle Plugin 4.0.37 functionality)

* Temporarily resolve gradle plugins from latest published snapshot.

* Temporarily resolve gradle plugins from latest published snapshot.

* Updates following feedback from PR review.

* Move constants into CordappInfo companion object.

* Contract and Workflow attribute `version` to `versionId` (as version is a reserved gradle variable)

* Clarified warning message on incorrect version identifier.

* Align version identifier processing logic with gradle cordapp plugin.

* Updated comment.

* Minor fixes following rebase from master.

* Fixed broken unit test.

* Improved exception reporting.

* Update to use 4.0.37 of Gradle Plugins.

* Added support for combined Contract and Workflow CorDapp info.

* Updated following discussions with Shams + cleanup.

* Updated following Shams PR review.

* Minor API improvements.

* Added missing cordapp info causing deployNodes to fail.
2018-12-14 09:39:23 +00:00
Matthew Nesbit
767e37a34e
Apply some protective measures to prevent the reconnection test failing when rogue RPC clients on the team city server connect. (#4413) 2018-12-14 09:28:26 +00:00
Rick Parker
b70740cf2e
ENT-2785 Create clearer separation between RPC and P2P classes to make it easier to adjust logging levels independently. (P2P remained as is, some internal node RPC classes moved to existing node RPC package from generic messaging package, which is now just for P2P). (#4401) 2018-12-12 17:36:44 +00:00
Michele Sollecito
ccaf9dfbfa
[CORDA-2323]: Make detectPublicIp disabled by default. (#4404) 2018-12-12 13:38:57 +00:00
szymonsztuka
4799df9b80
CORDA-2150 signature constraints non-downgrade rule (#4262)
Contract class version non-downgrade rule is check by LedgerTransaction.verify().
TransactionBuilder.toWireTransaction(services: ServicesForResolution) selects attachments for the transaction which obey non downgrade rule.
New ServiceHub method loadAttachmentConstraint(stateRef: StateRef, forContractClassName: ContractClassName? = null) retrieves the attachment contract related to transaction output states of given contract class name.
2018-12-11 10:23:07 +00:00
Viktor Kolomeyko
1f8582e99f
CORDA-2115: Straighten-up @Before invocation chain. (#4379)
Note: similar sort of changes already made in Ent codebase that fix execution against external DBs.
it will make sense to backport those change to OS to avoid merge conflicts in the future.
2018-12-07 11:24:44 +00:00
Michele Sollecito
e2ea97bae7
[CORDA-2265]: Nodes hangs when using AppServiceHub to start a flow in a blocking way (fix) (#4376) 2018-12-06 15:13:32 +00:00
josecoll
63e326aedb
CORDA-2157 - Hash to Signature constraints migration V2 (#4261)
* Hash to signature constraints migration #1

* After rebase from Attachments Classloader commit.

* Simplified implementation without CZ whitelisting and relaxing the no-overlap rule slightly.

* Further simplification.

* Detailed implementation.

* Use fully loaded Cash contract jar for hash to signature constraints migration test.
Additional debug logging.

* Minor cleanup.

* Address PR review feedback.

* Minor fix.

* Fixes following rebase from master.

* Implemented `calculateEntriesHashes` to improve classloader validation performance.

* Address minor PR review comments.

* Added integration tests and some minor fixes.

* Minor fixes following rebase from master.

* Updates and fixes following integration testing.

* Added changelog entry.

* Fix broken unit tests.

* Fix compilation errors in DriverDSL tests after rebase from master.

* Minor fix to test cordapp jar signing using explicit keystore.

* Run hash-to-signature constraints integration test out of process using a non-validating notary.

* Address PR review feedback: contract version from database + other minor changes.

* Address final PR review feedback: remove signed attachment field from attachmentWithContext

* Resolve conflicts following rebase from master.

* Fix failing junit test.

* Fix Kryo serialization error (forgot to write new `version` identifier field)

* Removed redundant query carried over from previous commit.

* Added documentation.

* Fix test case where explicit Hash Constraint input and Signature Constraint output explicitly configured.

* Addressing PR review comments from SA.

* AttachmentQueryCriteria API: added wither methods and Java Unit tests.

* Fixed compilation error caused by Unit tests being in wrong module.

* Added @CordaInternal to canBeTransitionedFrom function.

* Minimized AttachmentClassloader overlap duplicates checking.

* Moved JarSignatureTestUtils and ContractJarTestUtils to internal pending clean-up and documentation before public release.

* Minor fix following rebase from master.

* Removed redundant checkNotNull(networkParameters) checks now that these are always passed into the main (non-deprecated) constructor.

* Remove capitalization.
2018-12-04 18:45:29 +00:00
Andrius Dagys
838c99c6e4 CORDA-2115: Notary whitelist verification changes (#4293)
* CORDA-2115: Notary whitelist verification changes

- For regular and contract upgrade transactions: check that the notary is in the network parameter whitelist
- For notary change transactions: check the the new notary is in the network parameter whitelist. This enabled support for network merging: the old notary doesn't have to be in the current network's notary whitelist for re-pointing old states to another notary.

These checks are done during transaction construction/verification and also by the non-validating notary.

* Address comments

* Remove stale todo

* Use notary whitelist of current network parameters for platform versoin 3

* Cleanup test

* Move `getHistoricNotary` to `HistoricNetworkParameterStorage` in `core.internal`

* Require `newNotary` to be notary on the network map during notary change
2018-12-04 13:54:24 +00:00
Shams Asari
9100636b8c
CORDA-2269: Removed DriverParameters.initialiseSerialization as it's not needed (#4321) 2018-11-29 13:53:22 +00:00