3269 Commits

Author SHA1 Message Date
williamvigorr3
9d4d128f4e
CORDA-3597 add missed data to checkpoint class (#5995)
* Add extra fields to StateMachineState

* Move structures into checkpoint as this is a more natural place.
2020-02-26 10:48:52 +00:00
Viktor Kolomeyko
275e801094
ENT-4955: Hide usage of FlowHospital inside StateMachineManager (#5997) 2020-02-25 14:10:59 +00:00
Adel El-Beik
fe625d0f37
CORDA-3584: Now cope with 2 contract jars with same hash but different name (#5952)
* CORDA-3484: Now cope with 2 contract jars with same hash but different name, we just select one and use that.

* ENT-3584: Contract jars are now generated on the fly.

* CORDA-3584: Reverted changes to CordappProviderImpl. Exception is raised if node started with multiple jars with same hash.

* ENT-3584: Fixing test failure.

* CORDA-3584: Switch to test extension method instead of reflection to access internal member.

* ENT-3584: Address review comment. Dont fully qualify exception.

* CORDA-3584: Address review comment and converted lazy to a resettable one.

* CORDA-3584: Removed unused logger.

* CORDA-3584: Fixed visibility.

* CORDA-3584: Removed synchronized

* CORDA-3584: Removed CordappResolver

* CORDA-3584: Reverted change in gradle file and fixed test.

* CORDA-3584: Removed V3 from test description as it wasn't actually V3 specific.

* CORDA-3584: Address review comment. Let classes be garbage collected.
2020-02-25 11:18:01 +00:00
Kyriakos Tharrouniatis
2c9c2985c0
CORDA-3381: Errors in vault updates publisher are unsubscribing stopping observers from working (#5912)
* Throw SQLException or PersistenceException plain, that may come out of an unsafe subscriber

* Add explanatory comment about why we changed Observer.tee to use unsafe subscribe

* Introducing not unsubscribing version of Rx.Subscriber

* Wrap PublishSubjects with FlowSafeSubjects in all tests that test Observer.tee

* Minor code formatting

* Make rawUpdates Rx.Observers not unsubscribe when accessed from CordaServices - Do not allow rawUpdates subscribing from flows

* Warning fix: Add else block to when statement

* Revert "Wrap PublishSubjects with FlowSafeSubjects in all tests that test Observer.tee"

This reverts commit e419af86

* Correcting log message

* Improve log message

* Add fiber's id to log message and exception message

* Added test, asserting FlowSafeSubscriber is alive and re-accessed upon flow retry

* Logging flow name instead of flow id at VaultService.rawUpdates subscribing error

* Add kdoc to OnNextFailedException

* Minor text correction

* Update kdocs of FlowSafeSubject/ PreventSubscriptionsSubject

* Moved FlowSafeSubject under package node.internal as it is only used by NodeVaultService

* Add comment and update kdoc explaining how to subscribe with SafeSubscriber to FlowSafeSubject

* Change PreventSubscriptionsSubject#errorAction to be more specific; to return an Exception

* Minor text update

* Update messy comment

* Replace assertThat with assertEquals

* Splitting heartBeat to heartBeat1 and hearBeat2 for more clear asserting

* Correcting comment

* Update messy comment

* Splitting heartBeat into heartBeatOnNext and heartBeatOnError

* Update test name

* Add explanatory comment to test

* Update test name

* Update test and add test comment

* Moving NotarisedTxs from SendStateFlow to VaultObserverExceptionTest inside NodeHandle.getNotarisedTransactionIds

* Moving SubscribingRawUpdatesFlow from ErrorHandling to VaultObserverExceptionTest

* Update kdoc of FlowSafeSubscriber and FlowSafeSubscriber.onNext

* Make kdoc more clear

* Throw exception upon accessing VaultService.rawUpdates from within a flow

* Changing exception thrown when accessing VaultService.rawUpdates from within a flow to a CordaRuntimeException

* Minor kdoc update

* Update test comment

* Update kdoc of FlowSafeSubscriber

* Introducing Observable.flowSafeSubscribe public API method to subscribe with -non unsubscribing- Rx.Subscribers to Observables. It also replaced FlowSafeSubject

* Move CustomSafeSubscriber outside test methods

* Minor text update

* Add timeout to tests

* Update kdoc of flowSafeSubscribe

* Update kdoc of flowSafeSubscribe

* Update kdoc of flowSafeSubscribe

* Move FlowSafeSubscriber and flowSafeSubscribe under their own package

* Fix detekt issue

* Update Detekt baseline

* Revert "Update Detekt baseline"

This reverts commit 793a8ed9

* Fix Detekt issue

* Moved strictMode flag from flowSafeSubscribe to OnFlowSafeSubscribe

Moved OnFlowSafeSubscribe into internal package

Integration tested flowSafeLooseSubscribe

* Suppress Rx Deprecation

* Rename flowSafeSubscribe to flowSafeObservable

* Renaming flowSafeObservable to continueOnError and FlowSafeSubscriber to ResilientSubscriber
2020-02-21 10:02:34 +00:00
Chris Rankin
4c6f15c40c Merge commit '88008b71f03a539749e32e5e76b2dc5ea98f897f' into chrisr3-44-merge 2020-02-19 14:12:10 +00:00
Chris Rankin
afcab51fe3 Update "in-process" test on basis it can be made to work. 2020-02-19 10:42:44 +00:00
Chris Rankin
d89ce6608a Create a unit test for serialization whitelists via driver. 2020-02-19 10:42:34 +00:00
williamvigorr3
8d1b6cf499
CORDA-3432 update structure of checkpoint class (#5983)
* Split StateMachine State into 2 classes

The idea is this better reflects the database structure. Added a
few helper methods to copy and update state.

* Doc + Improve Checkpoint API

* Rename methods to be more clear
2020-02-18 15:31:46 +00:00
Chris Rankin
73e6a40158 Merge commit '21325dfa4fb8151e77f368e9356c16ae89c70aa3' into chrisr4-44-merge 2020-02-15 14:45:50 +00:00
James Higgs
3547b629c3
[NOTICK] Add a detekt rule to catch tests with no timeout (#5959)
* [NOTICK] Add a custom detekt rule for tests with no timeout, and fix remaining missing timeouts

* [NOTICK] Add a test for custom detekt rules and tidying

* add timeout annotation to new test

Co-authored-by: Stefano Franz <roastario@gmail.com>
2020-02-14 17:31:02 +00:00
James Higgs
da192bcf0d
[EG-140] Allow system property paths with multiple keys to be specified in node.conf (#5963)
* [EG-140] Allow system property paths with multiple keys to be specified in node.conf

* [EG-140] Split property paths to remove quotes

* [EG-140] Quote system properties in docs

* [EG-140] Rename path to key
2020-02-14 11:18:21 +00:00
stefano
13c52e4901 Merge remote-tracking branch 'origin/feature/checkpoint_table_improvements' into feature/checkpoint_table_improvements 2020-02-13 12:08:07 +00:00
Schife
a765510cf4 Merge branch 'release/os/4.4' of https://github.com/corda/corda into TM-197-4.5Merge 2020-02-12 14:33:32 +00:00
Razvan Codreanu
c6ab07d92c
TM-197 Fix failing test caused by regression and ignoring flaky DistributedServiceTests (#5934)
* TM-197 Setting bouncy castle provider in order for the test to pass

* TM-197 setting timeout for all builds at 3 hours

* TM-197 ignoring unstable tests
2020-02-12 14:26:06 +00:00
Razvan Codreanu
42eca48a02
TM-197 Build stability changes (#5947)
* TM-197 switching 4.3 to use local k8s instances and also make the maximum duration of builds 3 hours, fix 1 test and ignore 2 flaky ones

* update to use local-k8s version of the plugin

Co-authored-by: Stefano Franz <roastario@gmail.com>
2020-02-12 13:58:30 +00:00
Denis Rekalov
1705299bde ENT-4918: Support symbolic link for certificates directory during node registration 2020-02-11 13:14:34 +00:00
Stefano Franz
4a54374f86
Merge pull request #5940 from corda/release/os/4.4
Merge OS 4.4 -> OS 4.5
2020-02-11 11:55:53 +00:00
Stefano Franz
b23af5f0d2
add timeouts to all tests (#5875) 2020-02-11 10:14:05 +00:00
stefano
9e8ce6473d address dan review comments pt2 2020-02-10 13:22:51 +00:00
stefano
a28c15c2fd address dan review comments 2020-02-10 11:52:54 +00:00
stefano
546166e057 Merge branch 'release/os/4.4' of github.com:corda/corda into new_checkpoint_schema 2020-02-10 11:25:29 +00:00
Viktor Kolomeyko
90df56c173
CORDA-3565: Port ServiceStateSupport from ENT to OS (#5916)
* CORDA-3565: `ServiceStateSupport` and supporting classes

* CORDA-3565:Plug `ServiceLifecycleSupport` into `MessagingService`

* CORDA-3565: Detekt baseline update

* CORDA-3565: React to MessagingServer going up and addition logging for up/down

Co-authored-by: Matthew Nesbit <matthew.nesbit@r3.com>
2020-02-03 09:47:12 +00:00
Kyriakos Tharrouniatis
9ca1dd59da
CORDA-3381 Make internalUtilsKt Observer tee not wrap with SafeSubscriber (#5913)
* Make tee not wrap PublishSubjects in SafeSubscribers, otherwise a non Rx exception from an unsafe observer shuts down all other observers under the same PublishSubject

* Throw SQLException or PersistenceException plain, that may come out of an unsafe subscriber

* Revert "Throw SQLException or PersistenceException plain, that may come out of an unsafe subscriber"

This reverts commit c7b8af3fa6f94c5451924f5ba8c287e6472a39a7.

* Update Detekt baseline
2020-01-31 12:32:59 +00:00
Florian Friemel
cb6ed6042c
Move jimfs dependency from node to test-utils. (#5911) 2020-01-31 10:55:05 +00:00
stefano
eec9ada6ba persist some nonsense data to see if it breaks the statemachine 2020-01-30 18:18:45 +00:00
jmacmahonr3
9511329e3d
[EG-75] Not handled exception when NetworkMap not accessible (#5903)
* Just passing in the exception was printing the entire stacktrace to the log, now we just pring the exception and message

* Updating exception message
2020-01-30 17:26:22 +00:00
stefano
2b079bd92e working node startup with new tables and entities 2020-01-30 12:17:26 +00:00
Christian Sailer
cac96d81bf Remove unused dependency on IntelliJ forms 2020-01-30 10:48:00 +00:00
Oliver Knowles
286fffd67a
ENT-4368 - Improve error handling when required included config resource cannot be found. Add note in docs encouraging use of required syntax for includes. (#5907) 2020-01-30 10:10:24 +00:00
stefano
7b3da95456 flesh out entities for new checkpointing 2020-01-29 17:10:34 +00:00
Ryan Fowler
d16e1126db
CORDA-3542: Use the config values for reconnecting retry interval and max reconnect attempts (#5869) 2020-01-29 13:08:54 +00:00
stefano
78d83e9583 start adding hibernate entities 2020-01-28 16:55:14 +00:00
stefano
32bdd9728c Merge branch 'release/os/4.4' of github.com:corda/corda into new_checkpoint_schema 2020-01-28 16:05:29 +00:00
Nick Dunstone
a4cada4a2e ENT-4474 O/S Changes to support bulk backchain fetching (#5894)
* Ongoing dev multi backchain fetch

* Final dev pre logging message improvements

* Trace messages

* Trace messages

* Code tidy up

* Code review comments

* Code review comments

* Whitespace removed

* Code review changes

* Code reformatting
2020-01-28 09:04:18 +00:00
Ryan Fowler
e8d6701f87 CORDA-3558: Allow initial registration errors to propagate up so the node exits with a failure code (#5899) 2020-01-27 15:23:07 +00:00
stefano
a3bc624e16 add message to exceptions table 2020-01-27 12:19:14 +00:00
stefano
47c14673c4 address initial review comments 2020-01-27 11:20:13 +00:00
Kyriakos Tharrouniatis
7f62046c2f CORDA-3329 Exceptions thrown in raw vault observers can cause critical issues (#5816)
Observers registered on NodeVaultService#rawUpdates, if they throw an exception when called from serviceHub#recordTransactions and if this exception is not handled by the flow hospital, then this leads to the transaction not being recorded in the local vault. This could get the ledger in an out of sync state.

In the specific case this happens within FinalityFlow#notariseAndRecord this leads to the transaction being notarized but not recorded in the local vault nor broadcasted in any counter party. The -failed to be recorded locally- transaction and its output states are not visible to any vault, and its input states not able to consumed by a new transaction, since they are recorded as consumed within the Notary. In this specific case we need not loose, by any means, the current transaction.

We will handle all cases by catching all exceptions thrown from serviceHub#recordTransactions, wrapping them with a HospitalizeFlowException and throwing it instead. The flow will get to the hospital for observation to be retried from previous checkpoint on next node restart.
2020-01-24 18:21:07 +00:00
Viktor Kolomeyko
a4d63d1329 CORDA-2942: Remove multiple reps from the test (#5893)
... as it is proven to be stable now.
2020-01-24 16:57:31 +00:00
Dan Newton
ecc3b5e330 CORDA-3554 Use flowId from ExternalMessageEvent when failing to init sessions (#5890)
Use `flowId` from `ExternalMessageEvent` when failing to init sessions instead of generating
a new random UUID. The a `flowId` is generated and stored inside the event after the
state machine work that done previously.
2020-01-24 16:34:29 +00:00
stefano
5750c39348 start work on liquibase schema for new checkpoints 2020-01-24 13:24:08 +00:00
Dan Newton
4f1777adb4 NO-TICK Cap the default size of the external operation thread pool (#5878)
Cap the default size of the external operation thread pool to 10 or
the maximum number of available processors, whichever is smaller.

Set the minimum size of the thread pool to 1. Meaning that only a
single thread is used unless the node actually starts to use
`FlowExternalOperation` which consumes threads from this pool.
2020-01-24 12:11:53 +00:00
stefano
0619b4ce90 add 2020-01-24 12:02:16 +00:00
Viktor Kolomeyko
42a2ed98e2 CORDA-2942: Allow exception from CordaService creation to propagate (#5884)
* CORDA-2942: Allow exception from `CordaService` creation to propagate

It will ultimately be thrown from Node's `start()` method terminating the node start-up sequence.

* CORDA-2942: Be lenient when retrievign the name of the Notary

Some tests setup such that they do nto have Notary running.
2020-01-24 10:20:08 +00:00
Christian Sailer
1380779a9c
Remove unused schema options (#5885) 2020-01-24 08:49:59 +00:00
Viktor Kolomeyko
b72f71e7ac CORDA-3549: Re-write flaky CordaServiceLifecycleFatalTests (#5879)
* CORDA-3549: Improve stability of `CordaServiceLifecycleFatalTests`

* CORDA-3549: Bump-up reps count to ensure that test is definitely not flaky when executed by CI
(once proved the number of reps will be reduced)

* CORDA-3549: Making Detekt happier

* CORDA-2942: Ensure `NodeLifecycleEventsDistributor` cleans-up smoothly when node shuts down
2020-01-23 16:35:49 +00:00
James Higgs
9a1d46ddd1 EG-73 - Remove postgres dependency from the node and update docs (#5867) 2020-01-22 09:52:02 +00:00
Dan Newton
4bae045a58 ENT-4601 Public API to run external operations from a flow (#5833)
Deprecate FlowAsyncOperation and reimplement public versions FlowExternalOperation and FlowExternalAsyncOperation.

await added to FlowLogic to allow easy calling from both Java and Kotlin. There are two overrides of await (one for FlowExternalOperation and FlowExternalAsyncOperation).

Implementations of FlowExternalOperation return a result (written as blocking code) from their execute function. This operation will then be executed using a thread provided by the externalOperationExecutor.

Implementations of FlowExternalAsyncOperation return a future from their execute function. This operation must be executed on a newly spawned thread or one provided by a thread pool. It is up to developers to handle threading in this scenario.

The default thread pool (externalOperationExecutor) can be configured through the flowExternalOperationThreadPoolSize node config.

The current implementation leaves FlowAsyncOperation alone, meaning that any developers that have used it (even though it is internal) won't need to change their apps. If this was not concern I would delete it completely and replumb the state machine code. Instead, it has been marked with @DoNotImplement and executeAsync is annotated with @Deprecated
2020-01-22 09:27:17 +00:00
Viktor Kolomeyko
0978500a9a CORDA-2942: Node lifecycle events (#5846)
* CORDA-2942: Port minimal set of changes to make lifecycle events work

... and make codebase compile.

* CORDA-2942: Undo some changes which are not strictly speaking necessary

* CORDA-2942: Make `NodeServicesContext` leaner and delete `extensions-api` module

* CORDA-2942: Reduce even more number of files affected

* CORDA-2942: Integration test fix

* CORDA-2942: Make events `AfterStart` and `BeforeStop` generic w.r.t. `NodeServicesContext`

* CORDA-2942: `NodeLifecycleObserverService` and a set of integration tests.

Public API violations are expected as well as integration tests failing.

* CORDA-2942: Re-work to introduce `ServiceLifecycleObserver`

* CORDA-2942: Explicitly mention a type of exception that may be thrown for some events.

* CORDA-2942: Register `ServiceLifecycleObserver` through `AppServiceHub`

* CORDA-2942: Fix integration test + KDocs update

* CORDA-2942: Detekt and `api-current` update

* CORDA-2942: Improvement to `CordaServiceLifecycleFatalTests`

... or else it has side effects on other tests.

* CORDA-2942: Add an integration test for new API use in Java

Driver test is written in Kotlin, but services definition is written in Java.

Also KDocs improvements.

* CORDA-2942: Documentation and release notes update

* CORDA-2942: First set of changes following review by @mnesbit

* CORDA-2942: Second set of changes following review by @mnesbit

* CORDA-2942: Added multi-threaded test

* CORDA-2942: Fixes

* CORDA-2942: Undo changes to `api-current.txt`

* CORDA-2942: Bare mimimum change to `api-current.txt` for CI gate to pass.

* CORDA-2942: Address review feedback from @rick-r3

* CORDA-2942: Detekt update

* CORDA-2942: Delete `ServiceLifecycleObserverPriority` and replace it with `Int` after discussion with @mnesbit

* CORDA-2942: Introduce more `NodeLifecycleEvent` and switch services to listen for those events

* CORDA-2942: Few more changes after input from @rick-r3

* First stub on integration test
Unfinished - hang on issue and pay

* CORDA-2942: Switch to use out-of-process nodes for the inetgration test

Currently Alice and Notary stuck waiting to hear from each other.

* CORDA-2942: Extra log lines during event distribution

* CORDA-2942: Asynchronously distribute lifecycle events

* CORDA-2942: Await for complete P2P client start-up

Next step: Add vault query to integration test

* CORDA-2942: Asynchronously distribute lifecycle events

Next step: Improve integration test

* CORDA-2942: Fix test broken by recent changes and improve logging

* CORDA-2942: Improvement of the test to be able to monitor actions performed by @CordaService in the remote process

* CORDA-2942: Add node re-start step to the integration test

* CORDA-2942: Remove `CORDAPP_STOPPED` event for now

* CORDA-2942: s/CORDAPP_STARTED/STATE_MACHINE_STARTED/

* CORDA-2942: Inverse the meaning of `priority` as requested by @rick-r3

* CORDA-2942: Register `AppServiceHubImpl` for lifecycle events and put a warning when SMM is not ready.
2020-01-21 13:38:02 +00:00
Chris Rankin
a4d00b79d4 CORDA-3377: Upgrade to DJVM 1.0-RC06. (#5866) 2020-01-21 09:05:23 +00:00