Commit Graph

8911 Commits

Author SHA1 Message Date
Ryan Fowler
5f8cceae45 Merge branch 'release/os/4.4' into rfowler-os-4.4-os-4.5-20200511 2020-05-11 11:40:09 +01:00
Chris Rankin
9a5be9e47d
CORDA-3738: Upgrade to DJVM 1.1-RC03. (#6219) 2020-05-05 17:32:39 +01:00
Ryan Fowler
b6dc1d8c4a
NOTICK - Reduce flow count for test to improve run time (#6213) 2020-05-05 09:38:55 +01:00
nargas-ritu
54e4503866
Merge pull request #6212 from corda/dan/update-byteman-dependency
NOTICK Update byteman dependencies
2020-05-04 16:29:29 +01:00
Rick Parker
cfb985244a
Merge pull request #6211 from corda/CORDA-3662-backport-from-4.5
ENT-5263: Backport CORDA-3662: Use an INNER JOIN for network map cache queries
2020-05-04 16:15:14 +01:00
Ryan Fowler
55797612b2 ENT-5237: Remove DISTINCT (backport) 2020-05-04 15:02:33 +01:00
LankyDan
96c1120194 NOTICK Update byteman dependencies
Tests that were using byteman when running on Java 11 were failing.
Updating the byteman version fixed the issue while still working on Java
8.
2020-05-04 14:08:38 +01:00
Denis Rekalov
32407b2c46
CORDA-3753: Increase Artemis security-invalidation-interval to avoid frequent CRL checks (#6207) 2020-05-04 13:00:22 +01:00
Chris Rankin
8dccd4231d
CORDA-3750: Delete unwanted methods from the deterministic artifacts. (#6208) 2020-05-04 11:33:06 +01:00
Chris Rankin
14d5751591
CORDA-3758: Ensure that providerMap is immutable, at the very least. (#6209) 2020-05-04 11:19:33 +01:00
Ryan Fowler
ebdd40049c CORDA-3662: Use an INNER JOIN for network map cache queries, (#6062)
- rename add or update function for clarity
- put removal of old nodes after retrieval of new ones to avoid gaps in the map
- plus add a test
2020-05-04 11:10:21 +01:00
nargas-ritu
5c16e361fd
Merge pull request #6186 from xavierlepretre/release/os/4.5
CORDA-3754 Make TransactionBuilder's input references deep copied on copy.
2020-05-01 09:54:18 +01:00
Xavier Leprêtre
c4fd63ced6 Reverted comment change about immutable accessors. 2020-04-30 18:46:14 +04:00
jmacmahonr3
b989cfce43
EG-153 - Adding sshd by default to docker node.conf template (#6171)
* Adding value needed for the rapid deployment of Azure UAT enviroments
* Parameterizing Rpc user
* Removing duplicate config requirement
* Forcing users to provide sshPort for node config generation with docker image
2020-04-30 15:03:08 +01:00
Chris Rankin
107819f5b5
CORDA-3745: Modify DJVM serializers to support Enum Evolution. (#6189) 2020-04-30 14:59:10 +01:00
Joseph Zuniga-Daly
2ce76e407d
ENT-5237: Remove DISTINCT because Oracle cannot apply DISTINCT to BLOB fields (#6203) 2020-04-30 14:27:45 +01:00
Dan Newton
4639a8f68b
Revert "NOTICK Ignore JUnit time threads in RPCStabilityTests (#6201)" (#6204)
This reverts commit ecf74053
2020-04-30 13:17:40 +01:00
Katelyn Baker
99fd08909d
EG-433 - Error Reporting Framework (Merge)
Error Reporting Framework - Phase 1
2020-04-30 13:12:54 +01:00
Chris Rankin
83dd9a96da
CORDA-3738: Upgrade to DJVM 1.1-RC02. (#6195)
* CORDA-3738: Upgrade to DJVM 1.1-RC02.

* Update comment for DJVM 1.1's new requirements.
2020-04-30 11:58:03 +01:00
Adel El-Beik
9bcb9b2f54
CORDA-3191: When querying for attachment ids now only retrieve the attachment id columm not whole object. (#6199) 2020-04-30 11:52:47 +01:00
James Higgs
7db25b8986
[NOTICK] Fix a test error to allow tests to pass in Jenkins (#6202) 2020-04-30 10:57:47 +01:00
Dan Newton
ecf74053c0
NOTICK Ignore JUnit time threads in RPCStabilityTests (#6201) 2020-04-30 10:50:54 +01:00
Dan Newton
cb84fd86ee
CORDA-3291 Fix broken tests (#6197) 2020-04-30 10:27:36 +01:00
Dan Newton
66f241efe8
CORDA-3291 Support for new flow sleep in mock networks (#6200)
The new flow sleep made `NotaryWhitelistTests` flaky which highlighted
an issue with the change to how a flow sleeps. Messages were being
pumped while the flow was sleeping which lead to inconsistent behaviour.
Messages are no longer pumped while a flow sleeps.
2020-04-30 10:18:45 +01:00
Razvan Codreanu
7309949abf
INFRA-311 new jdk11 jenkinsfile (#6194) 2020-04-30 09:59:35 +01:00
Adel El-Beik
3259b595d7
CORDA-3715: Check contract classes hav… (#6155)
* CORDA-3715: When loading cordapps now check that contract classes have class version between 49 and 52

* CORDA-3715: Now check class version when contract verification takes place.

* CORDA-3715: Making detekt happy with number of levels in func

* CORDA-3715: Make use of new ClassGraph release which provides class file major version number.

* CORDA-3715: Changed package name in test jars

* CORDA-3715: Use ClassGraph when loading attachments.

* CORDA-3715: Reverted file to 4.5 version

* CORDA-3715: Updating method to match non deterministic version.

* CORDA-3715: Added in default param.

* CORDA-3715: Adjusted min JDK version to 1.1

* CORDA-3715: Switching check to JDK 1.2

* CORDA-3715: Now version check SerializationWhitelist classes.

* CORDA-3715: Switched default to null for range.
2020-04-30 08:57:37 +01:00
Matthew Nesbit
75d10fe99c
Merge pull request #6190 from corda/nikinagy/CORDA-3747-publish-sources-for-node-driver
CORDA-3747 - switch on the publishing of sources
2020-04-29 17:27:19 +01:00
James Higgs
2febf13ea4
Merge pull request #6196 from corda/jamesh/error-reporting-sync-29-04-20
[MERGE] Update Error Reporting feature branch to latest release/os/4.5
2020-04-29 13:51:50 +01:00
James Higgs
1956bdb5d4 [MERGE] Fix detekt issues 2020-04-29 11:57:37 +01:00
James Higgs
adbe030a2c Merge branch 'release/os/4.5' into jamesh/error-reporting-sync-29-04-20
# Conflicts:
#	node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt
2020-04-29 11:47:14 +01:00
James Higgs
ab95aa57a2
[EG-440] Add some error codes and the error resource generation tool (#6192)
* [EG-438] First commit of error code interface

* [EG-438] Implement error reporter and a few error codes

* [EG-438] Add unit tests and default properties files

* [EG-438] Add the error table builder

* [EG-438] Update initial properties files

* [EG-438] Add some Irish tests and the build.gradle

* [EG-438] Fall back for aliases and use different resource strategy

* [EG-438] Define the URL using a project-specific context

* [EG-438] Tidy up initialization code

* [EG-438] Add testing to generator and tidy up

* [EG-438] Remove direct dependency on core and add own logging config

* [EG-438] Fix compiler warnings and tidy up logging

* [EG-438] Fix detekt warnings

* [EG-438] Improve error messages

* [EG-438] Address first set of review comments

* [EG-438] Use enums and a builder for the reporter

* [EG-438] Address first set of review comments

* [EG-438] Use enums and a builder for the reporter

* [EG-438] Add kdocs for error resource static methods

* [EG-440] Add error code for duplicate CorDapp loading

* [EG-438] Handle enums defined with underscores

* [EG-440] Add errors for some CorDapp loading scenarios

* [EG-440] Finish adding errors for CorDapp loading

* [EG-440] Fix up errors in properties files

* [EG-440] Start change to error code definition

* [EG-440] Update error code definition and add resource generation tool

* [EG-440] Tidy up error resource generation tool frontend

* [EG-440] Small refactorings and add kdocs

* [EG-440] Generate all missing resources

* [EG-440] Some refactoring and start writing a test

* [EG-440] Update unit test for resource generator

* [EG-440] Renaming of various parts of the error tool

* [EG-440] Add testing for errors and fix an issue in resource generation

* [EG-440] Add a kdoc for context provider API

* [EG-440] Remove old code from repository

* [EG-440] Address some review comments
2020-04-29 11:21:50 +01:00
Chris Rankin
592d7238fd
CORDA-3685: Prevent ServiceHub.signInitialTransaction from throwing undeclared checked exceptions. (#6111) 2020-04-28 17:02:37 +01:00
Dan Newton
297e504740
CORDA-3291 isKilled flag and session errors for killed flows (#6170)
* CORDA-3291 `isKilled` flag and session errors for killed flows

## Summary

Two major improvements have been worked on:

- A new flag named `isKilled` has been added to `FlowLogic` to allow
developers to break out of loops without suspension points.
- Killed flows now send session errors to their counter parties allowing
their flows to also terminate without further coordination.

Achieving these changes required a __fundamental__ change to how flows are
killed as well as how they sleep.

## `isKilled` flag

The addition of `FlowLogic.isKilled` allows flows to check if the
current flow has been killed. They can then throw an exception to lead
to the flow's termination (following the standard error pathway). They
can also perform some extra logic or not throw an exception if they
really wanted to.

No matter what, once the flag is set, the flow will terminate. Due to
timing, a killed flow might successfully process its next suspension
event, but it will then process a killed transition and terminate.

## Send session errors when killing a flow

A flow will now send session errors to all of its counter parties. They
are transferred as `UnexpectedFlowEndException`s. This allows initiated
flows to handle these errors as they see fit, although they should
probably just terminate.

## How flows are killed

### Before

Originally we were relying on Quasar to interrupt a flow's fiber, we
could then handle the resulting `InterruptedException`. The problem with
this solution is that it only worked when a flow was already suspended
or when a flow moved into suspension. Flows stuck in loops did not work.

### After

We now *do not* use Quasar to interrupt a flow's fiber. Instead, we
switch `FlowStateMachine.isKilled` to true and schedule a new event.
Any event that is processed after switching this flag will now cause a
`KilledFlowTransition`. This transition follows similar logic to how
error propagation works. Note, the extra event allows a suspended flow
to be killed without waiting for the event that it was _really_ waiting
for.

This allows a lot of the tidy up code in `StateMachineManager.killFlow`
to be removed as tidy up is executed as part of removing a flow.
Deleting a flow's checkpoint and releasing related soft locks is still
handled manually in case of infinite loops but also triggered as part
of the actions executed in a transition.

This required flow sleeping to be changed as we no longer rely on
quasar.

## How flows now sleep

The reliance on Quasar to make a flow sleep has been removed.

Instead, when a flow sleeps we create a `ScheduledFuture` that is
delayed for the requested sleep duration. When the future executes it
schedules a `WakeUpFromSleep` event that wakes up the flow... Duh.

`FlowSleepScheduler` handles the future logic. It also uses the same
scheduled thread pool that timed flows uses.

A future field was added to `StateMachineState`. This removes the 
need for concurrency control around flow sleeps as the code path does
not need to touch any concurrent data structures.

To achieve this:

- `StateMachineState.future` added as a `var`
- When the `ScheduledFuture` is created to wake up the flow the passed
in `StateMachineState` has its `future` value changed
- When resumed `future` and `isWaitingForFuture` are set to `null` and
`false` respectively
- When cancelling a sleeping flow, the `future` is cancelled and nulled
out. `isWaitingForFuture` is not changed since the flow is ending anyway
so really the value of the field is not important.
2020-04-28 15:53:44 +01:00
Chris Rankin
1c3ec2eb18
CORDA-3738: Upgrade to DJVM 1.1-RC01. (#6182)
* Update for changes to API for DJVM 1.1-RC01.

* Tidy up generics for DJVM serialization.
2020-04-28 15:43:14 +01:00
James Higgs
ab43238420
[EG-438] Error Reporting Framework (#6125)
* [EG-438] First commit of error code interface

* [EG-438] Implement error reporter and a few error codes

* [EG-438] Add unit tests and default properties files

* [EG-438] Add the error table builder

* [EG-438] Update initial properties files

* [EG-438] Add some Irish tests and the build.gradle

* [EG-438] Fall back for aliases and use different resource strategy

* [EG-438] Define the URL using a project-specific context

* [EG-438] Tidy up initialization code

* [EG-438] Add testing to generator and tidy up

* [EG-438] Remove direct dependency on core and add own logging config

* [EG-438] Fix compiler warnings and tidy up logging

* [EG-438] Fix detekt warnings

* [EG-438] Improve error messages

* [EG-438] Address first set of review comments

* [EG-438] Use enums and a builder for the reporter

* [EG-438] Add kdocs for error resource static methods

* [EG-438] Handle enums defined with underscores

* [EG-438] Slight refactoring of startup code

* [EG-438] Port changes to error reporting code from future branch

* [EG-438] Also port test changes

* [EG-438] Suppress a deliberately unused parameter
2020-04-28 14:07:50 +01:00
nikinagy
274b958dad switch on the publishing of sources 2020-04-28 11:43:07 +01:00
Dan Newton
9a2ae8ae19
CORDA-3722 withEntityManager can rollback its session (#6187)
* CORDA-3722 withEntityManager can rollback its session

## Summary

Improve the handling of database transactions when using
`withEntityManager` inside a flow.

Extra changes have been included to improve the safety and
correctness of Corda around handling database transactions.

This focuses on allowing flows to catch errors that occur inside an
entity manager and handle them accordingly.

Errors can be caught in two places:

- Inside `withEntityManager`
- Outside `withEntityManager`

Further changes have been included to ensure that transactions are
rolled back correctly.

## Catching errors inside `withEntityManager`

Errors caught inside `withEntityManager` require the flow to manually
`flush` the current session (the entity manager's individual session).
By manually flushing the session, a `try-catch` block can be placed
around the `flush` call, allowing possible exceptions to be caught.

Once an error is thrown from a call to `flush`, it is no longer possible
to use the same entity manager to trigger any database operations. The
only possible option is to rollback the changes from that session.
The flow can continue executing updates within the same session but they
will never be committed. What happens in this situation should be handled
by the flow. Explicitly restricting the scenario requires a lot of effort
and code. Instead, we should rely on the developer to control complex
workflows.

To continue updating the database after an error like this occurs, a new
`withEntityManager` block should be used (after catching the previous
error).

## Catching errors outside `withEntityManager`

Exceptions can be caught around `withEntityManager` blocks. This allows
errors to be handled in the same way as stated above, except the need to
manually `flush` the session is removed. `withEntityManager` will
automatically `flush` a session if it has not been marked for rollback
due to an earlier error.

A `try-catch` can then be placed around the whole of the
`withEntityManager` block, allowing the error to be caught while not
committing any changes to the underlying database transaction.

## Savepoints / Transactionality

To make `withEntityManager` blocks work like mini database transactions,
save points have been utilised. A new savepoint is created when opening
a `withEntityManager` block (along with a new session). It is then used
as a reference point to rollback to if the session errors and needs to
roll back. The savepoint is then released (independently from
completing successfully or failing).

Using save points means, that either all the statements inside the
entity manager are executed, or none of them are.

## Some implementation details

- A new session is created every time an entity manager is requested,
but this does not replace the flow's main underlying database session.
- `CordaPersistence.transaction` can now determine whether it needs
to execute its extra error handling code. This is needed to allow errors
escape `withEntityManager` blocks while allowing some of our exception
handling around subscribers (in `NodeVaultService`) to continue to work.
2020-04-28 11:20:00 +01:00
nikinagy
ec96a844bd
CORDA-3659 - adding kdoc for RestrictedEntityManager and RestrictedConnection (#6179)
* adding kdoc for RestrictedEntityManager and RestrictedConnection

* adding kdoc for RestrictedEntityManager and RestrictedConnection
2020-04-28 09:35:47 +01:00
Rick Parker
e6e1caa58a
Merge pull request #6185 from corda/nnagy-os4.4-os4.5-20200426-2
os4.4->os4.5 2020-04-26
2020-04-28 09:02:36 +01:00
Xavier Leprêtre
9396ee9551 Make TransactionBuilder's input references deep copied on copy. 2020-04-25 22:50:09 +04:00
Denis Rekalov
6581815ec2
Merge pull request #6183 from corda/denis/ENT-5222-tls-handshake-error
ENT-5222: Prevent from blocking connections after transport failure on TLS handshake
2020-04-24 18:40:26 +01:00
Joseph Zuniga-Daly
e618416861 Move files to their new home 2020-04-24 17:11:15 +01:00
nikinagy
ee7700e28f Add Hibernate UTC Fix 2020-04-24 17:00:10 +01:00
nikinagy
8eab8653cd Merge branch 'release/os/4.4' of https://github.com/corda/corda into nnagy-os4.4-os4.5-20200426-2
 Conflicts:
	node-api/src/main/kotlin/net/corda/nodeapi/internal/persistence/HibernateConfiguration.kt
	node-api/src/main/kotlin/net/corda/nodeapi/internal/rpc/client/AMQPClientSerializationScheme.kt
2020-04-24 16:57:57 +01:00
Matthew Nesbit
8eda8b744f
Merge pull request #6181 from corda/nnagy-os-4.3-os-4.4-20200423
Nnagy os 4.3 os 4.4 20200423
2020-04-24 15:00:34 +01:00
Denis Rekalov
c86249adb3 ENT-5222: Prevent from blocking further connections after transport failure on TLS handshake 2020-04-24 13:59:19 +01:00
Matthijs van den Bos
a298a94960
Load CordaServices before NotaryService (#6173)
On node start, load CordaServices before starting the NotaryService,
so that the NotaryService can check that the services it requires are
available when starting.

Resolves #6172.
2020-04-24 09:15:38 +01:00
nikinagy
335372cfc1 Merge branch 'release/os/4.3' into nnagy-os-4.3-os-4.4-20200423 2020-04-23 16:03:17 +01:00
Joseph Zuniga-Daly
69a4f80cd2
ENT-5141: Fix ConcurrentModificationException in FetchDataFlow (#6176)
* ENT-5141: Fix ConcurrentModificationException in FetchDataFlow

* Make detekt happy

* Make CheckpointSerializationEnvironmentRule inheritable
2020-04-23 13:11:23 +01:00
Denis Rekalov
824c01daad
ENT-5219: Synchronize BCCryptoService between OS and ENT (#6178) 2020-04-23 12:44:27 +01:00