Commit Graph

2804 Commits

Author SHA1 Message Date
Chris Rankin
3f67e314c0 Merge commit 'ef00fa1388db37e155ab8cfed3763c14801f8aa9' into chrisr3-45-merge 2020-06-12 13:14:44 +01:00
James Higgs
6e349f298e
NOTICK - Ignore a potentially dodgy test (#6336) 2020-06-11 16:47:48 +01:00
James Higgs
ab023d0b07 Merge branch 'release/os/4.5' into jamesh/os-4.5-4.6-merge-11062020 2020-06-11 09:40:39 +01:00
James Higgs
58af87c988
EG-2225 - Create log directory in correct place with verbose flag set (#6321)
* Ensure logs directory is written to correct location
* Remove a superfluous set of log path property
* Add a unit test to catch bad log paths
* Address detekt issues
2020-06-10 10:46:57 +01:00
James Higgs
8b7275eb97
EG-2564 - Move printed error to logger (#6323) 2020-06-10 10:45:50 +01:00
Schife
fb184839f4 Merge branch 'release/os/4.5' of https://github.com/corda/corda into razvan-os-4.5-to-4.6-merge 2020-06-05 07:55:48 +01:00
nikinagy
caf5482244
ENT-4064, ENT-4608 - checking for unsigned cordapps in prod mode and checking the PV (#6291)
* checking for unsigned cordapps in prod mode and shutting down if it the contracts are not signed

* inheriting from CordaRuntimeException instead of Exception

* had the same tests twice, removed the shutdown for minimumplatformversion, modified some of the tests

* shut down when we get invalid platformversion and modified the test according to this

* making the message more accurate
2020-06-04 16:24:49 +01:00
Denis Rekalov
45614cf29e
Merge pull request #6266 from corda/denis/CORDA-3805-custom-migration-scripts
CORDA-3805: cut dependency from PersistentIdentityService for custom migration scripts
2020-06-04 14:20:26 +01:00
Dan Newton
f0d2c9fe71
NOTICK Correct StatemachineGeneralErrorHandlingtest (#6306) 2020-06-04 08:04:41 +01:00
Kyriakos Tharrouniatis
b8b462f68e
NOTICK Fix schema validation error for flow parameters (#6304)
Adding 'corda-blob' type, fixed 'Schema-validation: wrong column type'
2020-06-02 17:52:50 +01:00
williamvigorr3
0554c98d18
NOTICK Restrict to shell commands (#6303)
Remove shell command for pausing flows
2020-06-02 16:04:10 +01:00
James Higgs
04ddb267fd
[EG-2225] Prevent extra directories being created when relative base directories are specified (#6282)
Don't create an extra directory if a relative base path specified
2020-06-02 15:26:40 +01:00
Tamas Veingartner
9c4a76d367
CORDA-3176 Inefficient query generated on vault queries with custom p… (#6241)
* CORDA-3176 Inefficient query generated on vault queries with custom paging

a check added to avoid self joins
local postgres db tests were executed on large volume test data (50k states) to ensure that the DB optimizes the suspicious query and so it does not cuase performance issues.
A test added to check that a pagination query on large volume of sorted data is completed in a reasonable time

* foreach detekt fix
2020-06-02 10:33:59 +01:00
Denis Rekalov
98af7f10f9 CORDA-3805: Make custom migration scripts independent on the latest PersistentIdentityService schema 2020-06-01 16:22:21 +01:00
Rick Parker
9f2bd1dcae
Merge pull request #6295 from corda/feature/checkpoint_table_improvements
CORDA-3432 Feature/checkpoint table improvements
2020-06-01 11:31:13 +01:00
Kyriakos Tharrouniatis
4507b55857
CORDA-3725 Fix SQL deadlocks coming from soft locking states (#6287)
Adding to the query -explicitly- the flow's locked states resolved the SQL Deadlocks in SQL server. The SQL Deadlocks would come up  from `softLockRelease` when only the `lockId` was passed in as argument. In that case the query optimizer would use `lock_id_idx(lock_id, state_status)` to search and update entries in `VAULT_STATES` table. 

However, all rest of the queries would follow the opposite direction meaning they would use PK's `index(output_index, transaction_id)` but they would also update the `lock_id` column and therefore the `lock_id_idx` as well, because `lock_id` is a part of it. That was causing a circular locking among the different transactions (SQL processes) within the database.

To resolve this, whenever a flow attempts to reserve soft locks using their flow id (remember the flow id is always the flow id for the very first flow in the flow stack), we then save these states to the fiber. Then, upon releasing soft locks the fiber passes that set to the release soft locks query. That way the database query optimizer will use the primary key index of VAULT_STATES table, instead of lock_id_idx in order to search rows to update. That way the query will be aligned with the rest of the queries that are following that route as well (i.e. making use of the primary key), and therefore its locking order of resources within the database will be aligned with the rest queries' locking orders (solving SQL deadlocks).

* Fixed SQL deadlocks caused from softLockRelease, by saving locked states per fiber; NodeVaultService.softLockRelease query then uses VAULT_STATES PK index instead of lock_id_idx

Speed up SQL server by breaking down queries with > 16 elements in their IN clause, into sub queries with 16 elements max in their IN clauses

* Allow softLockedStates to remove states

* Add to softLockedStates only states soft locked under our flow id

* Fix softLockRelease not to take into account flowStateMachineImpl.softLockedStates when using lockId != ourFlowId

* Moved CriteriaBuilder.executeUpdate at the bottom of the file
2020-05-29 12:35:05 +01:00
Denis Rekalov
499c09e77b
Merge pull request #6285 from corda/denis/CORDA-3818-sync-identity-service
CORDA-3818: Synchronize OS implementation of PublicKeyToOwningIdentityCache with CE
2020-05-28 14:21:04 +01:00
Kyriakos Tharrouniatis
598228634f
CORDA-3608 Performance tune the new checkpoint schema (#6277)
Performance tuning of the new checkpoint schema;

- Checkpoint tables are now using `flowId` as join keys. 
- Indexes consist of a PK's index on `node_checkpoints(flow_id)` and then unique indexes on `node_checkpoint_blobs(flow_id)` and `node_flow_metadata(flow_id)`.
- Serialization of `checkpointState` is being done with `CHECKPOINT_CONTEXT` so that we can have compression. This is needed when messages get passed into `checkpointState.sessions` therefore `checkpointState` grows in size upon serialized and 
saved into the database.

* Deserialize checkpointState with CHECKPOINT_CONTEXT

* Align tests with schema update; We cannot add and update a checkpoint in the same session now, ends up with hibernate complaining: two different objects with same identifier

* Fix indentation and format

* Ignore tests that assert DBFlowResult or DBFlowException

* Set DBFlowCheckpoint.blob to null whenever the flow errors or hospitalizes; this way we save an extra SELECT in such cases;

* Fix test; cleared Hibernate session, it would fail at checkpoint with 'org.hibernate.NonUniqueObjectException'

* Changing VARCHAR to NVARCHAR

* Rename v17 liquibase scripts to v19 to resolve collision with ENT v17 scripts
2020-05-27 16:46:56 +01:00
Chris Rankin
6e156bc0ea Merge commit '6ebc6e9b16575a7afbb781b0d93a8f04b3affba5' into chrisr3-45-merge 2020-05-27 10:40:36 +01:00
Denis Rekalov
5afdb63c94 CORDA-3818: Synchronize OS implementation of PublicKeyToOwningIdentityCache with CE 2020-05-26 16:10:14 +01:00
Chris Rankin
6ebc6e9b16
CORDA-3750: Reimplement Corda's Crypto object for use inside the sandbox. (#6193)
* CORDA-3750: Use hand-written sandbox Crypto object that delegates to the node.

* CORDA-3750: Add integration test for deterministic CashIssueAndPayment flow.

* Tidy up generics for Array instances.

* Upgrade to DJVM 1.1-RC04.
2020-05-26 15:46:29 +01:00
Dan Newton
330a95cb68
ENT-4627 Log transaction context missing exception (#6278)
When a flow is missing its database transaction the _real_ stacktrace
of the exception is missing in the logs. This is due to the normal error
handling path way does not work due the transaction being missing. When
it goes to process the next error event (due to the original transaction
context missing error) it will fail for the same error as it needs the
context to be able to process the error event.

The extra logging should help diagnose future errors.
2020-05-26 11:32:42 +01:00
Kyriakos Tharrouniatis
57b1a5e0fc
ENT-5339 Failing tests against Oracle in VaultObserverExceptionTest (#6275)
* Fix erroneous sql statement for oracle; It was failing tests with 'ORA-00933: SQL command not properly ended'

* Fixed flaky test; it didn't wait for counter party flow to get hospitalized as the test implied
2020-05-22 10:15:51 +01:00
Rick Parker
67298eb780
Merge pull request #6270 from corda/nnagy-os-4.5-os-4.6-20200520
Nnagy os 4.5 os 4.6 20200520
2020-05-21 13:19:58 +01:00
Viktor Kolomeyko
31612901cf
CORDA-3800: Flaky ProtonWrapperTests test investigation (#6258) 2020-05-21 12:33:56 +01:00
nikinagy
301fc9f928 Merge branch 'release/os/4.5' into nnagy-os-4.5-os-4.6-20200520 2020-05-20 10:38:45 +01:00
nikinagy
938d038c97
ENT-5278 - add missed checks for empty lists (#6261)
* add missed checks for empty lists

* adding missing brackets

* adding missing timeout
2020-05-19 16:55:09 +01:00
Dan Newton
36a11e868a
NOTICK Fix timing issue in FlowIsKilledTest (#6264)
Also remove `Configurator` that seems to fail on some agents.
2020-05-19 16:54:34 +01:00
williamvigorr3
eb52de1b40
CORDA-3490 Add option to start node without starting checkpointed flows (#6136)
Added command-line option: `--pause-all-flows` to the Node to control this.
This mode causes all checkpoints to be set to status PAUSED when the
state machine starts up (in StartMode.Safe mode). 

Changed the state machine so that PAUSED checkpoints are loaded into
memory (the checkpoint is deserialised but the flow state is left serialised)
but not started.

Messages from peers are queued whilst the flow is paused and processed
once the flow is resumed.
2020-05-19 16:27:41 +01:00
Adel El-Beik
b9b0ca58e3 Merge remote-tracking branch 'origin/release/os/4.5' into adel/merge-from-4.5 2020-05-15 10:23:17 +01:00
Ryan Fowler
6dd72ac079
ENT-5275: Replace hardcoded platform versions with descriptive values (#6236)
* ENT-5275: Replace hardcoded platform versions with descriptive values

* PR updates

* missed versions

* missed versions

* fix

* add FIRST_VERSION
2020-05-14 14:43:52 +01:00
James Higgs
06467d04b1
EG-1980 - Remove stale docs links from OS (#6251)
Remove hardcoded docs references, push docs link to gradle
2020-05-14 12:57:59 +01:00
Denis Rekalov
8387e99535
[NOTICK]: Do not use Security.addProvider(BouncyCastleProvider()) in tests to avoid disruptions of other tests (#6250) 2020-05-14 10:53:54 +01:00
Ivan Schasny
bd657fad65
Merge pull request #6245 from corda/EG-1991-ConfigHelper-update
EG-1991 Ignore warning for CORDA environment variables in all caps
2020-05-13 15:37:42 +01:00
Dan Newton
c2ac22a41f
NOTICK Non-database error handling in withEntityManager (#6239)
When a non-database exception is thrown out of a `withEntityManager`
block, always check if the session needs to be rolled back.

This means if a database error is caught and a new non-database error is
thrown out of the `withEntityManager` block, the transaction is still
rolled back. The flow can then continue progressing as normal.
2020-05-13 15:17:51 +01:00
john.buckle@r3.com
febb5a44b7 Added comment 2020-05-13 21:55:04 +08:00
Ryan Fowler
9caf6538ce
ENT-5298: Increase timeout by two minutes because the test occasionally takes longer than five minutes (#6246) 2020-05-13 13:15:05 +01:00
john.buckle@r3.com
c3ea2c1470 Ignore warning for CORDA environment variables in all caps 2020-05-13 09:51:35 +08:00
nikinagy
e0b5319515 Merge branch 'release/os/4.4' into nnagy-os-4.4-os-4.5-20200511 2020-05-11 16:32:53 +01:00
nikinagy
350066d386
fix for handling empty lists in vault query (#6231) 2020-05-11 15:41:50 +01:00
Ryan Fowler
5f8cceae45 Merge branch 'release/os/4.4' into rfowler-os-4.4-os-4.5-20200511 2020-05-11 11:40:09 +01:00
LankyDan
8125b9bdb6 NOTICK Change platform version assert in FlowMetadataRecordingTest 2020-05-06 10:03:05 +01:00
LankyDan
565afc5fdb NOTICK Fix kill flow tests due to storing failed flows
Failed flows are stored after the checkpoint table rework. This meant
that some of the asserts in `FlowIsKilledTest` and `KillFlowTest` were
wrong.
2020-05-06 10:02:40 +01:00
Chris Rankin
9a5be9e47d
CORDA-3738: Upgrade to DJVM 1.1-RC03. (#6219) 2020-05-05 17:32:39 +01:00
LankyDan
217fd906b3 NOTICK Remove unused imports 2020-05-05 17:22:42 +01:00
LankyDan
7fe2845272 NOTICK Fix KilledFlowTransition after merge 2020-05-05 17:15:54 +01:00
LankyDan
4ccd0fd3df Merge branch 'release/os/4.6' into dan/4.6-into-checkpoint-feature-branch-2020-05-05
# Conflicts:
#	node/src/main/kotlin/net/corda/node/services/statemachine/Event.kt
#	node/src/main/kotlin/net/corda/node/services/statemachine/FlowStateMachineImpl.kt
#	node/src/main/kotlin/net/corda/node/services/statemachine/SingleThreadedStateMachineManager.kt
#	node/src/main/kotlin/net/corda/node/services/statemachine/transitions/TopLevelTransition.kt
2020-05-05 17:05:23 +01:00
Ryan Fowler
b6dc1d8c4a
NOTICK - Reduce flow count for test to improve run time (#6213) 2020-05-05 09:38:55 +01:00
nargas-ritu
54e4503866
Merge pull request #6212 from corda/dan/update-byteman-dependency
NOTICK Update byteman dependencies
2020-05-04 16:29:29 +01:00
Ryan Fowler
55797612b2 ENT-5237: Remove DISTINCT (backport) 2020-05-04 15:02:33 +01:00