Commit Graph

8328 Commits

Author SHA1 Message Date
Kyriakos Tharrouniatis
0174d996bd
CORDA-3598 Set Checkpoint.status to RUNNABLE (#6019)
* Set/ Reset Checkpoint.status to RUNNABLE after when suspending

* Removing/ Moving comment as it makes no longer sense to be there since, we now always create a new Checkpoint object in SingleThreadedStateMachineManager.createFlowFromCheckpoint through tryDeserializeCheckpoint

* Set -in memory- Checkpoint.status to RUNNABLE when a flow is retrying from Checkpoint
2020-03-12 17:57:35 +00:00
williamvigorr3
499b6cf17e
CORDA-3603 Save completed flow information (#6034)
When a flow is finished do not delete the checkpoint from the DB.

Instead, the FlowStatus is marked as Completed in the DB.

Updated numerous tests which relied on the flow being removed
when finished.
2020-03-12 11:45:30 +00:00
williamvigorr3
51db2c2d7f
CORDA-3600 Add flowIO request to checkpoint (#6017)
* Update Checkpoint DB to update flow io request

* Modify flow monitor to update Checkpoint DB with waiting flows

This happens periodically.

* Refactored code to avoid looping twice and updated tests

* Fix tests after rebasing

* Fix MR comments (non-functional refactor of tests + FlowMonitor).

* Made visible for testing method private in DBCheckpointStorage

This is not needed anymore.

* Explicity check if ioRequestType has changed in update method

* Fix shadowing warning

* Import non deprecated Assert into test

* Use AssertEquals not assert in test

* Address more comments (minor refactor) of DBCheckpointStorage

* Minor fix use it instead of referencing object explicitly

* Add null check to DBCheckpointStorage

* Revert changes to Flow Monitor.

We will instead store the information in the main thread of the
state machine.

* Remove now uneeded API and make statemachine update ioRequest

* Add Integration Test to check statemachine updates DB on Recieve

* Use simpleName in checkpoint storage instead of class.

Hibernate was previously resetting the class field this is now
set to null (when getting checkpoint form DB) and a new method
for getting back the simple name as a string.

* Update StateMachineState to store simple name.

* Fix after rebase broke stuff + renamed test

* Fix Detekt issue

* Remove uneeded null assertion
2020-03-10 15:46:37 +00:00
Dan Newton
d5e84a4f93
NOTICK Do not cascade checkpoint tables (#6040)
Do not cascade updates to checkpoint error and result tables to hopefully
improve database performance moving forward. Because the joined tables
are no longer being updated by updating the main `DBFlowCheckpoint` entity,
they must be created/updated/deleted manually.

The checkpoint blobs still cascade as they pretty much always evolve in 
tandem with the main checkpoint table.
2020-03-09 10:01:30 +00:00
williamvigorr3
ab000e0533
CORDA-3597 Replace old Checkpoint table with new one. (#5992)
* Replace old Checkpoint table with new one.

Adds some of the new fields into the table where needed (I have
guessed this stuff but we can update it as we go along).

* Fix database constraints + name table correctly opps.

* Fixed typos in Liquidbase script

Also corrected constraints and added missed fields in hibernate
checkpoint class and liquibase scripts.

* Update CheckpointStorage to pass in serialization context.

This is cleaner than passing both the checkpoint and the
serialized checkpoint into the methods. Also fixed
CordaPersistanceServiceTests which I accidentally broke.

* Fix detekt problem

* Revert "Update CheckpointStorage to pass in serialization context."

This reverts commit b71e78f202.

* Fix test broken by reverting commit

* CORDA-3597 Update metadata join, timestamp columns and serialization

- Change the metadata join to the checkpoints table to use
  `invocation_id` instead of `flow_id`. There were issues joining
  between the tables because `flow_id` was not the primary key of the
  metadata table. Switching over to `invocation_id` has at least allowed
  us to bypass this issue. The information about the `invocation_id` is
  stored in the `Checkpoint` class which makes it simple to save at
  runtime.
- Some of timestamp columns were nullable when they should always be
  populated, the nullable flags have now been removed.
- Previously the whole checkpoint was being serialized and stored into
  the `checkpoints_blob.checkpoint` column. This meant duplicated saving
  as the `flow_state` was contained in this object. Only the
  `CheckpointState` property of `Checkpoint` is now being serialized and
  saved to this field. Furthermore, it now uses the default
  `STORAGE_CONTEXT` serialization (AMQP) instead of Kryo (which is only
  used for serializing the `flow_state` / flow stack).
- The checkpoint database performance metrics recording has been
  abstracted to its own class.

* CORDA-3597 Make metadata join non optional

Remove the nullable declaration on the metadata field of
`DBFlowCheckpoint`

* CORDA-3597 Rename `node_checkpoints_blobs` to `node_checkpoint_blobs`

* CORDA-3597 Update some kdocs

Co-authored-by: Dan Newton <danknewton@hotmail.com>
2020-03-02 10:04:48 +00:00
williamvigorr3
9d4d128f4e
CORDA-3597 add missed data to checkpoint class (#5995)
* Add extra fields to StateMachineState

* Move structures into checkpoint as this is a more natural place.
2020-02-26 10:48:52 +00:00
williamvigorr3
8d1b6cf499
CORDA-3432 update structure of checkpoint class (#5983)
* Split StateMachine State into 2 classes

The idea is this better reflects the database structure. Added a
few helper methods to copy and update state.

* Doc + Improve Checkpoint API

* Rename methods to be more clear
2020-02-18 15:31:46 +00:00
stefano
13c52e4901 Merge remote-tracking branch 'origin/feature/checkpoint_table_improvements' into feature/checkpoint_table_improvements 2020-02-13 12:08:07 +00:00
Chris Rankin
117108f456
ENT-4921: Ensure that CorDapp classloader is not lost with graceful reconnect. (#5954) 2020-02-13 09:25:28 +00:00
Razvan Codreanu
c6ab07d92c
TM-197 Fix failing test caused by regression and ignoring flaky DistributedServiceTests (#5934)
* TM-197 Setting bouncy castle provider in order for the test to pass

* TM-197 setting timeout for all builds at 3 hours

* TM-197 ignoring unstable tests
2020-02-12 14:26:06 +00:00
Thomas Schroeter
af4d9c48a0
ENT-4933: Document notary.etaMessageThresholdSeconds (#5948)
* ENT-4933: Document `notary.etaMessageThresholdSeconds`

* Fix typo
2020-02-12 14:10:26 +00:00
Denis Rekalov
ba002a82ec
CORDA-3587: hashLookup command help misspelling (#5936) 2020-02-11 15:13:34 +00:00
Denis Rekalov
448e7baf0b
Merge pull request #5942 from corda/denis/ENT-4918-certificates-dir-symbolic-link
ENT-4918: Support symbolic link for certificates directory during node registration
2020-02-11 14:03:20 +00:00
Denis Rekalov
1705299bde ENT-4918: Support symbolic link for certificates directory during node registration 2020-02-11 13:14:34 +00:00
Waldemar Żurowski
e90058962f
Updating Artifactory URLs for Corda OS (#5930) (#5939)
* Switching Artifactory URLs from obsolete ci-artifactory to software.r3.com
2020-02-11 11:57:51 +00:00
Stefano Franz
b23af5f0d2
add timeouts to all tests (#5875) 2020-02-11 10:14:05 +00:00
Stefano Franz
c9f055aab4
Merge pull request #5892 from corda/new_checkpoint_schema
New checkpoint schema
2020-02-10 15:05:59 +00:00
stefano
9e8ce6473d address dan review comments pt2 2020-02-10 13:22:51 +00:00
stefano
a28c15c2fd address dan review comments 2020-02-10 11:52:54 +00:00
stefano
546166e057 Merge branch 'release/os/4.4' of github.com:corda/corda into new_checkpoint_schema 2020-02-10 11:25:29 +00:00
Stefano Franz
1d0918cdde
NOTICK - Speed up regression tests (#5918)
* speed up regression tests

* move to local-k8s for all Jenkins builds
2020-01-31 16:47:24 +00:00
Kyriakos Tharrouniatis
9ca1dd59da
CORDA-3381 Make internalUtilsKt Observer tee not wrap with SafeSubscriber (#5913)
* Make tee not wrap PublishSubjects in SafeSubscribers, otherwise a non Rx exception from an unsafe observer shuts down all other observers under the same PublishSubject

* Throw SQLException or PersistenceException plain, that may come out of an unsafe subscriber

* Revert "Throw SQLException or PersistenceException plain, that may come out of an unsafe subscriber"

This reverts commit c7b8af3fa6.

* Update Detekt baseline
2020-01-31 12:32:59 +00:00
Florian Friemel
cb6ed6042c
Move jimfs dependency from node to test-utils. (#5911) 2020-01-31 10:55:05 +00:00
stefano
eec9ada6ba persist some nonsense data to see if it breaks the statemachine 2020-01-30 18:18:45 +00:00
jmacmahonr3
9511329e3d
[EG-75] Not handled exception when NetworkMap not accessible (#5903)
* Just passing in the exception was printing the entire stacktrace to the log, now we just pring the exception and message

* Updating exception message
2020-01-30 17:26:22 +00:00
Denis Rekalov
609ae5664b
Merge pull request #5909 from corda/christians/NOTICK-remove-forms-dependency
Remove unused dependency on IntelliJ forms
2020-01-30 14:03:49 +00:00
Chris Rankin
1f9e54685d
CORDA-3377: Upgrade to DJVM 1.0-RC09. (#5910) 2020-01-30 13:59:16 +00:00
stefano
2b079bd92e working node startup with new tables and entities 2020-01-30 12:17:26 +00:00
Christian Sailer
cac96d81bf Remove unused dependency on IntelliJ forms 2020-01-30 10:48:00 +00:00
Oliver Knowles
286fffd67a
ENT-4368 - Improve error handling when required included config resource cannot be found. Add note in docs encouraging use of required syntax for includes. (#5907) 2020-01-30 10:10:24 +00:00
stefano
7b3da95456 flesh out entities for new checkpointing 2020-01-29 17:10:34 +00:00
Joel Dudley
5b186b1c65
Interfaces/classes/methods with internal in their name are non-public API (#5906) 2020-01-29 15:35:00 +00:00
Ryan Fowler
d16e1126db
CORDA-3542: Use the config values for reconnecting retry interval and max reconnect attempts (#5869) 2020-01-29 13:08:54 +00:00
stefano
78d83e9583 start adding hibernate entities 2020-01-28 16:55:14 +00:00
Chris Rankin
3a34b0d087 CORDA-3228: Upgrade to Quasar 0.8.0_r3 for JDK 11. (#5904) 2020-01-28 16:08:15 +00:00
stefano
32bdd9728c Merge branch 'release/os/4.4' of github.com:corda/corda into new_checkpoint_schema 2020-01-28 16:05:29 +00:00
Chris Rankin
eea8168672 CORDA-3377: Upgrade to DJVM 1.0-RC08. (#5905) 2020-01-28 13:54:34 +00:00
Viktor Kolomeyko
2e8f2ffc21 CORDA-3025: Permissions template for a read-only user (#5901) 2020-01-28 11:49:12 +00:00
Chris Rankin
c8a21cb8d2 CORDA-3536: Load DJVM serialization types more precisely to avoid runtime warnings. (#5896)
* Load DJVM serialization types more precisely to avoid runtime warnings.
* Remove unnecessary string concatenation from debug and trace log messages.
2020-01-28 09:26:13 +00:00
Nick Dunstone
a4cada4a2e ENT-4474 O/S Changes to support bulk backchain fetching (#5894)
* Ongoing dev multi backchain fetch

* Final dev pre logging message improvements

* Trace messages

* Trace messages

* Code tidy up

* Code review comments

* Code review comments

* Whitespace removed

* Code review changes

* Code reformatting
2020-01-28 09:04:18 +00:00
davidrapacchiale
16532299c5
Merge pull request #5902 from corda/vkolomeyko/CORDA-3559
CORDA-3559: Remove reference to `man run`
2020-01-27 15:36:01 +00:00
Ryan Fowler
e8d6701f87 CORDA-3558: Allow initial registration errors to propagate up so the node exits with a failure code (#5899) 2020-01-27 15:23:07 +00:00
Chris Rankin
08925ee0a5 ENT-4911: Split network verifier's sealed CorDapps into separate packages. (#5900) 2020-01-27 15:20:08 +00:00
Viktor Kolomeyko
ae3ce82e60 CORDA-3559: Remove reference to man run
`man` shell command is deemed to be unsafe.
2020-01-27 15:01:40 +00:00
Stefano Franz
d95c3e2b8b Use a local instance of k8s instead of remote cluster for parallel testing (#5897)
* switch to new version of plugin

* change plugin version to force refresh

* try using pre-populated cache

* do not preallocate

* no daemon

* reduce cores per fork to increase parallelism

* even more aggresive cpu allocation
2020-01-27 14:58:21 +00:00
Ryan Fowler
d0faf4d7cb NOTICK - allow resolveJar errors to be quieter (#5898) 2020-01-27 14:08:08 +00:00
stefano
e27e77463b :Merge branch 'switch_to_local_k8s' into new_checkpoint_schema 2020-01-27 13:18:42 +00:00
stefano
93623d7317 even more aggresive cpu allocation 2020-01-27 12:56:36 +00:00
stefano
a3bc624e16 add message to exceptions table 2020-01-27 12:19:14 +00:00
stefano
47c14673c4 address initial review comments 2020-01-27 11:20:13 +00:00