Commit Graph

613 Commits

Author SHA1 Message Date
josecoll
0280299104 Soft locking implementation using database coin selection
Fix broken IssuerFlowTest

Fix IssuerFlowTests after rebase.

Resolve conflicts after rebase.

Soft locking converted to use persistent store.
Added additional optional 'includeLockStates' parameter in VaultService states API call.
Added Vault softLocked states query API call.

Fixed commercial paper failing test.
Improved exception handling on soft locking UPDATE statement.

Using SELECT FOR UPDATE to ensure correct soft locking data visibility.
Db query operations moved out of mutex code (as locking managed by underlying DB)

Adjusted logging severity levels.

Adjusted logging severity levels.

GenerateSpending now performing fine grained query for unconsumed states by joining with contract_cash_states table.
Using H2 proprietary cummulative counting feature (using sessioni SET variables)
Refactored and simplified HibernateObserver constructor to enable usage in JUnit tests.

Event generator issues larger random amounts (10,000..1,000,000) to those than are spent (0..10,000)
Adjusted Issue (5:1) and Exit (10:1) generation frequency vs spending.

Minor fixes: added optional lockid into select for spending criteria, set notary, additional trace logging.

Generate Cash Schema by default upon node start-up (as part of NodeSchemaService initialisation).

Explicitly close JDBC statements in finally() blocks.

Tightened HibernateObserver constructor.

Fix CommercialPaper test (was missing auto-generation of CONTRACT_CASH table)

Revert default JVM size back to 200Mb.

Revert default number of iterations in Explorer Node Simulation mode (back to 10000 with .5 sec sleep interval).

Remove redundant setter function.

Added TODO messages indicating Requery / H2 restrictions & caveats.

Consumed states lock updates now performed in general consumed state Update.

Updated/added Soft Locking documentation.

Addressed initial PR comments: use THREAD_LOCAL_KRYO, use AbstractParty, extract helper method, improve readability, address some doc typos

Addressed PR comment: removed lockId from WireTransaction.

Fixed soft locking UPDATE statements.

Improvements to VaultSoftLockManager for auto-registration of soft locks for flows with spendable states (as notifications from vault).
Other optimisations (IssuerFlow no longer explicitly reserve/release issued state) and improvements (soft lock release management of soft locks, docs update)

Performance update: now using Requery for UPDATE in release soft locking (non-composite key statement)

Removed redundant TODO messages (TODO: revisit Kryo bug when using THREAD_LOCAL_KYRO)

Minor fixes following rebase

Fixed failing JUnit following rebase

Addressed MH PR review items (1st pass)

Fix broken JUnit

Significant changes to RDBMS operations within coin selection and soft locking as requested by PR review.
(Removed SELECT FOR UPDATE; added RETRY upon coin selection; reverting partial soft locks)

Addressed a number of PR review requests added by MH (comments/spelling, lockID instantiation, HibernateObserver instantiation, cash schema white-listing usage)

Addressed latest PR review comments from RP.

Minor fixes following rebase from master.

Fixed final failing JUnit (issuer flow concurrent).

Updated TraderDemo to trigger concurrent issuance of cash.

Fixed compiler warning on lockId null check.

Fixed subtle bug in coin selection intermittently surfaced in IntegrationTestTutorial.

Fixed small memory leak.

Removed stray } in logger trace message.

Slight rewording of description of Soft Locking in docs.

Renamed NoStatesAvailableException to StatesNotAvailableException.
generateSpend is now Suspendable (calls sleep method on flow upon coin selection retry).

Added companion function to enable a Strand to sleep but without locking transactional context.

Improved logging, changed to StateNotAvailableException, using Flow sleep upon retry, tweaked SELECT criteria in coin selection, fixed bug when insufficient states selectable, generateSpend is now @suspendable

Improved handling and logging of flow results in Simulation Mode.

Fixed minor error in sleep when not an active flow.

Retry coin selection when unavailable states (as these may become available as new states).
Additional debug logging to highlight and identify H2 coin selection sporadic bug.

Inlined sleep method due to intermittent Quasar error.

Re-introduce selection clause that prevents selection and temporary locking of already locked states (by other flows).
Improved trace logging for coin selection (SQL row level info).
Correctly calling FlowStateMachineImpl sleep (now inlined and working correctly)

Fixed rebase error.

Remove redundant TODO message.
2017-03-27 17:12:33 +01:00
Shams Asari
bb74f0ca34 Merge pull request #427 from corda/shams-consistent-guava-deps
Upgraded both guava and guava-testlib to 21.0 and made sure they shar…
2017-03-24 18:30:11 +00:00
Ross Nicoll
c3cf2226ea Extend identity service tests
RBS report problems with multiple nodes being able to find each other in the identity service. This extends the name
lookup test case to try three nodes, to check all of the stored values, and to use random keys rather than static.

Add trace log of identity registrations to aid with tracking state.
2017-03-24 18:25:49 +00:00
Shams Asari
52ea54f742 Upgraded both guava and guava-testlib to 21.0 and made sure they share the same version variable 2017-03-24 17:17:36 +00:00
Mike Hearn
68ff33549a Address review comments 2017-03-24 12:44:54 +01:00
Mike Hearn
ac90fe724e Shell: add support for uploading and downloading attachments. 2017-03-24 12:44:54 +01:00
Mike Hearn
347224c900 Improve attachment checks to ensure that a stream is actually a JAR/ZIP. 2017-03-24 12:44:54 +01:00
Mike Hearn
2cb02c75eb Minor: remove ANSI progress observer, delete some currently dead code in the shell. 2017-03-24 12:44:54 +01:00
Marek Skocovsky
262c87a5c6 Integrate CRaSH shell (SSHD). Joint effort between Mike Hearn and Marek Skocovsky.
The shell is embedded in the node and offers the ability to monitor
and control the node via the launching terminal.

Still to do:

* Switch to a fork of CRaSH that we can maintain ourselves, and merge in Marek's SSH patch so we can enable SSH access.
* Add persistent command history that survives restarts.
* Tab completion for the 'flow' and 'run' commands.
* Remove the 'jul' command and replace it with a command that lets you see and tail the log4j logs instead.
* Fix or remove the other crash commands that have bitrotted since 2015.
2017-03-24 12:44:54 +01:00
Andrius Dagys
71fe3e3d3b Record a metric for duration and rate of execution for every flow (#372)
* Record a metric for the duration and rate of execution for every flow. This is useful for task intense scenarios to detect performance degradation.

* Measure both successful and failed flow durations
2017-03-22 16:52:41 +00:00
Chris Rankin
98266da41c Refactor CordaRPCClient into new :client:rpc Gradle module. (#405)
* CORDA-305: Refactor CordaRPCClient into :client:rpc module

* CORDA-305: Remove the Kotlin test framework from the artifacts.

* CORDA-305: Migrate serialisation whitelist into node-api module.

* CORDA-305: Clean up unused RPC observables.

* CORDA-305: Add :client:rpc module to documentation tasks.

* CORDA-305: Include :finance into :client:rpc for its serialisable classes.

* CORDA-305: Move test classes into the correct directory.

* CORDA-305: Migrate :finance dependency from :client:rpc into DemoBench.

* CORDA-305: Update wording of TODO about handling Observables.
2017-03-22 15:52:54 +00:00
Ross Nicoll
7e0e7bc645 Fix warning in NodeAttachmentService 2017-03-22 12:20:29 +01:00
Chris Rankin
9e43df36d3 Refactor :client module into :client:javafx and :client:mock (#388)
* CORDA-304: Refactor :client modules into :client:javafx and :client:mock.

* CORDA-304: Add :client:mock to Dokka tasks, and remove unused integrationTest task.

* CORDA-304: Migrate DriverBasedTest from node into test-utils.

* CORDA-304: Rename .fx. package to .jfx. to prevent confusion with "exchange rate".

* CORDA-304: Rename module to ':client:jfx'.
2017-03-21 19:45:12 +00:00
Andras Slemmer
7a9bc5bfad driver: Add ShutdownManager, make network map service optional 2017-03-21 18:08:21 +00:00
Mike Hearn
1243118aa9 Minor: more banking jokes, surround legal name in quotes 2017-03-21 18:01:17 +01:00
Mike Hearn
4c6548107a Minor: don't repeat the legal name of the node multiple times in the output (which causes indentation issues) 2017-03-21 18:01:17 +01:00
Shams Asari
209f5c8a25 Logging fixes, including broken --logging-level cmd line arg 2017-03-21 15:36:32 +00:00
Mike Hearn
0d54832d88 FlowSessionException is sent over RPC so must be serializable and documented. 2017-03-20 15:17:31 +01:00
Andras Slemmer
4f6b44ceff Merge pull request #379 from corda/aslemmer-node-api
Add :node-api, factor out some artemis code
2017-03-20 13:31:21 +00:00
Andras Slemmer
d3f956ab60 Address PR 379 comments 2017-03-17 15:09:04 +00:00
Rick Parker
1de1f9095f Several tests were corrupting Kryo which was then returned to the common pool. (#378)
* We were leaving trailing attachmentStorage on pooled kryo instances after some tests.  Changed attachment storage logic to make it impossible to leave it behind.

* Some low level tests corrupt the Kryo config, so do not return to pool when this is the case.  Also, we discovered that Kryo is caching class name to class resolution.  We don't want to do this where attachments are involved.  The errors raised highlighted a class missing from the whitelist.  Need to write a unit test to test the class loader issue.

* Unit test for attachment class loading with kryo.
2017-03-17 12:43:11 +00:00
Andras Slemmer
c3c1f3d801 Add :node-api, factor out some artemis code 2017-03-17 11:47:05 +00:00
Ross Nicoll
486368d926 Clean up messaging/RPC port configuration and docs (#296)
* Non-ssl artemis acceptor for RPC connection. (#271)

* New non-ssl acceptor in artemis server for RPC connection.

* Rename artemisAddress with messagingAddress

Rename artemisAddress with messagingAddress so that the node configuration file properties match
the code variable names.
Rename artemisPort to messagingPort in Gradle configuration to match node configuration naming.

* Add rpcPort configuration option for Gradle

* Update docs to reflect changes to RPC port configuration

* Renumber ports in example CorDapp to match numbering used elsewhere

* Restructure upgrade guide

* added config file checks on corda startup to make the upgrade path a bit smoother.
2017-03-17 10:32:14 +00:00
Mike Hearn
eb21458885 Some fixes to emoji/ANSI renderer 2017-03-17 10:54:25 +01:00
Shams Asari
f079c05e6a Merge pull request #361 from corda/shams-nms-reg-error-response
Clean up of network map service  and its tests, and added error msg t…
2017-03-16 17:03:42 +00:00
Konstantinos Chalkias
5966610a6e DEFAULT_MAX_BUCKETS based on max heap size
MAX_DEFAULT_BUCKETS is linearly correlated to max heap size with a minimum of 256 buckets.
2017-03-16 16:40:34 +00:00
Shams Asari
5e3e7f6c1c Clean up of network map service and its tests, and added error msg to registration response 2017-03-16 16:37:21 +00:00
josecoll
391270ed71 RequeryConfiguration setting for table creation is now CREATE_NOT_EXISTS (was DROP_CREATE) (#374) 2017-03-16 16:23:37 +00:00
Rick Parker
f3a5f8e659 Pool Kryo instances for efficiency. (#352)
Pooled Kryo
2017-03-16 08:24:06 +00:00
Shams Asari
f581844f3f Flow registration takes in a Class object rather than a KClass 2017-03-15 16:32:13 +00:00
Andras Slemmer
6a6698b598 Merge pull request #339 from corda/aslemmer-fix-rpc-close-deliver-deadlock
rpc: Fix deadlock caused by deliver() and close()
2017-03-15 16:20:11 +00:00
Konstantinos Chalkias
195d69d0dc Print to screen and log basic info (#356)
Adding an extra logger to printBasicNodeInfo in both console and log file, respectively. Also, node's legal name is now printed to console as well.
2017-03-15 16:18:24 +00:00
josecoll
18e4620928 ArrayDeque FlowLogic references were not being removed on flow termination. (#353) 2017-03-14 15:25:34 +00:00
Andras Slemmer
389685a31e PR 339 Address comments 2017-03-14 15:17:15 +00:00
Andras Slemmer
60520412c8 rpc: Fix deadlock caused by deliver() and close() 2017-03-14 15:17:15 +00:00
Shams Asari
bbbc4d9eaa Fix in reading non-present manifest values in single tests 2017-03-13 18:37:16 +00:00
Konstantinos Chalkias
afd5521b00 Lazy NodeVaultService.states (#349)
Convert NodeVaultService states to return Iterable (backed by Sequence) Vs the old way using a List. Worth noting this relieves memory pressure as the number of vault states grows.

* remove toList in ContractUpgradeFlowTest
2017-03-13 14:06:47 +00:00
Mike Hearn
21f5f6d9d0 ANSIProgressRenderer: allow progressTracker to be set to null 2017-03-10 17:26:18 +01:00
Mike Hearn
5e19bad4b5 Minor: don't use a separate thread to start Metrics-to-JMX export.
This should hopefully resolve a shutdown hang due to the misnamed "WebServer" thread not being a daemon thread.
2017-03-10 17:23:43 +01:00
Shams Asari
1adc9e65ec Logging the PID and version info 2017-03-10 10:29:41 +00:00
Shams Asari
e9d63b2662 Added bits of versioning info to the node 2017-03-09 16:36:21 +00:00
Konstantinos Chalkias
4a9ff84fc7 Allowing multiple signature algorithms (#250)
Basic crypto API to support 5 signature schemes and MetaData-ed signatures.
Supported schemes: (1) RSA_SHA256, (2) ECDSA_SECP256K1_SHA256, (3) ECDSA_SECP256R1_SHA256, (4) EDDSA_ED25519_SHA512, (5) SPHINCS-256_SHA512.

To sign a transaction, a signer should create a MetaData wrapper that contains transaction's merkle root and some extra information, such as signer's public key, timestamp and visibleInputs. Actually, MetaData is utilised to support a practical partial, blind and extra-data attached signature model.
When a MetaData object is signed, the signer sends a TransactionSignature object that contains the signed output and the corresponding MetaData object.

Remarks: 
This is an temporary solution for signature algorithmic agility. Further development is required for a robust and extensible Crypto Manager/Provider PKI that will support certificate creation, key generation, signing/verifying, deterministic key derivation, encoding formats, SGX/HSM support, identity and key management, versioning, revocation, asynchronicity, metadata, partial sig. policies etc.
2017-03-08 17:45:23 +00:00
Rick Parker
dc8c6747d3 Allow escaped unsubscribed RPC Observables to be GC'd (#334)
* QueuedObservable could not leak due to hard references.  Have made it weak referenced but also had to reference count a hard reference to prevent early GC.  Explained in comments.

* Improved comments.

* Fixed typo

* Fixed another typo
2017-03-08 14:09:38 +00:00
Mike Hearn
35836d9926 Add a registeredFlows() RPC that lists the RPCs the node knows about. 2017-03-08 15:01:13 +01:00
Andrius Dagys
97b3c35ec2 Add public IP detection (from network interfaces), node outputs a warning if no public IP detected or specified in the config. 2017-03-07 17:41:23 +00:00
Chris Rankin
0e3540d174 CORDA-265: Implement "ALL" permission for RPC users. (#306)
* CORDA-265: Implement "ALL" permission for RPC users. Users with this permission in node.conf can use any flow.

* CORDA-265: Ensure that we always close the RPC proxy object after each test.

* CORDA-265: Refactor construction of dummy RPC client into an abstract base class.

* CORDA-265: Document RPC "ALL" permission.
2017-03-07 16:57:34 +00:00
Karel Hajek
1e78d6a3a7 Simple Attachment Storage implementation using Requery/H2 database 2017-03-07 16:12:38 +01:00
Andrius Dagys
907a893ca1 BFT notary prototype: add validation and signature collection (#279)
* BFT notary prototype: add a non-validating service.
Each replica now validates the transaction timestamp and returns an individual signature to the BFT client. The client then returns a list of signatures back to the notary service flow.

The validating variant is still incomplete - it requires the ability to suspend flows on arbitrary function calls.
2017-03-07 12:39:19 +00:00
Ross Nicoll
b5ba070301 Revert "Non-ssl artemis acceptor for RPC connection. (#271)"
This reverts commit f0d82e4918.
2017-03-06 14:03:23 +00:00
Shams Asari
059056de65 Removed "FLOW" from network map topic constants 2017-03-02 10:11:38 +00:00