corda/docs/source/key-concepts-vault.rst

57 lines
3.9 KiB
ReStructuredText
Raw Normal View History

Vault
=====
The vault contains data extracted from the ledger that is considered relevant to the nodes owner, stored in a relational model
that can be easily queried and worked with.
The vault keeps track of both unconsumed and consumed states:
* Unconsumed (or unspent) states represent fungible states available for spending (including spend-to-self transactions)
and linear states available for evolution (eg. in response to a lifecycle event on a deal) or transfer to another party.
* Consumed (or spent) states represent ledger immutable state for the purpose of transaction reporting, audit and archival, including the ability to perform joins with app-private data (like customer notes)
By fungible we refer to assets of measurable quantity (eg. a cash currency, units of stock) which can be combined
together to represent a single ledger state.
Soft locking implementation using database coin selection Fix broken IssuerFlowTest Fix IssuerFlowTests after rebase. Resolve conflicts after rebase. Soft locking converted to use persistent store. Added additional optional 'includeLockStates' parameter in VaultService states API call. Added Vault softLocked states query API call. Fixed commercial paper failing test. Improved exception handling on soft locking UPDATE statement. Using SELECT FOR UPDATE to ensure correct soft locking data visibility. Db query operations moved out of mutex code (as locking managed by underlying DB) Adjusted logging severity levels. Adjusted logging severity levels. GenerateSpending now performing fine grained query for unconsumed states by joining with contract_cash_states table. Using H2 proprietary cummulative counting feature (using sessioni SET variables) Refactored and simplified HibernateObserver constructor to enable usage in JUnit tests. Event generator issues larger random amounts (10,000..1,000,000) to those than are spent (0..10,000) Adjusted Issue (5:1) and Exit (10:1) generation frequency vs spending. Minor fixes: added optional lockid into select for spending criteria, set notary, additional trace logging. Generate Cash Schema by default upon node start-up (as part of NodeSchemaService initialisation). Explicitly close JDBC statements in finally() blocks. Tightened HibernateObserver constructor. Fix CommercialPaper test (was missing auto-generation of CONTRACT_CASH table) Revert default JVM size back to 200Mb. Revert default number of iterations in Explorer Node Simulation mode (back to 10000 with .5 sec sleep interval). Remove redundant setter function. Added TODO messages indicating Requery / H2 restrictions & caveats. Consumed states lock updates now performed in general consumed state Update. Updated/added Soft Locking documentation. Addressed initial PR comments: use THREAD_LOCAL_KRYO, use AbstractParty, extract helper method, improve readability, address some doc typos Addressed PR comment: removed lockId from WireTransaction. Fixed soft locking UPDATE statements. Improvements to VaultSoftLockManager for auto-registration of soft locks for flows with spendable states (as notifications from vault). Other optimisations (IssuerFlow no longer explicitly reserve/release issued state) and improvements (soft lock release management of soft locks, docs update) Performance update: now using Requery for UPDATE in release soft locking (non-composite key statement) Removed redundant TODO messages (TODO: revisit Kryo bug when using THREAD_LOCAL_KYRO) Minor fixes following rebase Fixed failing JUnit following rebase Addressed MH PR review items (1st pass) Fix broken JUnit Significant changes to RDBMS operations within coin selection and soft locking as requested by PR review. (Removed SELECT FOR UPDATE; added RETRY upon coin selection; reverting partial soft locks) Addressed a number of PR review requests added by MH (comments/spelling, lockID instantiation, HibernateObserver instantiation, cash schema white-listing usage) Addressed latest PR review comments from RP. Minor fixes following rebase from master. Fixed final failing JUnit (issuer flow concurrent). Updated TraderDemo to trigger concurrent issuance of cash. Fixed compiler warning on lockId null check. Fixed subtle bug in coin selection intermittently surfaced in IntegrationTestTutorial. Fixed small memory leak. Removed stray } in logger trace message. Slight rewording of description of Soft Locking in docs. Renamed NoStatesAvailableException to StatesNotAvailableException. generateSpend is now Suspendable (calls sleep method on flow upon coin selection retry). Added companion function to enable a Strand to sleep but without locking transactional context. Improved logging, changed to StateNotAvailableException, using Flow sleep upon retry, tweaked SELECT criteria in coin selection, fixed bug when insufficient states selectable, generateSpend is now @suspendable Improved handling and logging of flow results in Simulation Mode. Fixed minor error in sleep when not an active flow. Retry coin selection when unavailable states (as these may become available as new states). Additional debug logging to highlight and identify H2 coin selection sporadic bug. Inlined sleep method due to intermittent Quasar error. Re-introduce selection clause that prevents selection and temporary locking of already locked states (by other flows). Improved trace logging for coin selection (SQL row level info). Correctly calling FlowStateMachineImpl sleep (now inlined and working correctly) Fixed rebase error. Remove redundant TODO message.
2017-03-27 16:12:33 +00:00
Like with a cryptocurrency wallet, the Corda vault can create transactions that send value (eg. transfer of state) to
someone else by combining fungible states and possibly adding a change output that makes the values balance (this
process is usually referred to as coin selection). Vault spending ensures that transactions respect the fungibility
rules in order to ensure that the issuer and reference data is preserved as the assets pass from hand to hand.
A feature called **soft locking** provides the ability to automatically or explicitly reserve states to prevent
multiple transactions within the same node from trying to use the same output simultaneously. Whilst this scenario would
ultimately be detected by a notary, *soft locking* provides a mechanism of early detection for such unwarranted and
invalid scenarios. :doc:`soft-locking` describes this feature in detail.
.. note:: Basic 'coin selection' is currently implemented. Future work includes fungible state optimisation (splitting and
Soft locking implementation using database coin selection Fix broken IssuerFlowTest Fix IssuerFlowTests after rebase. Resolve conflicts after rebase. Soft locking converted to use persistent store. Added additional optional 'includeLockStates' parameter in VaultService states API call. Added Vault softLocked states query API call. Fixed commercial paper failing test. Improved exception handling on soft locking UPDATE statement. Using SELECT FOR UPDATE to ensure correct soft locking data visibility. Db query operations moved out of mutex code (as locking managed by underlying DB) Adjusted logging severity levels. Adjusted logging severity levels. GenerateSpending now performing fine grained query for unconsumed states by joining with contract_cash_states table. Using H2 proprietary cummulative counting feature (using sessioni SET variables) Refactored and simplified HibernateObserver constructor to enable usage in JUnit tests. Event generator issues larger random amounts (10,000..1,000,000) to those than are spent (0..10,000) Adjusted Issue (5:1) and Exit (10:1) generation frequency vs spending. Minor fixes: added optional lockid into select for spending criteria, set notary, additional trace logging. Generate Cash Schema by default upon node start-up (as part of NodeSchemaService initialisation). Explicitly close JDBC statements in finally() blocks. Tightened HibernateObserver constructor. Fix CommercialPaper test (was missing auto-generation of CONTRACT_CASH table) Revert default JVM size back to 200Mb. Revert default number of iterations in Explorer Node Simulation mode (back to 10000 with .5 sec sleep interval). Remove redundant setter function. Added TODO messages indicating Requery / H2 restrictions & caveats. Consumed states lock updates now performed in general consumed state Update. Updated/added Soft Locking documentation. Addressed initial PR comments: use THREAD_LOCAL_KRYO, use AbstractParty, extract helper method, improve readability, address some doc typos Addressed PR comment: removed lockId from WireTransaction. Fixed soft locking UPDATE statements. Improvements to VaultSoftLockManager for auto-registration of soft locks for flows with spendable states (as notifications from vault). Other optimisations (IssuerFlow no longer explicitly reserve/release issued state) and improvements (soft lock release management of soft locks, docs update) Performance update: now using Requery for UPDATE in release soft locking (non-composite key statement) Removed redundant TODO messages (TODO: revisit Kryo bug when using THREAD_LOCAL_KYRO) Minor fixes following rebase Fixed failing JUnit following rebase Addressed MH PR review items (1st pass) Fix broken JUnit Significant changes to RDBMS operations within coin selection and soft locking as requested by PR review. (Removed SELECT FOR UPDATE; added RETRY upon coin selection; reverting partial soft locks) Addressed a number of PR review requests added by MH (comments/spelling, lockID instantiation, HibernateObserver instantiation, cash schema white-listing usage) Addressed latest PR review comments from RP. Minor fixes following rebase from master. Fixed final failing JUnit (issuer flow concurrent). Updated TraderDemo to trigger concurrent issuance of cash. Fixed compiler warning on lockId null check. Fixed subtle bug in coin selection intermittently surfaced in IntegrationTestTutorial. Fixed small memory leak. Removed stray } in logger trace message. Slight rewording of description of Soft Locking in docs. Renamed NoStatesAvailableException to StatesNotAvailableException. generateSpend is now Suspendable (calls sleep method on flow upon coin selection retry). Added companion function to enable a Strand to sleep but without locking transactional context. Improved logging, changed to StateNotAvailableException, using Flow sleep upon retry, tweaked SELECT criteria in coin selection, fixed bug when insufficient states selectable, generateSpend is now @suspendable Improved handling and logging of flow results in Simulation Mode. Fixed minor error in sleep when not an active flow. Retry coin selection when unavailable states (as these may become available as new states). Additional debug logging to highlight and identify H2 coin selection sporadic bug. Inlined sleep method due to intermittent Quasar error. Re-introduce selection clause that prevents selection and temporary locking of already locked states (by other flows). Improved trace logging for coin selection (SQL row level info). Correctly calling FlowStateMachineImpl sleep (now inlined and working correctly) Fixed rebase error. Remove redundant TODO message.
2017-03-27 16:12:33 +00:00
merging of states in the background), and 'state re-issuance' (sending of states back to the
issuer for re-issuance, thus pruning long transaction chains and improving privacy).
There is also a facility for attaching descriptive textual notes against any transaction stored in the vault.
The vault supports the management of data in both authoritative ("on-ledger") form and, where appropriate, shadow ("off-ledger") form:
* "On-ledger" data refers to distributed ledger state (cash, deals, trades) to which a firm is participant.
* "Off-ledger" data refers to a firm's internal reference, static and systems data.
The following diagram illustrates the breakdown of the vault into sub-system components:
.. image:: resources/vault.png
Note the following:
* the vault "On Ledger" store tracks unconsumed state and is updated internally by the node upon recording of a transaction on the ledger
(following successful smart contract verification and signature by all participants)
* the vault "Off Ledger" store refers to additional data added by the node owner subsequent to transaction recording
* the vault performs fungible state spending (and in future, fungible state optimisation management including merging, splitting and re-issuance)
* vault extensions represent additional custom plugin code a developer may write to query specific custom contract state attributes.
* customer "Off Ledger" (private store) represents internal organisational data that may be joined with the vault data to perform additional reporting or processing
Vault Query API design (#522) * Added queryBy(QueryCriteria) Vault API and Junit tests. * Minor fix following rebase. * Spit out Vault Query tests into separate source file. * WIP * Enable composition of QueryCriteria specifications. Additional JUnit test cases to validate API. * Added Deprecating annotations. Added QueryCriteria for set of contractStateTypes * Minor tweaks and additional JUnit test cases (chain of linear id) * Added Java Junit tests and QueryCriteria builder support. * Added API documentation (including coding snippets and examples). * Added @JvmOverloads to QueryCriteria classes for easy of use from Java. * Refactored QueryCriteria API to use composition via sealed data classes. * Enable infix notation. * Fixed typo. * Clarified future work to enforce DB level permissioning. * Moved PageSpec and Order from QueryCriteria to become parameters of Query itself. * Moved PageSpec and Order from QueryCriteria to become parameters of Query itself. * TokenType now specified as set of <Class> (was non extensible enum). * Exposed new Vault Query API functions via RPC. * Fixed compiler error in java test. * Addressed a couple of minor PR review scomments from MH. * Major updates following PR discussion and recommendations. * All pagination and sorting arguments are optional (and constructed with sensible defaults). Added Java helper functions for queryBy and trackBy interfaces. Added Java trackBy unit tests. Miscellaneous cleanup. * Added Generic Index schema mapping and query support. * Query criteria referencing Party now references a String (until Identity framework built out). Added participants attribute to general query criteria. * Fleshed our IndexCriteria including PR recommendation to define column aliases for index mappings. * Removed all directly exposed API dependencies on requery. * Updated documentation. * Provide sensible defaults for all Query arguments. Add RPC Java helpers and increase range of Vault Service helpers. * Further improvements (upgrading notes) and updates to documentation. * RST documentation updates. * Updates to address RP latest set of review comments. * Updates to address MH latest set of review comments. * Updated to highlight use of VaultIndexQueryCriteria to directly reference a JPA-annotated entity (versus the indirect, explicitly mapped attribute to GenericIndexSchema approach) * Aesthetic updates requested by MH * Reverted Indexing approach: removed all references to VaultIndexedQueryCriteria and GenericVaultIndexSchemaV1 scheme. * Final clean-up and minor updates prior to merge. * Fixed compiler warnings (except deprecation warnings) * Reverted all changes to Vault Schemas (except simple illustrative VaultLinearState used in VaultQueryTests) * Reverted all changes to Vault Schemas (except simple illustrative VaultLinearState used in VaultQueryTests) * Commented out @Deprecated annotations (as a hedge against us releasing M12 with the work half-done) * Renamed RPC JavaHelper functions as RPCDispatcher does not allow more than one method with same name.
2017-05-05 14:14:43 +00:00
* a :doc:`vault-query` API is exposed to developers using standard Corda RPC and CorDapp plugin mechanisms
* a vault update API is internally used by transaction recording flows.
* the vault database schemas are directly accessible via JDBC for customer joins and queries
Soft locking implementation using database coin selection Fix broken IssuerFlowTest Fix IssuerFlowTests after rebase. Resolve conflicts after rebase. Soft locking converted to use persistent store. Added additional optional 'includeLockStates' parameter in VaultService states API call. Added Vault softLocked states query API call. Fixed commercial paper failing test. Improved exception handling on soft locking UPDATE statement. Using SELECT FOR UPDATE to ensure correct soft locking data visibility. Db query operations moved out of mutex code (as locking managed by underlying DB) Adjusted logging severity levels. Adjusted logging severity levels. GenerateSpending now performing fine grained query for unconsumed states by joining with contract_cash_states table. Using H2 proprietary cummulative counting feature (using sessioni SET variables) Refactored and simplified HibernateObserver constructor to enable usage in JUnit tests. Event generator issues larger random amounts (10,000..1,000,000) to those than are spent (0..10,000) Adjusted Issue (5:1) and Exit (10:1) generation frequency vs spending. Minor fixes: added optional lockid into select for spending criteria, set notary, additional trace logging. Generate Cash Schema by default upon node start-up (as part of NodeSchemaService initialisation). Explicitly close JDBC statements in finally() blocks. Tightened HibernateObserver constructor. Fix CommercialPaper test (was missing auto-generation of CONTRACT_CASH table) Revert default JVM size back to 200Mb. Revert default number of iterations in Explorer Node Simulation mode (back to 10000 with .5 sec sleep interval). Remove redundant setter function. Added TODO messages indicating Requery / H2 restrictions & caveats. Consumed states lock updates now performed in general consumed state Update. Updated/added Soft Locking documentation. Addressed initial PR comments: use THREAD_LOCAL_KRYO, use AbstractParty, extract helper method, improve readability, address some doc typos Addressed PR comment: removed lockId from WireTransaction. Fixed soft locking UPDATE statements. Improvements to VaultSoftLockManager for auto-registration of soft locks for flows with spendable states (as notifications from vault). Other optimisations (IssuerFlow no longer explicitly reserve/release issued state) and improvements (soft lock release management of soft locks, docs update) Performance update: now using Requery for UPDATE in release soft locking (non-composite key statement) Removed redundant TODO messages (TODO: revisit Kryo bug when using THREAD_LOCAL_KYRO) Minor fixes following rebase Fixed failing JUnit following rebase Addressed MH PR review items (1st pass) Fix broken JUnit Significant changes to RDBMS operations within coin selection and soft locking as requested by PR review. (Removed SELECT FOR UPDATE; added RETRY upon coin selection; reverting partial soft locks) Addressed a number of PR review requests added by MH (comments/spelling, lockID instantiation, HibernateObserver instantiation, cash schema white-listing usage) Addressed latest PR review comments from RP. Minor fixes following rebase from master. Fixed final failing JUnit (issuer flow concurrent). Updated TraderDemo to trigger concurrent issuance of cash. Fixed compiler warning on lockId null check. Fixed subtle bug in coin selection intermittently surfaced in IntegrationTestTutorial. Fixed small memory leak. Removed stray } in logger trace message. Slight rewording of description of Soft Locking in docs. Renamed NoStatesAvailableException to StatesNotAvailableException. generateSpend is now Suspendable (calls sleep method on flow upon coin selection retry). Added companion function to enable a Strand to sleep but without locking transactional context. Improved logging, changed to StateNotAvailableException, using Flow sleep upon retry, tweaked SELECT criteria in coin selection, fixed bug when insufficient states selectable, generateSpend is now @suspendable Improved handling and logging of flow results in Simulation Mode. Fixed minor error in sleep when not an active flow. Retry coin selection when unavailable states (as these may become available as new states). Additional debug logging to highlight and identify H2 coin selection sporadic bug. Inlined sleep method due to intermittent Quasar error. Re-introduce selection clause that prevents selection and temporary locking of already locked states (by other flows). Improved trace logging for coin selection (SQL row level info). Correctly calling FlowStateMachineImpl sleep (now inlined and working correctly) Fixed rebase error. Remove redundant TODO message.
2017-03-27 16:12:33 +00:00
Section 8 of the `Technical white paper`_ describes features of the vault yet to be implemented including private key managament, state splitting and merging, asset re-issuance and node event scheduling.
.. _`Technical white paper`: _static/corda-technical-whitepaper.pdf