corda/docs/source/api-vault-query.rst

API: Vault Query

Corda has been architected from the ground up to encourage usage of industry standard, proven query frameworks and libraries for accessing RDBMS backed transactional stores (including the Vault).

Corda provides a number of flexible query mechanisms for accessing the Vault:

• Vault Query API
• custom JPA/JPQL queries
• custom 3rd party Data Access frameworks such as Spring Data

The majority of query requirements can be satisfied by using the Vault Query API, which is exposed via the VaultQueryService for use directly by flows:

../../core/src/main/kotlin/net/corda/core/node/services/VaultQueryService.kt

and via CordaRPCOps for use by RPC client applications:

../../core/src/main/kotlin/net/corda/core/messaging/CordaRPCOps.kt

../../core/src/main/kotlin/net/corda/core/messaging/CordaRPCOps.kt

Helper methods are also provided with default values for arguments:

../../core/src/main/kotlin/net/corda/core/messaging/CordaRPCOps.kt

../../core/src/main/kotlin/net/corda/core/messaging/CordaRPCOps.kt

The API provides both static (snapshot) and dynamic (snapshot with streaming updates) methods for a defined set of filter criteria.

• Use queryBy to obtain a only current snapshot of data (for a given QueryCriteria)
• Use trackBy to obtain a both a current snapshot and a future stream of updates (for a given QueryCriteria)

Note

Streaming updates are only filtered based on contract type and state status (UNCONSUMED, CONSUMED, ALL)

Simple pagination (page number and size) and sorting (directional ordering using standard or custom property attributes) is also specifiable. Defaults are defined for Paging (pageNumber = 1, pageSize = 200) and Sorting (direction = ASC).

The QueryCriteria interface provides a flexible mechanism for specifying different filtering criteria, including and/or composition and a rich set of operators to include: binary logical (AND, OR), comparison (LESS_THAN, LESS_THAN_OR_EQUAL, GREATER_THAN, GREATER_THAN_OR_EQUAL), equality (EQUAL, NOT_EQUAL), likeness (LIKE, NOT_LIKE), nullability (IS_NULL, NOT_NULL), and collection based (IN, NOT_IN). Standard SQL-92 aggregate functions (SUM, AVG, MIN, MAX, COUNT) are also supported.

There are four implementations of this interface which can be chained together to define advanced filters.

1. VaultQueryCriteria provides filterable criteria on attributes within the Vault states table: status (UNCONSUMED, CONSUMED), state reference(s), contract state type(s), notaries, soft locked states, timestamps (RECORDED, CONSUMED).

   Note

   Sensible defaults are defined for frequently used attributes (status = UNCONSUMED, always include soft locked states).

2. FungibleAssetQueryCriteria provides filterable criteria on attributes defined in the Corda Core FungibleAsset contract state interface, used to represent assets that are fungible, countable and issued by a specific party (eg. Cash.State and CommodityContract.State in the Corda finance module). Filterable attributes include: participants(s), owner(s), quantity, issuer party(s) and issuer reference(s).

   Note

   All contract states that extend the FungibleAsset now automatically persist that interfaces common state attributes to the vault_fungible_states table.

3. LinearStateQueryCriteria provides filterable criteria on attributes defined in the Corda Core LinearState and DealState contract state interfaces, used to represent entities that continuously supercede themselves, all of which share the same linearId (eg. trade entity states such as the IRSState defined in the SIMM valuation demo). Filterable attributes include: participant(s), linearId(s), dealRef(s).

   Note

   All contract states that extend LinearState or DealState now automatically persist those interfaces common state attributes to the vault_linear_states table.

4. VaultCustomQueryCriteria provides the means to specify one or many arbitrary expressions on attributes defined by a custom contract state that implements its own schema as described in the Persistence </api-persistence> documentation and associated examples. Custom criteria expressions are expressed using one of several type-safe CriteriaExpression: BinaryLogical, Not, ColumnPredicateExpression, AggregateFunctionExpression. The ColumnPredicateExpression allows for specification arbitrary criteria using the previously enumerated operator types. The AggregateFunctionExpression allows for the specification of an aggregate function type (sum, avg, max, min, count) with optional grouping and sorting. Furthermore, a rich DSL is provided to enable simple construction of custom criteria using any combination of ColumnPredicate. See the Builder object in QueryCriteriaUtils for a complete specification of the DSL.

   Note

   It is a requirement to register any custom contract schemas to be used in Vault Custom queries in the associated CordaPluginRegistry configuration for the respective CorDapp using the requiredSchemas configuration field (which specifies a set of MappedSchema)

An example of a custom query is illustrated here:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

All QueryCriteria implementations are composable using and and or operators, as also illustrated above.

All QueryCriteria implementations provide an explicitly specifiable StateStatus attribute which defaults to filtering on UNCONSUMED states.

Note

Custom contract states that implement the Queryable interface may now extend common schemas types FungiblePersistentState or, LinearPersistentState. Previously, all custom contracts extended the root PersistentState class and defined repeated mappings of FungibleAsset and LinearState attributes. See SampleCashSchemaV2 and DummyLinearStateSchemaV2 as examples.

Examples of these QueryCriteria objects are presented below for Kotlin and Java.

Note

When specifying the Contract Type as a parameterised type to the QueryCriteria in Kotlin, queries now include all concrete implementations of that type if this is an interface. Previously, it was only possible to query on Concrete types (or the universe of all Contract States).

The Vault Query API leverages the rich semantics of the underlying JPA Hibernate based Persistence </api-persistence> framework adopted by Corda.

Note

Permissioning at the database level will be enforced at a later date to ensure authenticated, role-based, read-only access to underlying Corda tables.

Note

API's now provide ease of use calling semantics from both Java and Kotlin. However, it should be noted that Java custom queries are significantly more verbose due to the use of reflection fields to reference schema attribute types.

An example of a custom query in Java is illustrated here:

../../node/src/test/java/net/corda/node/services/vault/VaultQueryJavaTests.java

Note

Current queries by Party specify the AbstractParty which may be concrete or anonymous. In the later case, where an anonymous party does not have an associated X500Name, then no query results will ever be produced. For performance reasons, queries do not use PublicKey as search criteria. Ongoing design work on identity manangement is likely to enhance identity based queries (including composite key criteria selection).

Pagination

The API provides support for paging where large numbers of results are expected (by default, a page size is set to 200 results). Defining a sensible default page size enables efficient checkpointing within flows, and frees the developer from worrying about pagination where result sets are expected to be constrained to 200 or fewer entries. Where large result sets are expected (such as using the RPC API for reporting and/or UI display), it is strongly recommended to define a PageSpecification to correctly process results with efficient memory utilistion. A fail-fast mode is in place to alert API users to the need for pagination where a single query returns more than 200 results and no PageSpecification has been supplied.

Note

A pages maximum size MAX_PAGE_SIZE is defined as Int.MAX_VALUE and should be used with extreme caution as results returned may exceed your JVM's memory footprint.

Example usage

Kotlin

General snapshot queries using VaultQueryCriteria

Query for all unconsumed states (simplest query possible):

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Query for unconsumed states for some state references:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Query for unconsumed states for several contract state types:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Query for unconsumed states for a given notary:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Note

We are using the notaries X500Name as our search identifier.

Query for unconsumed states for a given set of participants:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Query for unconsumed states recorded between two time intervals:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Note

This example illustrates usage of a Between ColumnPredicate.

Query for all states with pagination specification (10 results per page):

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Note

The result set metadata field totalStatesAvailable allows you to further paginate accordingly.

LinearState and DealState queries using LinearStateQueryCriteria

Query for unconsumed linear states for given linear ids:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Query for all linear states associated with a linear id:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Query for unconsumed deal states with deals references:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Query for unconsumed deal states with deals parties:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

FungibleAsset and DealState queries using FungibleAssetQueryCriteria

Query for fungible assets for a given currency:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Query for fungible assets for a minimum quantity:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Note

This example uses the builder DSL.

Query for fungible assets for a specifc issuer party:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Aggregate Function queries using VaultCustomQueryCriteria

Note

Query results for aggregate functions are contained in the otherResults attribute of a results Page.

Aggregations on cash using various functions:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Note

otherResults will contain 5 items, one per calculated aggregate function.

Aggregations on cash grouped by currency for various functions:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Note

otherResults will contain 24 items, one result per calculated aggregate function per currency (the grouping attribute - currency in this case - is returned per aggregate result).

Sum aggregation on cash grouped by issuer party and currency and sorted by sum:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Note

otherResults will contain 12 items sorted from largest summed cash amount to smallest, one result per calculated aggregate function per issuer party and currency (grouping attributes are returned per aggregate result).

Dynamic queries (also using VaultQueryCriteria) are an extension to the snapshot queries by returning an additional QueryResults return type in the form of an Observable<Vault.Update>. Refer to ReactiveX Observable for a detailed understanding and usage of this type.

Track unconsumed cash states:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Track unconsumed linear states:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Note

This will return both Deal and Linear states.

Track unconsumed deal states:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt

Note

This will return only Deal states.

Java examples

Query for all unconsumed linear states:

../../node/src/test/java/net/corda/node/services/vault/VaultQueryJavaTests.java

Query for all consumed cash states:

../../node/src/test/java/net/corda/node/services/vault/VaultQueryJavaTests.java

Query for consumed deal states or linear ids, specify a paging specification and sort by unique identifier:

../../node/src/test/java/net/corda/node/services/vault/VaultQueryJavaTests.java

Aggregate Function queries using VaultCustomQueryCriteria

Aggregations on cash using various functions:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryJavaTests.kt

Aggregations on cash grouped by currency for various functions:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryJavaTests.kt

Sum aggregation on cash grouped by issuer party and currency and sorted by sum:

../../node/src/test/kotlin/net/corda/node/services/vault/VaultQueryJavaTests.kt

Track unconsumed cash states:

../../node/src/test/java/net/corda/node/services/vault/VaultQueryJavaTests.java

Track unconsumed deal states or linear states (with snapshot including specification of paging and sorting by unique identifier):

../../node/src/test/java/net/corda/node/services/vault/VaultQueryJavaTests.java

Behavioural notes

1. TrackBy updates do not take into account the full criteria specification due to different and more restrictive syntax in observables filtering (vs full SQL-92 JDBC filtering as used in snapshot views). Specifically, dynamic updates are filtered by contractType and stateType (UNCONSUMED, CONSUMED, ALL) only.
2. QueryBy and TrackBy snapshot views using pagination may return different result sets as each paging request is a separate SQL query on the underlying database, and it is entirely conceivable that state modifications are taking place in between and/or in parallel to paging requests. When using pagination, always check the value of the totalStatesAvailable (from the Vault.Page result) and adjust further paging requests appropriately.

Other use case scenarios

For advanced use cases that require sophisticated pagination, sorting, grouping, and aggregation functions, it is recommended that the CorDapp developer utilise one of the many proven frameworks that ship with this capability out of the box. Namely, implementations of JPQL (JPA Query Language) such as Hibernate for advanced SQL access, and Spring Data for advanced pagination and ordering constructs.

The Corda Tutorials provide examples satisfying these additional Use Cases:

1. Template / Tutorial CorDapp service using Vault API Custom Query to access attributes of IOU State
2. Template / Tutorial CorDapp service query extension executing Named Queries via JPQL
3. Advanced pagination queries using Spring Data JPA

Upgrading from previous releases

Here follows a selection of the most common upgrade scenarios:

1. ServiceHub usage to obtain Unconsumed states for a given contract state type

   Previously:

val yoStates = b.vault.unconsumedStates<Yo.State>()

This query returned an Iterable<StateAndRef<T>>

Now:

val yoStates = b.vault.queryBy<Yo.State>().states

The query returns a Vault.Page result containing:

• states as a List<StateAndRef<T : ContractState>> up to a maximum of DEFAULT_PAGE_SIZE (200) where no PageSpecification provided, otherwise returns results according to the parameters pageNumber and pageSize specified in the supplied PageSpecification.
• states metadata as a List<Vault.StateMetadata> containing Vault State metadata held in the Vault states table.
• a total number of results available if PageSpecification provided (otherwise returns -1). For pagination, this value can be used to issue subsequent queries with appropriately specified PageSpecification parameters (according to your paging needs and/or maximum memory capacity for holding large data sets). Note it is your responsibility to manage page numbers and sizes.
• status types used in this query: UNCONSUMED, CONSUMED, ALL
• other results as a [List] of any type (eg. aggregate function results with/without group by)

2. ServiceHub usage obtaining linear heads for a given contract state type

   Previously:

val iouStates = serviceHub.vaultService.linearHeadsOfType<IOUState>()
val iouToSettle = iouStates[linearId] ?: throw Exception("IOUState with linearId $linearId not found.")

Now:

val criteria = QueryCriteria.LinearStateQueryCriteria(linearId = listOf(linearId))
val iouStates = serviceHub.vaultService.queryBy<IOUState>(criteria).states

val iouToSettle = iouStates.singleOrNull() ?: throw Exception("IOUState with linearId $linearId not found.")

3. RPC usage was limited to using the vaultAndUpdates RPC method, which returned a snapshot and streaming updates as an Observable. In many cases, queries were not interested in the streaming updates.

   Previously:

val iouStates = services.vaultAndUpdates().first.filter { it.state.data is IOUState }

Now:

val iouStates = services.vaultQueryBy<IOUState>()