Vault Query API design (#522)

* Added queryBy(QueryCriteria) Vault API and Junit tests.

* Minor fix following rebase.

* Spit out Vault Query tests into separate source file.

* WIP

* Enable composition of QueryCriteria specifications.
Additional JUnit test cases to validate API.

* Added Deprecating annotations.
Added QueryCriteria for set of contractStateTypes

* Minor tweaks and additional JUnit test cases (chain of linear id)

* Added Java Junit tests and QueryCriteria builder support.

* Added API documentation (including coding snippets and examples).

* Added @JvmOverloads to QueryCriteria classes for easy of use from Java.

* Refactored QueryCriteria API to use composition via sealed data classes.

* Enable infix notation.

* Fixed typo.

* Clarified future work to enforce DB level permissioning.

* Moved PageSpec and Order from QueryCriteria to become parameters of Query itself.

* Moved PageSpec and Order from QueryCriteria to become parameters of Query itself.

* TokenType now specified as set of <Class> (was non extensible enum).

* Exposed new Vault Query API functions via RPC.

* Fixed compiler error in java test.

* Addressed a couple of minor PR review scomments from MH.

* Major updates following PR discussion and recommendations.

* All pagination and sorting arguments are optional (and constructed with sensible defaults).
Added Java helper functions for queryBy and trackBy interfaces.
Added Java trackBy unit tests.
Miscellaneous cleanup.

* Added Generic Index schema mapping and query support.

* Query criteria referencing Party now references a String (until Identity framework built out).
Added participants attribute to general query criteria.

* Fleshed our IndexCriteria including PR recommendation to define column aliases for index mappings.

* Removed all directly exposed API dependencies on requery.

* Updated documentation.

* Provide sensible defaults for all Query arguments.
Add RPC Java helpers and increase range of Vault Service helpers.

* Further improvements (upgrading notes) and updates to documentation.

* RST documentation updates.

* Updates to address RP latest set of review comments.

* Updates to address MH latest set of review comments.

* Updated to highlight use of VaultIndexQueryCriteria to directly reference a JPA-annotated entity (versus the indirect, explicitly mapped attribute to GenericIndexSchema approach)

* Aesthetic updates requested by MH

* Reverted Indexing approach: removed all references to VaultIndexedQueryCriteria and GenericVaultIndexSchemaV1 scheme.

* Final clean-up and minor updates prior to merge.

* Fixed compiler warnings (except deprecation warnings)

* Reverted all changes to Vault Schemas (except simple illustrative VaultLinearState used in VaultQueryTests)

* Reverted all changes to Vault Schemas (except simple illustrative VaultLinearState used in VaultQueryTests)

* Commented out @Deprecated annotations (as a hedge against us releasing M12 with the work half-done)

* Renamed RPC JavaHelper functions as RPCDispatcher does not allow more than one method with same name.
This commit is contained in:
josecoll
2017-05-05 15:14:43 +01:00
committed by GitHub
parent c062d48e6e
commit 8c3b9ac589
20 changed files with 1918 additions and 84 deletions

View File

@ -15,6 +15,9 @@ import net.corda.core.node.NodeInfo
import net.corda.core.node.services.NetworkMapCache
import net.corda.core.node.services.StateMachineTransactionMapping
import net.corda.core.node.services.Vault
import net.corda.core.node.services.vault.PageSpecification
import net.corda.core.node.services.vault.QueryCriteria
import net.corda.core.node.services.vault.Sort
import net.corda.core.serialization.CordaSerializable
import net.corda.core.transactions.SignedTransaction
import org.bouncycastle.asn1.x500.X500Name
@ -65,10 +68,64 @@ interface CordaRPCOps : RPCOps {
@RPCReturnsObservables
fun stateMachinesAndUpdates(): Pair<List<StateMachineInfo>, Observable<StateMachineUpdate>>
/**
* Returns a snapshot of vault states for a given query criteria (and optional order and paging specification)
*
* Generic vault query function which takes a [QueryCriteria] object to define filters,
* optional [PageSpecification] and optional [Sort] modification criteria (default unsorted),
* and returns a [Vault.Page] object containing the following:
* 1. states as a List of <StateAndRef> (page number and size defined by [PageSpecification])
* 2. states metadata as a List of [Vault.StateMetadata] held in the Vault States table.
* 3. the [PageSpecification] used in the query
* 4. a total number of results available (for subsequent paging if necessary)
*
* Note: a default [PageSpecification] is applied to the query returning the 1st page (indexed from 0) with up to 200 entries.
* It is the responsibility of the Client to request further pages and/or specify a more suitable [PageSpecification].
*/
// DOCSTART VaultQueryByAPI
fun <T : ContractState> vaultQueryBy(criteria: QueryCriteria = QueryCriteria.VaultQueryCriteria(),
paging: PageSpecification = PageSpecification(),
sorting: Sort = Sort(emptySet())): Vault.Page<T>
// DOCEND VaultQueryByAPI
/**
* Returns a snapshot (as per queryBy) and an observable of future updates to the vault for the given query criteria.
*
* Generic vault query function which takes a [QueryCriteria] object to define filters,
* optional [PageSpecification] and optional [Sort] modification criteria (default unsorted),
* and returns a [Vault.PageAndUpdates] object containing
* 1) a snapshot as a [Vault.Page] (described previously in [queryBy])
* 2) an [Observable] of [Vault.Update]
*
* Notes: the snapshot part of the query adheres to the same behaviour as the [queryBy] function.
* the [QueryCriteria] applies to both snapshot and deltas (streaming updates).
*/
// DOCSTART VaultTrackByAPI
@RPCReturnsObservables
fun <T : ContractState> vaultTrackBy(criteria: QueryCriteria = QueryCriteria.VaultQueryCriteria(),
paging: PageSpecification = PageSpecification(),
sorting: Sort = Sort(emptySet())): Vault.PageAndUpdates<T>
// DOCEND VaultTrackByAPI
// Note: cannot apply @JvmOverloads to interfaces nor interface implementations
// Java Helpers
// DOCSTART VaultQueryAPIJavaHelpers
fun <T : ContractState> vaultQueryByCriteria(criteria: QueryCriteria): Vault.Page<T> = vaultQueryBy(criteria = criteria)
fun <T : ContractState> vaultQueryByWithPagingSpec(criteria: QueryCriteria, paging: PageSpecification): Vault.Page<T> = vaultQueryBy(criteria, paging = paging)
fun <T : ContractState> vaultQueryByWithSorting(criteria: QueryCriteria, sorting: Sort): Vault.Page<T> = vaultQueryBy(criteria, sorting = sorting)
fun <T : ContractState> vaultTrackByCriteria(criteria: QueryCriteria): Vault.PageAndUpdates<T> = vaultTrackBy(criteria = criteria)
fun <T : ContractState> vaultTrackByWithPagingSpec(criteria: QueryCriteria, paging: PageSpecification): Vault.PageAndUpdates<T> = vaultTrackBy(criteria, paging = paging)
fun <T : ContractState> vaultTrackByWithSorting(criteria: QueryCriteria, sorting: Sort): Vault.PageAndUpdates<T> = vaultTrackBy(criteria, sorting = sorting)
// DOCEND VaultQueryAPIJavaHelpers
/**
* Returns a pair of head states in the vault and an observable of future updates to the vault.
*/
@RPCReturnsObservables
// TODO: Remove this from the interface
// @Deprecated("This function will be removed in a future milestone", ReplaceWith("vaultTrackBy(QueryCriteria())"))
fun vaultAndUpdates(): Pair<List<StateAndRef<ContractState>>, Observable<Vault.Update>>
/**

View File

@ -5,6 +5,9 @@ import com.google.common.util.concurrent.ListenableFuture
import net.corda.core.contracts.*
import net.corda.core.crypto.*
import net.corda.core.flows.FlowException
import net.corda.core.node.services.vault.PageSpecification
import net.corda.core.node.services.vault.QueryCriteria
import net.corda.core.node.services.vault.Sort
import net.corda.core.serialization.CordaSerializable
import net.corda.core.serialization.OpaqueBytes
import net.corda.core.toFuture
@ -16,6 +19,7 @@ import java.io.InputStream
import java.security.KeyPair
import java.security.PrivateKey
import java.security.PublicKey
import java.time.Instant
import java.util.*
/**
@ -88,8 +92,38 @@ class Vault<out T : ContractState>(val states: Iterable<StateAndRef<T>>) {
}
enum class StateStatus {
UNCONSUMED, CONSUMED
UNCONSUMED, CONSUMED, ALL
}
/**
* Returned in queries [VaultService.queryBy] and [VaultService.trackBy].
* A Page contains:
* 1) a [List] of actual [StateAndRef] requested by the specified [QueryCriteria] to a maximum of [MAX_PAGE_SIZE]
* 2) a [List] of associated [Vault.StateMetadata], one per [StateAndRef] result
* 3) the [PageSpecification] definition used to bound this result set
* 4) a total number of states that met the given [QueryCriteria]
* Note that this may be more than the specified [PageSpecification.pageSize], and should be used to perform
* further pagination (by issuing new queries).
*/
@CordaSerializable
data class Page<out T : ContractState>(val states: List<StateAndRef<T>>,
val statesMetadata: List<Vault.StateMetadata>,
val pageable: PageSpecification,
val totalStatesAvailable: Long)
@CordaSerializable
data class StateMetadata(val ref: StateRef,
val contractStateClassName: String,
val recordedTime: Instant,
val consumedTime: Instant?,
val status: Vault.StateStatus,
val notaryName: String,
val notaryKey: String,
val lockId: String?,
val lockUpdateTime: Instant?)
@CordaSerializable
data class PageAndUpdates<out T : ContractState> (val current: Vault.Page<T>, val future: Observable<Vault.Update>? = null)
}
/**
@ -129,12 +163,58 @@ interface VaultService {
* Atomically get the current vault and a stream of updates. Note that the Observable buffers updates until the
* first subscriber is registered so as to avoid racing with early updates.
*/
// TODO: Remove this from the interface
// @Deprecated("This function will be removed in a future milestone", ReplaceWith("trackBy(QueryCriteria())"))
fun track(): Pair<Vault<ContractState>, Observable<Vault.Update>>
// DOCSTART VaultQueryAPI
/**
* Generic vault query function which takes a [QueryCriteria] object to define filters,
* optional [PageSpecification] and optional [Sort] modification criteria (default unsorted),
* and returns a [Vault.Page] object containing the following:
* 1. states as a List of <StateAndRef> (page number and size defined by [PageSpecification])
* 2. states metadata as a List of [Vault.StateMetadata] held in the Vault States table.
* 3. the [PageSpecification] used in the query
* 4. a total number of results available (for subsequent paging if necessary)
*
* Note: a default [PageSpecification] is applied to the query returning the 1st page (indexed from 0) with up to 200 entries.
* It is the responsibility of the Client to request further pages and/or specify a more suitable [PageSpecification].
*/
fun <T : ContractState> queryBy(criteria: QueryCriteria = QueryCriteria.VaultQueryCriteria(),
paging: PageSpecification = PageSpecification(),
sorting: Sort = Sort(emptySet())): Vault.Page<T>
/**
* Generic vault query function which takes a [QueryCriteria] object to define filters,
* optional [PageSpecification] and optional [Sort] modification criteria (default unsorted),
* and returns a [Vault.PageAndUpdates] object containing
* 1) a snapshot as a [Vault.Page] (described previously in [queryBy])
* 2) an [Observable] of [Vault.Update]
*
* Notes: the snapshot part of the query adheres to the same behaviour as the [queryBy] function.
* the [QueryCriteria] applies to both snapshot and deltas (streaming updates).
*/
fun <T : ContractState> trackBy(criteria: QueryCriteria = QueryCriteria.VaultQueryCriteria(),
paging: PageSpecification = PageSpecification(),
sorting: Sort = Sort(emptySet())): Vault.PageAndUpdates<T>
// DOCEND VaultQueryAPI
// Note: cannot apply @JvmOverloads to interfaces nor interface implementations
// Java Helpers
fun <T : ContractState> queryBy(): Vault.Page<T> = queryBy()
fun <T : ContractState> queryBy(criteria: QueryCriteria): Vault.Page<T> = queryBy(criteria)
fun <T : ContractState> queryBy(criteria: QueryCriteria, paging: PageSpecification): Vault.Page<T> = queryBy(criteria, paging)
fun <T : ContractState> queryBy(criteria: QueryCriteria, sorting: Sort): Vault.Page<T> = queryBy(criteria, sorting)
fun <T : ContractState> trackBy(): Vault.PageAndUpdates<T> = trackBy()
fun <T : ContractState> trackBy(criteria: QueryCriteria): Vault.PageAndUpdates<T> = trackBy(criteria)
fun <T : ContractState> trackBy(criteria: QueryCriteria, paging: PageSpecification): Vault.PageAndUpdates<T> = trackBy(criteria, paging)
fun <T : ContractState> trackBy(criteria: QueryCriteria, sorting: Sort): Vault.PageAndUpdates<T> = trackBy(criteria, Sort(emptySet()))
/**
* Return unconsumed [ContractState]s for a given set of [StateRef]s
* TODO: revisit and generalize this exposed API function.
*/
// TODO: Remove this from the interface
// @Deprecated("This function will be removed in a future milestone", ReplaceWith("queryBy(VaultQueryCriteria(stateRefs = listOf(<StateRef>)))"))
fun statesForRefs(refs: List<StateRef>): Map<StateRef, TransactionState<*>?>
/**
@ -212,6 +292,8 @@ interface VaultService {
* Return [ContractState]s of a given [Contract] type and [Iterable] of [Vault.StateStatus].
* Optionally may specify whether to include [StateRef] that have been marked as soft locked (default is true)
*/
// TODO: Remove this from the interface
// @Deprecated("This function will be removed in a future milestone", ReplaceWith("queryBy(QueryCriteria())"))
fun <T : ContractState> states(clazzes: Set<Class<T>>, statuses: EnumSet<Vault.StateStatus>, includeSoftLockedStates: Boolean = true): Iterable<StateAndRef<T>>
// DOCEND VaultStatesQuery
@ -257,17 +339,25 @@ interface VaultService {
fun <T : ContractState> unconsumedStatesForSpending(amount: Amount<Currency>, onlyFromIssuerParties: Set<AbstractParty>? = null, notary: Party? = null, lockId: UUID, withIssuerRefs: Set<OpaqueBytes>? = null): List<StateAndRef<T>>
}
// TODO: Remove this from the interface
// @Deprecated("This function will be removed in a future milestone", ReplaceWith("queryBy(VaultQueryCriteria())"))
inline fun <reified T : ContractState> VaultService.unconsumedStates(includeSoftLockedStates: Boolean = true): Iterable<StateAndRef<T>> =
states(setOf(T::class.java), EnumSet.of(Vault.StateStatus.UNCONSUMED), includeSoftLockedStates)
// TODO: Remove this from the interface
// @Deprecated("This function will be removed in a future milestone", ReplaceWith("queryBy(VaultQueryCriteria(status = Vault.StateStatus.CONSUMED))"))
inline fun <reified T : ContractState> VaultService.consumedStates(): Iterable<StateAndRef<T>> =
states(setOf(T::class.java), EnumSet.of(Vault.StateStatus.CONSUMED))
/** Returns the [linearState] heads only when the type of the state would be considered an 'instanceof' the given type. */
// TODO: Remove this from the interface
// @Deprecated("This function will be removed in a future milestone", ReplaceWith("queryBy(LinearStateQueryCriteria(linearId = listOf(<UniqueIdentifier>)))"))
inline fun <reified T : LinearState> VaultService.linearHeadsOfType() =
states(setOf(T::class.java), EnumSet.of(Vault.StateStatus.UNCONSUMED))
.associateBy { it.state.data.linearId }.mapValues { it.value }
// TODO: Remove this from the interface
// @Deprecated("This function will be removed in a future milestone", ReplaceWith("queryBy(LinearStateQueryCriteria(dealPartyName = listOf(<String>)))"))
inline fun <reified T : DealState> VaultService.dealsWith(party: AbstractParty) = linearHeadsOfType<T>().values.filter {
it.state.data.parties.any { it == party }
}

View File

@ -0,0 +1,85 @@
package net.corda.core.node.services.vault
import net.corda.core.contracts.Commodity
import net.corda.core.contracts.ContractState
import net.corda.core.contracts.StateRef
import net.corda.core.contracts.UniqueIdentifier
import net.corda.core.node.services.Vault
import net.corda.core.node.services.vault.QueryCriteria.AndComposition
import net.corda.core.node.services.vault.QueryCriteria.OrComposition
import net.corda.core.serialization.CordaSerializable
import net.corda.core.serialization.OpaqueBytes
import java.time.Instant
import java.util.*
/**
* Indexing assumptions:
* QueryCriteria assumes underlying schema tables are correctly indexed for performance.
*/
@CordaSerializable
sealed class QueryCriteria {
/**
* VaultQueryCriteria: provides query by attributes defined in [VaultSchema.VaultStates]
*/
data class VaultQueryCriteria @JvmOverloads constructor (
val status: Vault.StateStatus = Vault.StateStatus.UNCONSUMED,
val stateRefs: List<StateRef>? = null,
val contractStateTypes: Set<Class<out ContractState>>? = null,
val notaryName: List<String>? = null,
val includeSoftlockedStates: Boolean? = true,
val timeCondition: Logical<TimeInstantType, Array<Instant>>? = null,
val participantIdentities: List<String>? = null) : QueryCriteria()
/**
* LinearStateQueryCriteria: provides query by attributes defined in [VaultSchema.VaultLinearState]
*/
data class LinearStateQueryCriteria @JvmOverloads constructor(
val linearId: List<UniqueIdentifier>? = null,
val latestOnly: Boolean? = true,
val dealRef: List<String>? = null,
val dealPartyName: List<String>? = null) : QueryCriteria()
/**
* FungibleStateQueryCriteria: provides query by attributes defined in [VaultSchema.VaultFungibleState]
*
* Valid TokenType implementations defined by Amount<T> are
* [Currency] as used in [Cash] contract state
* [Commodity] as used in [CommodityContract] state
*/
data class FungibleAssetQueryCriteria @JvmOverloads constructor(
val ownerIdentity: List<String>? = null,
val quantity: Logical<*,Long>? = null,
val tokenType: List<Class<out Any>>? = null,
val tokenValue: List<String>? = null,
val issuerPartyName: List<String>? = null,
val issuerRef: List<OpaqueBytes>? = null,
val exitKeyIdentity: List<String>? = null) : QueryCriteria()
/**
* VaultCustomQueryCriteria: provides query by custom attributes defined in a contracts
* [QueryableState] implementation.
* (see Persistence documentation for more information)
*
* Params
* [indexExpression] refers to a (composable) JPA Query like WHERE expression clauses of the form:
* [JPA entityAttributeName] [Operand] [Value]
*
* Refer to [CommercialPaper.State] for a concrete example.
*/
data class VaultCustomQueryCriteria<L,R>(val indexExpression: Logical<L,R>? = null) : QueryCriteria()
// enable composition of [QueryCriteria]
data class AndComposition(val a: QueryCriteria, val b: QueryCriteria): QueryCriteria()
data class OrComposition(val a: QueryCriteria, val b: QueryCriteria): QueryCriteria()
// timestamps stored in the vault states table [VaultSchema.VaultStates]
@CordaSerializable
enum class TimeInstantType {
RECORDED,
CONSUMED
}
}
infix fun QueryCriteria.and(criteria: QueryCriteria): QueryCriteria = AndComposition(this, criteria)
infix fun QueryCriteria.or(criteria: QueryCriteria): QueryCriteria = OrComposition(this, criteria)

View File

@ -0,0 +1,113 @@
package net.corda.core.node.services.vault
import net.corda.core.serialization.CordaSerializable
@CordaSerializable
enum class Operator {
AND,
OR,
EQUAL,
NOT_EQUAL,
LESS_THAN,
LESS_THAN_OR_EQUAL,
GREATER_THAN,
GREATER_THAN_OR_EQUAL,
IN,
NOT_IN,
LIKE,
NOT_LIKE,
BETWEEN,
IS_NULL,
NOT_NULL
}
interface Condition<L, R> {
val leftOperand: L
val operator: Operator
val rightOperand: R
}
interface AndOr<out Q> {
infix fun <V> and(condition: Condition<V, *>): Q
infix fun <V> or(condition: Condition<V, *>): Q
}
@CordaSerializable
sealed class Logical<L, R> : Condition<L, R>, AndOr<Logical<*, *>>
class LogicalExpression<L, R>(leftOperand: L,
operator: Operator,
rightOperand: R? = null) : Logical<L, R>() {
init {
if (rightOperand == null) {
check(operator in setOf(Operator.NOT_NULL, Operator.IS_NULL),
{ "Must use a unary operator (${Operator.IS_NULL}, ${Operator.NOT_NULL}) if right operand is null"} )
}
else {
check(operator !in setOf(Operator.NOT_NULL, Operator.IS_NULL),
{ "Cannot use a unary operator (${Operator.IS_NULL}, ${Operator.NOT_NULL}) if right operand is not null"} )
}
}
override fun <V> and(condition: Condition<V, *>): Logical<*, *> = LogicalExpression(this, Operator.AND, condition)
override fun <V> or(condition: Condition<V, *>): Logical<*, *> = LogicalExpression(this, Operator.OR, condition)
override val operator: Operator = operator
override val rightOperand: R = rightOperand as R
override val leftOperand: L = leftOperand
}
/**
* Pagination and Ordering
*
* Provide simple ability to specify an offset within a result set and the number of results to
* return from that offset (eg. page size) together with (optional) sorting criteria at column level.
*
* Note: it is the responsibility of the calling client to manage page windows.
*
* For advanced pagination it is recommended you utilise standard JPA query frameworks such as
* Spring Data's JPARepository which extends the [PagingAndSortingRepository] interface to provide
* paging and sorting capability:
* https://docs.spring.io/spring-data/commons/docs/current/api/org/springframework/data/repository/PagingAndSortingRepository.html
*/
val DEFAULT_PAGE_NUM = 0L
val DEFAULT_PAGE_SIZE = 200L
/**
* Note: this maximum size will be configurable in future (to allow for large JVM heap sized node configurations)
* Use [PageSpecification] to correctly handle a number of bounded pages of [MAX_PAGE_SIZE].
*/
val MAX_PAGE_SIZE = 512L
/**
* PageSpecification allows specification of a page number (starting from 0 as default) and page size (defaulting to
* [DEFAULT_PAGE_SIZE] with a maximum page size of [DEFAULT_PAGE_SIZE]
*/
@CordaSerializable
data class PageSpecification(val pageNumber: Long = DEFAULT_PAGE_NUM, val pageSize: Long = DEFAULT_PAGE_SIZE)
/**
* Sort allows specification of a set of entity attribute names and their associated directionality
* and null handling, to be applied upon processing a query specification.
*/
@CordaSerializable
data class Sort(val columns: Collection<SortColumn>) {
@CordaSerializable
enum class Direction {
ASC,
DESC
}
@CordaSerializable
enum class NullHandling {
NULLS_FIRST,
NULLS_LAST
}
/**
* [columnName] should reference a persisted entity attribute name as defined by the associated mapped schema
* (for example, [VaultSchema.VaultStates::txId.name])
*/
@CordaSerializable
data class SortColumn(val columnName: String, val direction: Sort.Direction = Sort.Direction.ASC,
val nullHandling: Sort.NullHandling = if (direction == Sort.Direction.ASC) Sort.NullHandling.NULLS_LAST else Sort.NullHandling.NULLS_FIRST)
}