corda/docs/source/api-persistence.rst

API: Persistence

Corda offers developers the option to expose all or some parts of a contract state to an Object Relational Mapping (ORM) tool to be persisted in a Relational Database Management System (RDBMS).

The purpose of this, is to assist vault development and allow for the persistence of state data to a custom database table. Persisted states held in the vault are indexed for the purposes of executing queries. This also allows for relational joins between Corda tables and the organization's existing data.

The Object Relational Mapping is specified using Java Persistence API (JPA) annotations. This mapping is persisted to the database as a table row (a single, implicitly structured data item) by the node automatically every time a state is recorded in the node's local vault as part of a transaction.

Note

By default, nodes use an H2 database which is accessed using Java Database Connectivity JDBC. Any database with a JDBC driver is a candidate and several integrations have been contributed to by the community. Please see the info in "node-database" for details.

Schemas

Every ContractState may implement the QueryableState interface if it wishes to be inserted into a custom table in the node's database and made accessible using SQL.

../../core/src/main/kotlin/net/corda/core/schemas/PersistentTypes.kt

The QueryableState interface requires the state to enumerate the different relational schemas it supports, for instance in situations where the schema has evolved. Each relational schema is represented as a MappedSchema object returned by the state's supportedSchemas method.

Nodes have an internal SchemaService which decides what data to persist by selecting the MappedSchema to use. Once a MappedSchema is selected, the SchemaService will delegate to the QueryableState to generate a corresponding representation (mapped object) via the generateMappedObject method, the output of which is then passed to the ORM.

../../node/src/main/kotlin/net/corda/node/services/api/SchemaService.kt

../../core/src/main/kotlin/net/corda/core/schemas/PersistentTypes.kt

With this framework, the relational view of ledger states can evolve in a controlled fashion in lock-step with internal systems or other integration points and is not dependant on changes to the contract code.

It is expected that multiple contract state implementations might provide mappings within a single schema. For example an Interest Rate Swap contract and an Equity OTC Option contract might both provide a mapping to a Derivative contract within the same schema. The schemas should typically not be part of the contract itself and should exist independently to encourage re-use of a common set within a particular business area or Cordapp.

Note

It's advisable to avoid cross-references between different schemas as this may cause issues when evolving MappedSchema or migrating its data. At startup, nodes log such violations as warnings stating that there's a cross-reference between MappedSchema's. The detailed messages incorporate information about what schemas, entities and fields are involved.

MappedSchema offer a family name that is disambiguated using Java package style name-spacing derived from the class name of a schema family class that is constant across versions, allowing the SchemaService to select a preferred version of a schema.

The SchemaService is also responsible for the SchemaOptions that can be configured for a particular MappedSchema. These allow the configuration of database schemas or table name prefixes to avoid clashes with other MappedSchema.

Note

It is intended that there should be plugin support for the SchemaService to offer version upgrading, implementation of additional schemas, and enable active schemas as being configurable. The present implementation does not include these features and simply results in all versions of all schemas supported by a QueryableState being persisted. This will change in due course. Similarly, the service does not currently support configuring SchemaOptions but will do so in the future.

Custom schema registration

Custom contract schemas are automatically registered at startup time for CorDapps. The node bootstrap process will scan for states that implement the Queryable state interface. Tables are then created as specified by the MappedSchemas identified by each states supportedSchemas method.

For testing purposes it is necessary to manually register the packages containing custom schemas as follows:

• Tests using MockNetwork and MockNode must explicitly register packages using the cordappPackages parameter of MockNetwork
• Tests using MockServices must explicitly register packages using the cordappPackages parameter of the MockServices makeTestDatabaseAndMockServices() helper method.

Note

Tests using the DriverDSL will automatically register your custom schemas if they are in the same project structure as the driver call.

Object relational mapping

To facilitate the ORM, the persisted representation of a QueryableState should be an instance of a PersistentState subclass, constructed either by the state itself or a plugin to the SchemaService. This allows the ORM layer to always associate a StateRef with a persisted representation of a ContractState and allows joining with the set of unconsumed states in the vault.

The PersistentState subclass should be marked up as a JPA 2.1 Entity with a defined table name and having properties (in Kotlin, getters/setters in Java) annotated to map to the appropriate columns and SQL types. Additional entities can be included to model these properties where they are more complex, for example collections (Persisting Hierarchical Data<persisting-hierarchical-data>), so the mapping does not have to be flat. The MappedSchema constructor accepts a list of all JPA entity classes for that schema in the MappedTypes parameter. It must provide this list in order to initialise the ORM layer.

Several examples of entities and mappings are provided in the codebase, including Cash.State and CommercialPaper.State. For example, here's the first version of the cash schema.

../../finance/src/main/kotlin/net/corda/finance/schemas/CashSchemaV1.kt

Note

If Cordapp needs to be portable between Corda OS (running against H2) and Corda Enterprise (running against a standalone database), consider database vendors specific requirements. Ensure that table and column names are compatible with the naming convention of the database vendors for which the Cordapp will be deployed, e.g. for Oracle database, prior to version 12.2 the maximum length of table/column name is 30 bytes (the exact number of characters depends on the database encoding).

Persisting Hierarchical Data

You may wish to persist hierarchical relationships within states using multiple database tables

You may wish to persist hierarchical relationships within state data using multiple database tables. In order to facillitate this, multiple PersistentState subclasses may be implemented. The relationship between these classes is defined using JPA annotations. It is important to note that the MappedSchema constructor requires a list of all of these subclasses.

An example Schema implementing hierarchical relationships with JPA annotations has been implemented below. This Schema will cause parent_data and child_data` tables to be created. .. container:: codeset .. sourcecode:: java @CordaSerializable public class SchemaV1 extends MappedSchema { /** * This class must extend the MappedSchema class. Its name is based on the SchemaFamily name and the associated version number abbreviation (V1, V2... Vn). * In the constructor, use the super keyword to call the constructor of MappedSchema with the following arguments: a class literal representing the schema family, * a version number and a collection of mappedTypes (class literals) which represent JPA entity classes that the ORM layer needs to be configured with for this schema. */ public SchemaV1() { super(Schema.class, 1, ImmutableList.of(PersistentParentToken.class, PersistentChildToken.class)); } /** * The @entity annotation signifies that the specified POJO class' non-transient fields should be persisted to a relational database using the services * of an entity manager. The @table annotation specifies properties of the table that will be created to contain the persisted data, in this case we have * specified a name argument which will be used the table's title. */ @Entity @Table(name = "parent_data") public static class PersistentParentToken extends PersistentState { /** * The @Column annotations specify the columns that will comprise the inserted table and specify the shape of the fields and associated * data types of each database entry. */ @Column(name = "owner") private final String owner; @Column(name = "issuer") private final String issuer; @Column(name = "amount") private final int amount; @Column(name = "linear_id") public final UUID linearId; /** * The @OneToMany annotation specifies a one-to-many relationship between this class and a collection included as a field. * The @JoinColumn and @JoinColumns annotations specify on which columns these tables will be joined on. */ @OneToMany(cascade = CascadeType.PERSIST) @JoinColumns({ @JoinColumn(name = "output_index", referencedColumnName = "output_index"), @JoinColumn(name = "transaction_id", referencedColumnName = "transaction_id"), }) private final List<PersistentChildToken> listOfPersistentChildTokens; public PersistentParentToken(String owner, String issuer, int amount, UUID linearId, List<PersistentChildToken> listOfPersistentChildTokens) { this.owner = owner; this.issuer = issuer; this.amount = amount; this.linearId = linearId; this.listOfPersistentChildTokens = listOfPersistentChildTokens; } // Default constructor required by hibernate. public PersistentParentToken() { this.owner = ""; this.issuer = ""; this.amount = 0; this.linearId = UUID.randomUUID(); this.listOfPersistentChildTokens = null; } public String getOwner() { return owner; } public String getIssuer() { return issuer; } public int getAmount() { return amount; } public UUID getLinearId() { return linearId; } public List<PersistentChildToken> getChildTokens() { return listOfPersistentChildTokens; } } @Entity @CordaSerializable @Table(name = "child_data") public static class PersistentChildToken { // The @Id annotation marks this field as the primary key of the persisted entity. @Id private final UUID Id; @Column(name = "owner") private final String owner; @Column(name = "issuer") private final String issuer; @Column(name = "amount") private final int amount; /** * The @ManyToOne annotation specifies that this class will be present as a member of a collection on a parent class and that it should * be persisted with the joining columns specified in the parent class. It is important to note the targetEntity parameter which should correspond * to a class literal of the parent class. */ @ManyToOne(targetEntity = PersistentParentToken.class) private final TokenState persistentParentToken; public PersistentChildToken(String owner, String issuer, int amount) { this.Id = UUID.randomUUID(); this.owner = owner; this.issuer = issuer; this.amount = amount; this.persistentParentToken = null; } // Default constructor required by hibernate. public PersistentChildToken() { this.Id = UUID.randomUUID(); this.owner = ""; this.issuer = ""; this.amount = 0; this.persistentParentToken = null; } public UUID getId() { return Id; } public String getOwner() { return owner; } public String getIssuer() { return issuer; } public int getAmount() { return amount; } public TokenState getPersistentToken() { return persistentToken; } } } .. sourcecode:: kotlin @CordaSerializable object SchemaV1 : MappedSchema(schemaFamily = Schema::class.java, version = 1, mappedTypes = listOf(PersistentParentToken::class.java, PersistentChildToken::class.java)) { @Entity @Table(name = "parent_data") class PersistentParentToken( @Column(name = "owner") var owner: String, @Column(name = "issuer") var issuer: String, @Column(name = "amount") var currency: Int, @Column(name = "linear_id") var linear_id: UUID, @JoinColumns(JoinColumn(name = "transaction_id", referencedColumnName = "transaction_id"), JoinColumn(name = "output_index", referencedColumnName = "output_index")) var listOfPersistentChildTokens: MutableList<PersistentChildToken> ) : PersistentState() @Entity @CordaSerializable @Table(name = "child_data") class PersistentChildToken( @Id var Id: UUID = UUID.randomUUID(), @Column(name = "owner") var owner: String, @Column(name = "issuer") var issuer: String, @Column(name = "amount") var currency: Int, @Column(name = "linear_id") var linear_id: UUID, @ManyToOne(targetEntity = PersistentParentToken::class) var persistentParentToken: TokenState ) : PersistentState() Identity mapping ---------------- Schema entity attributes defined by identity types (AbstractParty,Party,AnonymousParty) are automatically processed to ensure only theX500Nameof the identity is persisted where an identity is well known, otherwise a null value is stored in the associated column. To preserve privacy, identity keys are never persisted. Developers should use theIdentityServiceto resolve keys from well know X500 identity names. .. _jdbc_session_ref: JDBC session ------------ Apps may also interact directly with the underlying Node's database by using a standard JDBC connection (session) as described by the `Java SQL Connection API <https://docs.oracle.com/javase/8/docs/api/java/sql/Connection.html>`_ Use theServiceHubjdbcSessionfunction to obtain a JDBC connection as illustrated in the following example: .. literalinclude:: ../../node/src/test/kotlin/net/corda/node/services/persistence/HibernateConfigurationTest.kt :language: kotlin :start-after: DOCSTART JdbcSession :end-before: DOCEND JdbcSession JDBC sessions can be used in flows and services (see ":doc:`flow-state-machines`"). The following example illustrates the creation of a custom Corda service using ajdbcSession: .. literalinclude:: ../../docs/source/example-code/src/main/kotlin/net/corda/docs/kotlin/vault/CustomVaultQuery.kt :language: kotlin :start-after: DOCSTART CustomVaultQuery :end-before: DOCEND CustomVaultQuery which is then referenced within a custom flow: .. literalinclude:: ../../docs/source/example-code/src/main/kotlin/net/corda/docs/kotlin/vault/CustomVaultQuery.kt :language: kotlin :start-after: DOCSTART TopupIssuer :end-before: DOCEND TopupIssuer For examples on testing@CordaServiceimplementations, see the oracle example :doc:`here <oracles>`. JPA Support ----------- In addition tojdbcSession,ServiceHubalso exposes the Java Persistence API to flows via thewithEntityManagermethod. This method can be used to persist and query entities which inherit fromMappedSchema. This is particularly useful if off-ledger data must be maintained in conjunction with on-ledger state data. .. note:: Your entity must be included as a mappedType as part of aMappedSchemafor it to be added to Hibernate as a custom schema. If it's not included as a mappedType, a corresponding table will not be created. See Samples below. The code snippet below defines aPersistentFootype insideFooSchemaV1. Note thatPersistentFoois added to a list of mapped types which is passed toMappedSchema. This is exactly how state schemas are defined, except that the entity in this case should not subclassPersistentState(as it is not a state object). See examples: .. container:: codeset .. sourcecode:: java public class FooSchema {} public class FooSchemaV1 extends MappedSchema { FooSchemaV1() { super(FooSchema.class, 1, ImmutableList.of(PersistentFoo.class)); } @Entity @Table(name = "foos") class PersistentFoo implements Serializable { @Id @Column(name = "foo_id") String fooId; @Column(name = "foo_data") String fooData; } } .. sourcecode:: kotlin object FooSchema object FooSchemaV1 : MappedSchema(schemaFamily = FooSchema.javaClass, version = 1, mappedTypes = listOf(PersistentFoo::class.java)) { @Entity @Table(name = "foos") class PersistentFoo(@Id @Column(name = "foo_id") var fooId: String, @Column(name = "foo_data") var fooData: String) : Serializable } Instances ofPersistentFoocan be manually persisted inside a flow as follows: .. container:: codeset .. sourcecode:: java PersistentFoo foo = new PersistentFoo(new UniqueIdentifier().getId().toString(), "Bar"); serviceHub.withEntityManager(entityManager -> { entityManager.persist(foo); return null; }); .. sourcecode:: kotlin val foo = FooSchemaV1.PersistentFoo(UniqueIdentifier().id.toString(), "Bar") serviceHub.withEntityManager { persist(foo) } And retrieved via a query, as follows: .. container:: codeset .. sourcecode:: java node.getServices().withEntityManager((EntityManager entityManager) -> { CriteriaQuery<PersistentFoo> query = entityManager.getCriteriaBuilder().createQuery(PersistentFoo.class); Root<PersistentFoo> type = query.from(PersistentFoo.class); query.select(type); return entityManager.createQuery(query).getResultList(); }); .. sourcecode:: kotlin val result: MutableList<FooSchemaV1.PersistentFoo> = services.withEntityManager { val query = criteriaBuilder.createQuery(FooSchemaV1.PersistentFoo::class.java) val type = query.from(FooSchemaV1.PersistentFoo::class.java) query.select(type) createQuery(query).resultList } Please note that suspendable flow operations such as: *FlowSession.send*FlowSession.receive*FlowLogic.receiveAll*FlowLogic.sleep*FlowLogic.subFlowCannot be used within the lambda function passed towithEntityManager``.