From 6a07576c96575afa4c67f88c6b3a3cfcc105d2c9 Mon Sep 17 00:00:00 2001 From: Katelyn Baker Date: Fri, 5 Jan 2018 17:44:53 +0000 Subject: [PATCH] CORDA-834 - Class default evolution serialisation documentation (#2209) CORDA-834 - Class default evolution serialisation documentation --- .../serialization-default-evolution.rst | 237 ++++++++++++++++++ docs/source/serialization.rst | 7 +- 2 files changed, 242 insertions(+), 2 deletions(-) create mode 100644 docs/source/serialization-default-evolution.rst diff --git a/docs/source/serialization-default-evolution.rst b/docs/source/serialization-default-evolution.rst new file mode 100644 index 0000000000..282ca452b8 --- /dev/null +++ b/docs/source/serialization-default-evolution.rst @@ -0,0 +1,237 @@ +Default Class Evolution +======================= + +.. contents:: + +Whilst more complex evolutionary modifications to classes require annotating, Corda's serialization +framework supports several minor modifications to classes without any external modification save +the actual code changes. These are: + + #. Adding nullable properties + #. Adding non nullable properties if, and only if, an annotated constructor is provided + #. Removing properties + #. Reordering constructor parameters + +Adding Nullable Properties +-------------------------- + +The serialization framework allows nullable properties to be freely added. For example: + +.. container:: codeset + + .. sourcecode:: kotlin + + // Initial instance of the class + data class Example1 (val a: Int, b: String) // (Version A) + + // Class post addition of property c + data class Example1 (val a: Int, b: String, c: Int?) // (Version B) + +A node with version A of class ``Example1`` will be able to deserialize a blob serialized by a node with it +at version B as the framework would treat it as a removed property. + +A node with the class at version B will be able to deserialize a serialized version A of ``Example1`` without +any modification as the property is nullable and will thus provide null to the constructor. + +Adding Non Nullable Properties +------------------------------ + +If a non null property is added, unlike nullable properties, some additional code is required for +this to work. Consider a similar example to our nullable example above + +.. container:: codeset + + .. sourcecode:: kotlin + + // Initial instance of the class + data class Example2 (val a: Int, b: String) // (Version A) + + // Class post addition of property c + data class Example1 (val a: Int, b: String, c: Int) { // (Version B) + @DeprecatedConstructorForDeserialization(1) + constructor (a: Int, b: String) : this(a, b, 0) // 0 has been determined as a sensible default + } + +For this to work we have had to add a new constructor that allows nodes with the class at version B to create an +instance from serialised form of that class from an older version, in this case version A as per our example +above. A sensible default for the missing value is provided for instantiation of the non null property. + +.. note:: The ``@DeprecatedConstructorForDeserialization`` annotation is important, this signifies to the + serialization framework that this constructor should be considered for building instances of the + object when evolution is required. + + Furthermore, the integer parameter passed to the constructor if the annotation indicates a precedence + order, see the discussion below. + +As before, instances of the class at version A will be able to deserialize serialized forms of example B as it +will simply treat them as if the property has been removed (as from its perspective, they will have been.) + + +Constructor Versioning +~~~~~~~~~~~~~~~~~~~~~~ + +If, over time, multiple non nullable properties are added, then a class will potentially have to be able +to deserialize a number of different forms of the class. Being able to select the correct constructor is +important to ensure the maximum information is extracted. + +Consider this example: + + +.. container:: codeset + + .. sourcecode:: kotlin + + // The original version of the class + data class Example3 (val a: Int, val b: Int) + +.. container:: codeset + + .. sourcecode:: kotlin + + // The first alteration, property c added + data class Example3 (val a: Int, val b: Int, val c: Int) + +.. container:: codeset + + .. sourcecode:: kotlin + + // The second alteration, property d added + data class Example3 (val a: Int, val b: Int, val c: Int, val d: Int) + +.. container:: codeset + + .. sourcecode:: kotlin + + // The third alteration, and how it currently exists, property e added + data class Example3 (val a: Int, val b: Int, val c: Int, val d: Int, val: Int e) { + // NOTE: version number purposefully omitted from annotation for demonstration purposes + @DeprecatedConstructorForDeserialization + constructor (a: Int, b: Int) : this(a, b, -1, -1, -1) // alt constructor 1 + @DeprecatedConstructorForDeserialization + constructor (a: Int, b: Int, c: Int) : this(a, b, c, -1, -1) // alt constructor 2 + @DeprecatedConstructorForDeserialization + constructor (a: Int, b: Int, c: Int, d) : this(a, b, c, d, -1) // alt constructor 3 + } + +In this case, the deserializer has to be able to deserialize instances of class ``Example3`` that were serialized as, for example: + +.. container:: codeset + + .. sourcecode:: kotlin + + Example3 (1, 2) // example I + Example3 (1, 2, 3) // example II + Example3 (1, 2, 3, 4) // example III + Example3 (1, 2, 3, 4, 5) // example IV + +Examples I, II, and III would require evolution and thus selection of constructor. Now, with no versioning applied there +is ambiguity as to which constructor should be used. For example, example II could use 'alt constructor 2' which matches +it's arguments most tightly or 'alt constructor 1' and not instantiate parameter c. + +``constructor (a: Int, b: Int, c: Int) : this(a, b, c, -1, -1)`` + +or + +``constructor (a: Int, b: Int) : this(a, b, -1, -1, -1)`` + +Whilst it may seem trivial which should be picked, it is still ambiguous, thus we use a versioning number in the constructor +annotation which gives a strict precedence order to constructor selection. Therefore, the proper form of the example would +be: + +.. container:: codeset + + .. sourcecode:: kotlin + + // The third alteration, and how it currently exists, property e added + data class Example3 (val a: Int, val b: Int, val c: Int, val d: Int, val: Int e) { + // NOTE: version number purposefully omitted from annotation for demonstration purposes + @DeprecatedConstructorForDeserialization(1) + constructor (a: Int, b: Int) : this(a, b, -1, -1, -1) // alt constructor 1 + @DeprecatedConstructorForDeserialization(2) + constructor (a: Int, b: Int, c: Int) : this(a, b, c, -1, -1) // alt constructor 2 + @DeprecatedConstructorForDeserialization(3) + constructor (a: Int, b: Int, c: Int, d) : this(a, b, c, d, -1) // alt constructor 3 + } + +Constructors are selected in strict descending order taking the one that enables construction. So, deserializing examples I to IV would +give us: + +.. container:: codeset + + .. sourcecode:: kotlin + + Example3 (1, 2, -1, -1, -1) // example I + Example3 (1, 2, 3, -1, -1) // example II + Example3 (1, 2, 3, 4, -1) // example III + Example3 (1, 2, 3, 4, 5) // example IV + +Removing Properties +------------------- + +Property removal is effectively a mirror of adding properties (both nullable and non nullable) given that this functionality +is required to facilitate the addition of properties. When this state is detected by the serialization framework, properties +that don't have matching parameters in the main constructor are simply omitted from object construction. + +.. container:: codeset + + .. sourcecode:: kotlin + + // Initial instance of the class + data class Example4 (val a: Int?, val b: String?, val c: Int?) // (Version A) + + + // Class post removal of property 'a' + data class Example4 (val b: String?, c: Int?) // (Version B) + + +In practice, what this means is removing nullable properties is possible. However, removing non nullable properties isn't because +a node receiving a message containing a serialized form of an object with fewer properties than it requires for construction has +no capacity to guess at what values should or could be used as sensible defaults. When those properties are nullable it simply sets +them to null. + +Reordering Constructor Parameter Order +-------------------------------------- + +Properties (in Kotlin this corresponds to constructor parameters) may be reordered freely. The evolution serializer will create a +mapping between how a class was serialized and its current constructor parameter order. This is important to our AMQP framework as it +constructs objects using their primary (or annotated) constructor. The ordering of whose parameters will have determined the way +an object's properties were serialised into the byte stream. + +For an illustrative example consider a simple class: + +.. Container:: codeset + + .. sourcecode:: kotlin + + data class Example5 (val a: Int, val b: String) + + val e = Example5(999, "hello") + +When we serialize ``e`` its properties will be encoded in order of its primary constructors parameters, so: + +``999,hello`` + +Were those parameters to be reordered post serialisation then deserializing, without evolution, would fail with a basic +type error as we'd attempt to create the new value of ``Example5`` with the values provided in the wrong order: + +.. Container:: codeset + + .. sourcecode:: kotlin + + // changed post serialisation + data class Example5 (val b: String, val a: Int) + + .. sourcecode:: shell + + | 999 | hello | <--- Extract properties to pass to constructor from byte stream + | | + | +--------------------------+ + +--------------------------+ | + | | + deserializedValue = Example5(999, "hello") <--- Resulting attempt at construction + | | + | \ + | \ <--- Will clearly fail as 999 is not a + | \ string and hello is not an integer + data class Example5 (val b: String, val a: Int) + diff --git a/docs/source/serialization.rst b/docs/source/serialization.rst index 347152c175..1b51f95110 100644 --- a/docs/source/serialization.rst +++ b/docs/source/serialization.rst @@ -373,6 +373,9 @@ Future Enhancements Type Evolution -------------- -When we move to AMQP as the serialization format, we will be adding explicit support for interoperability of different versions of the same code. -We will describe here the rules and features for evolvability as part of a future update to the documentation. +Type evolution is the mechanisms by which classes can be altered over time yet still remain serializable and deserializable across +all versions of the class. This ensures an object serialized with an older idea of what the class "looked like" can be deserialized +and a version of the current state of the class instantiated. + +More detail can be found in :doc:`serialization-default-evolution`