Commit Graph

166 Commits

Author SHA1 Message Date
Stefano Franz
b23af5f0d2
add timeouts to all tests (#5875) 2020-02-11 10:14:05 +00:00
Chris Rankin
c8a21cb8d2 CORDA-3536: Load DJVM serialization types more precisely to avoid runtime warnings. (#5896)
* Load DJVM serialization types more precisely to avoid runtime warnings.
* Remove unnecessary string concatenation from debug and trace log messages.
2020-01-28 09:26:13 +00:00
Chris Rankin
01666ed068 CORDA-3523: DJVM custom serializers and whitelists (#5858) 2020-01-16 17:48:15 +00:00
Chris Rankin
a7147c1ffd ENT-4652: Provide an "attachment fixup" mechanism to repair broken transactions. (#5825)
* Do not register cordapp custom serialisers when using attachment classloader.

* Record the URLs of CorDapp JARs that contain custom serialisers. Include these JARs as extra attachments if we discover that we're missing a custom serialiser during transaction verification.

* Check for disabled serializer when explicitly requesting a custom serializer.
Refactor test case to force use of a custom serializer.

* Tidy up basic custom serializer test.

* Also test that TransactionBuilder rejects missing custom serializers.

* Remove test whitelists, which should not be needed with custom serialisers.

* Add changelog entry. Also align TestCordappImpl.findRoots() with OS backports.

* Second approach based around CorDapps inside AttachmentStorage - report missing type descriptor or any non-composable types.

* Initial implementation of Corda-Fixup rules inside a CorDapp jar.

* Replace original "automatic attachment fixing" mechanism completely.

* First review comments: restore "missing class" logic to TransactionBuilder.

* Restore "missing class" mechanism as fallback for SignedTransaction too.
2020-01-14 15:18:51 +00:00
Dimos Raptis
ce774e459a [CORDA-3501] - Provide option for user to specify custom serializers (#5837)
* [CORDA-3501] - Provide option for user to specify custom serializers

* Remove not needed integration test shemas
2020-01-09 11:29:16 +00:00
Zoltan Kiss
885bc534af CORDA-3489 State Evolution: Support adding new mandatory field and removal of optional (#5817)
* Reproduce

* Another failing test

* rename test

* rename test

* slot mapping

* pass all tests

* remove comments

* refactor

* broken test

* broken test

* detekt

* simplify

* simplify

* detekt baseline cleanup

* Add check

* requireForSer

* simplify

* Remove check

* use indices
2019-12-20 11:28:16 +00:00
Christian Sailer
14050826e9 ENT-4595 harmonize core and serialization (#5792)
* Harmonize serialization/core and deterministic counterparts

* Fix test for changed private alias key behaviour

* Detekt errors

* roll back project.xml
2019-12-09 14:17:48 +00:00
Chris Rankin
5ce1535ea5 CORDA-3469: Allow EvolutionSerializer to handle boxed types becoming primitive. (#5766) 2019-11-28 16:39:45 +00:00
Chris Rankin
f226ddc4f2 CORDA-2876: Integrate the DJVM with the Corda Node. (#5633)
* * CORDA-2876: Migrate DJVM serialization modules into Corda.
* Pre-generate Corda classes for DJVM sandbox when node boots in production mode.
* Ensure that all DJVM test contract CorDapps are signed.
* Test examining attachments within DJVM sandbox.
* Test Contract.verify() using cryptographic verify function.
* Add test cases for more non-determinism in Contract.verify().
* Update node-driver to support testing nodes with DJVM support.
* Modify Node to allow alternative DJVM configurations for testing.
* Refactor DeterministicVerifierFactoryService for default use-case.
* Small whitespace and code-style refactors.
* Create and activate a DJVM execution profile for the Node.
* Revert making Verifier implement AutoCloseable.
* Allow the node to cache sandboxed Corda byte-code for reuse.
* Use updated Quasar agent that knows not to touch DJVM classloaders.
* Fix Quasar's package exclusions globs for DJVM.
* Deserialise LedgerTransaction into the sandbox for Contract.verify().
* Add the DJVM's serialisation modules to the Corda node.
* Update the node for the latest DJVM API, and preserve the ConstructorForDeserialization annotation on user contract classes.
* Add corda-dev to repositories while DJVM is SNAPSHOT.
* Migrate DJVM specialisation into AbstractNode's ServiceHubInternalImpl.
* Exclude sandbox.** and shaded djvm.** classes from Quasar agent.
* Add the corda-dev repository to :node for the deterministic runtime.
* Turn Verifier into an abstract base class that is specialised by BasicVerifier and DeterministicVerifier.
* Add the Corda deterministic libraries to the Node, and split the DJVM sandbox across two SandboxClassLoader instances.
* Add DJVM to contract verification path inside Corda Node.
* Minor lambda simplifications and removing unused import.
* CORDA-2871: Remove @CordaSerializable from LedgerTransaction.
* CORDA-2871: Add a callback to ServicesForResolution to allow the Node to modify a LedgerTransaction object.
* CORDA-2871: Refactor the contract verification code into a separate class,
 and allow LedgerTransaction to choose different Verifier objects.
* Update DJVM to use Corda 4.4-SNAPSHOT. (#95)
* CORDA-3330: Allow DJVM to preload / pregenerate classes from selected jars. (#92)
* Add support for SourceClassLoader.getResources() to DJVM.
* Allow a SandboxConfiguration to preload sandbox byte-code for all classes inside jars containing META-INF/DJVM-preload.
* CORDA-3309: Remove explicit try-catch in favour of UncaughtExceptionHandler. (#91)
* CORDA-3309: Install UncaughtExceptionHandler for DJVM tasks. (#88)
* Fix tests broken by Windows line endings. (#82)
* CORDA-3292: Reimplement ExecutionProfile as a data class. (#80)
* CORDA-2877: Refactor how we create child SandboxConfiguration objects. (#76)
* CORDA-2877: Load bytecode from a persistent cache to prevent repeated rewriting. (#75)
* Refactor byte-code cache to SandboxConfiguration instead of AnalysisConfiguration. We cannot "mix and match" byte-code generated by different sets of rules.
* CORDA-3137: Enhance annotation handling so that we can allow some annotations to be mapped into the sandbox without also needing to be stitched. (#72)
* CORDA-2871: Minor cosmetic fixes. (#69)
* CORDA-3218: Align DJVM with internal Corda Serialisation API. (#68)
* Ensure we get the latest SNAPSHOT of the serialisation code.
* CORDA-2871: Refactor SourceClassLoader to define source classes. (#66)
* Rewrite SourceClassLoader to support parent/child relationships.
* Revert catching TypNotPresebtException - it was a symptom of a bigger problem.
* Remove AutoCloseable from AnalysisConfiguration and SourceClassLoader.
* SourceClassLoader.getResource() must delegate to its parent first.
* CORDA-2871: Ensure ClassLoader.loadClass() throws ClassNotFoundException for all cases where the class cannot be found. (#64)
* CORDA-2871: Modify sandbox tasks to implement both java.Function and sandbox.Function (#62)
* Make TaskExecutors implement BiFunction to make them composable.
* Create ImportTask to wrap a java.Function inside a sandbox.Function.
* Add createExecutor() and createRawExecutor() APIs to SandboxClassLoader.
* Update serialization to use SandboxClassLoader.toSandboxClass().
* Remove a layer of lambdas from the serialisation code.
* Update SandboxExecutor and SandboxRawExecutor.
* Rename Executor to TaskFactory.
* Rename dangling executor -> taskFactory.
* CORDA-2871: Sanity fixes! (#63)
* Improve message for SandboxClassLoadingException.
* Fix serialisation API for using sandboxed environment.
* CORDA-3174: Extend serialisation to include InputStream and OpaqueBytesSubSequence. (#60)
* Update DJVM Example project for serialisation.
* Add serializers for InputStream and OpaqueBytesSubSequence.
* Support ZIP Inflater and CRC32 inside the sandbox.
* Allow the DJVM to wrap java.io.InputStream as sandbox.java.io.InputStream.
* Configure tests also to preserve @DeprecatedConstructorForDeserialization.
* CORDA-3174: Implement Corda serialization modules. (#59)
* Create DJVM serialization modules.
* Create test cases for Array<T>, List<T> and List<Array<T>>.
* Refactor SandboxPrimiveSerializer for all primitive types.
* Implement SandboxCollectionSerializer to support Collection types.
* Implement SandboxMapSerializer to support Map types.
* Attempt to fix infinite loop when computing Collection and Map fingerprints.
* Apply special handling when deserialising sandbox.java.lang.Character.
* Remap Java primitive types to sandbox Java object types to deter evolution.
* Use Class.getPackage().getName() to determine sandbox package name.
* Implement SandboxEnumSerializer to support Enum types.
* Implement SandboxPublicKeySerializer to support Java security keys.
* Add serialization projects to the composite example project.
* Implement serializers for BigInteger, BigDecimal, Currency and StringBuffer.
* Test that deserialising does not instantiate the untrusted user classes.
* Implement serializers for java.time.* types.
* Add serialiser for BitSet - currently disabled until BitSet itself is supported.
* Add serialisers for EnumSet and Class.
* Include support for EnumMap in the SandboxMapSerializer.
* Ensure the DJVM Example project's tests preserve @CordaSerializable.
* Add support for UUID as a primitive type.
* Use common abortReadOnly() method for declaring serialization as unsupported.
* Streamline the API for deserialising into the sandbox.
* Add preliminary support for deserialising X.509 certificates.
* Implement serializer for java.util.Optional.
* Refactor configuration of the sandbox serialization scheme.
* Add tests for deserialising arrays of basic types.
* Include method annotations in annotation stitching. This ensures that `@ConstructorForDeserialization` is not dropped.
* Enable test for SandboxBitSetSerializer.
* Enable tests for X.509 serializers.
* Implement serializers for ProtonJ primitive types.
* Serialize java.util.Date as a primitive type.
* Add the bintray Gradle plugin to the serialisation modules.
* Do not publish serialisation modules - they will become part of Corda itself.

* CORDA-2876: Only apply DJVM sources to Node Driver when devMode=true.

* Resolve DeteKT warnings.

* Require Node's JVM to set -Dnet.corda.djvm=true in order to enable DJVM.

* Enable DJVM for DemoBench nodes.

* Disable Quasar instrumentation verification for DemoBench nodes.

* Upgrade to DJVM 1.0-RC01.

* Try to modify DriverParameters in a more "ABI friendly" way.

* Refactor and simplify sandbox deserialisation of primitive objects.

* Review fixes.

* Update EvolutionSerializerFactory to handle sandboxed primitive boxed types.
2019-11-05 13:44:18 +00:00
stefano
5ac1e50135 Merge remote-tracking branch 'origin/release/os/4.3' into my_merge_branch 2019-11-04 17:48:15 +00:00
Chris Rankin
b2c944bb0e CORDA-3390: Allow EvolutionSerializer to handle primitive types becoming nullable. (#5661) 2019-11-04 16:19:46 +00:00
Chris Rankin
43856319bd CORDA-3392: Fix caching of local AMQPSerializer<*> instances. (#5657) 2019-11-01 14:08:27 +00:00
Chris Rankin
2895283500 CORDA-3388: Restore mapping of 'java.lang.Void -> void' (#5650) 2019-10-30 17:47:13 +00:00
Ryan Fowler
bfa460bc07 CORDA-3152: Register custom serializers for jackson as well as amqp 2019-10-15 15:52:31 +01:00
Chris Rankin
6d467bdbb1 CORDA-2050: Ensure that ArraySerializer.elementType is resolved for GenericArray. (#5512) 2019-09-24 11:28:23 +01:00
Chris Rankin
a9d9b668bc CORDA-2050: Do not add java.lang.Class fields and properties to local type cache. (#5511)
* Fix SLF4J logging for example Gradle project.

* Stop adding java.lang.Class's internal fields and properties to the local type cache. This allows us to fix ArraySerializer for the DJVM.
2019-09-23 18:26:21 +01:00
josecoll
3fafbe551c
CORDA-2050 Upgrade Corda to Java 11 (compatibility mode) (#5356)
Upgrade Corda to run with Java 11 (compatibility mode) - see https://github.com/corda/corda/pull/5356
2019-09-18 10:26:26 +01:00
Jonathan Locke
e2fcc1edd5
CORDA-3218: Make set of serializer types considered suitable for object reference to be configurable.
CORDA-3218: Make set of serializer types considered suitable for object reference to be configurable.
2019-09-13 10:27:57 +01:00
Chris Rankin
842888ac1b Fix typo. (Doh!) 2019-09-12 16:44:15 +01:00
josecoll
c096dcab3b
CORDA-3206: Move serialization tests into separate module to break de… (#5452)
* CORDA-3206: Move serialization tests into separate module to break dependency on `node-driver` (and associated transitive dependencies).
Required to complete CORDA-2050: Upgrade Corda to Java 11 (compatibility mode)

* Keep all serialization tests in same module UNLESS they require access to `node-driver` (and transitive dependencies of).

* Move helper object to internal package.

* Gradle build file clean-up and additional README.

* Addressing CS PR review comments.
2019-09-12 16:14:44 +01:00
Chris Rankin
7af8011649 CORDA-3218: Make set of serializer types considered suitable for object reference to be configurable. 2019-09-12 15:50:13 +01:00
Zoltan Kiss
f171de7b69 TM-23 Fail build on compiler warnings (#5453)
* java compile respects compilation.allWarningsAsErrors

* suppress or cleanup warnings

* suppress warning

* use non-deprecated kotlin dependency

* rename property

* handle property existence check

* Deal with warnings
2019-09-11 16:34:51 +01:00
Dan Newton
4cbe22949d CORDA-3018 Whitelisting attachments by public key - phase two tooling (#5386)
Allow node operators to blacklist signing keys (using blacklistedAttachmentSigningKeys config option). These blacklisted keys prevent attachments that are received over the network from being trusted. The docs have been updated to detail how to generate the key hashes that the config requires.

A new shell command attachments trustRoots has been added to see what attachments exist on the node along with information about their trust and where it comes from.

run dumpCheckpoints has been replaced by checkpoints dump as InternalCordaRPCOps needed to change to prevent a function that is meant to be internal from being visible on the shell.
2019-09-10 12:16:34 +01:00
Jonathan Locke
35d0170b0a Merge branch 'master' into jlocke/merge_4.1_20190827 2019-08-27 15:21:21 +01:00
Jonathan Locke
e5f314ffbf Merge branch 'release/4' into jlocke/merge_4.1_20190827
Merged release/4 (Corda OS version 4.1) into a branch created from master (Corda OS version 4.3)

Conflicts:
	build.gradle
	client/rpc/src/main/kotlin/net/corda/client/rpc/internal/ReconnectingCordaRPCOps.kt
	common/logging/build.gradle
	common/logging/src/main/kotlin/net/corda/common/logging/CordaVersion.kt
	constants.properties
	core-tests/src/test/kotlin/net/corda/coretests/utilities/ProgressTrackerTest.kt
	core/src/main/kotlin/net/corda/core/transactions/TransactionBuilder.kt
	docs/source/api-contract-constraints.rst
	docs/source/api-stability-guarantees.rst
	docs/source/app-upgrade-notes.rst
	docs/source/changelog.rst
	docs/source/clientrpc.rst
	docs/source/conf.py
	docs/source/corda-network/UAT.md
	docs/source/getting-set-up.rst
	docs/source/network-builder.rst
	docs/source/upgrading-cordapps.rst
	docs/source/versioning.rst
	node/src/integration-test/kotlin/net/corda/node/services/rpc/RpcReconnectTests.kt
	serialization/src/main/kotlin/net/corda/serialization/internal/amqp/PropertyDescriptor.kt
	serialization/src/main/kotlin/net/corda/serialization/internal/model/LocalTypeInformationBuilder.kt
	tools/network-builder/src/main/resources/node-Dockerfile
	tools/network-builder/src/main/resources/notary-Dockerfile
2019-08-27 14:51:22 +01:00
Chris Rankin
e25d9a1d4e Update KDocs. 2019-08-27 11:24:41 +01:00
Chris Rankin
92ae45a949 Fix typo decimal62 -> decimal64. 2019-08-26 15:26:36 +01:00
Chris Rankin
4ebd02bc04 Tidy up changes for review. 2019-08-23 11:11:11 +01:00
Chris Rankin
b2d335c518 Make the choice of AMQP serializer for primitive types configurable. 2019-08-23 08:52:16 +01:00
Chris Rankin
99074b5a49 Modify the fingerprinter not to use ConcurrentHashMap.computeIfAbsent() because we cannot guarantee that the cache is not reentered by the computation. 2019-08-23 08:52:04 +01:00
Chris Rankin
a5d5e0d476 Allow custom serialization for all subclasses of a configurable set of classes. 2019-08-23 08:51:56 +01:00
Chris Rankin
bdd5d13655 Provide a map of Java primitive types as a configuration value. 2019-08-23 08:51:47 +01:00
Chris Rankin
aa2f1029a6 Use LocalTypeIdentifier information where available to lookup CustomSerializer. 2019-08-23 08:51:37 +01:00
Chris Rankin
be64c89515 Implement generic CustomerSerializers that create more specific AMQPSerializer instances at runtime. 2019-08-23 08:51:27 +01:00
Chris Rankin
e4f38d1945 Ensure that described properties are associated with a descriptor. 2019-08-23 08:51:16 +01:00
Chris Rankin
8f0c7c947a Allow custom serializers to be registered with type aliases for deserializing. 2019-08-23 08:51:06 +01:00
Shams Asari
71c316ca5a
CORDA-2954: Upgrade to common-lang3 (#5152)
Utilised Fields.getAllFieldsList where we were doing the equivalent
2019-05-21 17:48:32 +01:00
josecoll
0263650a5f
Revert "Merge pull request #5145 from corda/jdk11-migration-gradle5-upgrade" (#5146)
This reverts commit f46377556c, reversing
changes made to ba675e73b9.
2019-05-20 16:16:49 +01:00
josecoll
ac56f302eb TEMP disable use of "sun.security, sun.misc.Unsafe" in serializer lists (re-instate with module add-exports directive) 2019-05-20 15:08:27 +01:00
Dan Newton
cd73161513 CORDA-2870 improve error messages for non composable types (#5120)
* CORDA-2870 Add `reason` and `remedy` to `LocalTypeInformation.NonComposable`

When creating `LocalTypeInformation.NonComposable` pass in the `reason`
a type was not composable and the `remedy` to fix it. This required
changes in `LocalTypeInformationBuilder` to pass in this extra
information so that it can be used later.

The message that the `ObjectSerializer` includes in its
`NotSerializableException` now includes the extra information about the
non composable type.

* CORDA-2870 Include custom serializers in serialization error message

In `ObjectSerializer`, when a serialization exception is thrown,
include the registered custom serializers + their classloaders as part
of the error message.

This required making properties on `CustomSerializerRegistry` and
`LocalSerializerFactory` public.

Tidy up `LocalTypeInformationBuilder` error message text for
transitive non-composable types.

* CORDA-2870 Tidy up error thrown for unserializable objects

Fix `DeserializeSimpleTypesTests` and tidy up the code in
`ObjectSerializer` a bit.

* CORDA-2870 Remove non-composable warning logs in `LocalTypeInformationBuilder`

The flag `warnIfNonComposable` and its corresponding log lines are not
needed now that the non-composable error messages contain a lot of
information in them.

The `warnIfNonComposable` flag is now incorrect and has been renamed to
`validateProperties` and the function `suppressWarningsAnd` has been
changed to `suppressValidation`.

`propertyDescriptors` has also had its input boolean changed to
`validateProperties` to better represent what it is doing.

* CORDA-2870 Remove need for casting by moving variable to interface

Expose `customSerializerNames` in `LocalSerializerFactory` and
`CustomSerializerFactory`.
2019-05-16 10:05:49 +01:00
Dimos Raptis
a468bee0ed [CORDA-2473] - Remove AMQP system property (#5112) 2019-05-14 13:22:22 +01:00
Tudor Malene
67808d985e CORDA-2363 - Fix ClassNotFound handling (#5078) (#5107)
(cherry picked from commit 25f335861b)
2019-05-09 15:23:21 +01:00
JamesHR3
b4e96778bf [CORDA-2517] Whitelist attachments signed by keys that already sign existing trusted attachments (#5068)
This allows a different signed version of the same CorDapp to be automatically trusted.

This reverts "[CORDA-2575] Allow users to whitelist attachments by public key config (#5035)"
2019-05-07 12:55:27 +01:00
Tudor Malene
25f335861b Fix ClassNotFound handling (#5078) 2019-05-02 18:03:02 +01:00
JamesHR3
7a7c471ebf [CORDA-2575] Allow users to whitelist attachments by public key config (#5035) 2019-04-25 16:55:43 +01:00
Dominic Fox
a29f417a67 CORDA-2860 relax property type checking (#5028) (#5038)
* CORDA-2860 relax property type checking

* Remove redundant check

* Unit test

* Comment linking ticket to test

* More descriptive comment

* Fix test that was broken by jacocoData field being silently added to class

* Revert to previous behaviour around opaques, suppress validation
2019-04-18 14:20:00 +01:00
Dominic Fox
a416d5025f CORDA-2860 relax property type checking (#5028)
* CORDA-2860 relax property type checking

* Remove redundant check

* Unit test

* Comment linking ticket to test

* More descriptive comment

* Fix test that was broken by jacocoData field being silently added to class

* Revert to previous behaviour around opaques, suppress validation
2019-04-17 16:14:24 +00:00
JamesHR3
4afbf78c79 CORDA-2633 - Restructure evolution serialization errors to print reason first (#4978) (#5014)
(cherry picked from commit 9963a6609f)
2019-04-15 10:27:39 +01:00
Dominic Fox
13f6bc0c4d CORDA-2848 - relax fingerprinter strictness (#5001 (#5007)) 2019-04-12 11:54:47 +01:00
Dominic Fox
2e97eaee0d CORDA-2848 relax fingerprinter strictness (#5001)
* CORDA-2848 relax fingerprinter strictness

* Unit test for non-serializable type parameter case
2019-04-11 15:42:56 +01:00