Commit Graph

301 Commits

Author SHA1 Message Date
Katelyn Baker
414f2b4b40 Review Comments - give test methods a better name 2017-08-29 16:23:10 +01:00
Katelyn Baker
1e023fcf54 Add alt serialise method that returns the schema and the bytes
For future testing it would be nice, post serialisation, to have easy
access to the serialised objects schema so we can check how it was
serialised. Adding a helper function to return a data class that does
this in the same way we can for deserialize
2017-08-29 15:20:43 +01:00
Rick Parker
c61b036844 Some custom serializers, fixes to generics handling, some annotations and added NonEmptySet as a special case of Set. (#1330) 2017-08-25 15:09:43 +01:00
Katelyn Baker
dda9d22264 Review Comments
* Typos in comments
 * Refactor some one liners
2017-08-24 17:36:01 +01:00
Katelyn Baker
e4a2529de0 More Review Comments
Move more logic into the constructor factory for the evolver seraliser
rather than everytime we read
2017-08-24 17:36:01 +01:00
Katelyn Baker
cf9fbc715d Review Comments
Refactor small bits of the serialiser, move things out of the read
methods and into the make factory companion object
2017-08-24 17:36:01 +01:00
Katelyn Baker
299b8026c7 Add constructor annotation for evolve selection
Current implementation is such that if we find *a* constructor that
works for us we use that, this is of course rather non deterministic, it
also means we may not select the best constructor

for example

old versions had constructors that took

V1: A B
V2: A B C D

if current version, V3 is

V3: A B C D E

which provides secondary constructors for the above then there is a
chance we'd de-serialise objects that were serialised as V2 using the V1
constructor and thus throw away information we don't need to

Additional Changes:

Fixes following rebase onto master
2017-08-24 17:36:01 +01:00
Katelyn Baker
7894e723e8 Fix evolver to work on nested schema types
Dumb assumption in the initial implementation meant it could only
evolve a top level type in the schema
2017-08-24 17:36:01 +01:00
Katelyn Baker
6bcfb2eddf Add better test, add support for constructor determination 2017-08-24 17:36:01 +01:00
Katelyn Baker
1d131eced5 Add Evolvability to Serializer
If we attempt to deserialize a class and find that since it's
serialization the definition has changed we need to create a serializer
capable of evolving the serialised data and constructing an instance of
the new type

We currently cope with

* Removing members
* Adding nullable members
* Adding non nullable members if a constructor is provided that
  allows us to set the old arguments and defaults the new (mandatory)
  fields
* Reordering paramters
2017-08-24 17:35:03 +01:00
Rick Parker
db07d717e0 Make constructors always accessible to deserialization code. (#1320) 2017-08-24 17:33:09 +01:00
Viktor Kolomeyko
d6d7fb52b4 Make sure members of type Class correctly serialized using AMQP (#1314)
* Minor changes and expose the problem with class serialization
* Custom serializer for Class
* More changes to make TransactionEncumbranceTests pass in AMQP mode
2017-08-23 17:37:49 +01:00
Rick Parker
0458e5415a java.time.* custom AMQP serializers and tests. Excludes pure enums. (#1308)
* java.time.* custom AMQP serializers and tests.  Excludes pure enums.

* Register java.time.* serializers with scheme.

* Provide default implementation of `additionalSerializers`
2017-08-23 12:08:09 +01:00
Viktor Kolomeyko
86fdbb9f22 Make OpaqueBytesSubSequence AMQP serializable (#1303) 2017-08-23 08:40:09 +01:00
Viktor Kolomeyko
66e6f90140 Make WireTransaction @CordaSerializable (#1299)
And do everything necessary for AttachmentClassLoaderTests to be passing in AMQP mode

* Changes following code review by @fenryka and @rick-r3
2017-08-22 16:53:55 +01:00
Viktor Kolomeyko
970ecf1bae Extend unit test to cover SecureHash and Opaque bytes 2017-08-22 10:04:36 +01:00
Chris Rankin
b33e299e3a Refactor serialisation contexts into separate classes, according to their use-cases. (#1293)
* Refactor Kryo contexts into separate classes, according to their use-cases. This prevents Kotlin from trying to instantiate them all every time.
* Also refactor AMQP contexts accordingly.
* Expand comments to explain why these serialisation contexts have been separated.
2017-08-21 14:19:21 +01:00
josecoll
a2ede0fc73 Requery removal (#1276)
* Removed Requery object relational mapping usage (and associated schemas including node-schemas module)

* Fixed issues with NodeAttachmentService tests.
Cannot use JPA custom converters with Primary Key fields.
Hibernate entities require explicit call to flush() to persist to disk.

* Removed redundant requery converters (equivalents not even required in Hibernate).

* Removed remaining gradle requery dependency definitions.

* Fixed broken tests.

* Fixes for failing NodeVaultService tests:
- Dynamic SQL updates (in soft locking code)
- Explicit request by session to participate in transaction (causing "TransactionRequiredException" Executing an update/delete query)
- Explicit flush() required to persist to disk

* Updated changelog.
Fixed compiler warning.

* Fixed WHERE clause AND/OR condition.
Enforced immediate data visibility through transaction commit.

* Final fixes to address failing tests.

* Deferred all hibernate session/txn management to DatabaseTransactionManager.

* Fixed transaction boundaries in failing Cash tests.

* Fixes to address failing tests (transaction boundaries, merge detached object, config clean-up).

* Final adjustment to transaction boundaries in JUnit tests.

* Refactored AttachmentSchemaV1 into NodeAttachmentService itself and referenced from NodeServicesV1.

* Refactored HSQL UPDATE statements to use CriteriaUpdate API.

* Updated all criteria API getters to reference attribute names by type.

* Remove redundant VaultSchema entity name (required when previously using HSQL UPDATE syntax)

* Fix compiler warnings.

* Minor changes following rebase from master.

* Fixed suppress warning type.
2017-08-21 10:42:59 +01:00
Rick Parker
56a84882a7 AMQP fixes: Don't attempt to include non-serializable interfaces in schemas (#1281)
* Ignore interfaces that are not serializable

* Annotate so test still works.

* Make methods accessible to serializer.

* Make sure interfaces are annotated in the carpenter.  Expand tests to check whitelisting.  Object is now whitelisted by default since it has no fields.

* Prevented Object from being whitelisted but allow arrays of Objects (i.e. pretty much an untyped array)
2017-08-21 10:27:27 +01:00
Viktor Kolomeyko
06ad2ddd45 Eliminate warnings in Java and Kotlin code
Functionally this change is a NOP
2017-08-21 09:36:24 +01:00
Viktor Kolomeyko
ee76de0957 Resolve a problem when Exceptions with suppression passed through Kryo serialization (#1280)
Also added some unit tests that expose the problem
2017-08-18 15:58:35 +01:00
Andrzej Cichocki
2829faa01f Retire legalContractReference (#1188)
* Add LegalProseReference annotation
* Migrate code from autogenerated javascript to TypeScript source
* Add instructions to rebuild the web resources
* Make installWeb more reproducible
2017-08-18 13:30:39 +01:00
Viktor Kolomeyko
5864a22d05 Move dependency on "proton-j" library from "core" sub-project to "node-api" 2017-08-18 09:49:46 +01:00
Viktor Kolomeyko
51ed943272 Remove SerializationFactory parameter from the places where it is not needed
Also minor improvements to the looping/if-branching code and documentation changes

Functionally, this change is a NOP
2017-08-17 17:53:40 +01:00
Viktor Kolomeyko
4a3848cc2a Delete KryoAMQPSerializer and its registration
As discussed with @rick-r3 this class is no longer fit for purpose because it may break reference equality if part of the graph been serialized with Kryo and part with AMQP
2017-08-17 17:52:47 +01:00
Katelyn Baker
e6e6644fc9 Fixes to bring into line with changes API
Can't have default factories now, they need building with whitelist and
classloader.

Also remove completely spurious import that broke everything, have a
feeling that was something IntelliJ pulled in "to be helpful"
2017-08-17 14:17:59 +01:00
Katelyn Baker
6076d39ee4 Remove spurious printlns 2017-08-17 13:53:35 +01:00
Katelyn Baker
c72ac2a102 Add WeakHashMap to the blacklist 2017-08-17 13:53:35 +01:00
Katelyn Baker
8bebd6ea0b Rework To better deal with types we won't serialise
* Add depricated jvm types to the blacklist
* Add concrete types for user convenience
* Add better "We won't serialise this because" error handling
2017-08-17 13:53:35 +01:00
Katelyn Baker
d2933ca8a3 Finalise which map containers to serialize 2017-08-17 13:53:35 +01:00
Katelyn Baker
de05a11511 Move toward allowing serialisation of concrete types 2017-08-17 13:53:35 +01:00
Katelyn Baker
4b8d6e32c3 Merge pull request #1269 from corda/bugfix/kat/missingFactory
Re-re-move Java test to the correct location, fix compile issue
2017-08-17 13:25:30 +01:00
Viktor Kolomeyko
4286065872 Eliminate explicit Kryo configuration to allow flexibility for AMQP 2017-08-17 12:01:27 +01:00
Chris Rankin
1672b4aa0a Simple deserialisation fixes. (#1243)
* Fix typo: prefered -> preferred
* Simplify KryoVerifierSerializationScheme and resolve warning.
* Add a custom serialiser for PrivacySeed so that we can avoid invoking RNG.
2017-08-17 11:15:04 +01:00
Katelyn Baker
f38954e550 Re-re-move Java test to the correct location, fix compile issue
Somehow this slipped past Gradle that I hadn't changed the serializer
factory here to take the needed argument now it's no longer defaulted
2017-08-17 11:04:30 +01:00
Katelyn Baker
2dc362a409 Merge pull request #1252 from corda/feature/kat/moveAMQPSerialiserTowardCommonInterface
Move AMQP serialiser toward common interface
2017-08-17 09:43:30 +01:00
Shams Asari
dc8d232480 Creating a PartyAndCertificate only requires a CertPath 2017-08-16 16:36:03 +01:00
Katelyn Baker
4d54d1a8dc Review Comments
Removing unused default method on the factory, it was added for testing
although given there are easier ways to build factories for the tests it
was never used and doesn't need leaving in
2017-08-16 11:39:07 +01:00
Katelyn Baker
49c23a58d1 Final changes integrating tests into the new API
Fix some rebase issues
Ensure tests are using the correct versions of the factories
2017-08-15 15:11:26 +01:00
Katelyn Baker
825908cf15 Refactoring changes to work with interface changes
Specifically we don't need the factory factory as that functionality
lives in the global defaults
2017-08-15 15:11:26 +01:00
Katelyn Baker
139eef2ccb Move AMQP serializer toward the new common abstract itnerface
Allow unique serializer factories to be created per unique set of
WhiteList / Class Loader pair. Remove default consruction of the
SerializerFactory to force use of the FactoryFactory to get the generic
factory and thus access to it's ClassLoader
2017-08-15 15:11:20 +01:00
Michal Kit
b6fd5ede58 Merge community-master 2017-08-15 12:04:09 +01:00
Katelyn Baker
ae72d4d788 Review comments
Rename rtn to serializer
Some whatspace reformatting
2017-08-14 17:35:36 +01:00
Katelyn Baker
2d28cb01d5 Optimise by caching class when serialising
This removed the need to calculate anything on de-serialisation if using
the same factory
2017-08-14 17:35:36 +01:00
Rick Parker
2f08425c43 Refactor KryoAMQPSerializer to go through generic APIs to access AMQP serialization (#1225) 2017-08-14 17:24:04 +01:00
Rick Parker
31df43bd76 Some preparation work for AMQP serialization integration & a refactor fix (#1214) 2017-08-11 13:23:00 +01:00
Matthew Nesbit
af13371510 Move certificate builder code from core to node utilities.
Address PR comments

Fixup dead reference
2017-08-11 12:02:38 +01:00
Katelyn Baker
52aa145c91 Ensure deserialised obj isn't cached serialised version
if it is (it's not) then we wouldn't be testing the carpetner
2017-08-10 12:17:10 +01:00
Katelyn Baker
4d6230f22c Post rebase fixes
- Add missing import
- Deal with files that need to move from core to nodeapi.internal
- Fix up the imports
2017-08-09 16:39:28 +01:00
Katelyn Baker
1f9c04544d Make the carpenter a member of the serialiser factory
If it's not then the caroenter's class loader isn't persistent and thus
we will be constantly recarpenting objects
2017-08-09 15:25:17 +01:00
Katelyn Baker
36cbcae99c Review Comments 2017-08-09 15:09:59 +01:00
Katelyn Baker
45ed78b1a8 Serialiser / Synthesiser work and cleanup
- Adding test case that shows it should work
      AS it works with concrete classes, not if we carpent up the back end
    - WIP
    - The current test fails because the carpenter is creating the map (for
      some reason) as a linked hash map and not a normal map that we can't
      serialise. Added a test to confirm that it's not the carpenter here.

      Two questions really, why are we creating it as a linked has map, AFAIK
      the carpenter should just be whipping up what it's told to and it's not
      being given a linked map

      Secondly, should we support serialisation of these objects

    - Remove debug
    - Remove yet more debug
2017-08-09 15:09:59 +01:00
Katelyn Baker
92ba0c33e1 Adding more / better tests, splitting existing tests up
Squashed commit messages
    - Split up the serialiser/synthesiser tests into more manageable sizes
    - WIP - found test which breaks deserialiser
      Looks to be soemthign to do with synthesising a class with a container
      of Interfaces
2017-08-09 15:09:59 +01:00
Katelyn Baker
ae70d73e56 Synthesiser / Serializer integration working 2017-08-09 15:09:59 +01:00
Katelyn Baker
0541d128e4 Fixes for nullable Prim Types
Commit messages from Squashed Commits
    - Carpenter doesn't work with nullable prim types
    - Fix to get Cherry picks up to date with Master
2017-08-09 15:06:48 +01:00
Katelyn Baker
a5106d74a8 Integration of the serialiser and carpenter working
Tests added
2017-08-09 15:05:56 +01:00
Katelyn Baker
4cfa376d7d Add the carpenter into the deserializer
Deseializing any blob that contains a type unknown to the classloader
will cause a class object to be carpented up and a suitable object
returned
2017-08-09 15:04:50 +01:00
Rick Parker
28610868c4 Remove Kryo from core module completely, including gradle dependencies (#1161) 2017-08-09 14:53:46 +01:00
Andrzej Cichocki
3a3ead2dfe Remove ListenableFuture from core/main (#1030) 2017-08-07 14:31:24 +01:00
Patrick Kuo
3a63f0c606 * remove x500 name validation when loading config file (#1137)
* added x500 name validation in Network registration.
* fix for github issue #1130
2017-07-28 16:18:49 +01:00
Shams Asari
407b467f67 Moved the majority of the contents of Utils.kt into either InternalUtils.kt or KotlinUtils.kt.
What remains is being dealt with in another PR.
2017-07-24 19:25:56 +01:00
Rick Parker
fe9db6f1f7 Remove links to Kryo from serialization "clients" (#1079) 2017-07-21 14:23:05 +01:00
Clinton
0ccfae252f POMs generated by publishing are now correct. (#1055)
* POMs generated by publishing are now correct. The publish extension now requires an explicit call to configure the publishing instead of waiting until after evaluation. This prevents evaluation order issues with the artifact renaming code that causes the POM to have the original, incorrect, artifact names.

* Fixed new test compile issues caused by removal of some dependencies in test utils that caused webserver code to be automatically included in any project also compiling test utils.
2017-07-18 12:34:56 +01:00
Shams Asari
195189070a Moved the Path extension methods that were in Utils.kt into the internal package 2017-07-17 16:01:19 +01:00
Shams Asari
7caee508ec Refactored ErrorOr into Try, with Success and Failure data sub-classes, and moved it into core.utilities 2017-07-11 12:10:38 +01:00
Patrick Kuo
7e8de79848 Legal name validation for X500Name (#983)
* * Legal name validation for X500Name while loading from config file.

* * Removed unintended changes.
2017-07-11 12:09:30 +01:00
Shams Asari
f718acb939 Moved CordaException.kt to base core package 2017-07-10 10:36:59 +01:00
Andrzej Cichocki
cefa14507a Retire HostAndPort (#962)
* Don't attempt to parse a resolved InetSocketAddress toString
* A mock node isn't reachable via an address
2017-07-07 15:11:07 +01:00
Clinton Alexander
b4e7d7ca1b Can now publish to artifactory. 2017-07-03 18:12:56 +01:00
Katarzyna Streich
58da76c052 Network map redesign: Change field types in NodeInfo, move away messaging data from NodeInfo (#921)
* First stage of changing fields in NodeInfo.

Part of work related to NetworkMapService upgrade. Create slots for
multiple IP addresses and legalIdentities per node.

* NodeInfo stores HostAndPort.

Move information specific to messaging layer away from NodeInfo.
Only HostAndPort addresses are stored. Add peer name - peer handle
mapping to MockNetwork to reflect that change.
2017-06-27 18:14:51 +01:00
Andrius Dagys
aaf7de0d02 RPC server: buffer response messages until the client queue is fully set up.
The issue arises when the server restarts, and the client is sometimes
not able to recreate its queue in time, so the server is unable to send
back a response message and just drops it, causing the client to hang.
2017-06-22 11:26:04 +01:00
Clinton
08cbcac40c Corda publications and JARs now have cord or corda at the start. (#749)
Core corda publications and JARs now have cord or corda at the start (excluding gradle plugins). Removed an unnecessary dependency on test-utils in node-schemas to prevent an evaluation order bug in gradle.
2017-06-06 15:05:47 +01:00
Andrzej Cichocki
c510e67ed5 Move ShutdownHook to node-api. (#770) 2017-06-02 15:32:20 +01:00
Rick Parker
d3f027cea8 Custom serialiser support for AMQP (#741) 2017-05-31 18:33:57 +01:00
Chris Rankin
bb0cc85295 Allow RPC for internal classes with special serialisers. (#751)
Also ensure that HashCheckingStream validates the hash afterwards.
2017-05-26 15:28:22 +01:00
Chris Rankin
ea53bab7d7 Forbid internal classes from being serialised. (#655) 2017-05-25 17:18:51 +01:00
Andrzej Cichocki
ca36b4676d Unduplicate certificatesDirectory (#740) 2017-05-25 10:30:28 +01:00
Patrick Kuo
246de55433 Create client CA certificate with X509 name constraint (#731)
* The node will be issued a CA certificate with name constraint which will allow the node to create keys with a valid certificate chain.
2017-05-24 16:13:37 +01:00
Mike Hearn
b461a89ac1 Whitelist some more java.time types for serialisation. 2017-05-19 16:45:22 +02:00
Andrzej Cichocki
d3bb040355 Refactoring related to BFT notary demo (#680)
* Fix: Add missing @StartableByRPC to fix the Raft notary demo
* Make loadConfig take a Config object, for cordformation Node
* Unduplicate User.toMap
* Unduplicate WHITESPACE regex, choose possessive form
* Use slash to make a Path
* Remove Companion where redundant
* Remove unused code
2017-05-16 11:30:50 +01:00
Andrius Dagys
885f951dc3 Node IP auto-detection (#666)
* If the p2pAddress provided by the configuration is not public, the node tries to discover the public IP:
- First by checking the network interfaces
- If not found, sends a request to the network map service

* Enable initial connection retry

* Improve error handling

* Update docs
2017-05-12 14:09:02 +01:00
Shams Asari
e75732af91 Making sure non-serialisable objects in FlowException do not interfere with the flow session (#651)
Also TransactionVerificationException no longer has reference to non-serialisable LedgerTransaction
2017-05-10 11:07:28 +01:00
Andras Slemmer
3a2afcdbb2 #592: Address more comments 2017-05-05 17:10:52 +01:00
Andras Slemmer
de88ad4f40 RPC muxing, multithreading, RPC driver, performance tests 2017-05-05 17:10:52 +01:00
Ross Nicoll
25dbac0f07 Change party to hold an X.500 name
Change the legal name of parties to be an X500 name. This ensures that we aren't converting between
common names and X500 names in various places, eliminating substantial scope for error in the conversion
process. As a result, all node names must now be full X500 names, which has impact on most configurations.
2017-05-05 16:13:29 +01:00
Ross Nicoll
b64e7f51f6 Enforce X.500 distinguished names in configuration 2017-05-05 16:13:29 +01:00
Shams Asari
c5a9312e07 Merge pull request #582 from corda/shams-flow-counterpartymarker-cleanup
Deprecated FlowLogic.getCounterpartyMarker as it's complicated and probably not used
2017-04-26 09:40:48 +01:00
Matthew Nesbit
7e3b9a8934 Fix duplicate content root problems in IntelliJ 2017-04-26 10:37:16 +02:00
Shams Asari
913487cb32 Deprecated FlowLogic.getCounterpartyMarker as it's complicated and probably not used (replacement is to use sub-flows).
Also made flow registration require the client flow class rather than any old class.
2017-04-25 18:53:33 +01:00
kasiastreich
36d5d0d7b2 Refactor of CompositeKeys to implement PublicKey interface. (#433)
* Make CompositeKey implement PublicKey

The initial implementation of composite keys as their own distinct class separate from PublicKey
means that the keys cannot be used on standard classes such as Certificate. This work is a beginning
to modifying CompositeKey to being a PublicKey implementation, although significant further work
is required to integrate this properly with the standard Java APIs, especially around verifying
signatures using the new key type.

* First stage of making CompositeKey implement PublicKey interface. Revert to using PublicKey everywhere we expect a key.

* Move algorithm and format into companion object (#432)

Move algorithm and format into companion object so that they can be referenced from other
classes (i.e. the upcoming signature class).

* Add simple invariants to construction of CompositeKey.
Builder emits CompositeKeys in simplified normalised form. Forbid keys with single child node, force ordering on children and forbid duplicates on the same level. It's not full semantical normalisation.

* Make constructor of CompositeKey private, move NodeWeight inside the class.
Add utility function for Kryo deserialization to read list with length constraints.
2017-04-12 11:13:20 +01:00
Mike Hearn
eec8b9e4da Minor: auto-format of module: node-api, node-schemas 2017-04-11 15:53:05 +02:00
Shams Asari
c17fe29a62 Converted FullNodeConfiguration into a data class and added ability to parse Configs into data classes 2017-04-07 17:54:03 +01:00
Shams Asari
faef877a8d Converted sealed data types to be data classes 2017-04-04 11:37:56 +01:00
Chris Rankin
b3936a2159 Exclude commons-logging for all projects at top level. (#473) 2017-03-30 20:59:50 +01:00
Chris Rankin
f15a22d828 Replace kotlinx-support-jdk8 with kotlin-stdlib-jre8. (#463)
* Replace kotlinx-support-jdk8 with kotlin-stdlib-jre8.
* Add kotlinVersion to constants.properties.

* Remove unused variable.
2017-03-30 16:27:01 +01:00
Patrick Kuo
78a0024e00 Fix various deprecated warnings
* Changed deprecated kotlin.reflect.primaryConstructor to kotlin.reflect.full.primaryConstructor
* Changed deprecated kotlin.reflect.memberProperties to kotlin.reflect.full.memberProperties
* Changed deprecated kotlin.reflect.declaredMemberProperties to kotlin.reflect.full.declaredMemberProperties
* Changed deprecated AllComposition to AllOf
* Changed deprecated AnyComposition to AnyOf
* Changed deprecated kotlin.reflect.KotlinReflectionInternalError to kotlin.reflect.jvm.internal.KotlinReflectionInternalError
* Changed deprecated HostAndPort.hostText to HostAndPort.host
* Removed duplicated method net.corda.core.node.recordTransactions
2017-03-30 10:00:46 +01:00
Rick Parker
8ef1d767c9 Consolidate maven repositories (#445)
* Consolidate repositories into root build.gradle
2017-03-28 17:17:40 +01:00
Andras Slemmer
48952dfc02 Add node-api, split minimal node functionality, OutOfProcessTransactionVerifierService 2017-03-27 17:42:51 +01:00
Chris Rankin
98266da41c Refactor CordaRPCClient into new :client:rpc Gradle module. (#405)
* CORDA-305: Refactor CordaRPCClient into :client:rpc module

* CORDA-305: Remove the Kotlin test framework from the artifacts.

* CORDA-305: Migrate serialisation whitelist into node-api module.

* CORDA-305: Clean up unused RPC observables.

* CORDA-305: Add :client:rpc module to documentation tasks.

* CORDA-305: Include :finance into :client:rpc for its serialisable classes.

* CORDA-305: Move test classes into the correct directory.

* CORDA-305: Migrate :finance dependency from :client:rpc into DemoBench.

* CORDA-305: Update wording of TODO about handling Observables.
2017-03-22 15:52:54 +00:00
Andras Slemmer
d3f956ab60 Address PR 379 comments 2017-03-17 15:09:04 +00:00
Andras Slemmer
c3c1f3d801 Add :node-api, factor out some artemis code 2017-03-17 11:47:05 +00:00