Commit Graph

178 Commits

Author SHA1 Message Date
Anthony Keenan
0a194cfb51 Merge branch 'release/os/4.3' into release/os/4.4
# Conflicts:
#	docs/source/docker-image.rst
2019-10-21 13:44:32 +01:00
Dimos Raptis
42e364386d
Merge branch 'release/os/4.3' into CORDA-3304-rpc-max-retries 2019-10-15 14:04:40 +01:00
Zoltan Kiss
a1dd6abe17 TM-40 Ephemeral workspace for k8s workers that survives restarts (#5567)
* Simplify

* Mount shared dir to worker

* format

* podnames with separators

* refactor parameters

* Use PVC for storage

* pvc in namespace

* KubesTest simplify

* no tolowercase

* no private

* lowercase

* RetryStrategy

* minor changes

* wait forever

* undo .idea

* elvis

* add comment

* regcred

* use correct ConfigBuilder

* delete java, will migrate later

* Revert "delete java, will migrate later"

This reverts commit e3bab1f3

* Merging changes in groovy to new java file

* format

* rename variable

* fix log

* private

* remove bak

* move java files

* Revert "move java files"

This reverts commit 89aa4c35
2019-10-15 10:52:44 +01:00
Dimos Raptis
a7884c53aa [CORDA-3304] Introduce max number of retries per invocation for reconnecting rpc 2019-10-14 11:12:08 +01:00
stefano
0488a5ac5d Merge remote-tracking branch 'origin/release/os/4.3' into merge_4.3_to_4.4
# Conflicts:
#	constants.properties
2019-10-08 15:57:14 +01:00
Schife
28852ce47d Razvan/merge 4.3 into 4.4 (#5494)
* NOTICK: Corda 4.3-RC01

Created first release candidate of Corda 4.3 - RC01.

* CORDA-3141: Add GracefulReconnect callbacks which allow logic to be performed when RPC disconnects unexpectedly (#5430)

Also removed potential for growing stack trace on reconnects.

* CORDA-2050 Upgrade Corda to Java 11 (compatibility mode) (#5356)

Upgrade Corda to run with Java 11 (compatibility mode) - see https://github.com/corda/corda/pull/5356

* ENT-4198 Adding legal text

Signed-off-by: Ed Prosser <edward.prosser@r3.com>

* TM-29 new baseline for 4.3 since new debt has been added with the last few commits (#5487)

* TM-23 compileAll task to compile all code (#5490)

* Add simple compileAll task to be used by warning check

* lazy configure compileAll

* TM-32 Merge OS 4.3 into 4.4

* TM-32 fixed detekt issue

* Downgrade Dokka back to 0.9.17 due to failing docs_builder.

* add ability to group test types together (#5459)

* add ability to group test types together

* add ability to specify podCount for use in parallel testing

* remove compiler xml

* add Jenkinsfile to enable scanning

* trigger build

* add ability to specify what docker tag to use from outside of the build

* fix docker work dir

* fix pipeline syntax issues

* use environment rather than `def`

* move agent restrictor outside of stages block

* use steps block

* more pipeline syntax fixes

* even more pipeline syntax fixes

* even more pipeline syntax fixes

* add kubenetize as property to image build

* move clear of docker image to end of build rather than start to prevent colocated builds

* escape dollar on docker image remove command

* attempt to kill all existing jobs

* fix compile issue due to killall_jobs

* fix compile issue due to killall_jobs pt2

* fix spelling

* make all variables environment variables

* add logic to delete images locally after pushing

* wrap testing phase with try / finally so that junit reports are always evaluated

* change the behaviour around post build actions

* break implicit link between testing phase and image building phase, allowing testing to occur without a rebuild and push of image

* prepend registry name to provided tag

* allow tasks to specify whether they wish to stream output from containers

* add timestamps directive to Jenkinsfile to have timing info on output

* make KubesTest resilient against transient pod failures in k8s

* increase CPU request

* add logic to allow specifying container resource requests

* attempt to run unit and integration tests in parallel

* change unit tests to use 3 cores to allow co-location on 8c machines

* join grouped tests together to give pod meaningful name

* add step to renew token with GKE

* change renew step to use pods instead of nodes

* fix bug where memory request is not correctly passed to pod

* disable unit tests for now

* [CORDA-2368] Added exception handling for missing files that displays appropriate messages rather than defaulting to file names. (#5472)

* NOTIK Minor adjustments to Detekt rules to reflect current working practises  (#5498)

* Minor adjustments to rules to reflect current working practises (including IntelliJ code style alignment)

* Adjust another rule in line with existing code style.

* rebaseline with changed detekt ruleset

* rebaseline with NodeStartup changes
2019-09-20 14:10:15 +00:00
Stefano Franz
dea00fd107
remove compiler xml (#5426) 2019-09-12 13:40:49 +00:00
willhr3
79c675fb41 CORDA-3200 Fix postgres oid/ bytea column issue (#5465)
* CORDA-3200 Add postgres specific migration to change from @Lob to @Type(type = corda-blob)

* CORDA-3200 Remove unwanted changes

* CORDA-3200 Change column length to 64000 for long cert chains
2019-09-12 11:23:28 +00:00
Shams Asari
90284a6143
CORDA-2919: JacksonSupport, for CordaSerializable classes, improved to only uses those properties that are part of Corda serialisation (#5397) 2019-08-27 13:06:28 +01:00
Katelyn Baker
913ba1d46e
NOTICK - Initial work on a non JVM (C++) serialiser (#5368)
Currently, it's more of a blob inspector than a full-blown
implementation, but this code does have the ability to understand a Corda
serialized blob and build a set of deserialisers to handle a blob. It
does this in the same way the native java implementation works by
interrogating the types (as per the definition in the envelope) and
building recursive type readers that are able to pull the relevant
information out of the byte stream.

Lots of future work to undertake, but as a starting point its a little
bit useful with the scope it could be made much more useful with some
work.
2019-08-15 21:16:20 +01:00
josecoll
99e802c5c5 CORDA-3071 - Checkpoint agent tool (#5295)
* Initial commit based on experimental kryo hook agent.

* WIP

* Added documentation.

* Additional improvements and documentation following more testing.

* Added field level instrumentation + basic type handlers for String, byteArray, charArray, primitive types.

* Working version (without array type handling)

* Missing build.gradle file.

* Handle display of Arrays and String.
Pruning output to avoid repetition (by loop depth, object count).

* Added configurable StackDepth (for display purposes) and filter out ProgressTracker stacks.

* Further array handling (Object arrays, 2D, 3D), improved display and general code cleanup.

* Various fixes and improvements following demo to RP.

* Clean-up

* Further clean-up

* Set checkpoint id before deserialization.

* Update documentation

* Final clean-up.

* Minor documentation fixes.

* Updates following PR review feedback.

* Add changelog entry.
2019-08-01 09:40:52 +01:00
Dimos Raptis
8962d930d4 [CORDA-2923] - Make the RPC client reconnect with gracefulReconnect param (#5244)
When set to true the RPC client will:

* automatically reconnect when the connection is broken
* simple RPC calls will block until connection is established
* Observables returned from RPC will automatically resubscribe on reconnect so the client continues to receive events. This doesn't guarantee that events will not be lost during the reconnect.
2019-07-16 11:29:21 +01:00
Christian Sailer
687fa81894 ETO-39 Move core tests (#5255)
* Moved core.flows and core.contract tests

* move internal tests

* move node, schemas and serialization tests

* Move transactions tests

* Move test access helper into test package

* Move remaining tests, and create access helpers to test internals.

* Fix test dependencies.

* Delete some cruft that flagged up on the API checker

* Add readme explaing where tests need to go.

* fix typo
2019-07-09 10:09:21 +01:00
Shams Asari
4e835d5483
CORDA-2333: Updated the majority of the dependencies that were out of date (#5235)
Some where not touched, including the crypto ones, libraries related to messaging and Kotlin.
2019-06-20 16:19:16 +01:00
Shams Asari
d685c2154b
ENT-3489: Removing unnecessary @CordaSerializable annotation from exceptions (#5209) 2019-06-11 11:51:30 +01:00
Shams Asari
71c316ca5a
CORDA-2954: Upgrade to common-lang3 (#5152)
Utilised Fields.getAllFieldsList where we were doing the equivalent
2019-05-21 17:48:32 +01:00
josecoll
0263650a5f
Revert "Merge pull request #5145 from corda/jdk11-migration-gradle5-upgrade" (#5146)
This reverts commit f46377556c, reversing
changes made to ba675e73b9.
2019-05-20 16:16:49 +01:00
josecoll
fa738c1354 Update IDE compiler dependencies to run tests within IntelliJ. 2019-05-20 15:08:27 +01:00
dazraf
270444ab3e CORDA-2769 - fixed infinite recursive call for copy implementation. (#4912) 2019-03-27 18:51:35 +00:00
Shams Asari
b2d0ba7e14 CORDA-2613: FinalityHandler is no longer gated on app target versions
To allow rolling upgrades where members of a BN may not all upgrade at the same time, the FinalityHandler is now always enabled, and not disabled if no old apps are installed.

Previously any attempt to use the FinalityHandler in such a scenerio would send the flow to the hospital, where manual intervention would be required to recover the transaction.
2019-02-18 11:58:23 +01:00
Shams Asari
e20278fbfd
Made FlowCheckpointVersionNodeStartupCheckTest leaner (#4640)
* Removed `restart node successfully with suspended flow` as it duplicates `TraderDemoTest#Test restart node during flow works properly`
* Removed the need for a notary
2019-01-28 14:08:54 +00:00
Christian Sailer
b5fc095288
Use FlowTimeOut exception rather than SQL deadlock in unit tests to avoid hitting the exponential back-off. (#4527) 2019-01-10 09:59:21 +00:00
Shams Asari
b4c3fa1948
Fixed incorrect attachment loading integration test (#4453)
* Fixed incorrect attachment loading integration test

`AttachmentLoadingTests.test that attachments retrieved over the network are not used for code` was a false-positive - it was incorrect on multiple levels. Fixing it required updating the finance:isolated CorDapp, at which point it was given the new MANIFEST metadata for V4, and moved out of the net.corda.finance namespace to avoid package sealing issues.

The new test exposed a bug in the LedgerTransaction verification logic. This was cleaned up as it was too easy to verify on the wrong instance.
2018-12-31 15:02:11 +00:00
Shams Asari
830959c9f7
CORDA-2345: Simplified TestCordapp to make it inline with the recent CorDapp versioning changes (#4434)
TestCordapp has now two implementations to clearly separate the two use cases it has in the Corda repo:

* TestCordappImpl which implements the revised public API of TestCordapp; namely that a TestCordapp instance references a real CorDapp jar on the classpath. This is either an external dependency jar in which case it’s taken as is and given to the node, or it’s a local gradle project in which case it’s compiled using the gradle “jar” task to generate the CorDapp jar. This approach means the jar has all the original CorDapp versioning information, which is important that it’s correct when testing. To this end, TestCordapp only needs to expose the ability to specify the app’s config. All the remaining properties have moved to CustomCordapp.

* CustomCordapp for creating arbitrary custom CorDapps, including specifying the jar’s MANIFEST values. This is internal API and only used for testing the platform. Technically this shouldn’t implement TestCordapp but does so to reduce the complexity of the driver and mock network.
2018-12-20 09:49:58 +00:00
szymonsztuka
4aaefb4fe9 CORDA-2331 Split Workflow and contracts of Finance App into separate Cordapps (#4422)
* Split Workflow and contracts of Finance App into separate Cordapps, part 1 - content which is different between OS and ENT is still in contract Cordapp.

* Move CashSelection implementations to workflow module.

* Move CashSelection implmentations to workflow module.

* Move finance module to finance-flows, top level finance module is empty.

* Move finance module to finance-flows, top level finance module is empty.

* Updated build comment.

* Revert publication of combined (contracts and flows) corda-finance.jar (to maintain backwards compatibility with 3rd party cordapps dependent on finance)

* Added backwards compatibility clarification comment.

* Re-instate new cordapp metadata.

* Global rename of `finance-flows` to `finance-workflows` to follow adopted naming conventions.

* Addressed final review comments.

* Rename application to "Corda Finance Demo"

* Generation of original corda-finance jar from new sub-modules.

* Fixed and tested demobench with new split finance contract and workflow jars.

* Renamed finance sub-modules to contracts and workflows.

* Remove Michele!!!

* Minor fix to filtering logic.

* Align CorDapp configuration filename with workflows jar.

* Fix breaks caused by finance module naming changes.

* Final alignment between OS/ENT of finance contract code.
2018-12-19 18:02:51 +00:00
Shams Asari
49d1cee6d4
Removed crypto service config (#4284) 2018-11-22 20:30:37 +00:00
Stefano Franz
99e9864975
add kotlin 1.3 codestyle to git (#4204) 2018-11-09 09:38:41 +00:00
Konstantinos Chalkias
da8dec0d63
[CORDA-2192] Corda specific OID for AliasPrivateKey (#4175) 2018-11-07 09:48:13 +00:00
Konstantinos Chalkias
106eb9df4a
[CORDA-2011] [CORDA-2057] CryptoService interface and BC HSM simulation (#4099) 2018-11-06 12:57:13 +00:00
Michele Sollecito
28dd3ac873
[CORDA-1778, CORDA-1835]: Decoupled configuration parsing mechanism (#4093) 2018-10-25 16:45:14 +01:00
Thomas Schroeter
f685df46b5
[ENT-1774] FlowAsyncOperation deduplication ID (#4068) 2018-10-19 11:40:59 +01:00
Katelyn Baker
7cfd44e383
CORDA-2113 - Include PNM ID in CSR (#4086)
* CORDA-2113 - Include PNM ID in CSR

If Compatibility Zone operator is using private networks and the node
should be joining one, optionally the ID (a UUID) of that network can be
included as part of the node's CSR to to the Doorman.

* fix broken test
2018-10-18 15:39:42 +01:00
Andrius Dagys
9ebeac1ad8
CORDA-535: Extract notary implementations into CorDapps (#3978)
* Move Raft and BFT notaries into separate modules

* Move schemas

* Fix tests & demos

* Modified logic for creating notary services:

Added a new field 'className' to the notary configuration. The node now
loads the specified implementation via reflection. The default className
value points to the simple notary implementation for backwards compatibility.
Relevant schemas are loaded in a similar fashion.

For backwards compatibility purposes the default SimpleNotaryService will
remain built-in to node, but its cordapp will be generated on startup – so
the loading of notary services is streamlined.

* Move test namedcache factory to test utils
2018-10-10 10:04:22 +01:00
Thomas Schroeter
057ee74611
Single node notary thread safety (#3924) 2018-09-12 13:36:04 +01:00
Christian Sailer
bc330bd989
ENT-2414 Named caches (#3848)
* Add named caches and apply to NonInvalidingUnboundCache and all usages.

* Add named caches and apply to NonInvalidingCache and all usages.

* Add named caches and apply to NonInvalidingWeightBasedCache and all usages.

* Move NamedCache to core/internal

* Remove type `NamedCache` and `NamedLoadingCache`

* Suppressed 'name not used' warning, added comment, and fixed generic parameters on the buildNamed functions.

* Use `buildNamed` in all caffeine instances in production code. Not using it for caches that are created in test code.

* Add checks for the cache name

* Formatting

* Minor code review revisions
2018-08-24 17:17:22 +01:00
Anthony Keenan
bcfadfeebf
CORDA-1833: Create a picocli base class (#3826)
* Add shell extensions to CLI utils class and move into its own module

* Fix issue with completion script generation and slight refactor

* Fix autocompletion for logging level

* Delete uneeded comment

* More tidying up

* Make run function final

* Fixed an issue with the program being run twice.

* Address review comments
2018-08-22 21:51:25 +01:00
Tommy Lillehagen
d2ef16cbfd
Deterministic JVM (#3386)
* CID-251 - Deterministic JVM

* CID-251 - Add DJVM documentation

* CID-251 - Address review comments from @chrisr3

* CID-251 - Address further review comments from @chrisr3

* CID-251 - Use shadowJar to generate fat JAR

* CID-251 - Address review comments from @exFalso

* CID-251 - Improve naming in ReferenceMap

* CID-251 - Add test for Kotlin meta-class behaviour

* CID-251 - Address review comments from @shamsasari

* CID-251 - Add description of high-level flow

* CID-251 - Refactoring

* CID-251 - Rename package to net.corda.djvm

* CID-251 - Include deterministic-rt.jar as runtime dependency

* CID-251 - Add Gradle task for generating whitelist from deterministic rt.jar

* CID-251 - Error messages for StackOverflow/OutOfMemory, update whitelist

* CID-251 - Reduce set definition of pinned classes

* CID-251 - Tidy up logic around pinned classes

* CID-251 - Shade ASM dependency and split out CLI tool

* CID-251 - Address review comments from @mikehearn (part 1)

* CID-251 - Address review comments from @mikehearn (part 2)

* CID-251 - Address review comments from @mikehearn (part 3)

* CID-251 - Address review comments from @exFalso

* CID-251 - Address review comments from @mikehearn (part 4)

* CID-251 - Address review comments from @exFalso and @mikehearn

* CID-251 - Address review comments from @mikehearn (part 5)
2018-08-22 16:01:39 +01:00
Stefano Franz
fffa063803
Cli backwards compatibility testing (#3733)
* first pass at infrastructure around cli compatibility

* add example unit test

* inspect enum types

* add a basic unit test to verify behaviour of the cli checker

* revert root build.gradle
2018-08-16 15:44:40 +01:00
Shams Asari
93bb24ed17
Internal mock node clean up (#3715)
* InMemoryMessagingNetwork.InMemoryMessaging renamed to MockNodeMessagingService and moved to internal package
* start method added to MockNodeMessagingService which enables AbstractNode to call makeMessagingService in its c'tor
* Removed TopicStringValidator as it's no longer used
* Clean up of TestStartedNode
* Merged InMemoryMessagingTests into InternalMockNetworkTests as it's testing InternalMockNetwork
2018-07-30 10:35:03 +01:00
Anthony Keenan
857ed1897a CORDA-1712 - Make documentation clearer for Network Bootstrapper (#3504) 2018-07-09 17:38:31 +01:00
Shams Asari
244167d3e9
Enabled warnings as errors (#3514) 2018-07-04 17:17:27 +01:00
Shams Asari
3046843d40
CORDA-1602: Added cmd line flags to the network bootstrapper (#3419)
The list of CorDapps jars is no longer passed in via the cmd line but is now expected to be placed in the bootstrapped directory.

Ended up being a bit of a refactor to cater for unit testing, and also tidied up the bootstrapper docs.
2018-06-23 11:36:10 +01:00
Stefano Franz
66294df34f
add experimental bootstrapper (#3009)
* add experimental bootstrapper

(cherry picked from commit 63665d1)

refactor code to be less Azure Specific
Use node classes for notaries

add local docker backend

move to tools directoy

apply fixes for local docker RPC admin port

add extraParams field to context to allow dynamic backend selection

begin refactor to move all common node/notary functionality into single implementations

node and notaries now share the same code path as much as possible.

refactor network building logic into api class

port Main.kt to use new networkbuilder api

add gui

fix issues with local docker ports not being exposed on localhost

make push and instantiate async operations

add ability to "add" a node after network has been built to gui

tidy up backend selection via command line and GUI

allow region selection for AZURE instantiations

remove old network map based node.conf and network map dockerfile

tidy up constructors of the various node stage objects

tidy up artefact name

add network-name selection dialog

* print out help

* exclude transitive dep onto log4j to suppress error print out

* windows fixes for local docker

* fixes to allow "devs.XXXX" resource groups in line with the new devops policy of having named resourceGroups

* add extra logging around constructing azure backend
2018-06-12 13:59:27 +01:00
josecoll
002d5a8f10
Publish tools to Artifactory: Blob Inspector, Network Bootstrapper (#3321)
* Added missing plugin to enable publishing of artifact to Artifactory.

* Remove references to experimental blobinspector

* Remove disableDefaultJar for default jar.

* Add additional artifacts to be published.
2018-06-08 14:02:57 +01:00
Thomas Schroeter
141d45c39d
Add simulation of the avalanche consensus protocol to experimental (#3283) 2018-06-04 13:33:34 +01:00
Shams Asari
7b09795795
CORDA-1383: Cleaned up the JSON format of WireTransaction and SignedTransaction (#3248)
In particular correctly outputs the custom state and command objects in the txs. Also fixed up deserialisation back into the transaction objects.
2018-05-30 16:37:41 +01:00
Shams Asari
4e0378de9c
CORDA-1238: Moved the blob inspector out of experimental and wired it to JackonSupport (#3224)
The existing output format was not complete and so was deleted to avoid it becoming a tech debt. We can always resurrect it at a later point.
2018-05-24 18:26:55 +01:00
Shams Asari
3136e973a7
CORDA-1511: Kryo only used for node checkpoints and so moved out of serialization module and into node (#3228) 2018-05-24 12:18:07 +01:00
Ivan Schasny
8990e9f783
Merge pull request #3170 from corda/CORDA-1459
Corda 1459
2018-05-21 10:45:29 +01:00
Shams Asari
3cdd908714
Moved the serialisation logic in nodeapi.internal.serialization into its own module: serialization (#3179)
node-api now depends on this module and upcoming changes will use this as well rather than having to depend on node-api.

EnumEvolveTests.deserializeWithRename and EnumEvolveTests.multiOperations are temporarily ignored since their test resources can't be regenerated due to bugs.
2018-05-17 16:18:07 +01:00