Commit Graph

535 Commits

Author SHA1 Message Date
Nick Dunstone
e6f9b46584 https://r3-cev.atlassian.net/browse/ENT-3829 (#5763)
Open Source
2019-12-03 10:28:00 +00:00
Ed Prosser
63b7f2093e Merge branch 'release/os/4.3' into EdP/typo-merge-4.4
# Conflicts:
#	buildSrc/build.gradle
2019-11-29 13:53:06 +00:00
Razvan Codreanu
d33dbb2ea9 TM-104 Switching the distributed testing plugin to the one released in artifactory (#5764)
* TM-104 switch to using the published plugin

* TM-104 switching to artifactory plugin

* TM-104 remove unused plugin

* TM-104 adding docker plugin

* TM-104 adding docker plugin take 2

* add dependencies-dev and set distributed build plugin to changing
2019-11-28 14:49:39 +00:00
stefano
40b1a188f1 Merge remote-tracking branch 'origin/release/os/4.3' into EdP/CORDA-3446-4.4
# Conflicts:
#	docs/source/changelog.rst
2019-11-25 09:41:50 +00:00
Stefano Franz
16eb2fce78
Allow taints on kubernetes nodes to determine scheduling of tests (#5755)
* allow a test group to specify taints for the nodes

* specify "effect" of toleration
modify test groups to use taints
modify preallocation to use taints

* add extra command line flags for unit test run
2019-11-25 09:39:37 +00:00
Ed Prosser
e692ec292a Merge branch 'release/os/4.3' of https://github.com/corda/corda into EdP/CORDA-3446-4.4 2019-11-20 10:36:38 +00:00
Chris Rankin
ce8ce2ba7c CORDA-2876: Upgrade to deterministic-rt 1.0-RC01. (#5738) 2019-11-19 15:02:35 +00:00
Stefano Franz
138a2f114d
Refresh token by invoking KubeCTL before each Kuberenetes API invocation (#5732)
* fix merge issue with sidecar image work

* add logic to refresh token in GKE

* set streaming to true

* set streaming to true

* set streaming to false, and increment number of workers for regression test as approaching 1h limit

* set PR build to run unit and integration tests

* try method distribution to see if faster

* rename test stage to reflect the fact it runs unit tests

* remove commented out parallel unit test code
2019-11-19 15:01:42 +00:00
Chris Rankin
6380feee41 CORDA-3406: Modify DemoBench to make using the DJVM obvious and optional. 2019-11-07 12:22:40 +00:00
stefano
6a3a9eded3 Merge remote-tracking branch 'origin/release/os/4.3' into my_merge_branch
# Conflicts:
#	core/src/main/kotlin/net/corda/core/internal/TransactionVerifierServiceInternal.kt
#	node/src/integration-test/kotlin/net/corda/node/services/rpc/RpcExceptionHandlingTest.kt
#	testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/DriverDSLImpl.kt
2019-11-05 17:19:18 +00:00
Razvan Codreanu
7106b0f1b9 Converting Groovy to Java in the Distributed testing plugin (#5642)
* TM-74 Converting ParallelTestGroup

* TM-74 private fields, getters, distributedTesting refactoring

* TM-74 More conversions

* TM-74 reinstating ListTests groovy class

* TM-74 extracting enum and clas from ListTests

* TM-74 switching to java ListTests

* TM-74 null check

* TM-74 new access modifiers

* TM-74 minor changes

* TM-74 scrapping the conversions of ListTests to java

* TM-74 reverting build.gradle

* TM-74 formatting main gradle

* TM-74 removing the commented out code. It has been linked in the jira

* TM-74 adding list tests back for investigation

* TM-74 collecting after every operation

* TM-74 collecting after every operation

* TM-74 more debugging

* TM-74 more debugging with lifecycle now

* TM-74 debugging on the existing groovy file

* TM-74 with stream output

* TM-74 switching back to java

* TM-74 reverting java code, fixing cast in distributed testing file?

* TM-74 nailing down the line causing the problem

* TM-74 casting

* TM-74 changing types

* TM-74 stacktrace at pod lvl

* TM-74 stacktrace at pod lvl

* TM-74 fix issue with immutable list

* TM-74 reverting changes now that fix is in place

* TM-74 switching to generic task

* TM-74 turning off debugging

* TM-74 allocating by method
2019-11-05 16:59:19 +00:00
Chris Rankin
f226ddc4f2 CORDA-2876: Integrate the DJVM with the Corda Node. (#5633)
* * CORDA-2876: Migrate DJVM serialization modules into Corda.
* Pre-generate Corda classes for DJVM sandbox when node boots in production mode.
* Ensure that all DJVM test contract CorDapps are signed.
* Test examining attachments within DJVM sandbox.
* Test Contract.verify() using cryptographic verify function.
* Add test cases for more non-determinism in Contract.verify().
* Update node-driver to support testing nodes with DJVM support.
* Modify Node to allow alternative DJVM configurations for testing.
* Refactor DeterministicVerifierFactoryService for default use-case.
* Small whitespace and code-style refactors.
* Create and activate a DJVM execution profile for the Node.
* Revert making Verifier implement AutoCloseable.
* Allow the node to cache sandboxed Corda byte-code for reuse.
* Use updated Quasar agent that knows not to touch DJVM classloaders.
* Fix Quasar's package exclusions globs for DJVM.
* Deserialise LedgerTransaction into the sandbox for Contract.verify().
* Add the DJVM's serialisation modules to the Corda node.
* Update the node for the latest DJVM API, and preserve the ConstructorForDeserialization annotation on user contract classes.
* Add corda-dev to repositories while DJVM is SNAPSHOT.
* Migrate DJVM specialisation into AbstractNode's ServiceHubInternalImpl.
* Exclude sandbox.** and shaded djvm.** classes from Quasar agent.
* Add the corda-dev repository to :node for the deterministic runtime.
* Turn Verifier into an abstract base class that is specialised by BasicVerifier and DeterministicVerifier.
* Add the Corda deterministic libraries to the Node, and split the DJVM sandbox across two SandboxClassLoader instances.
* Add DJVM to contract verification path inside Corda Node.
* Minor lambda simplifications and removing unused import.
* CORDA-2871: Remove @CordaSerializable from LedgerTransaction.
* CORDA-2871: Add a callback to ServicesForResolution to allow the Node to modify a LedgerTransaction object.
* CORDA-2871: Refactor the contract verification code into a separate class,
 and allow LedgerTransaction to choose different Verifier objects.
* Update DJVM to use Corda 4.4-SNAPSHOT. (#95)
* CORDA-3330: Allow DJVM to preload / pregenerate classes from selected jars. (#92)
* Add support for SourceClassLoader.getResources() to DJVM.
* Allow a SandboxConfiguration to preload sandbox byte-code for all classes inside jars containing META-INF/DJVM-preload.
* CORDA-3309: Remove explicit try-catch in favour of UncaughtExceptionHandler. (#91)
* CORDA-3309: Install UncaughtExceptionHandler for DJVM tasks. (#88)
* Fix tests broken by Windows line endings. (#82)
* CORDA-3292: Reimplement ExecutionProfile as a data class. (#80)
* CORDA-2877: Refactor how we create child SandboxConfiguration objects. (#76)
* CORDA-2877: Load bytecode from a persistent cache to prevent repeated rewriting. (#75)
* Refactor byte-code cache to SandboxConfiguration instead of AnalysisConfiguration. We cannot "mix and match" byte-code generated by different sets of rules.
* CORDA-3137: Enhance annotation handling so that we can allow some annotations to be mapped into the sandbox without also needing to be stitched. (#72)
* CORDA-2871: Minor cosmetic fixes. (#69)
* CORDA-3218: Align DJVM with internal Corda Serialisation API. (#68)
* Ensure we get the latest SNAPSHOT of the serialisation code.
* CORDA-2871: Refactor SourceClassLoader to define source classes. (#66)
* Rewrite SourceClassLoader to support parent/child relationships.
* Revert catching TypNotPresebtException - it was a symptom of a bigger problem.
* Remove AutoCloseable from AnalysisConfiguration and SourceClassLoader.
* SourceClassLoader.getResource() must delegate to its parent first.
* CORDA-2871: Ensure ClassLoader.loadClass() throws ClassNotFoundException for all cases where the class cannot be found. (#64)
* CORDA-2871: Modify sandbox tasks to implement both java.Function and sandbox.Function (#62)
* Make TaskExecutors implement BiFunction to make them composable.
* Create ImportTask to wrap a java.Function inside a sandbox.Function.
* Add createExecutor() and createRawExecutor() APIs to SandboxClassLoader.
* Update serialization to use SandboxClassLoader.toSandboxClass().
* Remove a layer of lambdas from the serialisation code.
* Update SandboxExecutor and SandboxRawExecutor.
* Rename Executor to TaskFactory.
* Rename dangling executor -> taskFactory.
* CORDA-2871: Sanity fixes! (#63)
* Improve message for SandboxClassLoadingException.
* Fix serialisation API for using sandboxed environment.
* CORDA-3174: Extend serialisation to include InputStream and OpaqueBytesSubSequence. (#60)
* Update DJVM Example project for serialisation.
* Add serializers for InputStream and OpaqueBytesSubSequence.
* Support ZIP Inflater and CRC32 inside the sandbox.
* Allow the DJVM to wrap java.io.InputStream as sandbox.java.io.InputStream.
* Configure tests also to preserve @DeprecatedConstructorForDeserialization.
* CORDA-3174: Implement Corda serialization modules. (#59)
* Create DJVM serialization modules.
* Create test cases for Array<T>, List<T> and List<Array<T>>.
* Refactor SandboxPrimiveSerializer for all primitive types.
* Implement SandboxCollectionSerializer to support Collection types.
* Implement SandboxMapSerializer to support Map types.
* Attempt to fix infinite loop when computing Collection and Map fingerprints.
* Apply special handling when deserialising sandbox.java.lang.Character.
* Remap Java primitive types to sandbox Java object types to deter evolution.
* Use Class.getPackage().getName() to determine sandbox package name.
* Implement SandboxEnumSerializer to support Enum types.
* Implement SandboxPublicKeySerializer to support Java security keys.
* Add serialization projects to the composite example project.
* Implement serializers for BigInteger, BigDecimal, Currency and StringBuffer.
* Test that deserialising does not instantiate the untrusted user classes.
* Implement serializers for java.time.* types.
* Add serialiser for BitSet - currently disabled until BitSet itself is supported.
* Add serialisers for EnumSet and Class.
* Include support for EnumMap in the SandboxMapSerializer.
* Ensure the DJVM Example project's tests preserve @CordaSerializable.
* Add support for UUID as a primitive type.
* Use common abortReadOnly() method for declaring serialization as unsupported.
* Streamline the API for deserialising into the sandbox.
* Add preliminary support for deserialising X.509 certificates.
* Implement serializer for java.util.Optional.
* Refactor configuration of the sandbox serialization scheme.
* Add tests for deserialising arrays of basic types.
* Include method annotations in annotation stitching. This ensures that `@ConstructorForDeserialization` is not dropped.
* Enable test for SandboxBitSetSerializer.
* Enable tests for X.509 serializers.
* Implement serializers for ProtonJ primitive types.
* Serialize java.util.Date as a primitive type.
* Add the bintray Gradle plugin to the serialisation modules.
* Do not publish serialisation modules - they will become part of Corda itself.

* CORDA-2876: Only apply DJVM sources to Node Driver when devMode=true.

* Resolve DeteKT warnings.

* Require Node's JVM to set -Dnet.corda.djvm=true in order to enable DJVM.

* Enable DJVM for DemoBench nodes.

* Disable Quasar instrumentation verification for DemoBench nodes.

* Upgrade to DJVM 1.0-RC01.

* Try to modify DriverParameters in a more "ABI friendly" way.

* Refactor and simplify sandbox deserialisation of primitive objects.

* Review fixes.

* Update EvolutionSerializerFactory to handle sandboxed primitive boxed types.
2019-11-05 13:44:18 +00:00
Chris Rankin
346410a2e1 CORDA-3327: Fix API scanning for Java 11. (#5645) 2019-10-30 17:24:34 +00:00
Stefano Franz
f9890a5359
PreAllocate pod resources during image build phase (#5587)
* use zulu for jdk
add some parallel groups

* port kubesTest to Java
remove asterix from tests listed by ListTests, instead add after allocation

* attempt to setup unit test builds with correct github integrations

# Conflicts:
#	.ci/dev/unit/Jenkinsfile

* fix issue with github context

* add credentials block

* start pre-allocating pods for builds

* test

* add blocks for reporting build stages

* add logic to preallocate pods during image building

* tidy up Jenkinsfile for unit tests

* add magic command line flag to enable preallocation of pods

* make docker tag deterministic

* fix issue concatenating docker tag inputs

* add build type specific Jenkinsfile

* try new preallocation approach

* make pre-allocation prefix group specific

* force deAllocator to wait for pods to be actually deleted

* revert jenkinsfiles in .ci

* use smarter waiting logic to address review comments

* add --stacktrace to builds to help debugging

* fix issue with closed stream

* add some logging around preallocation

* tidy up by refactoring (de)allocate task generation into method

* change default from 20 pods to 5 pods

* fix issue where docker tag was unstable between building and running tests

* more documentation

* add some infrastructure around setting the log level for a given build

* change preallocation pod duration to 5min

* see if fast enough if using combined unit and integration tests

* disable unit tests

* print out test summaries

* try and make the kubes client a per-use object, rather than a long lived object. This is step one of making GKE use possible

* add log line about what command is executed in the pod
2019-10-28 11:48:04 +00:00
Chris Rankin
971eb56a98 CORDA-3346: Remove the JitPack repository from Corda. 2019-10-21 13:15:03 +01:00
Razvan Codreanu
ee09cd8762 TM-45 Make detektBaseline pass (#5561)
* TM-45 make the baseline generating task show a successful build regardless of the existing detekt violations

* TM-45 address PR feedback
2019-10-15 15:49:31 +01:00
Stefano Franz
6e98adb085
use zulu for jdk in testing image (#5583)
* use zulu for jdk
add some parallel groups

* port kubesTest to Java
remove asterix from tests listed by ListTests, instead add after allocation
2019-10-14 13:35:22 +01:00
Stefano Franz
298c91ce82
Webook test branch (#5528)
* add ability to index by class OR method

* disable unit tests again

* pending commit

* stream output of build for debugging

* remove git from the base image

* re-enable NodeRPCTests

* add ability to distribute tests based on TeamCity CSV output for test times

* try enabling unit tests again after test distribution has been fixed

* refactor BucketingAllocator to be a class, which bucketingAllocatorTask delegates to.

* finishing touches for improved test distribution

* create new pipelines for builds
2019-10-08 16:33:24 +02:00
josecoll
246ec8766e
CORDA-3224 JDK11 built and published artifacts to include classifier. (#5488)
* JDK11 built and published artifacts to include classifier.

* Added additional non-intuitive gradle dust sprinkled recommended by CR.

* Added additional non-intuitive gradle dust sprinkled recommended by CR.

* Bump corda gradle plugins version.

* Incorporating PR review feedback.

* jdk11Classifier
2019-10-08 10:45:10 +01:00
josecoll
42e104ca4f
Publish checkpoint agent jar and allow for inclusion of version id in jar upon run-time execution. (#5553) 2019-10-04 14:28:42 +01:00
Tudor Malene
298d8ba69c CORDA-3024 Rename the webserver (#5489) 2019-09-26 10:20:49 +01:00
josecoll
6666ee1a46
Set JFX 3rd party library dependency (fontawesomefx) according to Java version. (#5513) 2019-09-24 12:09:01 +01:00
Jonathan Locke
d5d313148d
NOTIK Downgrade Dokka back to 0.9.17 due to failing docs_builder
NOTIK Downgrade Dokka back to 0.9.17 due to failing docs_builder
2019-09-19 20:34:29 +01:00
Stefano Franz
974c45bb3c
add ability to group test types together (#5459)
* add ability to group test types together

* add ability to specify podCount for use in parallel testing

* remove compiler xml

* add Jenkinsfile to enable scanning

* trigger build

* add ability to specify what docker tag to use from outside of the build

* fix docker work dir

* fix pipeline syntax issues

* use environment rather than `def`

* move agent restrictor outside of stages block

* use steps block

* more pipeline syntax fixes

* even more pipeline syntax fixes

* even more pipeline syntax fixes

* add kubenetize as property to image build

* move clear of docker image to end of build rather than start to prevent colocated builds

* escape dollar on docker image remove command

* attempt to kill all existing jobs

* fix compile issue due to killall_jobs

* fix compile issue due to killall_jobs pt2

* fix spelling

* make all variables environment variables

* add logic to delete images locally after pushing

* wrap testing phase with try / finally so that junit reports are always evaluated

* change the behaviour around post build actions

* break implicit link between testing phase and image building phase, allowing testing to occur without a rebuild and push of image

* prepend registry name to provided tag

* allow tasks to specify whether they wish to stream output from containers

* add timestamps directive to Jenkinsfile to have timing info on output

* make KubesTest resilient against transient pod failures in k8s

* increase CPU request

* add logic to allow specifying container resource requests

* attempt to run unit and integration tests in parallel

* change unit tests to use 3 cores to allow co-location on 8c machines

* join grouped tests together to give pod meaningful name

* add step to renew token with GKE

* change renew step to use pods instead of nodes

* fix bug where memory request is not correctly passed to pod

* disable unit tests for now
2019-09-19 17:41:06 +00:00
josecoll
94a1d357ec Downgrade Dokka back to 0.9.17 due to failing docs_builder. 2019-09-19 17:34:53 +01:00
Zoltan Kiss
7bbbc71fdf TM-23 compileAll task to compile all code (#5490)
* Add simple compileAll task to be used by warning check

* lazy configure compileAll
2019-09-19 10:50:29 +01:00
josecoll
3fafbe551c
CORDA-2050 Upgrade Corda to Java 11 (compatibility mode) (#5356)
Upgrade Corda to run with Java 11 (compatibility mode) - see https://github.com/corda/corda/pull/5356
2019-09-18 10:26:26 +01:00
Schife
3b1a49a2cf TM-20 rebasing the detekt changes to be able to merge into OS 4.3. The changes include: detekt integration, rule configurations, baseline of the current issues that exist in 4.3 and a MaxLineLength rule violation fix to ANSIProgress test since it was causing the baseline to fail to load due to the special characters in the test (#5464) 2019-09-12 14:08:50 +01:00
Zoltan Kiss
f171de7b69 TM-23 Fail build on compiler warnings (#5453)
* java compile respects compilation.allWarningsAsErrors

* suppress or cleanup warnings

* suppress warning

* use non-deprecated kotlin dependency

* rename property

* handle property existence check

* Deal with warnings
2019-09-11 16:34:51 +01:00
Stefano Franz
a842740c9e
WIP Kubenetes parallel build (#5396)
* Split integration tests

* add simple example of printing all methods annotated with @Test

* add docker plugin to root project
remove docker plugin from child projects
add Dockerfile for image to use when testing
add task to build testing image to root project

* add comment describing proposed testing workflow

* simple attempt at running tests in docker container

* add my first k8s interaction script

* add fabric8 as dependnency to buildSrc

* before adding classpath

* collect reports from containers and run through testReports

* re-enable kubes backed testing

* for each project
1. add a list tests task
2. use this list tests task to modify the included tests
3. add a parallel version of the test task

* tweak logic for downloading test report XML files

* use output of parallel testing tasks in report tasks to determine build resultCode

* prepare for jenkins test

* prepare for jenkins test

* make docker reg password system property

* add logging to print out docker reg creds

* enable docker build

* fix gradle build file

* gather xml files into root project

* change log level for gradle modification

* stop printing gradle docker push passwd

* tidy up report generation

* fix compilation errors

* split signature constraints test into two

* change Sig constraint tests type hierarchy

* tidy up build.gradle

* try method based test includes

* add unit test for test listing

* fix  bug with test slicing

* stop filtering ignored tests to make the numbers match existing runs

* change log level to ensure print out

* move all plugin logic to buildSrc files

* tidy up test modification
add comments to explain what DistributedTesting plugin does

* move new plugins into properly named packages

* tidy up runConfigs

* fix compile errors due to merge with slow-integration-test work

* add system parameter to enable / disable build modification

* add -Dkubenetise to build command

* address review comments

* type safe declaration of parameters in KubesTest
2019-09-03 15:40:08 +00:00
James Brown
b8e278680d CORDA-3120 Update owasp scanner (#5335) 2019-08-08 16:05:54 +01:00
Stefano Franz
88894bc592
add a shared memory port allocator to allow multiple processes to sha… (#5223)
* add a shared memory port allocator to allow multiple processes to share a single allocation pool

* remove dangerous reset function on port allocator

* set forkCount = 2 in node integration test

* only allow one build of a cordapp at any given time for Driver tests

* make all portallocation requests use same starting point

* globally set forks to 6

* tweak forking parameters to allow parallel builds

* tweak unit test parallelism

* 2 workers for integrationTest

* some more tweaks for parallel builds

* some more tweaks for parallel builds

* seems that 49K is not the start of ephemeral ports on all kernels

* tweak parallel settings

* try fix RPC shutdown test in parallel env

* add some logging for RPC shutdown test

* added some logging around PortAllocation tests - try figure out where they are getting stuck

* added some logging around PortAllocation tests - try figure out where they are getting stuck

* fix api-scanner tests

* minimize api changes

* revert to complying with existing API

* add the AtomicInteger for api compatibility reasons

* make sizing script executable

* address review comments pt1

* address review comments pt2

* fix compile errors after review comments

* return to using home dir as temp dir seemed to interact badly with gradle
2019-07-02 18:38:33 +00:00
szymonsztuka
f89008c070 CORDA-2998 fix network builder (#5265)
CORDA-2998 fix network builder
2019-07-02 11:32:48 +01:00
Dimos Raptis
5db70192ec [CORDA-3042] - Revert upgrade of dokka (#5259) 2019-06-27 18:28:12 +01:00
Dan Newton
5e48ad0f52 Upgrade jackson_version to 2.9.7 (#5261)
`jackson-module-kotlin` version `2.9.5` was causing runtime errors.
Updating to `2.9.7` resolved these errors. The dependency was not
updated to a more recent version as they require kotlin 1.3.
2019-06-27 16:48:35 +01:00
Shams Asari
4e835d5483
CORDA-2333: Updated the majority of the dependencies that were out of date (#5235)
Some where not touched, including the crypto ones, libraries related to messaging and Kotlin.
2019-06-20 16:19:16 +01:00
Dominic Fox
f01f8a129e ENT-3444 create test-db module (#5093)
* ENT-3444 define RequiresDB annotation and junit5 extension

* Move to internal

* info to trace

* Limit exposure of gradle imports

* Enable annotation inheritance, and multiple SQL scripts per class or method

* Get the test context class globally for all groups

* usingRemoteDatabase flag
2019-06-18 10:57:20 +01:00
Shams Asari
df19b444dd
CORDA-2333: Reverting jersey and mockito as it currently causes issues with ENT (#5228) 2019-06-17 19:27:35 +01:00
Shams Asari
bb3997ed5b
CORDA-2333: Reverting ClassGraph version back to 4.6.12 (#5221)
4.8.38 not working with the changes from 2152961d
2019-06-13 17:22:09 +01:00
Shams Asari
2152961d46
CORDA-2333: Dependency update pass for tests and demos (#5211) 2019-06-13 10:15:13 +01:00
Shams Asari
8480d49e7e
CORDA-2333: Bumped ClassGraph version to latest (#5214) 2019-06-12 13:27:51 +01:00
josecoll
8678bad88d
CORDA-2970 Revert usage of Gradle JUnit 5 Platform Runner (#5173)
* Revert usage of Gradle 5 useJUnitPlatform() which causes significant test execution performance degradation.

* Remove completely top-level Java CordaCaplet unit tests causing consistent TC failure on Azure VMs:

 java.security.NoSuchAlgorithmException: no such algorithm: NONEwithEdDSA for provider BC
2019-06-05 10:17:02 +01:00
James Brown
24c8a5a2dd ENT-3540 Reverting jackson - kotlin runtime issue (#5133)
Jackson was updated to 2.9.8, however this version has a transitive dependency on kotlin 1.3.10 which is incompatible with corda (which uses 1.2 currently).
2019-06-05 09:04:13 +01:00
josecoll
06db5973d5
Reduce test execution times by explicitly configure quasar package exclusions (#5161)
* Reduce test execution times by explicitly configure quasar package exclusions (using new quasar util extension introduced in Corda gradle plugins 5.0.1-SNAPSHOT).

* Remove SNAPSHOT from Corda gradle plugins version identifier.

* Incorporating PR review feedback from CR.

* Minor consolidation clean-up.

* Minor consolidation clean-up.

* Minor consolidation clean-up.
2019-05-30 10:09:11 +01:00
Anthony Keenan
b06fa62860
Update OWASP dependency checker to v4.0.2 to fix clash with Gradle 5 upgrade (#5177) 2019-05-29 10:43:32 +00:00
josecoll
0263650a5f
Revert "Merge pull request #5145 from corda/jdk11-migration-gradle5-upgrade" (#5146)
This reverts commit f46377556c, reversing
changes made to ba675e73b9.
2019-05-20 16:16:49 +01:00
josecoll
84ca6405c2 Rebase from OS master to incorporate upgrade to Gradle 5.2.1 2019-05-20 15:09:36 +01:00
josecoll
d2f2f4aaca Display JAVA_HOME 2019-05-20 15:09:36 +01:00
josecoll
b03b5133f6 Added configurable flag to continue on test failure so TC can perform complete test execution sweep. 2019-05-20 15:09:34 +01:00
josecoll
9183e401c3 Upgraded Mockito and targetCompatibility to 11 (REVISIT). 2019-05-20 15:08:27 +01:00
josecoll
8d4816f203 Allow corda gradle plugin snapshot version resolution from artifactory 'corda-dev' 2019-05-20 15:08:27 +01:00
josecoll
8c921cfe71 Upgrade jacoco to JDK11 supported version 2019-05-20 15:08:27 +01:00
josecoll
a56ef580b2 Temp remove usage of java modularity plugin. 2019-05-20 15:08:27 +01:00
josecoll
57fb63940f Include Gradle JPMS plugin (v1.5.0) 2019-05-20 15:08:27 +01:00
josecoll
ba675e73b9
CORDA-2893 Upgrade Corda to use Gradle 5.4.1 (Take 2) (#5137)
Upgrade Corda to use Gradle 5.4.1
2019-05-20 14:26:42 +01:00
Chris Rankin
b9f7c1a08a CORDA-2893, ENT-3422: Tweak JUnit 5 configurations to keep vintage engine off compile classpaths. (#5138)
* Tweak JUnit 5 configurations to keep vintage engine off compile classpaths.

* Configure Gradle tests to use JUnit 5.
2019-05-20 11:57:56 +01:00
josecoll
0eab42b63e
Revert "CORDA-2893 Upgrade Corda to use Gradle 5.x (#5063)" (#5135)
This reverts commit 88babefd89.
2019-05-17 11:11:05 +01:00
josecoll
88babefd89
CORDA-2893 Upgrade Corda to use Gradle 5.x (#5063)
* Upgrade gradle wrapper to Gradle 5.0

* Upgrade to use locally deployed version of Capsule plugin (using Gradle 5.0)

* Upgrade to use Corda Gradle Plugins 5.x (inclusive of Gradle 5.0 compatibility fixes)

* Fix compile-time problems resolving log4j packages.

* Update to use Artifactory deployed version of Capsule plugin (using Gradle 5.0)

* Upgrade to use Gradle 4.2.1 (excluding default memory setting change: command line client now starts with 64MB of heap instead of 1GB.)

* Added explicit compile-time dependency on 'de.jensd:fontawesomefx-commons:8.13' (compile-time error in tools:explorer without)

* Update "wrapper" gradleVersion to 5.4.1

* Update Capsule plugin version label to reference R3 forked build.

* Commit all gradle upgrade changes following: ./gradlew wrapper --gradle-version 5.4.1

* Increase maximum heap memory for Test tasks to 1GB, and upgrade build-scan plugin to 2.2.1.

* Increase Test tasks' maximum memory to 1.5GB - what could go wrong?

* Up maxHeapSize to 2g for gradle test runner (global setting).

* Added explicit compile-time dependency on 'de.jensd:fontawesomefx-commons:8.13' (compile-time error in tools:demobench without)

* Added forkEvery for node Unit tests.

* Up :node test task 'forkEvery' to 10.

* TC test execution tuning (:core, :node => forkEvery 10, default JVM heap size)

* TC test execution tuning (bump JVM heap size up to 1g for :node)

* TC test execution tuning (bump JVM heap size up to 1.5g for :node)

* TC test execution tuning (re-instate global JVM heap size of 1Gb)

* TC test execution tuning (re-instate JVM heap size of 2Gb for :node)

* Update Corda Gradle plugins to 5.0.0

* Updated plugin resolution order + renamed artifactory URL to use "software.r3.com"

* Reorder plugin resolution such that mavenlocal() is always queried first.
2019-05-17 10:36:21 +01:00
Dominic Fox
0b63157a4b ENT-3422 Add JUnit 5 dependencies to all projects (#5084)
* ENT-3422 Add JUnit 5 dependencies to all projects

* Remove explicit junit 4 imports where not needed

* Minor PR fixes

* Limit import scope
2019-05-15 16:40:12 +01:00
James Brown
ee3befcc6e ENT-3543 Update okhttp dependency (#5089) 2019-05-15 10:54:44 +01:00
Matthew Nesbit
7bcff70864
Update the proton-j library to latest version (#5105) 2019-05-08 13:04:49 +01:00
James Brown
4838fb1bee ENT-3540 update Jackson dependency (#5083) 2019-05-02 11:21:37 +01:00
James Brown
4e3b6f01c1 ENT-3535 Update Hibernate dependency (#5080) 2019-05-01 13:47:27 +01:00
Anthony Keenan
e61a8424da Publish corda-common-logging (#5002)
Merging this because team city is broken and i need the change.
2019-04-11 10:04:16 +01:00
Stefan Iliev
746fcc32e5 CORDA-2519 Changed crash version to our latest. (#4976)
* Changed crash version to our latest.

* Changed short revision to long revision on the crash dependency from jitpack.
2019-04-09 18:56:01 +00:00
Chris Rankin
4b41bd0189 BUILD: Resolve warnings with Gradle and system property API usage. (#4904) 2019-03-20 13:39:51 +00:00
Chris Rankin
abbf2562c2 CORDA-2750: Modularise the DJVM build. (#4897)
* CORDA-2750: Modularise the DJVM build.

* CORDA-2750: Update documentation and scripts for modular DJVM.
2019-03-17 06:06:51 +00:00
Chris Rankin
03967742e4 CORDA-2747: Package DJVM CLI for delivery. (#4893)
* Modify DJVM CLI so that it runs as executable JAR.
 * Package DJVM CLI as a distributable ZIP.
 * Add djvm.bat file for Windows.
2019-03-14 17:42:39 +00:00
Mike Hearn
785755067a Docs: use version numbers read from the build system, not duplicated. 2019-03-07 16:44:50 +01:00
Andrius Dagys
b52c7a09a3 CORDA-2668 - TestCordapp – use Gradle tooling API for builds (#4794)
TestCordapp – disable daemon on internal gradle process

The TestCordappImpl runs gradle to build cordapp jars required for tests.
The started gradle process reuses a Gradle daemon that's potentially already
used for running the tests causing the JVM to die with SIGBUS.
2019-02-22 16:48:28 +00:00
Rick Parker
6c4433d0b5 CORDA-2646 - Database connection pools leaking memory on every checkpoint (#4773)
* ENT-3053 Database connection pools leaking memory on every checkpoint. Flip in the thread local from the thread into the fiber.

* Back port to OS (needs some gradle changes) and added TODO, ability for it to avoid erroring if not using Hikari.

* Review feedback to remove warning.
2019-02-20 11:28:32 +00:00
josecoll
db35f73bcc
Remove old reference to publishing a single corda finance jar. (#4704) 2019-02-07 18:33:20 +00:00
James Brown
8722e9f0cf
ENT-2945 Update postgresql dependency (#4550)
Update postgresql to 42.2.5
2019-01-24 14:58:30 +00:00
Shams Asari
c6a7d14f4f
Reducing the size of the CorDapp jars used in tests (#4600)
The helper method enclosedCordapp is a replacement to scanning the current package (which pulls in a lot more into the jar than intended).
2019-01-21 13:54:26 +00:00
Katelyn Baker
8e4dbc7060 RELEASE - Post V4 branch cut update platform version to 5 2019-01-13 20:34:22 +00:00
Andrius Dagys
fa025dedeb
ENT-2822: Move experimental raft and bft-smart notaries back into node, fix reference state support (#4509)
Move Raft and BFT-Smart notaries back into node to preserve backwards compatibility.

* Allow overriding full node config when using internal mock network parameters.

* Make BFT-Smart notary start up in prod mode as well

* Move raft & bftsmart notaries to net.corda.notary.experimental package

* Make sure Raft notary handles reference state edge cases correctly.

* Make sure BFT-Smart notary handles reference state edge cases correctly.

* Include notary schemas in node internal schemas

* Undo Raft notary table schema changes to maintain compatibility.
2019-01-09 15:52:42 +00:00
Stefano Franz
e98dbcc2af move back into test-utils 2019-01-04 17:58:43 +00:00
Stefano Franz
5081fcc9a8 break out the FungibleAsset testing schemas into module to prevent having to publish a test-jar. 2019-01-04 17:58:43 +00:00
szymonsztuka
4aaefb4fe9 CORDA-2331 Split Workflow and contracts of Finance App into separate Cordapps (#4422)
* Split Workflow and contracts of Finance App into separate Cordapps, part 1 - content which is different between OS and ENT is still in contract Cordapp.

* Move CashSelection implementations to workflow module.

* Move CashSelection implmentations to workflow module.

* Move finance module to finance-flows, top level finance module is empty.

* Move finance module to finance-flows, top level finance module is empty.

* Updated build comment.

* Revert publication of combined (contracts and flows) corda-finance.jar (to maintain backwards compatibility with 3rd party cordapps dependent on finance)

* Added backwards compatibility clarification comment.

* Re-instate new cordapp metadata.

* Global rename of `finance-flows` to `finance-workflows` to follow adopted naming conventions.

* Addressed final review comments.

* Rename application to "Corda Finance Demo"

* Generation of original corda-finance jar from new sub-modules.

* Fixed and tested demobench with new split finance contract and workflow jars.

* Renamed finance sub-modules to contracts and workflows.

* Remove Michele!!!

* Minor fix to filtering logic.

* Align CorDapp configuration filename with workflows jar.

* Fix breaks caused by finance module naming changes.

* Final alignment between OS/ENT of finance contract code.
2018-12-19 18:02:51 +00:00
James Brown
351207145c
CORDA-2322 disable owasp assembly scan (#4403) 2018-12-12 13:02:50 +00:00
Michele Sollecito
5f02425ef5
[CORDA-2285]: Enabled jUnit parallel execution. (#4344) 2018-12-03 13:31:17 +00:00
szymonsztuka
759fd9b5bb
Update to corda-gradle-plugins v4.0.36 (#4292)
* Update to corda-gradle-plugins v4.0.36 (new development key for JAR signing).
* Changed maven repo from https://dl.bintray.com/kotlin/kotlin-eap/   to https://kotlin.bintray.com/kotlinx.
2018-11-27 11:41:05 +00:00
Anthony Keenan
b7d04b1c6e
[CORDA-2235]: Add overrides for network parameters via command line and file (#4279)
* Temp commit

* Print the error message first by default, makes error output more natural.

* Polishing

* Further modifications after testing

* Documentation updates

* Couple of fixes after review

* Removing unnecessary tests

* Fix broken test

* Add interface to bootstrapper for testign

* Added unit tests

* Remove unused class

* Fix up bootstrapper unit tests and add a couple more

* Refactor the tests slightly

* Review comments

* Couple of minor tweaks
2018-11-26 17:11:05 +00:00
Michele Sollecito
97c1e5babc
[CORDA-2211]: Added error for older bash versions when attempting to install shell extensions. (#4263) 2018-11-20 14:01:39 +00:00
Anthony Keenan
3ea03068b8
Update picocli to latest version (#4241) 2018-11-16 10:10:52 +00:00
Michele Sollecito
2f833f589c
Fixed missing entries for common modules in parent build.gradle file. (#4214)
* Fixed missing entries for common modules in parent `build.gradle` file.

* Fixed publication names to match existing convention.
2018-11-12 13:38:35 +00:00
James Brown
c205a10ec0
ENT-2684 Fix owasp dependency checker to fail builds based on CVSS level (#4169)
OWASP Dependency Checker has been updated so that it can be configured to automatically fail a build when a vulnerable dependency is detected. This option is exposed through gradle settings, so that a build can be configured in TeamCity to pass/fail. This change is backward-compatible - i.e. it does not affect a build by default unless configured to do so.

Ability to fail a build is exposed by the new owasp.failBuildOnCVSS gradle property
By default this is set to '11.0' which will always pass a build even if a vulnerability is found (so will not affect anything by default)
Reduce the CVSS level between 0-10 to indicate what level to fail a build on
Example usage to catch Medium severity (and above) issues:

Configure gradle with the option -Powasp.failBuildOnCVSS=4
2018-11-07 09:00:19 +00:00
bpaunescu
16453ebfcc
update OWASP to latest version after kotlin update (#4109) 2018-10-24 14:36:01 +01:00
Anthony Keenan
0ab644783e CORDA-1838: Add subcommands to node (#4091)
* Tidy up

* Add install-shell-extensions command

* Make cli tests use same version of picocli as everything else

* Remove initLogging from NodeStartup, it is ran earlier by CordaCLIWrapper

* Use picocli snapshot for testing

* Use RunLast() parser to invoke correct subcommands

* Deprecate old clear-network-map-cache parameter

* Restructure NodeStartup for commands

* Get rid of -c option since the flag method has been deprecated and that didn't exist in last release

* Update documentation

* Update backwards compatibility test

* Get all subcommands working

* Refactor sub commands into seperate classes

* Update docs and fix some tests

* Docs changes

* Fix merge conflicts with master

* Fix renamed parameters

* Fix test failure

* Fix compatibility tests

* Add missing compatibility test for blob inspector

* Remove blob inspector compatibility test as there are import conflicts

* Assorted doc fixes

* Addressing review comments

* More review comments

* Couple more bits

* Fix broken tests

* Fix compilation error

* More merge conflicts

* Make startup logging function a bit more sensible

* Fix broken shell extensions

* Make shell extensions work with subcommands

* Make sure parameters for deprecated options are carried through

* More review comments

* Adding some s's

* One last go

* Fix compilation error on Windows

* Revert logging changes

* Revert docs back to their original imperatively moody state
2018-10-24 13:58:19 +01:00
Tudor Malene
391c6bf66f
Feature/corda 1947/add package ownership (#4097)
* Upgrade hibernate and fix tests

CORDA-1947 Address code review changes

CORDA-1947 Address code review changes

(cherry picked from commit ab98c03d1a)

* ENT-2506 Changes signers field type

ENT-2506 Clean up some docs

ENT-2506 Fix tests and api

ENT-2506 Fix compilation error

ENT-2506 Fix compilation error

(cherry picked from commit 32f279a243)

* CORDA-1947 added packageOwnership parameter

CORDA-1947 add signers field to DbAttachment. Add check when importing attachments

CORDA-1947 add signers field to DbAttachment. Add check when importing attachments

CORDA-1947 add tests

CORDA-1947 fix comment

CORDA-1947 Fix test

CORDA-1947 fix serialiser

CORDA-1947 fix tests

CORDA-1947 fix tests

CORDA-1947 fix serialiser

CORDA-1947 Address code review changes

CORDA-1947 Address code review changes

CORDA-1947 Revert test fixes

CORDA-1947 address code review comments

CORDA-1947 move verification logic to LedgerTransaction.verify

CORDA-1947 fix test

CORDA-1947 fix tests

CORDA-1947 fix tests

CORDA-1947 address code review comments

CORDA-1947 address code review comments

(cherry picked from commit 86bc0d9606)

CORDA-1947 fix merge
2018-10-22 15:00:08 +01:00
Shams Asari
aced03df54
CORDA-1274: Migrated usage of FastClasspathScanner to ClassGraph (#4060)
FastClasspathScanner was renamed to ClassGraph for the version 4 release
2018-10-11 19:50:26 +01:00
Andrius Dagys
9ebeac1ad8
CORDA-535: Extract notary implementations into CorDapps (#3978)
* Move Raft and BFT notaries into separate modules

* Move schemas

* Fix tests & demos

* Modified logic for creating notary services:

Added a new field 'className' to the notary configuration. The node now
loads the specified implementation via reflection. The default className
value points to the simple notary implementation for backwards compatibility.
Relevant schemas are loaded in a similar fashion.

For backwards compatibility purposes the default SimpleNotaryService will
remain built-in to node, but its cordapp will be generated on startup – so
the loading of notary services is streamlined.

* Move test namedcache factory to test utils
2018-10-10 10:04:22 +01:00
Shams Asari
bffac331a3
Moved the PLATFORM_VERSION constant to core and added some missing usages (#4026) 2018-10-05 09:28:00 +01:00
Christian Sailer
92d2e4ae38
Corda-1931 Asynchronous logging (#3989)
* Provide AsyncLoggingContextSelector that inhibits use of thread local storage

* Turn on async logging via log4j properties file

* Mention async logging in the documentation and also explain how to turn it off when required.

* Formatting

* Typo

* Add shutdown hook to flush loggers.

* code review rework

* Ring buffer size to 256kB

* Set maximal log file size to 100MB - should slow down the rolling of log files and give us a bit more history on the cluster.
The old limit of max 10GB of compressed log files still stands.
2018-10-02 10:33:17 +01:00
josecoll
6f27898664
CORDA-1509 Configure and enable Gradle Build Cache (#3908)
* Fix to enable gradle build caching of test runs.

* Configure gradle build caching to be enabled.

* Generate dependency report and graph:
1) ./gradlew htmlDependencyReport
2) ./gradlew generateDependencyGraphCorda

* Strip out all Jacoco references to prevent Gradle Build Cache error:
"Caching disabled for task ':<module>:test': 'JaCoCo agent configured with `append = true`' satisfied"

* Revert jacoco back into jarFilter gradle build file.
Disable building deterministic modules (including jarFilter).

* Added Gradle build scan plugin.

* Set file encoding to prevent incorrect Gradle build cache keys across different machines.
https://guides.gradle.org/using-build-cache/#system_file_encoding

* Apply gradle build cache settings to buildSrc.

* Added targetted gradle build tasks to leverage cache.

* Updated URL's of several different test Gradle Cache instances.

* Updated CI batch build scripts.

* Updated script perms to be executable.

* Added CI smoke tests batch script.

* Use TestDev Labs gradle cache.

* Echo URL of Gradle Build Repository.
Echo exit status of each Gradle build command.

* Use environment variables to define Gradle Build Cache usage and URL.

* Customisation through parameters.

* Remove dependency graph generation plugin.
Align build cache settings across project and buildSrc gradle files.

* Remove buildSrc gradle build cache config.

* Revert definition of gradle build cache variables back to settings.gradle.

* Fix incorrect path.

* Aligned gradle build cache configuration across buildSrc and project.

* Minor updates to test scope.

* Minor updates to test scope.

* Update scripts to use GRADLE_HOME

* Exit on unset GRADLE_HOME

* Remove duplication following rebase from master.

* Remove fine-grained build task scripts.

* Added back Jacoco reporting.

* Revert jdk8u-deterministic module.

* Incorporating changes from PR review feedback.

* Workaround Jacoco issue associated with Gradle Build Cache test task.

* Update init script.

* Remove redundant build-scan declarations.

* Updates from PR review feedback.

* Remove GRADLE_HOME as no longer needed - everything is driven via gradle wrapper.

* Use CORDA prefix in system environment variable.
2018-09-26 13:38:23 +01:00
Dan Newton
6775ffa2e6 Check java version is above 1.8.0_171 on node startup (#3870) 2018-09-19 16:34:03 +01:00
Chris Rankin
df4699c69a
CORDA-1985: Configure Gradle capsule plugin to use Capsule 1.0.3. (#3949) 2018-09-17 13:55:31 +01:00
Chris Rankin
7e4c5d4bb0
CORDA-1983: Upgrade to Gradle 4.10.1. (#3947) 2018-09-15 14:23:16 +01:00
Anthony Keenan
ec0fc849ed
Publish cliutils package (#3930) 2018-09-13 11:14:19 +01:00
Chris Rankin
1c7dfd4b7b
CORDA-1964: Unify versioning of the shadow plugin, and resolve more issues for Gradle 5.0. (#3918) 2018-09-13 10:55:52 +01:00