Commit Graph

8373 Commits

Author SHA1 Message Date
ac8a4fc08c TM-170 Generic on-demand-test configuration (#5863)
* Generic on-demand test configuration

* Rename library "magic" string from existing-build-control to
corda-shared-build-pipeline-steps
2020-01-24 11:26:52 +00:00
8842be29ac NOTICK disable max line length check in detekt. (#5882) 2020-01-24 11:26:52 +00:00
fe52a415fb Manual fix of mergr for PR-5797 (#5856) 2020-01-24 11:26:52 +00:00
2809c0484d TM-137 Daily email report for release branch regression tests (#5797)
* Daily email report for release branch regression tests

* Manual merge fix
2020-01-24 11:26:52 +00:00
9ae60a934c Strategic fix for allure report generation memory issues (#5845) 2020-01-24 11:26:52 +00:00
b512a2981d ENT-3828: Remove ability to SSH into standalone shell (#5876) 2020-01-24 10:44:25 +00:00
ffe75db464 TM-170 Generic on-demand-test configuration (#5863)
* Generic on-demand test configuration

* Rename library "magic" string from existing-build-control to
corda-shared-build-pipeline-steps
2020-01-24 10:39:23 +00:00
42a2ed98e2 CORDA-2942: Allow exception from CordaService creation to propagate (#5884)
* CORDA-2942: Allow exception from `CordaService` creation to propagate

It will ultimately be thrown from Node's `start()` method terminating the node start-up sequence.

* CORDA-2942: Be lenient when retrievign the name of the Notary

Some tests setup such that they do nto have Notary running.
2020-01-24 10:20:08 +00:00
1380779a9c Remove unused schema options (#5885) 2020-01-24 08:49:59 +00:00
f02bfc134b NOTICK disable max line length check in detekt. (#5882) 2020-01-23 16:40:38 +00:00
b72f71e7ac CORDA-3549: Re-write flaky CordaServiceLifecycleFatalTests (#5879)
* CORDA-3549: Improve stability of `CordaServiceLifecycleFatalTests`

* CORDA-3549: Bump-up reps count to ensure that test is definitely not flaky when executed by CI
(once proved the number of reps will be reduced)

* CORDA-3549: Making Detekt happier

* CORDA-2942: Ensure `NodeLifecycleEventsDistributor` cleans-up smoothly when node shuts down
2020-01-23 16:35:49 +00:00
c1909722f6 CORDA-3552: Reuse a single SerializerFactory when deserialising a LedgerTransaction into the DJVM sandbox. (#5874) 2020-01-23 09:21:01 +00:00
6a2249e697 CORDA-3520, CORDA-3550: SSH memory leak and security (#5873)
* CORDA-3520: Closing RPC connection on SSH disconnect
CORDA-3550: Remove support for outdated ciphers and algorithms from SSH

* CORDA-3550: Remove support for outdated ciphers and algorithms from SSH
2020-01-22 14:40:25 +00:00
597658c4ab ENT-4822: Don't attempt RPC reconnect for NotSerializableException (#2991) (#5870)
* ENT-4822: Don't attempt RPC reconnect for NotSerializableException

* ENT-4822: Don't attempt RPC reconnect for NotSerializableException
2020-01-22 11:37:38 +00:00
9a1d46ddd1 EG-73 - Remove postgres dependency from the node and update docs (#5867) 2020-01-22 09:52:02 +00:00
4bae045a58 ENT-4601 Public API to run external operations from a flow (#5833)
Deprecate FlowAsyncOperation and reimplement public versions FlowExternalOperation and FlowExternalAsyncOperation.

await added to FlowLogic to allow easy calling from both Java and Kotlin. There are two overrides of await (one for FlowExternalOperation and FlowExternalAsyncOperation).

Implementations of FlowExternalOperation return a result (written as blocking code) from their execute function. This operation will then be executed using a thread provided by the externalOperationExecutor.

Implementations of FlowExternalAsyncOperation return a future from their execute function. This operation must be executed on a newly spawned thread or one provided by a thread pool. It is up to developers to handle threading in this scenario.

The default thread pool (externalOperationExecutor) can be configured through the flowExternalOperationThreadPoolSize node config.

The current implementation leaves FlowAsyncOperation alone, meaning that any developers that have used it (even though it is internal) won't need to change their apps. If this was not concern I would delete it completely and replumb the state machine code. Instead, it has been marked with @DoNotImplement and executeAsync is annotated with @Deprecated
2020-01-22 09:27:17 +00:00
0978500a9a CORDA-2942: Node lifecycle events (#5846)
* CORDA-2942: Port minimal set of changes to make lifecycle events work

... and make codebase compile.

* CORDA-2942: Undo some changes which are not strictly speaking necessary

* CORDA-2942: Make `NodeServicesContext` leaner and delete `extensions-api` module

* CORDA-2942: Reduce even more number of files affected

* CORDA-2942: Integration test fix

* CORDA-2942: Make events `AfterStart` and `BeforeStop` generic w.r.t. `NodeServicesContext`

* CORDA-2942: `NodeLifecycleObserverService` and a set of integration tests.

Public API violations are expected as well as integration tests failing.

* CORDA-2942: Re-work to introduce `ServiceLifecycleObserver`

* CORDA-2942: Explicitly mention a type of exception that may be thrown for some events.

* CORDA-2942: Register `ServiceLifecycleObserver` through `AppServiceHub`

* CORDA-2942: Fix integration test + KDocs update

* CORDA-2942: Detekt and `api-current` update

* CORDA-2942: Improvement to `CordaServiceLifecycleFatalTests`

... or else it has side effects on other tests.

* CORDA-2942: Add an integration test for new API use in Java

Driver test is written in Kotlin, but services definition is written in Java.

Also KDocs improvements.

* CORDA-2942: Documentation and release notes update

* CORDA-2942: First set of changes following review by @mnesbit

* CORDA-2942: Second set of changes following review by @mnesbit

* CORDA-2942: Added multi-threaded test

* CORDA-2942: Fixes

* CORDA-2942: Undo changes to `api-current.txt`

* CORDA-2942: Bare mimimum change to `api-current.txt` for CI gate to pass.

* CORDA-2942: Address review feedback from @rick-r3

* CORDA-2942: Detekt update

* CORDA-2942: Delete `ServiceLifecycleObserverPriority` and replace it with `Int` after discussion with @mnesbit

* CORDA-2942: Introduce more `NodeLifecycleEvent` and switch services to listen for those events

* CORDA-2942: Few more changes after input from @rick-r3

* First stub on integration test
Unfinished - hang on issue and pay

* CORDA-2942: Switch to use out-of-process nodes for the inetgration test

Currently Alice and Notary stuck waiting to hear from each other.

* CORDA-2942: Extra log lines during event distribution

* CORDA-2942: Asynchronously distribute lifecycle events

* CORDA-2942: Await for complete P2P client start-up

Next step: Add vault query to integration test

* CORDA-2942: Asynchronously distribute lifecycle events

Next step: Improve integration test

* CORDA-2942: Fix test broken by recent changes and improve logging

* CORDA-2942: Improvement of the test to be able to monitor actions performed by @CordaService in the remote process

* CORDA-2942: Add node re-start step to the integration test

* CORDA-2942: Remove `CORDAPP_STOPPED` event for now

* CORDA-2942: s/CORDAPP_STARTED/STATE_MACHINE_STARTED/

* CORDA-2942: Inverse the meaning of `priority` as requested by @rick-r3

* CORDA-2942: Register `AppServiceHubImpl` for lifecycle events and put a warning when SMM is not ready.
2020-01-21 13:38:02 +00:00
a4d00b79d4 CORDA-3377: Upgrade to DJVM 1.0-RC06. (#5866) 2020-01-21 09:05:23 +00:00
49b4f66515 CORDA-3377: Upgrade to DJVM 1.0-RC05 (#5862) 2020-01-17 09:46:15 +00:00
01666ed068 CORDA-3523: DJVM custom serializers and whitelists (#5858) 2020-01-16 17:48:15 +00:00
f027fd5c77 ENT-4652: Add unit tests for parsing attachment fix-up rules. (#5859) 2020-01-16 17:19:58 +00:00
b195e4f703 Manual fix of mergr for PR-5797 (#5856) 2020-01-16 11:55:01 +00:00
a7147c1ffd ENT-4652: Provide an "attachment fixup" mechanism to repair broken transactions. (#5825)
* Do not register cordapp custom serialisers when using attachment classloader.

* Record the URLs of CorDapp JARs that contain custom serialisers. Include these JARs as extra attachments if we discover that we're missing a custom serialiser during transaction verification.

* Check for disabled serializer when explicitly requesting a custom serializer.
Refactor test case to force use of a custom serializer.

* Tidy up basic custom serializer test.

* Also test that TransactionBuilder rejects missing custom serializers.

* Remove test whitelists, which should not be needed with custom serialisers.

* Add changelog entry. Also align TestCordappImpl.findRoots() with OS backports.

* Second approach based around CorDapps inside AttachmentStorage - report missing type descriptor or any non-composable types.

* Initial implementation of Corda-Fixup rules inside a CorDapp jar.

* Replace original "automatic attachment fixing" mechanism completely.

* First review comments: restore "missing class" logic to TransactionBuilder.

* Restore "missing class" mechanism as fallback for SignedTransaction too.
2020-01-14 15:18:51 +00:00
86347abe79 TM-137 Daily email report for release branch regression tests (#5797)
* Daily email report for release branch regression tests

* Manual merge fix
2020-01-14 11:52:04 +00:00
38fb51b74b Strategic fix for allure report generation memory issues (#5845) 2020-01-14 11:10:42 +00:00
4669a699c0 CORDA-3507: Use the config value for connectionRetryInterval (#5850)
* CORDA-3507: Use the config value for connectionRetryInterval rather than a hardcoded value

* CORDA-3507: Use the config value for connectionRetryInterval rather than a hardcoded value
2020-01-14 09:57:11 +00:00
672106e3dc [NOTICK] Print out local version (#5851)
* Print out local version when running install or publishToMavenLocal

* Just always print the current version - so we also see it when publishing to artifactory.
2020-01-14 08:36:00 +00:00
73a0782f5d CORDA-3452: Node: Configure the input of custom string in CSR (#5844)
* CORDA-3452: Node: Configure the input of custom string in CSR to be used by Identity Service

* CORDA-3452: Remove unused import

* CORDA-3452: Add test for networkServices configuration
2020-01-13 09:52:51 +00:00
ce774e459a [CORDA-3501] - Provide option for user to specify custom serializers (#5837)
* [CORDA-3501] - Provide option for user to specify custom serializers

* Remove not needed integration test shemas
2020-01-09 11:29:16 +00:00
d0543d7270 [CORDA-3436] Allow CorDapps access to node diagnostic information (#5834)
* [CORDA-3436] Allow CorDapps access to node diagnostic information

* [CORDA-3436] Fix API breakages

* [CORDA-3436] Improve documentation around diagnostics service

* [CORDA-3436] Remove CorDapps from the diagnostics information

* [CORDA-3436] Silence detekt warning
2020-01-09 11:18:32 +00:00
e0eac8fa0d Corda 3513 rpc flow without permission (#5828)
* CORDA-3513: Don't try to reconnect for PermissionExceptions

* CORDA-3513: Don't try to reconnect for PermissionExceptions

* CORDA-3513: Add test for not reconnecting for PermissionExceptions

* CORDA-3513: Update exception message and test
2020-01-07 11:35:45 +00:00
5df5d01f14 [ENT-4489] Add notary doctor (#5836) 2020-01-06 12:21:50 +00:00
7f4c2ca974 CORDA-2942: Improve tests coverage for Checkpoint dumper (#5830)
* CORDA-2942: Switch to use predictable timestamp

* CORDA-2942: Validate content of dumped checkpoint

* CORDA-2942: First stub on the integration test
(no checkpoints dumped for some reason using RPC)

* CORDA-2942: Reduce checkpointing code to bare minimum

* CORDA-2942: Minor refactoring

* CORDA-2942: Verify dump checkpoint content
2019-12-30 10:43:59 +00:00
d5e7c9abbd DOCS - typo (#5815) 2019-12-20 15:07:31 +00:00
edd1205db1 OS 4.3 -> OS 4.4 Merge (#5819)
* Prevent on-demand tests re-triggering from branch indexing

* Mark integration test tasks with "big" node taint

* Jenkins file for nightly regression tests (#5786)

* Jenkins file for nightly regression tests

* Use k8s instead of gke cluster

* DOCS: Update UAT.md (#5602)

* Fix report generation against regression builds (#5818)

* Prevent on-demand tests re-triggering from branch indexing

* Mark integration test tasks with "big" node taint

* Jenkins file for nightly regression tests (#5786)

* Jenkins file for nightly regression tests

* Use k8s instead of gke cluster

Co-authored-by: Stefano Franz <roastario@gmail.com>
Co-authored-by: carolynequinn <44175553+carolynequinn@users.noreply.github.com>
2019-12-20 14:44:45 +00:00
885bc534af CORDA-3489 State Evolution: Support adding new mandatory field and removal of optional (#5817)
* Reproduce

* Another failing test

* rename test

* rename test

* slot mapping

* pass all tests

* remove comments

* refactor

* broken test

* broken test

* detekt

* simplify

* simplify

* detekt baseline cleanup

* Add check

* requireForSer

* simplify

* Remove check

* use indices
2019-12-20 11:28:16 +00:00
bd436e640d Fix report generation against regression builds (#5818) 2019-12-20 10:11:26 +00:00
35c58f1b9b DOCS: Update UAT.md (#5602) 2019-12-19 18:28:11 +00:00
bc96bea24a ENT-4628: Harmonize CryptoService and BCCryptoService between OS and ENT (#5822) 2019-12-18 16:54:39 +00:00
8d5781db43 ENT-4628: Harmonize net.corda.nodeapi.internal.crypto between OS and ENT (#5820)
* ENT-4628: Harmonize net.corda.nodeapi.internal.crypto between OS and ENT

* ENT-4628: Fix detekt
2019-12-18 13:59:30 +00:00
af30e40397 CORDA-3485 Restore CollectSignaturesFlow.kt to allow multiple collections from Well Known sessions (#5800)
* modify CollectSignaturesFlow.kt to allow multiple collections from wellKnown party initiated sessions

* detekt fixes

* review comments

* move require lambdas back outside of the function definition of requires

* address review comments

* fix detekt

* fix api scanner
2019-12-17 16:26:09 +00:00
5458f11998 Jenkins file for nightly regression tests (#5786)
* Jenkins file for nightly regression tests

* Use k8s instead of gke cluster
2019-12-17 13:02:56 +00:00
545a463a5b NOTICK: Remove buildscript block from Node's build.gradle. (#5809) 2019-12-16 16:24:47 +00:00
91cdffd429 ENT-4382: Core module changes to support multi-RPC work (#5814)
* ENT-4382: Move `InvocationHandlerTemplate` into `core`

This is an internal helper which is general enough and does not have any Node specific code.

* ENT-4382: Make @CordaInternal applicable to classes

And apply it on `AttachmentTrustCalculator` which is `core/internal` interface anyway.
2019-12-16 16:03:48 +00:00
5d47466051 NOTICK: Refactor Gradle script for building Docker images. (#5813) 2019-12-16 14:34:51 +00:00
4a35b99283 CORDA-3479: Added timestamp to the node_transactions table. (#2660) (#5812)
* ENT-4237: Added timestamp to the node_transactions table.

* ENT-4237: Clock for timestamp now retrieved from ServiceHub. And now record verification time as well.

* ENT-4237: Fixed tests. Also enabled stream output in allParallelIntegrationTest.

* ENT-4237: Changed timestamp to a val.

* ENT-4237: Changed streamOutput to false for allParallelIntegrationTest

* ENT-4237: Unit tests added for new timestamp column. Also now passing a clock into DBTransactionStorage.

* ENT-4237: Added more unit tests to check timestamp

* ENT-4237: Fix test to actually change clock time when testing transaction time does not change.
2019-12-13 15:14:21 +00:00
ec8f0a6db1 Merge pull request #5802 from corda/ramzi/TM-141
Prevent on-demand tests re-triggering from branch indexing
2019-12-13 15:09:45 +00:00
c65497ee4d Merge pull request #5810 from corda/ramzi/TM-140
Mark integration test tasks with "big" node taint
2019-12-13 15:09:15 +00:00
575df97c52 Mark integration test tasks with "big" node taint 2019-12-13 10:08:51 +00:00
8f7346f84c NOTICK - Add C++ Serialiser Support for Maps and Arrays (#5775)
Adds support for understanding both Maps and Arrays

Irritatingly, whilst arrays are mostly serialized as lists, we cannot simply use a restricted List reader to deserialize them because there are subtle differences in the way we need to work out if its actually a restricted type or not. Rather than add a bunch of random logic into the factory method I've isolated it in the class hierarchy. So a little bit more code makes the implementations a lot neater. We also need to deal with the fact arras of unboxed primitives exist, which whilst Java really gets excited about, we don't need to care about. An int, is an int, is an int!.

Map support required we add a slightly better Value dumper, essentially the "key" component of the KV pair needs to be more flexible than a simple string when we're dumping out param:value pairs.

Testing
Added a lot more unit tests to both the ordered type notation code to ensure we build up the schema dependency struct in the correct order. Quite important as we rely on that in the composite factory to be strictly ordered to ensure we're not constructing a reader for a type we don't yet understand... and there were some small bugs in the version that predates this PR

Also added a lot higher level tests to ensure actual reading out of the blob works
2019-12-13 00:32:25 +00:00