Allow node operators to blacklist signing keys (using blacklistedAttachmentSigningKeys config option). These blacklisted keys prevent attachments that are received over the network from being trusted. The docs have been updated to detail how to generate the key hashes that the config requires.
A new shell command attachments trustRoots has been added to see what attachments exist on the node along with information about their trust and where it comes from.
run dumpCheckpoints has been replaced by checkpoints dump as InternalCordaRPCOps needed to change to prevent a function that is meant to be internal from being visible on the shell.
* Split integration tests
* add simple example of printing all methods annotated with @Test
* add docker plugin to root project
remove docker plugin from child projects
add Dockerfile for image to use when testing
add task to build testing image to root project
* add comment describing proposed testing workflow
* simple attempt at running tests in docker container
* add my first k8s interaction script
* add fabric8 as dependnency to buildSrc
* before adding classpath
* collect reports from containers and run through testReports
* re-enable kubes backed testing
* for each project
1. add a list tests task
2. use this list tests task to modify the included tests
3. add a parallel version of the test task
* tweak logic for downloading test report XML files
* use output of parallel testing tasks in report tasks to determine build resultCode
* prepare for jenkins test
* prepare for jenkins test
* make docker reg password system property
* add logging to print out docker reg creds
* enable docker build
* fix gradle build file
* gather xml files into root project
* change log level for gradle modification
* stop printing gradle docker push passwd
* tidy up report generation
* fix compilation errors
* split signature constraints test into two
* change Sig constraint tests type hierarchy
* tidy up build.gradle
* try method based test includes
* add unit test for test listing
* fix bug with test slicing
* stop filtering ignored tests to make the numbers match existing runs
* change log level to ensure print out
* move all plugin logic to buildSrc files
* tidy up test modification
add comments to explain what DistributedTesting plugin does
* move new plugins into properly named packages
* tidy up runConfigs
* fix compile errors due to merge with slow-integration-test work
* add system parameter to enable / disable build modification
* add -Dkubenetise to build command
* address review comments
* type safe declaration of parameters in KubesTest
* migrate PersistentIdentityService to use key.toShortString()
update definition of PublicKeyToExternalId mapping to allow fast lookup by externalId/publicKey
* fix misspelled table name
* add test of migration script
* add design document for proposal to move IdentityService to using the correct PK.toStringShort() method for hashing a publickey
* add enterprise testing considerations to design
* address review comments
* fix compilation errors
* modify PublicKeyToOwningIdentityCache to use toStringShort() as it's lookup key
* address syzmon's code review comments
* CORDA-3139: Cater for port already bound scenario during port allocation
Also moved `SharedMemoryIncremental` into a separate file as it getting bigger
and improved readability of logic and added some logging.
* CORDA-3139: Fix the unit test
* CORDA-3139: Improve logging when failing
* CORDA-3139: Improve stability of the test
* [ENT-3801] Store transactions in the database during transaction resolution (#2305)
* ENT-3801: Store downloaded txns as part of the backchain resolution into the db rather than the checkpoint
It's very inefficient to store the downloaded backchain in the checkpoint as more of it downloaded. Instead, if a threshold is reached (which currently defaults at 0) then the backchain is stored in the transactions table as unverified. A new is_verified column has been added to track this. Initially testing on the OS codebase has been very promising but unfortunately this current code is not quite ready. I had to quickly port it to ENT as this is meant to be an ENT-only optimisation.
To that effect, there is a TransactionResolver abstraction with two implementations: an in-memory one which has the old behaviour, and which will be the behaviour for OS, and a db one.
DBTransactionStorage hasn't been fully updated and I had to comment out the optimistic path for now.
Most of these changes will need to be ported to OS to keep the merge conflicts in check, but obviously not DbTransactionsResolver and the "is_verified" changes in DBTransactionStorage. DBTransactionStorage does have other refactoring which will make sense to port though.
* [ENT-3801] Start work on allowing modifications in AppendOnlyPersistentMap
* [ENT-3801] Add transaction resolver tests
* [ENT-3801] Adjust suspendable annotations
* [ENT-3801] Fix the ResolveTransactionFlow tests
* [ENT-3801] Update ResolveTransactionsFlow tests
* [ENT-3801] Add a liquibase migration script for isVerified
* [ENT-3801] Ensure the migration runs in the correct place
* [ENT-3801] Handle resolution of already present transactions
* [ENT-3801] Fix compile error in performance test app
* [ENT-3801] Logging and comment updates, plus a test case
* [ENT-3801] Add a notary change resolution test
* [ENT-3801] Add a contract upgrade transaction test
* [ENT-3801] Change new column to be a character based status
* [ENT-3801] Migration script type change
* [ENT-3801] Address first round of review comments
* [ENT-3801] Update variable names in AppendOnlyPersistentMap
* [ENT-3801] Another variable name clarification
* [ENT-3801] Fix missing name changes
* [ENT-3801] Make the signature list immutable when constructing cache value
* [ENT-3801] Add a locking strategy for unverified transactions
* [ENT-3801] Address tidying up review comments
* [ENT-3801] First attempt at ensuring locks are released after commit
* [ENT-3801] Remove references to old cache name
* [ENT-3801] Update locking logic
* [ENT-3801] Fix potential deadlock with read/write transaction locks
* [ENT-3801] Remove read locks, and ensure minimal extra suspends
* [ENT-3801] Fix build issues in tests
* [ENT-3801] Use the correct clock when calculating sleep durations
* [ENT-3801] Add a pessimism flag for writing verified transactions
* [ENT-3801] Change logging statement to debug
(cherry picked from commit 8ab6a55e17)
* [NOTICK] Fix up imports for some changed files
* [NOTICK] Fix transaction resolution tests
* [NOTICK] Reinstate the DBTransactionsResolver
* [NOTICK] Add the topological sort back to recordTransactions
* [NOTICK] Adjust test case to remove dependency on query ordering
* [NOTICK] Make test code match that in ENT
* Fix test execution flakiness on fast hardware: set node info polling interval to 1 second in DriverDSL Node Startup (was only being set to 1 sec in Node Registration). Follow-up to https://github.com/corda/corda/pull/5240
* Diagnostics: add additional logging to NodeInfoFilersCopier.
* Diagnostics: add additional logging to NodeInfoWatcher
* Downgrade logging level severity to debug.
* Make atomic counter unique across instances.
* Moved core.flows and core.contract tests
* move internal tests
* move node, schemas and serialization tests
* Move transactions tests
* Move test access helper into test package
* Move remaining tests, and create access helpers to test internals.
* Fix test dependencies.
* Delete some cruft that flagged up on the API checker
* Add readme explaing where tests need to go.
* fix typo
* add a shared memory port allocator to allow multiple processes to share a single allocation pool
* remove dangerous reset function on port allocator
* set forkCount = 2 in node integration test
* only allow one build of a cordapp at any given time for Driver tests
* make all portallocation requests use same starting point
* globally set forks to 6
* tweak forking parameters to allow parallel builds
* tweak unit test parallelism
* 2 workers for integrationTest
* some more tweaks for parallel builds
* some more tweaks for parallel builds
* seems that 49K is not the start of ephemeral ports on all kernels
* tweak parallel settings
* try fix RPC shutdown test in parallel env
* add some logging for RPC shutdown test
* added some logging around PortAllocation tests - try figure out where they are getting stuck
* added some logging around PortAllocation tests - try figure out where they are getting stuck
* fix api-scanner tests
* minimize api changes
* revert to complying with existing API
* add the AtomicInteger for api compatibility reasons
* make sizing script executable
* address review comments pt1
* address review comments pt2
* fix compile errors after review comments
* return to using home dir as temp dir seemed to interact badly with gradle
* Initial commit with a test that can detect a logger that isn't initialised with by lazy that gets called during startup.
* Test improvement. Test fix for api stability.
* Added explanatory comment for the test.
* Changes according to PR review. Added empty file check to make test more robust.
* CORDA-2838 Set Artemis memory config.
* CORDA-2838 Cannot have page size larger than max size.
* CORDA-2838 Use real slow consumers. Need to see if the old config settings can work with a global limit to avoid this.
This allows a different signed version of the same CorDapp to be automatically trusted.
This reverts "[CORDA-2575] Allow users to whitelist attachments by public key config (#5035)"
* ENT-3057: Log hibernate warns and errors in different log
If a hibernate error occurs (deadlock, for example) that would cause a flow to be sent to the hospital, hibernate logs the warnings and errors before we do. This results in duplication in the logs, and pollutes the log. To solve this, we create a new log appender named diagnostic-{node-name}.log and log any org.hibernate messages of warn and above to that file. This way, messages are not lost, which means that the information can be retrieved if need be.
* Corrected indentation of comment (changed tab to space)
* Updated node-administration document to mention diagnostic logging change
* Fixed integration test. It was breaking because it was fetching the first log file in the folder, assuming there would be only one. This assumption is now invalid because the diagnostic log file that was introduced. Two tests were found that used similar logic to find a log file to examine, hence both were corrected to look for log files beginning with "node"
* Updated documentation as per review comments.
There's a bug with the ServiceLoader which leaks a file handle to the app jar on shutdown. This causes an issue if a mock node is restarted in Windows. To avoid the problem completely we no longer overwrite any existing jars, as the jar to be copied will be same anyway.
(cherry picked from commit 0038a86481)
TestCordapp – disable daemon on internal gradle process
The TestCordappImpl runs gradle to build cordapp jars required for tests.
The started gradle process reuses a Gradle daemon that's potentially already
used for running the tests causing the JVM to die with SIGBUS.
* First pass at fixing 2563.
* In memory KMS now maps keys to IDs.
* CreateDatabaseAndMockServices now creates a persistent key management service and a can take a persistent identity service, so now the external id mapping works for mock services.
* * Created a helper for mock services which allows the creation of a mock services with persistent identity management service key management service and vault.
* MockNode now uses persistent key management service - not sure why it didn't do before?
* * MockNode now uses BasicHSMKeyManagementService
* Updated api-current file
* Little fix required after rebase to master.
* Fixed broken test.
* Added informative error messages to UnsupportedOperationExceptions thrown by E2ETestKeyManagementService.
* Removed redundant private constructor for mock services from api-current.txt.
* Addressed Rick's comments.
Previous implementation was in LedgerTransaction and focused only on contract classes,
but every package matters.
Also fixes some exception types and does misc refactorings.
