* CORDA-3028: Add Node Diagnostics Info RPC Call - Backport a diff from Corda ENT
* CORDA-3028: Add Node Diagnostics Info RPC Call - Rename a value for consistency
* CORDA-3028: Add Node Diagnostics Info RPC Call - One more change
* Moved core.flows and core.contract tests
* move internal tests
* move node, schemas and serialization tests
* Move transactions tests
* Move test access helper into test package
* Move remaining tests, and create access helpers to test internals.
* Fix test dependencies.
* Delete some cruft that flagged up on the API checker
* Add readme explaing where tests need to go.
* fix typo
* add a shared memory port allocator to allow multiple processes to share a single allocation pool
* remove dangerous reset function on port allocator
* set forkCount = 2 in node integration test
* only allow one build of a cordapp at any given time for Driver tests
* make all portallocation requests use same starting point
* globally set forks to 6
* tweak forking parameters to allow parallel builds
* tweak unit test parallelism
* 2 workers for integrationTest
* some more tweaks for parallel builds
* some more tweaks for parallel builds
* seems that 49K is not the start of ephemeral ports on all kernels
* tweak parallel settings
* try fix RPC shutdown test in parallel env
* add some logging for RPC shutdown test
* added some logging around PortAllocation tests - try figure out where they are getting stuck
* added some logging around PortAllocation tests - try figure out where they are getting stuck
* fix api-scanner tests
* minimize api changes
* revert to complying with existing API
* add the AtomicInteger for api compatibility reasons
* make sizing script executable
* address review comments pt1
* address review comments pt2
* fix compile errors after review comments
* return to using home dir as temp dir seemed to interact badly with gradle
* CORDA-3021: Introduce `SignOnlyCryptoService` and use it whenever possible
Also modify `CryptoServiceFactory` to show how sign only implementation can be created.
* CORDA-3021: Undo some of my earlier changes
Which after discussion with @dimosr and @fowlerrr proven to be contradictory.
Dumps all the node's checkpoints as JSON into a single zip file in the node's directory. The output contains:
* All the fields for the top-level flow
* The current sub-flow call stack, along with the current progress tracker step for each sub-flow
* The event that suspended the flow, which if it's a send or sendAndReceive will show the payload that was sent
* Low level information on the active sessions with other peers
* Initial commit with a test that can detect a logger that isn't initialised with by lazy that gets called during startup.
* Test improvement. Test fix for api stability.
* Added explanatory comment for the test.
* Changes according to PR review. Added empty file check to make test more robust.
Party and AnonymousParty have been retrofitted to implement this interface and are currently the only supported types. A new FlowLogic.initiateFlow(Destination) method overload to easily support the addition of new destination types in future versions.
To prevent making `dumpCheckpoints` part of the public API a new
interface, `InternalCordaRPCOps` has been created and the function
has been moved there. `InternalCordaRPCOps` inherits from
`CordaRPCOps`.
`CordaRPCOpsImpl` now implements `InternalCordaRPCOps`.
`RunShellCommand` and `StringToMethodCallParser` required additional
changes due to issues handling inherited functions. This has only been
raised now due to `InternalCordaRPCOps` inheriting from `CordaRPCOps`.
Many classes have had references to `CordaRPCOps` changed to
`InternalCordaRPCOps`.
* ENT-3642: move the crypto service builder method to node-api
* ENT-3642: add arg for different crypto services
* ENT-3642: add arg for cryptoservice config
* ENT-3642: add null check for BCC requirement
* ENT-3642: remove double-bang operator
* ENT-3642: cryptoservice factory method moved to its own class, improve API
* ENT-3642: remove import
Store the flow dump zip in the logs directory.
Only allow one dump to be created at a time. This is protected by
using a `AtomicInteger` lock and checking for the zip's existence.
* Revert usage of Gradle 5 useJUnitPlatform() which causes significant test execution performance degradation.
* Remove completely top-level Java CordaCaplet unit tests causing consistent TC failure on Azure VMs:
java.security.NoSuchAlgorithmException: no such algorithm: NONEwithEdDSA for provider BC
* Added periodic message logged from a timer which is switched on and off by the node drain events.
* Timer also spawns on start() to ensure that if a node is shutdown in draining mode and started with it, it will continue logging messages.
* Reduce test execution times by explicitly configure quasar package exclusions (using new quasar util extension introduced in Corda gradle plugins 5.0.1-SNAPSHOT).
* Remove SNAPSHOT from Corda gradle plugins version identifier.
* Incorporating PR review feedback from CR.
* Minor consolidation clean-up.
* Minor consolidation clean-up.
* Minor consolidation clean-up.
Dumps all the node's checkpoints as JSON into a single zip file in the node's directory. The output contains:
* All the fields for the top-level flow
* The current sub-flow call stack, along with the current progress tracker step for each sub-flow
* The event that suspended the flow, which if it's a send or sendAndReceive will show the payload that was sent
* Low level information on the active sessions with other peers
* CORDA-2838 Set Artemis memory config.
* CORDA-2838 Cannot have page size larger than max size.
* CORDA-2838 Use real slow consumers. Need to see if the old config settings can work with a global limit to avoid this.
* What went wrong:
Caused by: org.gradle.api.GradleException: failed to read class file /Users/josecoll/IdeaProjects/corda-jdk11/node/build/classes/java/test/net/corda/node/services/events/FlowLogicRefFromJavaTest$JavaNoArgFlowLogic.class
* Upgrade gradle wrapper to Gradle 5.0
* Upgrade to use locally deployed version of Capsule plugin (using Gradle 5.0)
* Upgrade to use Corda Gradle Plugins 5.x (inclusive of Gradle 5.0 compatibility fixes)
* Fix compile-time problems resolving log4j packages.
* Update to use Artifactory deployed version of Capsule plugin (using Gradle 5.0)
* Upgrade to use Gradle 4.2.1 (excluding default memory setting change: command line client now starts with 64MB of heap instead of 1GB.)
* Added explicit compile-time dependency on 'de.jensd:fontawesomefx-commons:8.13' (compile-time error in tools:explorer without)
* Update "wrapper" gradleVersion to 5.4.1
* Update Capsule plugin version label to reference R3 forked build.
* Commit all gradle upgrade changes following: ./gradlew wrapper --gradle-version 5.4.1
* Increase maximum heap memory for Test tasks to 1GB, and upgrade build-scan plugin to 2.2.1.
* Increase Test tasks' maximum memory to 1.5GB - what could go wrong?
* Up maxHeapSize to 2g for gradle test runner (global setting).
* Added explicit compile-time dependency on 'de.jensd:fontawesomefx-commons:8.13' (compile-time error in tools:demobench without)
* Added forkEvery for node Unit tests.
* Up :node test task 'forkEvery' to 10.
* TC test execution tuning (:core, :node => forkEvery 10, default JVM heap size)
* TC test execution tuning (bump JVM heap size up to 1g for :node)
* TC test execution tuning (bump JVM heap size up to 1.5g for :node)
* TC test execution tuning (re-instate global JVM heap size of 1Gb)
* TC test execution tuning (re-instate JVM heap size of 2Gb for :node)
* Update Corda Gradle plugins to 5.0.0
* Updated plugin resolution order + renamed artifactory URL to use "software.r3.com"
* Reorder plugin resolution such that mavenlocal() is always queried first.
This allows a different signed version of the same CorDapp to be automatically trusted.
This reverts "[CORDA-2575] Allow users to whitelist attachments by public key config (#5035)"
When a `UnexpectedFlowEndException` or a `FlowException` is received the
peer that the exception was thrown from will be added to the stacktrace.
This is due to it being easier to see and a field that developers
are much less likely to override.
A nullable field `peer` has been added to `FlowException` and
`UnexpectedFlowEndException`. This is read later on (when peer info
is not available) to append the peer info to the stacktrace.
* Initial version of new(old) dialog that won't print a stacktrace for rpc exceptions.
* Decoupled CordaVersionProvider. Moved common files to common-logging to lower dependencies on the node explorer.
* Removed unused import and duplicate documentation comment.
* Moved error code rewrite policy in the new common/logging module according to PR review.
* Removed extra line.
* Updated log4j configurations with new package name where logging policies will be contained.
* Included common-logging module with cliutils.
* setting non existant base directory now yields suitable error message
* corrections to missing base directory message and comments added for clarity
* added check for valid base directory before node loggin and bootstrapping
* removed uneeded import
* quantum computer joke back by popular demand and not back at the same time
A test was failing due to `assumeFalse` (to stop a test running on windows)
being inside of a `assertThatExceptionOfType` block. The exception it
threw was consumed by the block instead of junit.
If a single whitelisted constraint is being used by input states and the version of the cordapp changes + is signed, then the constraint will transition to a signature constraint.
When a `UnexpectedFlowEndException` or a `FlowException` is received the
peer that the exception was thrown from will be added to the stacktrace.
This is due to it being easier to see and a field that developers
are much less likely to override.
A nullable field `peer` has been added to `FlowException` and
`UnexpectedFlowEndException`. This is read later on (when peer info
is not available) to append the peer info to the stacktrace.
`ExceptionSerialisingRpcOpsProxy` was removing information about the
original exception in dev mode. This code has been removed. Although
there is no check in `ExceptionSerialisingRpcOpsProxy` for dev mode
(and also due to it being about serialising),
`ExceptionMaskingRpcOpsProxy` will handle the removal of exception data
in non dev mode (production mode).
* [CORDA-2522] Improve error reporting around failed flows (#5000)
* Improve error reporting around failed flows
* Fix an index to start from 1 when printed
* Address first set of review comments
(cherry picked from commit 24699cd7f4)
* [CORDA-2522] Follow up changes to error reporting around failed flows (#5006)
* Follow up changes to error reporting around failed flows
* Have FinalityDoctor report stack trace
* Revert changes to the DumpHistoryOnErrorInterceptor
(cherry picked from commit 2da597a5b7)
* Follow up changes to error reporting around failed flows
* Have FinalityDoctor report stack trace
* Revert changes to the DumpHistoryOnErrorInterceptor
* Initial version of new(old) dialog that won't print a stacktrace for rpc exceptions.
* Decoupled CordaVersionProvider. Moved common files to common-logging to lower dependencies on the node explorer.
* Removed unused import and duplicate documentation comment.
* Moved error code rewrite policy in the new common/logging module according to PR review.
* Removed extra line.
* Updated log4j configurations with new package name where logging policies will be contained.
* Included common-logging module with cliutils.
* setting non existant base directory now yields suitable error message
* corrections to missing base directory message and comments added for clarity
* added check for valid base directory before node loggin and bootstrapping
* removed uneeded import
* quantum computer joke back by popular demand and not back at the same time
* CORDA-2813 Add DbException handler to terminate process when the DB goes away or gives up.
Also add a helper to terminate the node and use it instead of calling Runtime.halt() in random places.
* CORDA-2813 Address threading/blocking concerns raised during review
* Tests for custom registry restrictions
* ENT-3121 restrict custom serialisation
* Remove redundant code
* Only count declared annotations
* Check annotation on superclasses, remove annotation from ByteArray
* Forbid custom serialization of primitive types
* Remove @CordaSerializable from another class that is always handled by custom serialisation
* Add log warnings to aid diagnosis of custom serialization issues
* Remove another annotation
* Remove another annotation
* Remove another annotation
* Remove another annotation
* Fixup api-current
* Fixup api-current
* KDocs on exceptions
Addressed review comments - removed nested if and added parameter name for the usage of requiresCertificates.
Removed unnecessary import and white space.
Normally, these requests are left unacknowledged in the MQ broker to allow recovery via installing the missing CorDapp. If this is not applicable then the request be dropped (and the other side informed of the error).
While the node is starting up, we now check for the presence of the certificates directory. This allows us to print out an easily understandable error message if the directory is not present. An exception is made for devMode, as devMode will result in the directory being created in any case.
* ENT-3057: Log hibernate warns and errors in different log
If a hibernate error occurs (deadlock, for example) that would cause a flow to be sent to the hospital, hibernate logs the warnings and errors before we do. This results in duplication in the logs, and pollutes the log. To solve this, we create a new log appender named diagnostic-{node-name}.log and log any org.hibernate messages of warn and above to that file. This way, messages are not lost, which means that the information can be retrieved if need be.
* Corrected indentation of comment (changed tab to space)
* Updated node-administration document to mention diagnostic logging change
* Fixed integration test. It was breaking because it was fetching the first log file in the folder, assuming there would be only one. This assumption is now invalid because the diagnostic log file that was introduced. Two tests were found that used similar logic to find a log file to examine, hence both were corrected to look for log files beginning with "node"
* Updated documentation as per review comments.
Add a cache for notary identities in the PersistentIdentityService. This
solves a reported problem where notary identity lookup fails if its network
map entry is missing, which results in an exception when trying to insert
a state into the vault after recording a transaction.
Test if we have access to the logging path (baseDirectory/logs) before attempting to write to them. This allows us to shut down gracefully with an easily understandable error message.
Without doing this, the log4j2 will attempt to access the logPath when it first uses the logger in the call() function.
* CORDA-2629: Provide a better error msg when notary type misconfig
If a notary service is misconfigured with the type (either validating or not validating) stored in the node's configuration not matching that advertised in the network map cache, the notary will throw an exception and fail on startup.
Previously, this misconfiguration would result in an exception being thrown when attempting to notarise a transaction. This change results in the exception being thrown at node startup and the node operator being aware of the misconfiguration earlier.
* Corrected exception message when the notary does not have a party
* Changed exception message to include configured and advertised values.
Fixed unit test that was failing because of the new check.
* Plumb through the crlCheckSoftFail configuration option to bridge manager
* Add crlCheckSoftFail test to bridge manager and fix equivalent proton wrapper test
* Update documentation and set the node configuration default to true
* Revert default change and clarify consequences of setting option to false
* Remove NodeConfiguration default to leave only AMQPConfiguration default
There's a bug with the ServiceLoader which leaks a file handle to the app jar on shutdown. This causes an issue if a mock node is restarted in Windows. To avoid the problem completely we no longer overwrite any existing jars, as the jar to be copied will be same anyway.
(cherry picked from commit 0038a86481)
Only log a line about looking for notary implementations if at least notary implementation is found.
It might be unnecessary and confusing to log when no implementations are found.
Replaced TrustedAuthorityNotaryService with SinglePartyNotaryService in comment
* CORDA-2665: Updated OwnableState relevancy check is now gated on target version 4
https://github.com/corda/corda/pull/3789 changed the relevancy check of OwnableState to include the participants list in addition to the owner. This however breaks existing apps which assume (in their vault query) an OwnableState is recorded to the vault if-and-only-if the owner matches.
* CORDA-2665 Don't switch to new behaviour on target version upgrade.
When an RPC client disconnects from the RPC server, the log entry created now has its log level set to info rather than warning. It is perfectly normal for an RPC client to disconnect - only the RPC client knows if it was intentional.
* CORDA-2669 - pendingFlowsCount not in public API
Reintroduce `pendingFlowsCount` to public API (as deprecated). Advise
to use the `gracefulShutdown` command in the shell instead.
* CORDA-2669 - Add pendingFlowsCount to api-current.txt
* ENT-3053 Database connection pools leaking memory on every checkpoint. Flip in the thread local from the thread into the fiber.
* Back port to OS (needs some gradle changes) and added TODO, ability for it to avoid erroring if not using Hikari.
* Review feedback to remove warning.
* CORDA-2634 - Fix default currentTargetVersion
* CORDA-2634 - Include abstract flow classes in CorDapp class scanning
* CORDA-2634 - Run test for target platform version 4
To allow rolling upgrades where members of a BN may not all upgrade at the same time, the FinalityHandler is now always enabled, and not disabled if no old apps are installed.
Previously any attempt to use the FinalityHandler in such a scenerio would send the flow to the hospital, where manual intervention would be required to recover the transaction.
* Use the attachments classloader to deserialize contract states in migrations
* Added some comments to explain serialisation behaviour and how tests work.
* Add debug log to indicate when attachment classloading has failed.
* Use a servicesForResolution to load states for compatibility with notary changes and contract upgrades
* Add test case to cover notary change transactions
* Address review comments
* Change logging message in MigrationServicesForResolution
* Read the network-parameters file if there is nothing in the database
* Update documentation and provide a warning if there are many states.
* Handle case where file containing network parameters is missing
* Improve logging if attachments are not available and the file path doesn't exist
* Get logs the right way around
* Fix build error in tests
* Improve error when transaction deserialisation fails and move migrations for finance to contracts CorDapp
* Revert move of migrations and errors thrown from CordaRPCOps
* Ensure VaultQueryException is thrown from vault queries and remove unused import
* Improve error reporting from VaultQueryException
* Fix API break
* Fix vault query test failure due to exception change
CORDA-2595 - Fix test and api.
CORDA-2595 add test
CORDA-2595 fix tests
CORDA-2595 fix test and address code review comments
CORDA-2595 address code review comments
* First pass at fixing 2563.
* In memory KMS now maps keys to IDs.
* CreateDatabaseAndMockServices now creates a persistent key management service and a can take a persistent identity service, so now the external id mapping works for mock services.
* * Created a helper for mock services which allows the creation of a mock services with persistent identity management service key management service and vault.
* MockNode now uses persistent key management service - not sure why it didn't do before?
* * MockNode now uses BasicHSMKeyManagementService
* Updated api-current file
* Little fix required after rebase to master.
* Fixed broken test.
* Added informative error messages to UnsupportedOperationExceptions thrown by E2ETestKeyManagementService.
* Removed redundant private constructor for mock services from api-current.txt.
* Addressed Rick's comments.
* Create a new custom migration for populating the state_party table and the relevancy_status column within the vault_states table when migrating from V3 to V4.
* Add a corresponding unit test suite.
* Small refactorings to expose data required by the migration: the isRelevant function in the NodeVaultService, and the node's legal name from the configuration.
Previous implementation was in LedgerTransaction and focused only on contract classes,
but every package matters.
Also fixes some exception types and does misc refactorings.
Take out a useless parameter from a method that was added to the public
API, document it. Add some comments explaining more about why we are
looking up attachment versions in WireTransaction.toLedgerTransaction.
* CORDA-2547: WIP Migrate old notary schema correctly.
Modify liquibase scripts to rename the old committed state table instead of creating a new one.
* Rename liquibase change sets
* Change 3.3 baseline before Liquibase.
* Aligin with ENT, orginal node-notary.changelog-v1.xml moved to node-notary.changelog-v2.xml and it's compatible with ENT (which already has changes from v2).
* Aligin with ENT, orginal node-notary.changelog-v1.xml moved to node-notary.changelog-v2.xml and it's compatible with ENT (which already has changes from v2) - adding v2 to master file.
* Ensure setting up Liquibase on pre-existing pre-Liquibase database
* Add FetchParametersFlow
* No downgrade parameters in ResolveTransactionsFlow
Make sure that parameters in the transaction
graph are ordered (this is to prevent the downgrade attack, when the
malicious notary and participants sign transaction that shouldn't be
notarised otherwise). We ensure that by checking that epochs of network
parameters in the transaction chain are ordered.
* Addressed some minor items from RP review feedback.
* Refactoring following rebase from master.
* Address RP PR review comments (round 2)
* Addressed a couple of minor PR review points.
* Renaming of unit tests and cleanup.
* Changes discusses with RP to ensure Network Param checking is applied at txn verify time + resolve order checking gated on existence of tagged NPs in txn and associated minimum platform version.
* Do not fail on missing ServiceHub impl + return nothing if txn not NP tagged.
* Unify HistoricNetworkParametersStorage and
NetworkParametersStorageInternal
* SignedDataWithCert implements NamedByHash
* Cleanup
* Move parameters ordering check to signed transaction resolution
* Fixes after merge, address comments
* Address Andrius comments
This includes all of the cash selection logic, JSON support using jackson and a bunch of utilities which are not relevant to contract verification. The exception to this are Interpolator which has been moved to the IRS demo, and PhysicalLocationStructures.kt which is now in explorer.
* Allow for duplicate (contract class, version) signed attachments in devMode.
* Code clean-up.
* Fix compilation error in test code.
* Additional gating/warning and added Unit test for development mode behaviour.
* Remove locks around database access in AppendOnlyPersistentMap and introduce
a unit test that checks that known deadlock scenarios of the old version
are avoided.
* Fix Deadlock unit test
* Add some extra latching to try and make timing less fragile. Can never be perfect though.
* Review feedback, and some thread safety fixes.
* Start pooling classpath scanning
Quickly patch synchronisation of attachment class loader cache. Needs a revisit but more complicated due to DJVM.
Annotate away for DJVM
Take ClassGraph utils into their own file so we can exclude for DJVM
Clean up a little
* Daemonize the threads
* Seems to be some concurrency problems with use of ClassGraph. Using a mutex for now to work around.
Merged three tests into one to avoid repeating the same node startups. The assertions the original tests made can be done in one test without any contamination.
* Removed `restart node successfully with suspended flow` as it duplicates `TraderDemoTest#Test restart node during flow works properly`
* Removed the need for a notary
This is only testing the internal cordappForClasses method which is used for internal testing. This method is covered by other unit and integration tests.
CORDA-2497 fix
* Added test to show the fix working.
* Now backwards compatible.
* Refactored out some duped code.
* Added better explanations for what's going on.
* Fixed test which was failing due to the serializationEnvRule problem.
* Addressed Tudor's review comments.
The column 'uploader' was marked as not updatable entity so any update wasn't propagated to the database when the attachment was uploaded again and the uploader field was changed to a trusted one. Also 'saveOrUpdate' removed, the entity is already managed, saveOrUpdate is for detached/new entity.
