ENT-6947: Implement interning for SecureHash, CordaX500Name, PublicKey, AsbtractParty and SignatureAttachmentConstraint, including automatic detection of internable types off companion objects in AMQP & Kyro deserialization. In some cases, add new factory methods to companion objects, and make main code base use them.
Performance tested in performance cluster with no negative impact visible (so default concurrency setting seems okay).
Testing suggests 5-6x memory saving for tokens in TokensSDK in memory selector. Should see approx. 1 million tokens per GB or better (1.5 million for the tokens we tested with).
* TM-51 Prep for reading and writing test results to artifactory.
* TM-51 Tests from target branch if no tests for current branch
* TM-51 Placeholder for test averaging over runs.
* TM-51 Replace slashes in branch names used as tags.
* TM-51 More placeholder work for the mean duration work.
* TM-51 Write out average tests results as as csv.
The csv file should grow and be updated on each run. This includes whether or not we are running unit tests, integration tests and so on.
* TM-51 Comment out old junit test archiving, add more comments.
* TM-51 Zip task needs to depend on a csv creation task.
If there isn't a csv file present, then the zip task doesn't run due to 'NO-SOURCE'
* TM-51 Zip task should ignore empty dirs
* TM-51 Fix up loading of test results.
We were looking for the wrong artifact name.
Add a bit more logging.
* TM-51 Fix up possible problem with allocating by class distribution.
If we encounter a class we haven't seen before, there won't be any tests.
This means we should give it some weight. '1' is far too small.
* TM-51 Test that we are definitely increment the run count.
Tracking down whether the zipped csv file should have incremented.
* TM-51 Better default value for missing test/class names.
Begin by using mean unit test duration, but we have the option to bump
that to the mean class unit tests duration.
* TM-51 More debug information around csv writing.
We should be incrementing the tests.
* TM-51 Reload the csv before updating it.
* TM-51 Reduce verbosity of logging.
* TM-51 Reinstate unit tests. Remove logging verbosity.
* TM-51 Load tests from artifactory in memory and avoid interim file.
* TM-51 Better handling of zero duration tests.
Ensure we return zero times from junit artifacts which may either be zero or have no recorded time. Before writing the tests duration csv file, store those with a known time, and then store those with zero using the average time.
* TM-51 Log whether we have recorded a test.
Tracking down the curious case where we seem to not be rerunning the
same set of tests on the second run.
* TM-51 Capture junit files as well.
Trying to track down whether some tests are intermittently run.
* TM-51 Change task dependencies to ensure ziptask is triggered.
* TM-51 Remove test assertion, and trigger build
* TM-51 Add corda/enterprise to artifactory tag name.
Moved properties to own file.
* TM-51 Remove unnecessary mean class-based duration.
* TM-51 Add more BucketingAllocator tests.
We need these to nail down its behaviour some more.
* TM-51 Further log information.
We don't seem to be finding the tests in the 'production' runs which is odd.
* TM-51 corda type double set?
* TM-51 do not set the project type in the properties.
SRP and all that.
* TM-51 better plan reporting
* TM-51 duration may be zero
Another runtime problem that doesn't show in tests.
* TM-51 better plan reporting
* fix missing space after image id
* fix merge issue in DistributedTesting
* TM-51 remove unused code when GET/PUT-ting to Artifactory.
* TM-51 put tasks in gradle group and tidy up zip task creation
* TM-51 Fix the junit XML path.
* TM-51 Fix the task graph
* TM-51 Less logging
* Simplify
* Mount shared dir to worker
* format
* podnames with separators
* refactor parameters
* Use PVC for storage
* pvc in namespace
* KubesTest simplify
* no tolowercase
* no private
* lowercase
* RetryStrategy
* minor changes
* wait forever
* undo .idea
* elvis
* add comment
* regcred
* use correct ConfigBuilder
* delete java, will migrate later
* Revert "delete java, will migrate later"
This reverts commit e3bab1f3
* Merging changes in groovy to new java file
* format
* rename variable
* fix log
* private
* remove bak
* move java files
* Revert "move java files"
This reverts commit 89aa4c35
* NOTICK: Corda 4.3-RC01
Created first release candidate of Corda 4.3 - RC01.
* CORDA-3141: Add GracefulReconnect callbacks which allow logic to be performed when RPC disconnects unexpectedly (#5430)
Also removed potential for growing stack trace on reconnects.
* CORDA-2050 Upgrade Corda to Java 11 (compatibility mode) (#5356)
Upgrade Corda to run with Java 11 (compatibility mode) - see https://github.com/corda/corda/pull/5356
* ENT-4198 Adding legal text
Signed-off-by: Ed Prosser <edward.prosser@r3.com>
* TM-29 new baseline for 4.3 since new debt has been added with the last few commits (#5487)
* TM-23 compileAll task to compile all code (#5490)
* Add simple compileAll task to be used by warning check
* lazy configure compileAll
* TM-32 Merge OS 4.3 into 4.4
* TM-32 fixed detekt issue
* Downgrade Dokka back to 0.9.17 due to failing docs_builder.
* add ability to group test types together (#5459)
* add ability to group test types together
* add ability to specify podCount for use in parallel testing
* remove compiler xml
* add Jenkinsfile to enable scanning
* trigger build
* add ability to specify what docker tag to use from outside of the build
* fix docker work dir
* fix pipeline syntax issues
* use environment rather than `def`
* move agent restrictor outside of stages block
* use steps block
* more pipeline syntax fixes
* even more pipeline syntax fixes
* even more pipeline syntax fixes
* add kubenetize as property to image build
* move clear of docker image to end of build rather than start to prevent colocated builds
* escape dollar on docker image remove command
* attempt to kill all existing jobs
* fix compile issue due to killall_jobs
* fix compile issue due to killall_jobs pt2
* fix spelling
* make all variables environment variables
* add logic to delete images locally after pushing
* wrap testing phase with try / finally so that junit reports are always evaluated
* change the behaviour around post build actions
* break implicit link between testing phase and image building phase, allowing testing to occur without a rebuild and push of image
* prepend registry name to provided tag
* allow tasks to specify whether they wish to stream output from containers
* add timestamps directive to Jenkinsfile to have timing info on output
* make KubesTest resilient against transient pod failures in k8s
* increase CPU request
* add logic to allow specifying container resource requests
* attempt to run unit and integration tests in parallel
* change unit tests to use 3 cores to allow co-location on 8c machines
* join grouped tests together to give pod meaningful name
* add step to renew token with GKE
* change renew step to use pods instead of nodes
* fix bug where memory request is not correctly passed to pod
* disable unit tests for now
* [CORDA-2368] Added exception handling for missing files that displays appropriate messages rather than defaulting to file names. (#5472)
* NOTIK Minor adjustments to Detekt rules to reflect current working practises (#5498)
* Minor adjustments to rules to reflect current working practises (including IntelliJ code style alignment)
* Adjust another rule in line with existing code style.
* rebaseline with changed detekt ruleset
* rebaseline with NodeStartup changes
* CORDA-3200 Add postgres specific migration to change from @Lob to @Type(type = corda-blob)
* CORDA-3200 Remove unwanted changes
* CORDA-3200 Change column length to 64000 for long cert chains
Currently, it's more of a blob inspector than a full-blown
implementation, but this code does have the ability to understand a Corda
serialized blob and build a set of deserialisers to handle a blob. It
does this in the same way the native java implementation works by
interrogating the types (as per the definition in the envelope) and
building recursive type readers that are able to pull the relevant
information out of the byte stream.
Lots of future work to undertake, but as a starting point its a little
bit useful with the scope it could be made much more useful with some
work.
* Initial commit based on experimental kryo hook agent.
* WIP
* Added documentation.
* Additional improvements and documentation following more testing.
* Added field level instrumentation + basic type handlers for String, byteArray, charArray, primitive types.
* Working version (without array type handling)
* Missing build.gradle file.
* Handle display of Arrays and String.
Pruning output to avoid repetition (by loop depth, object count).
* Added configurable StackDepth (for display purposes) and filter out ProgressTracker stacks.
* Further array handling (Object arrays, 2D, 3D), improved display and general code cleanup.
* Various fixes and improvements following demo to RP.
* Clean-up
* Further clean-up
* Set checkpoint id before deserialization.
* Update documentation
* Final clean-up.
* Minor documentation fixes.
* Updates following PR review feedback.
* Add changelog entry.
When set to true the RPC client will:
* automatically reconnect when the connection is broken
* simple RPC calls will block until connection is established
* Observables returned from RPC will automatically resubscribe on reconnect so the client continues to receive events. This doesn't guarantee that events will not be lost during the reconnect.
* Moved core.flows and core.contract tests
* move internal tests
* move node, schemas and serialization tests
* Move transactions tests
* Move test access helper into test package
* Move remaining tests, and create access helpers to test internals.
* Fix test dependencies.
* Delete some cruft that flagged up on the API checker
* Add readme explaing where tests need to go.
* fix typo
To allow rolling upgrades where members of a BN may not all upgrade at the same time, the FinalityHandler is now always enabled, and not disabled if no old apps are installed.
Previously any attempt to use the FinalityHandler in such a scenerio would send the flow to the hospital, where manual intervention would be required to recover the transaction.
* Removed `restart node successfully with suspended flow` as it duplicates `TraderDemoTest#Test restart node during flow works properly`
* Removed the need for a notary
* Fixed incorrect attachment loading integration test
`AttachmentLoadingTests.test that attachments retrieved over the network are not used for code` was a false-positive - it was incorrect on multiple levels. Fixing it required updating the finance:isolated CorDapp, at which point it was given the new MANIFEST metadata for V4, and moved out of the net.corda.finance namespace to avoid package sealing issues.
The new test exposed a bug in the LedgerTransaction verification logic. This was cleaned up as it was too easy to verify on the wrong instance.
TestCordapp has now two implementations to clearly separate the two use cases it has in the Corda repo:
* TestCordappImpl which implements the revised public API of TestCordapp; namely that a TestCordapp instance references a real CorDapp jar on the classpath. This is either an external dependency jar in which case it’s taken as is and given to the node, or it’s a local gradle project in which case it’s compiled using the gradle “jar” task to generate the CorDapp jar. This approach means the jar has all the original CorDapp versioning information, which is important that it’s correct when testing. To this end, TestCordapp only needs to expose the ability to specify the app’s config. All the remaining properties have moved to CustomCordapp.
* CustomCordapp for creating arbitrary custom CorDapps, including specifying the jar’s MANIFEST values. This is internal API and only used for testing the platform. Technically this shouldn’t implement TestCordapp but does so to reduce the complexity of the driver and mock network.
* Split Workflow and contracts of Finance App into separate Cordapps, part 1 - content which is different between OS and ENT is still in contract Cordapp.
* Move CashSelection implementations to workflow module.
* Move CashSelection implmentations to workflow module.
* Move finance module to finance-flows, top level finance module is empty.
* Move finance module to finance-flows, top level finance module is empty.
* Updated build comment.
* Revert publication of combined (contracts and flows) corda-finance.jar (to maintain backwards compatibility with 3rd party cordapps dependent on finance)
* Added backwards compatibility clarification comment.
* Re-instate new cordapp metadata.
* Global rename of `finance-flows` to `finance-workflows` to follow adopted naming conventions.
* Addressed final review comments.
* Rename application to "Corda Finance Demo"
* Generation of original corda-finance jar from new sub-modules.
* Fixed and tested demobench with new split finance contract and workflow jars.
* Renamed finance sub-modules to contracts and workflows.
* Remove Michele!!!
* Minor fix to filtering logic.
* Align CorDapp configuration filename with workflows jar.
* Fix breaks caused by finance module naming changes.
* Final alignment between OS/ENT of finance contract code.
* CORDA-2113 - Include PNM ID in CSR
If Compatibility Zone operator is using private networks and the node
should be joining one, optionally the ID (a UUID) of that network can be
included as part of the node's CSR to to the Doorman.
* fix broken test
* Move Raft and BFT notaries into separate modules
* Move schemas
* Fix tests & demos
* Modified logic for creating notary services:
Added a new field 'className' to the notary configuration. The node now
loads the specified implementation via reflection. The default className
value points to the simple notary implementation for backwards compatibility.
Relevant schemas are loaded in a similar fashion.
For backwards compatibility purposes the default SimpleNotaryService will
remain built-in to node, but its cordapp will be generated on startup – so
the loading of notary services is streamlined.
* Move test namedcache factory to test utils
* Add named caches and apply to NonInvalidingUnboundCache and all usages.
* Add named caches and apply to NonInvalidingCache and all usages.
* Add named caches and apply to NonInvalidingWeightBasedCache and all usages.
* Move NamedCache to core/internal
* Remove type `NamedCache` and `NamedLoadingCache`
* Suppressed 'name not used' warning, added comment, and fixed generic parameters on the buildNamed functions.
* Use `buildNamed` in all caffeine instances in production code. Not using it for caches that are created in test code.
* Add checks for the cache name
* Formatting
* Minor code review revisions
* Add shell extensions to CLI utils class and move into its own module
* Fix issue with completion script generation and slight refactor
* Fix autocompletion for logging level
* Delete uneeded comment
* More tidying up
* Make run function final
* Fixed an issue with the program being run twice.
* Address review comments
* first pass at infrastructure around cli compatibility
* add example unit test
* inspect enum types
* add a basic unit test to verify behaviour of the cli checker
* revert root build.gradle
* InMemoryMessagingNetwork.InMemoryMessaging renamed to MockNodeMessagingService and moved to internal package
* start method added to MockNodeMessagingService which enables AbstractNode to call makeMessagingService in its c'tor
* Removed TopicStringValidator as it's no longer used
* Clean up of TestStartedNode
* Merged InMemoryMessagingTests into InternalMockNetworkTests as it's testing InternalMockNetwork
The list of CorDapps jars is no longer passed in via the cmd line but is now expected to be placed in the bootstrapped directory.
Ended up being a bit of a refactor to cater for unit testing, and also tidied up the bootstrapper docs.
* add experimental bootstrapper
(cherry picked from commit 63665d1)
refactor code to be less Azure Specific
Use node classes for notaries
add local docker backend
move to tools directoy
apply fixes for local docker RPC admin port
add extraParams field to context to allow dynamic backend selection
begin refactor to move all common node/notary functionality into single implementations
node and notaries now share the same code path as much as possible.
refactor network building logic into api class
port Main.kt to use new networkbuilder api
add gui
fix issues with local docker ports not being exposed on localhost
make push and instantiate async operations
add ability to "add" a node after network has been built to gui
tidy up backend selection via command line and GUI
allow region selection for AZURE instantiations
remove old network map based node.conf and network map dockerfile
tidy up constructors of the various node stage objects
tidy up artefact name
add network-name selection dialog
* print out help
* exclude transitive dep onto log4j to suppress error print out
* windows fixes for local docker
* fixes to allow "devs.XXXX" resource groups in line with the new devops policy of having named resourceGroups
* add extra logging around constructing azure backend
* Added missing plugin to enable publishing of artifact to Artifactory.
* Remove references to experimental blobinspector
* Remove disableDefaultJar for default jar.
* Add additional artifacts to be published.
