From cc75a65f9233b5ac0bcdc5e03c2af032c43fbba5 Mon Sep 17 00:00:00 2001 From: Katelyn Baker Date: Wed, 17 Oct 2018 15:09:48 +0100 Subject: [PATCH 1/8] RELEASE - Merge 3.3 upgrade / notes / changelog backto master (#4085) --- docs/source/changelog.rst | 116 ++++++++++++++++++++++---- docs/source/release-notes.rst | 150 +++++++++++++++++++++++++++++++++- docs/source/upgrade-notes.rst | 36 +++++--- 3 files changed, 273 insertions(+), 29 deletions(-) diff --git a/docs/source/changelog.rst b/docs/source/changelog.rst index f5fd3d3ae3..827dee6db5 100644 --- a/docs/source/changelog.rst +++ b/docs/source/changelog.rst @@ -26,8 +26,6 @@ Unreleased * Removed experimental feature ``CordformDefinition`` -* Vault query fix: support query by parent classes of Contract State classes (see https://github.com/corda/corda/issues/3714) - * Added ``registerResponderFlow`` method to ``StartedMockNode``, to support isolated testing of responder flow behaviour. * "app", "rpc", "p2p" and "unknown" are no longer allowed as uploader values when importing attachments. These are used @@ -57,9 +55,6 @@ Unreleased interfaces that will have unimplemented methods. This is useful, for example, for object viewers. This can be turned on with ``SerializationContext.withLenientCarpenter``. -* Introduced a grace period before the initial node registration fails if the node cannot connect to the Doorman. - It retries 10 times with a 1 minute interval in between each try. At the moment this is not configurable. - * Added a ``FlowMonitor`` to log information about flows that have been waiting for IO more than a configurable threshold. * H2 database changes: @@ -173,7 +168,7 @@ Unreleased * Added public support for creating ``CordaRPCClient`` using SSL. For this to work the node needs to provide client applications a certificate to be added to a truststore. See :doc:`tutorial-clientrpc-api` -*The node RPC broker opens 2 endpoints that are configured with ``address`` and ``adminAddress``. RPC Clients would connect +* The node RPC broker opens 2 endpoints that are configured with ``address`` and ``adminAddress``. RPC Clients would connect to the address, while the node will connect to the adminAddress. Previously if ssl was enabled for RPC the ``adminAddress`` was equal to ``address``. @@ -227,6 +222,73 @@ Unreleased normal state when it occurs in an input or output position. *This feature is only available on Corda networks running with a minimum platform version of 4.* +Version 3.3 +----------- + +* Vault query fix: support query by parent classes of Contract State classes (see https://github.com/corda/corda/issues/3714) + +* Fixed an issue preventing Shell from returning control to the user when CTRL+C is pressed in the terminal. + +* Fixed a problem that sometimes prevented nodes from starting in presence of custom state types in the database without a corresponding type from installed CorDapps. + +* Introduced a grace period before the initial node registration fails if the node cannot connect to the Doorman. + It retries 10 times with a 1 minute interval in between each try. At the moment this is not configurable. + +* Fixed an error thrown by NodeVaultService upon recording a transaction with a number of inputs greater than the default page size. + +* Changes to the JSON/YAML serialisation format from ``JacksonSupport``, which also applies to the node shell: + + * ``Instant`` and ``Date`` objects are serialised as ISO-8601 formatted strings rather than timestamps + * ``PublicKey`` objects are serialised and looked up according to their Base58 encoded string + * ``Party`` objects can be deserialised by looking up their public key, in addition to their name + * ``NodeInfo`` objects are serialised as an object and can be looked up using the same mechanism as ``Party`` + * ``NetworkHostAndPort`` serialised according to its ``toString()`` + * ``PartyAndCertificate`` is serialised as the name + * ``SerializedBytes`` is serialised by materialising the bytes into the object it represents, and then serialising that + object into YAML/JSON + * ``X509Certificate`` is serialised as an object with key fields such as ``issuer``, ``publicKey``, ``serialNumber``, etc. + The encoded bytes are also serialised into the ``encoded`` field. This can be used to deserialise an ``X509Certificate`` + back. + * ``CertPath`` objects are serialised as a list of ``X509Certificate`` objects. + +* ``fullParties`` boolean parameter added to ``JacksonSupport.createDefaultMapper`` and ``createNonRpcMapper``. If ``true`` + then ``Party`` objects are serialised as JSON objects with the ``name`` and ``owningKey`` fields. For ``PartyAndCertificate`` + the ``certPath`` is serialised. + +* Several members of ``JacksonSupport`` have been deprecated to highlight that they are internal and not to be used + +* ``ServiceHub`` and ``CordaRPCOps`` can now safely be used from multiple threads without incurring in database transaction problems. + +* Fixed an issue preventing out of process nodes started by the ``Driver`` from logging to file. + +* The Vault Criteria API has been extended to take a more precise specification of which class contains a field. This primarily impacts Java users; Kotlin users need take no action. The old methods have been deprecated but still work - the new methods avoid bugs that can occur when JPA schemas inherit from each other. + +* Removed -xmx VM argument from Explorer's Capsule setup. This helps avoiding out of memory errors. + +* Node will now gracefully fail to start if one of the required ports is already in use. + +* Fixed incorrect exception handling in ``NodeVaultService._query()``. + +* Avoided a memory leak deriving from incorrect MappedSchema caching strategy. + +* Fix CORDA-1403 where a property of a class that implemented a generic interface could not be deserialised in + a factory without a serialiser as the subtype check for the class instance failed. Fix is to compare the raw + type. + +* Fix CORDA-1229. Setter-based serialization was broken with generic types when the property was stored + as the raw type, List for example. + +.. _changelog_v3.2: + +Version 3.2 +----------- + +* Doorman and NetworkMap URLs can now be configured individually rather than being assumed to be + the same server. Current ``compatibilityZoneURL`` configurations remain valid. See both :doc:`corda-configuration-file` + and :doc:`permissioning` for details. + +* Table name with a typo changed from ``NODE_ATTCHMENTS_CONTRACTS`` to ``NODE_ATTACHMENTS_CONTRACTS``. + .. _changelog_v3.1: Version 3.1 @@ -235,7 +297,7 @@ Version 3.1 * Update the fast-classpath-scanner dependent library version from 2.0.21 to 2.12.3 .. note:: Whilst this is not the latest version of this library, that being 2.18.1 at time of writing, versions -later than 2.12.3 (including 2.12.4) exhibit a different issue. + later than 2.12.3 (including 2.12.4) exhibit a different issue. * Updated the api scanner gradle plugin to work the same way as the version in master. These changes make the api scanner more accurate and fix a couple of bugs, and change the format of the api-current.txt file slightly. Backporting these changes @@ -254,21 +316,25 @@ later than 2.12.3 (including 2.12.4) exhibit a different issue. Version 3.0 ----------- -* Per CorDapp configuration is now exposed. ``CordappContext`` now exposes a ``CordappConfig`` object that is populated - at CorDapp context creation time from a file source during runtime. +* Due to a security risk, the `conflict` property has been removed from `NotaryError.Conflict` error object. It has been replaced + with `consumedStates` instead. The new property no longer specifies the original requesting party and transaction id for + a consumed state. Instead, only the hash of the transaction id is revealed. For more details why this change had to be + made please refer to the release notes. -* Introduced Flow Draining mode, in which a node continues executing existing flows, but does not start new. This is to - support graceful node shutdown/restarts. In particular, when this mode is on, new flows through RPC will be rejected, - scheduled flows will be ignored, and initial session messages will not be consumed. This will ensure that the number of - checkpoints will strictly diminish with time, allowing for a clean shutdown. +* Added ``NetworkMapCache.getNodesByLegalName`` for querying nodes belonging to a distributed service such as a notary cluster + where they all share a common identity. ``NetworkMapCache.getNodeByLegalName`` has been tightened to throw if more than + one node with the legal name is found. -* Make the serialisation finger-printer a pluggable entity rather than hard wiring into the factory +* Introduced Flow Draining mode, in which a node continues executing existing flows, but does not start new. This is to support graceful node shutdown/restarts. + In particular, when this mode is on, new flows through RPC will be rejected, scheduled flows will be ignored, and initial session messages will not be consumed. + This will ensure that the number of checkpoints will strictly diminish with time, allowing for a clean shutdown. * Removed blacklisted word checks in Corda X.500 name to allow "Server" or "Node" to be use as part of the legal name. * Separated our pre-existing Artemis broker into an RPC broker and a P2P broker. * Refactored ``NodeConfiguration`` to expose ``NodeRpcOptions`` (using top-level "rpcAddress" property still works with warning). + * Modified ``CordaRPCClient`` constructor to take a ``SSLConfiguration?`` additional parameter, defaulted to ``null``. * Introduced ``CertificateChainCheckPolicy.UsernameMustMatchCommonName`` sub-type, allowing customers to optionally enforce @@ -284,6 +350,28 @@ Version 3.0 * JPA Mapping annotations for States extending ``CommonSchemaV1.LinearState`` and ``CommonSchemaV1.FungibleState`` on the `participants` collection need to be moved to the actual class. This allows to properly specify the unique table name per a collection. See: DummyDealStateSchemaV1.PersistentDummyDealState +* Database schema changes - an H2 database instance of Corda 1.0 and 2.0 cannot be reused for Corda 3.0, listed changes for Vault and Finance module: + + * ``NODE_TRANSACTIONS``: + column ``"TRANSACTION”`` renamed to ``TRANSACTION_VALUE``, serialization format of BLOB stored in the column has changed to AMQP + * ``VAULT_STATES``: + column ``CONTRACT_STATE`` removed + * ``VAULT_FUNGIBLE_STATES``: + column ``ISSUER_REFERENCE`` renamed to ``ISSUER_REF`` and the field size increased + * ``"VAULTSCHEMAV1$VAULTFUNGIBLESTATES_PARTICIPANTS"``: + table renamed to ``VAULT_FUNGIBLE_STATES_PARTS``, + column ``"VAULTSCHEMAV1$VAULTFUNGIBLESTATES_OUTPUT_INDEX"`` renamed to ``OUTPUT_INDEX``, + column ``"VAULTSCHEMAV1$VAULTFUNGIBLESTATES_TRANSACTION_ID"`` renamed to ``TRANSACTION_ID`` + * ``VAULT_LINEAR_STATES``: + type of column ``"UUID"`` changed from ``VARBINARY`` to ``VARCHAR(255)`` - select varbinary column as ``CAST("UUID" AS UUID)`` to get UUID in varchar format + * ``"VAULTSCHEMAV1$VAULTLINEARSTATES_PARTICIPANTS"``: + table renamed to ``VAULT_LINEAR_STATES_PARTS``, + column ``"VAULTSCHEMAV1$VAULTLINEARSTATES_OUTPUT_INDEX"`` renamed to ``OUTPUT_INDEX``, + column ``"VAULTSCHEMAV1$VAULTLINEARSTATES_TRANSACTION_ID"`` renamed to ``TRANSACTION_ID`` + * ``contract_cash_states``: + columns storing Base58 representation of the serialised public key (e.g. ``issuer_key``) were changed to store Base58 representation of SHA-256 of public key prefixed with `DL` + * ``contract_cp_states``: + table renamed to ``cp_states``, column changes as for ``contract_cash_states`` * X.509 certificates now have an extension that specifies the Corda role the certificate is used for, and the role hierarchy is now enforced in the validation code. See ``net.corda.core.internal.CertRole`` for the current implementation diff --git a/docs/source/release-notes.rst b/docs/source/release-notes.rst index 3f3e0e9d60..b1db04faf5 100644 --- a/docs/source/release-notes.rst +++ b/docs/source/release-notes.rst @@ -1,11 +1,14 @@ Release notes ============= -Unreleased ----------- +.. _release_notes_v4_0: + +Release 4.0 (Unreleased) +------------------------ Significant Changes in 4.0 ~~~~~~~~~~~~~~~~~~~~~~~~~~ + * **Retirement of non-elliptic Diffie-Hellman for TLS** The TLS_DHE_RSA_WITH_AES_128_GCM_SHA256 family of ciphers is retired from the list of allowed ciphers for TLS as it is a legacy cipher family not supported by all native SSL/TLS implementations. @@ -18,9 +21,150 @@ Significant Changes in 4.0 for "current-ness". In other words, the contract logic isn't run for the referencing transaction only. It's still a normal state when it occurs in an input or output position. +<< MORE TO COME >> + +.. _release_notes_v3_3: + +Release 3.3 +----------- + +Corda 3.3 brings together many small improvements, fixes, and community contributions to deliver a stable and polished release +of Corda. Where both the 3.1 and 3.2 releases delivered a smaller number of critical bug fixes addressing immediate and impactful error conditions, 3.3 +addresses a much greater number of issues, both small and large, that have been found and fixed since the release of 3.0 back in March. Rolling up a great +many improvements and polish to truly make the Corda experience just that much better. + +In addition to work undertaken by the main Corda development team, we've taken the opportunity in 3.3 to bring back many of the contributions made +by community members from master onto the currently released stable branch. It has been said many times before, but the community and its members +are the real life-blood of Corda and anyone who takes the time to contribute is a star in our eyes. Bringing that code into the current version we hope +gives people the opportunity to see their work in action, and to help their fellow community members by having these contributions available in a +supported release. + +Changes of Note +~~~~~~~~~~~~~~~ + +* **Serialization fixes** + + Things "in the lab" always work so much better than they do in the wild, where everything you didn't think of is thrown at your code and a mockery + is made of some dearly held assumptions. A great example of this is the serialization framework which delivers Corda's wire stability guarantee + that was introduced in 3.0 and has subsequently been put to a rigorous test by our users. Corda 3.3 consolidates a great many fixes in that framework, + both programmatically in terms of fixing bugs, but also in the documentation, hopefully making things clearer and easier to work with. + +* **Certificate Hierarchy** + + After consultation, collaboration, and discussion with industry experts, we have decided to alter the default Certificate Hierarchy (PKI) utilized by + Corda and the Corda Network. To facilitate this, the nodes have had their certificate path verification logic made much more flexible. All existing + certificate hierarchy, certificates, and networks will remain valid. The possibility now exists for nodes to recognize a deeper certificate chain and + thus Compatibility Zone operators can deploy and adhere to the PKI standards they expect and are comfortable with. + + Practically speaking, the old code assumed a 3-level hierarchy of Root -> Intermediate CA (Doorman) -> Node, and this was hard coded. From 3.3 onward an + arbitrary depth of certificate chain is supported. For the Corda Network, this means the introduction of an intermediate layer between the root and the + signing certificates (Network Map and Doorman). This has the effect of allowing the root certificate to *always* be kept offline and never retrieved or + used. Those new intermediate certificates can be used to generate, if ever needed, new signing certs without risking compromise of the root key. + +Special Thanks +~~~~~~~~~~~~~~ + +The Corda community is a vibrant and exciting ecosystem that spreads far outside the virtual walls of the +R3 organisation. Without that community, and the most welcome contributions of its members, the Corda project +would be a much poorer place. + +We're therefore happy to extend thanks to the following members of that community for their contributions + + * `Dan Newton `_ for a fix to cleanup node registration in the test framework. The changes can be found `here `_. + * `Tushar Singh Bora `_ for a number of `documentation tweaks `_. In addition, some updates to the tutorial documentation `here `_. + * `Jiachuan Li `_ for a number of corrections to our documentation. Those contributions can be found `here `_ and `here `_. + * `Yogesh `_ for a documentation tweak that can be see `here `_. + * `Roman Plášil `_ for speeding up node shutdown when connecting to an http network map. This fix can be found `here `_. + * `renlulu `_ for a small `PR `_ to optimize some of the imports. + * `cxyzhang0 `_ for making the ``IdentitySyncFlow`` more useful. See `here `_. + * `Venelin Stoykov `_ with updates to the `documentation `_ around the progress tracker. + * `Mohamed Amine Legheraba `_ for updates to the Azure documentation that can be seen `here `_. + * `Stanly Johnson `_ with a `fix `_ to the network bootstrapper. + * `Tittu Varghese `_ for adding a favicon to the docsite. This commit can be found `here `_ + Issues Fixed ~~~~~~~~~~~~ -* Cordform Gradle task (`deployNodes`) doesn't work when `configFile` element was used. + +* Refactoring ``DigitalSignatureWithCertPath`` for more performant storing of the certificate chain. [`CORDA-1995 `_] +* The serializers class carpenter fails when superclass has double-size primitive field. [`Corda-1945 `_] +* If a second identity is mistakenly created the node will not start. [`CORDA-1811 `_] +* Demobench profile load fails with stack dump. [`CORDA-1948 `_] +* Deletes of NodeInfo can fail to propagate leading to infinite retries. [`CORDA-2029 `_] +* Copy all the certificates from the network-trust-store.jks file to the node's trust store. [`CORDA-2012 `_] +* Add SNI (Server Name Indication) header to TLS connections. [`CORDA-2001 `_] +* Fix duplicate index declaration in the Cash schema. [`CORDA-1952 `_] +* Hello World Tutorial Page mismatch between code sample and explanatory text. [`CORDA-1950 `_] +* Java Instructions to Invoke Hello World CorDapp are incorrect. [`CORDA-1949 `_] +* Add ``VersionInfo`` to the ``NodeInfo`` submission request to the network map element of the Compatibility Zone. [`CORDA-1938 `_] +* Rename current INTERMEDIATE_CA certificate role to DOORMAN_CA certificate role. [`CORDA-1934 `_] +* Make node-side network map verification agnostic to the certificate hierarchy. [`CORDA-1932 `_] +* Corda Shell incorrectly deserializes generic types as raw types. [`CORDA-1907 `_] +* The Corda web server does not support asynchronous servlets. [`CORDA-1906 `_] +* Amount is deserialized from JSON and YAML as Amount, for all values of T. [`CORDA-1905 `_] +* ``NodeVaultService.loadStates`` queries without a ``PageSpecification`` property set. This leads to issues with large transactions. [`CORDA-1895 `_] +* If a node has two flows, where one's name is a longer version of the other's, they cannot be started [`CORDA-1892 `_] +* Vault Queries across ``LinearStates`` and ``FungibleState`` tables return incorrect results. [`CORDA-1888 `_] +* Checking the version of the Corda jar file by executing the jar with the ``--version`` flag without specifying a valid node configuration file causes an exception to be thrown. [`CORDA-1884 `_] +* RPC deadlocks after a node restart. [`CORDA-1875 `_] +* Vault query fails to find a state if it extends some class (``ContractState``) and it is that base class that is used as the predicate (``vaultService.queryBy()``). [`CORDA-1858 `_] +* Missing unconsumed states from linear id when querying vault caused by a the previous transaction failing with an SQL exception. [`CORDA-1847 `_] +* Inconsistency in how a web path is written. [`CORDA-1841 `_] +* Cannot use ``TestIdentities`` with same organization name in ``net.corda.testing.driver.Driver``. [`CORDA-1837 `_] +* Docs page typos. [`CORDA-1834 `_] +* Adding flexibility to the serialization frameworks unit tests support and utility code. [`CORDA-1808 `_] +* Cannot use ``--initial-registration`` with the ``networkServices`` configuration option in place of the older ``compatibilityzone`` option within ``node.conf``. [`CORDA-1789 `_] +* Document more clearly the supported version of both IntelliJ and the IntelliJ Kotlin Plugins. [`CORDA-1727 `_] +* DemoBench's "Launch Explorer" button is not re-enabled when you close Node Explorer. [`CORDA-1686 `_] +* It is not possible to run ``stateMachinesSnapshot`` from the shell. [`CORDA-1681 `_] +* Node won't start if CorDapps generate states prior to deletion [`CORDA-1663 `_] +* Serializer Evolution breaks with Java classes adding nullable properties. [`CORDA-1662 `_] +* Add Java examples for the creation of proxy serializers to complement the existing kotlin ones. [`CORDA-1641 `_] +* Proxy serializer documentation isn't clear on how to write a proxy serializer. [`CORDA-1640 `_] +* Node crashes in ``--initial-registration`` polling mode if doorman returns a transient HTTP error. [`CORDA-1638 `_] +* Nodes started by gradle task are not stopped when the gradle task exits. [`CORDA-1634 `_] +* Notarizations time out if notary doesn't have up-to-date network map. [`CORDA-1628 `_] +* Node explorer: Improve error handling when connection to nodes cannot be established. [`CORDA-1617 `_] +* Validating notary fails to resolve an attachment. [`CORDA-1588 `_] +* Out of process nodes started by the driver do not log to file. [`CORDA-1575 `_] +* Once ``--initial-registration`` has been passed to a node, further restarts should assume that mode until a cert is collected. [`CORDA-1572 `_] +* An array of primitive byte arrays (an array of arrays) won't deserialize in a virgin factory (i.e. one that didn't build the serializer for serialization). [`CORDA-1545 `_] +* Ctrl-C in the shell fails to aborts the flow. [`CORDA-1542 `_] +* One transaction with two identical cash outputs cannot be save in the vault. [`CORDA-1535 `_] +* The unit tests for the enum evolver functionality cannot be regenerated. This is because verification logic added after their initial creation has a bug that incorrectly identifies a cycle in the graph. [`CORDA-1498 `_] +* Add in a safety check that catches flow checkpoints from older versions. [`CORDA-1477 `_] +* Buggy ``CommodityContract`` issuance logic. [`CORDA-1459 `_] +* Error in the process-id deletion process allows multiple instances of the same node to be run. [`CORDA-1455 `_] +* Node crashes if network map returns HTTP 50X error. [`CORDA-1414 `_] +* Delegate Property doesn't serialize, throws an erroneous type mismatch error. [`CORDA-1403 `_] +* If a vault query throws an exception, the stack trace is swallowed. [`CORDA-1397 `_] +* Node can fail to fully start when a port conflict occurs, no useful error message is generated when this occurs. [`CORDA-1394 `_] +* Running the ``deployNodes`` gradle task back to back without a clean doesn't work. [`CORDA-1389 `_] +* Stripping issuer from Amount> does not preserve ``displayTokenSize``. [`CORDA-1386 `_] +* ``CordaServices`` are instantiated multiple times per Party when using ``NodeDriver``. [`CORDA-1385 `_] +* Out of memory errors can be seen when using Demobench + Explorer. [`CORDA-1356 `_] +* Reduce the amount of classpath scanning during integration tests execution. [`CORDA-1355 `_] +* SIMM demo throws "attachment too big" errors. [`CORDA-1346 `_] +* Fix vault query paging example in ``ScheduledFlowTests``. [`CORDA-1344 `_] +* The shell doesn't print the return value of a started flow. [`CORDA-1342 `_] +* Provide access to database transactions for CorDapp developers. [`CORDA-1341 `_] +* Error with ``VaultQuery`` for entity inheriting from ``CommonSchemaV1.FungibleState``. [`CORDA-1338 `_] +* The ``--network-root-truststore`` command line option not defaulted. [`CORDA-1317 `_] +* Java example in "Upgrading CorDapps" documentation is wrong. [`CORDA-1315 `_] +* Remove references to ``registerInitiatedFlow`` in testing documentation as it is not needed. [`CORDA-1304 `_] +* Regression: Recording a duplicate transaction attempts second insert to vault. [`CORDA-1303 `_] +* Columns in the Corda database schema should have correct NULL/NOT NULL constraints. [`CORDA-1297 `_] +* MockNetwork/Node API needs a way to register ``@CordaService`` objects. [`CORDA-1292 `_] +* Deleting a ``NodeInfo`` from the additional-node-infos directory should remove it from cache. [`CORDA-1093 `_] +* ``FailNodeOnNotMigratedAttachmentContractsTableNameTests`` is sometimes failing with database constraint "Notary" is null. [`CORDA-1976 `_] +* Revert keys for DEV certificates. [`CORDA-1661 `_] +* Node Info file watcher should block and load ``NodeInfo`` when node startup. [`CORDA-1604 `_] +* Improved logging of the network parameters update process. [`CORDA-1405 `_] +* Ensure all conditions in cash selection query are tested. [`CORDA-1266 `_] +* ``NodeVaultService`` bug. Start node, issue cash, stop node, start node, ``getCashBalances()`` will not show any cash +* A Corda node doesn't re-select cluster from HA Notary. +* Event Horizon is not wire compatible with older network parameters objects. +* Notary unable to resolve Party after processing a flow from same Party. +* Misleading error message shown when a node is restarted after a flag day event. .. _release_notes_v3_2: diff --git a/docs/source/upgrade-notes.rst b/docs/source/upgrade-notes.rst index f2c51349b9..09ccfb1c9f 100644 --- a/docs/source/upgrade-notes.rst +++ b/docs/source/upgrade-notes.rst @@ -9,33 +9,31 @@ first public Beta (:ref:`Milestone 12 `), to :ref:`V1.0 >> - * Database upgrade - Change the type of the ``checkpoint_value``. This will address the issue that the `vacuum` function is unable to clean up deleted checkpoints as they are still referenced from the ``pg_shdepend`` table. @@ -86,6 +84,17 @@ For H2: No action is needed for default node tables as ``PersistentStateRef`` is used as Primary Key only and the backing columns are automatically not nullable or custom Cordapp entities using ``PersistentStateRef`` as Primary Key. +V3.2 to v3.3 +------------ + +* Update the Corda Release version + + The ``corda_release_version`` identifier in your projects gradle file will need changing as follows: + + .. sourcecode:: shell + + ext.corda_release_version = '3.3-corda' + v3.1 to v3.2 ------------ @@ -118,8 +127,11 @@ Database schema changes ALTER TABLE node_checkpoints ALTER COLUMN checkpoint_value set data type bytea using null; - .. important:: - The Corda node will fail on startup if the database was not updated with the above commands. + .. note:: + This change will also need to be run when migrating from version 3.0. + +.. important:: + The Corda node will fail on startup if the database was not updated with the above commands. v3.0 to v3.1 ------------ From 55731ef81606e93d2f7d413a08f600b6c3536091 Mon Sep 17 00:00:00 2001 From: Rick Parker Date: Thu, 18 Oct 2018 10:38:43 +0100 Subject: [PATCH 2/8] ENT-2431 Tidy up buildNamed and CacheFactory --- .../client/jfx/model/NetworkIdentityModel.kt | 3 +- .../client/rpc/internal/ClientCacheFactory.kt | 19 +++++ .../rpc/internal/RPCClientProxyHandler.kt | 8 +-- .../net/corda/core/internal/NamedCache.kt | 37 ++++------ .../net/corda/core/internal/NamedCacheTest.kt | 14 +++- .../notary/raft/RaftUniquenessProvider.kt | 2 +- .../raft/RaftTransactionCommitLogTests.kt | 2 +- .../nodeapi/internal/DeduplicationChecker.kt | 9 ++- .../internal/persistence/CordaPersistence.kt | 4 +- .../persistence/HibernateConfiguration.kt | 5 +- .../messaging/ArtemisMessagingTest.kt | 4 +- .../network/PersistentNetworkMapCacheTest.kt | 4 +- .../node/services/rpc/ArtemisRpcTests.kt | 4 +- .../net/corda/node/internal/AbstractNode.kt | 28 +++----- .../kotlin/net/corda/node/internal/Node.kt | 6 +- .../security/RPCSecurityManagerImpl.kt | 33 ++++----- .../node/services/api/ServiceHubInternal.kt | 2 +- .../identity/PersistentIdentityService.kt | 2 +- .../keys/PersistentKeyManagementService.kt | 2 +- .../messaging/InternalRPCMessagingClient.kt | 5 +- .../messaging/P2PMessageDeduplicator.kt | 2 +- .../services/messaging/P2PMessagingClient.kt | 2 +- .../node/services/messaging/RPCServer.kt | 23 ++---- .../network/PersistentNetworkMapCache.kt | 2 +- .../persistence/DBTransactionStorage.kt | 2 +- .../persistence/NodeAttachmentService.kt | 1 - .../NodePropertiesPersistentStore.kt | 10 +-- .../PersistentUniquenessProvider.kt | 2 +- .../upgrade/ContractUpgradeServiceImpl.kt | 10 +-- .../node/utilities/AppendOnlyPersistentMap.kt | 2 +- .../corda/node/utilities/NodeNamedCache.kt | 71 +++++++++++++------ .../node/utilities/NonInvalidatingCache.kt | 1 + .../utilities/NonInvalidatingUnboundCache.kt | 10 +-- .../net/corda/node/utilities/PersistentMap.kt | 7 +- .../net/corda/node/CordaRPCOpsImplTest.kt | 3 +- .../corda/node/internal/AbstractNodeTests.kt | 2 +- .../net/corda/node/internal/NodeTest.kt | 1 + .../node/services/RPCSecurityManagerTest.kt | 4 +- .../events/NodeSchedulerServiceTest.kt | 2 +- .../PersistentScheduledFlowRepositoryTest.kt | 3 +- .../PersistentIdentityServiceTests.kt | 2 +- .../AppendOnlyPersistentMapTest.kt | 2 +- .../persistence/DBCheckpointStorageTests.kt | 6 +- .../persistence/DBTransactionStorageTests.kt | 2 +- .../persistence/HibernateConfigurationTest.kt | 2 +- .../persistence/NodeAttachmentServiceTest.kt | 2 +- .../persistence/TransactionCallbackTest.kt | 2 +- .../schema/PersistentStateServiceTests.kt | 9 +-- .../PersistentUniquenessProviderTests.kt | 2 +- .../node/services/vault/VaultQueryTests.kt | 2 +- .../corda/node/utilities/ObservablesTests.kt | 2 +- .../node/utilities/PersistentMapTests.kt | 6 +- .../corda/irs/api/NodeInterestRatesTest.kt | 5 +- .../internal/SerializationScheme.kt | 3 +- .../net/corda/testing/node/MockServices.kt | 2 +- .../corda/testing/node/internal/RPCDriver.kt | 7 +- .../testing/internal/InternalTestUtils.kt | 41 ++++++++++- .../internal/TestingNamedCacheFactory.kt | 13 ++-- .../explorer/identicon/IdenticonRenderer.kt | 3 +- 59 files changed, 269 insertions(+), 197 deletions(-) create mode 100644 client/rpc/src/main/kotlin/net/corda/client/rpc/internal/ClientCacheFactory.kt diff --git a/client/jfx/src/main/kotlin/net/corda/client/jfx/model/NetworkIdentityModel.kt b/client/jfx/src/main/kotlin/net/corda/client/jfx/model/NetworkIdentityModel.kt index aa7f7e50f2..af2acc1aac 100644 --- a/client/jfx/src/main/kotlin/net/corda/client/jfx/model/NetworkIdentityModel.kt +++ b/client/jfx/src/main/kotlin/net/corda/client/jfx/model/NetworkIdentityModel.kt @@ -7,7 +7,6 @@ import javafx.collections.FXCollections import javafx.collections.ObservableList import net.corda.client.jfx.utils.* import net.corda.core.identity.AnonymousParty -import net.corda.core.internal.buildNamed import net.corda.core.node.NodeInfo import net.corda.core.node.services.NetworkMapCache.MapChange import java.security.PublicKey @@ -32,7 +31,7 @@ class NetworkIdentityModel { private val rpcProxy by observableValue(NodeMonitorModel::proxyObservable) private val identityCache = Caffeine.newBuilder() - .buildNamed>("NetworkIdentityModel_identity", CacheLoader { publicKey: PublicKey -> + .build>(CacheLoader { publicKey: PublicKey -> publicKey.let { rpcProxy.map { it?.cordaRPCOps?.nodeInfoFromParty(AnonymousParty(publicKey)) } } }) val notaries = ChosenList(rpcProxy.map { FXCollections.observableList(it?.cordaRPCOps?.notaryIdentities() ?: emptyList()) }, "notaries") diff --git a/client/rpc/src/main/kotlin/net/corda/client/rpc/internal/ClientCacheFactory.kt b/client/rpc/src/main/kotlin/net/corda/client/rpc/internal/ClientCacheFactory.kt new file mode 100644 index 0000000000..bad74ecd2b --- /dev/null +++ b/client/rpc/src/main/kotlin/net/corda/client/rpc/internal/ClientCacheFactory.kt @@ -0,0 +1,19 @@ +package net.corda.client.rpc.internal + +import com.github.benmanes.caffeine.cache.Cache +import com.github.benmanes.caffeine.cache.CacheLoader +import com.github.benmanes.caffeine.cache.Caffeine +import com.github.benmanes.caffeine.cache.LoadingCache +import net.corda.core.internal.NamedCacheFactory + +class ClientCacheFactory : NamedCacheFactory { + override fun buildNamed(caffeine: Caffeine, name: String): Cache { + checkCacheName(name) + return caffeine.build() + } + + override fun buildNamed(caffeine: Caffeine, name: String, loader: CacheLoader): LoadingCache { + checkCacheName(name) + return caffeine.build(loader) + } +} \ No newline at end of file diff --git a/client/rpc/src/main/kotlin/net/corda/client/rpc/internal/RPCClientProxyHandler.kt b/client/rpc/src/main/kotlin/net/corda/client/rpc/internal/RPCClientProxyHandler.kt index be74d7b316..6ceff2b9d0 100644 --- a/client/rpc/src/main/kotlin/net/corda/client/rpc/internal/RPCClientProxyHandler.kt +++ b/client/rpc/src/main/kotlin/net/corda/client/rpc/internal/RPCClientProxyHandler.kt @@ -83,7 +83,8 @@ class RPCClientProxyHandler( private val sessionId: Trace.SessionId, private val externalTrace: Trace?, private val impersonatedActor: Actor?, - private val targetLegalIdentity: CordaX500Name? + private val targetLegalIdentity: CordaX500Name?, + private val cacheFactory: NamedCacheFactory = ClientCacheFactory() ) : InvocationHandler { private enum class State { @@ -169,8 +170,7 @@ class RPCClientProxyHandler( } observablesToReap.locked { observables.add(observableId) } } - return Caffeine.newBuilder(). - weakValues().removalListener(onObservableRemove).executor(SameThreadExecutor.getExecutor()).buildNamed("RpcClientProxyHandler_rpcObservable") + return cacheFactory.buildNamed(Caffeine.newBuilder().weakValues().removalListener(onObservableRemove).executor(SameThreadExecutor.getExecutor()), "RpcClientProxyHandler_rpcObservable") } private var sessionFactory: ClientSessionFactory? = null @@ -179,7 +179,7 @@ class RPCClientProxyHandler( private var rpcProducer: ClientProducer? = null private var rpcConsumer: ClientConsumer? = null - private val deduplicationChecker = DeduplicationChecker(rpcConfiguration.deduplicationCacheExpiry) + private val deduplicationChecker = DeduplicationChecker(rpcConfiguration.deduplicationCacheExpiry, cacheFactory = cacheFactory) private val deduplicationSequenceNumber = AtomicLong(0) private val sendingEnabled = AtomicBoolean(true) diff --git a/core/src/main/kotlin/net/corda/core/internal/NamedCache.kt b/core/src/main/kotlin/net/corda/core/internal/NamedCache.kt index 2a96de4835..632b0c3060 100644 --- a/core/src/main/kotlin/net/corda/core/internal/NamedCache.kt +++ b/core/src/main/kotlin/net/corda/core/internal/NamedCache.kt @@ -6,30 +6,21 @@ import com.github.benmanes.caffeine.cache.Caffeine import com.github.benmanes.caffeine.cache.LoadingCache /** - * Restrict the allowed characters of a cache name - this ensures that each cache has a name, and that - * the name can be used to create a file name or a metric name. + * Allow extra functionality to be injected to our caches. */ -internal fun checkCacheName(name: String) { - require(!name.isBlank()) - require(allowedChars.matches(name)) +interface NamedCacheFactory { + /** + * Restrict the allowed characters of a cache name - this ensures that each cache has a name, and that + * the name can be used to create a file name or a metric name. + */ + fun checkCacheName(name: String) { + require(!name.isBlank()) + require(allowedChars.matches(name)) + } + + fun buildNamed(caffeine: Caffeine, name: String): Cache + + fun buildNamed(caffeine: Caffeine, name: String, loader: CacheLoader): LoadingCache } private val allowedChars = Regex("^[0-9A-Za-z_.]*\$") - -/* buildNamed is the central helper method to build caffeine caches in Corda. - * This allows to easily add tweaks to all caches built in Corda, and also forces - * cache users to give their cache a (meaningful) name that can be used e.g. for - * capturing cache traces etc. - * - * Currently it is not used in this version of CORDA, but there are plans to do so. - */ - -fun Caffeine.buildNamed(name: String): Cache { - checkCacheName(name) - return this.build() -} - -fun Caffeine.buildNamed(name: String, loader: CacheLoader): LoadingCache { - checkCacheName(name) - return this.build(loader) -} diff --git a/core/src/test/kotlin/net/corda/core/internal/NamedCacheTest.kt b/core/src/test/kotlin/net/corda/core/internal/NamedCacheTest.kt index 3358696bce..4d9f9297ab 100644 --- a/core/src/test/kotlin/net/corda/core/internal/NamedCacheTest.kt +++ b/core/src/test/kotlin/net/corda/core/internal/NamedCacheTest.kt @@ -1,9 +1,21 @@ package net.corda.core.internal +import com.github.benmanes.caffeine.cache.Cache +import com.github.benmanes.caffeine.cache.CacheLoader +import com.github.benmanes.caffeine.cache.Caffeine +import com.github.benmanes.caffeine.cache.LoadingCache import org.junit.Test import kotlin.test.assertEquals -class NamedCacheTest { +class NamedCacheTest : NamedCacheFactory { + override fun buildNamed(caffeine: Caffeine, name: String): Cache { + throw IllegalStateException("Should not be called") + } + + override fun buildNamed(caffeine: Caffeine, name: String, loader: CacheLoader): LoadingCache { + throw IllegalStateException("Should not be called") + } + fun checkNameHelper(name: String, throws: Boolean) { var exceptionThrown = false try { diff --git a/experimental/notary-raft/src/main/kotlin/net/corda/notary/raft/RaftUniquenessProvider.kt b/experimental/notary-raft/src/main/kotlin/net/corda/notary/raft/RaftUniquenessProvider.kt index c7670d0c18..8146178559 100644 --- a/experimental/notary-raft/src/main/kotlin/net/corda/notary/raft/RaftUniquenessProvider.kt +++ b/experimental/notary-raft/src/main/kotlin/net/corda/notary/raft/RaftUniquenessProvider.kt @@ -18,6 +18,7 @@ import net.corda.core.contracts.TimeWindow import net.corda.core.crypto.SecureHash import net.corda.core.flows.NotarisationRequestSignature import net.corda.core.identity.Party +import net.corda.core.internal.NamedCacheFactory import net.corda.core.internal.notary.NotaryInternalException import net.corda.core.internal.notary.UniquenessProvider import net.corda.core.schemas.PersistentStateRef @@ -27,7 +28,6 @@ import net.corda.core.utilities.contextLogger import net.corda.core.utilities.debug import net.corda.node.services.config.RaftConfig import net.corda.node.utilities.AppendOnlyPersistentMap -import net.corda.node.utilities.NamedCacheFactory import net.corda.nodeapi.internal.config.MutualSslConfiguration import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.NODE_DATABASE_PREFIX diff --git a/experimental/notary-raft/src/test/kotlin/net/corda/notary/raft/RaftTransactionCommitLogTests.kt b/experimental/notary-raft/src/test/kotlin/net/corda/notary/raft/RaftTransactionCommitLogTests.kt index 8b0f31f12d..f8848a4746 100644 --- a/experimental/notary-raft/src/test/kotlin/net/corda/notary/raft/RaftTransactionCommitLogTests.kt +++ b/experimental/notary-raft/src/test/kotlin/net/corda/notary/raft/RaftTransactionCommitLogTests.kt @@ -14,7 +14,6 @@ import net.corda.core.internal.concurrent.asCordaFuture import net.corda.core.internal.concurrent.transpose import net.corda.core.utilities.NetworkHostAndPort import net.corda.core.utilities.getOrThrow -import net.corda.node.internal.configureDatabase import net.corda.node.services.schema.NodeSchemaService import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.nodeapi.internal.persistence.CordaPersistence @@ -23,6 +22,7 @@ import net.corda.testing.core.ALICE_NAME import net.corda.testing.core.SerializationEnvironmentRule import net.corda.testing.driver.PortAllocation import net.corda.testing.internal.LogHelper +import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import org.hamcrest.Matchers.instanceOf import org.junit.After diff --git a/node-api/src/main/kotlin/net/corda/nodeapi/internal/DeduplicationChecker.kt b/node-api/src/main/kotlin/net/corda/nodeapi/internal/DeduplicationChecker.kt index 71d99e0fec..8b913c6ea2 100644 --- a/node-api/src/main/kotlin/net/corda/nodeapi/internal/DeduplicationChecker.kt +++ b/node-api/src/main/kotlin/net/corda/nodeapi/internal/DeduplicationChecker.kt @@ -2,7 +2,7 @@ package net.corda.nodeapi.internal import com.github.benmanes.caffeine.cache.CacheLoader import com.github.benmanes.caffeine.cache.Caffeine -import net.corda.core.internal.buildNamed +import net.corda.core.internal.NamedCacheFactory import java.time.Duration import java.util.concurrent.TimeUnit import java.util.concurrent.atomic.AtomicLong @@ -10,11 +10,10 @@ import java.util.concurrent.atomic.AtomicLong /** * A class allowing the deduplication of a strictly incrementing sequence number. */ -class DeduplicationChecker(cacheExpiry: Duration, name: String = "DeduplicationChecker") { +class DeduplicationChecker(cacheExpiry: Duration, name: String = "DeduplicationChecker", cacheFactory: NamedCacheFactory) { // dedupe identity -> watermark cache - private val watermarkCache = Caffeine.newBuilder() - .expireAfterAccess(cacheExpiry.toNanos(), TimeUnit.NANOSECONDS) - .buildNamed("${name}_watermark", WatermarkCacheLoader) + private val watermarkCache = cacheFactory.buildNamed(Caffeine.newBuilder() + .expireAfterAccess(cacheExpiry.toNanos(), TimeUnit.NANOSECONDS), "${name}_watermark", WatermarkCacheLoader) private object WatermarkCacheLoader : CacheLoader { override fun load(key: Any) = AtomicLong(-1) diff --git a/node-api/src/main/kotlin/net/corda/nodeapi/internal/persistence/CordaPersistence.kt b/node-api/src/main/kotlin/net/corda/nodeapi/internal/persistence/CordaPersistence.kt index e1558971eb..d3320d46ee 100644 --- a/node-api/src/main/kotlin/net/corda/nodeapi/internal/persistence/CordaPersistence.kt +++ b/node-api/src/main/kotlin/net/corda/nodeapi/internal/persistence/CordaPersistence.kt @@ -1,6 +1,7 @@ package net.corda.nodeapi.internal.persistence import co.paralleluniverse.strands.Strand +import net.corda.core.internal.NamedCacheFactory import net.corda.core.schemas.MappedSchema import net.corda.core.utilities.contextLogger import rx.Observable @@ -52,6 +53,7 @@ class CordaPersistence( databaseConfig: DatabaseConfig, schemas: Set, val jdbcUrl: String, + cacheFactory: NamedCacheFactory, attributeConverters: Collection> = emptySet() ) : Closeable { companion object { @@ -61,7 +63,7 @@ class CordaPersistence( private val defaultIsolationLevel = databaseConfig.transactionIsolationLevel val hibernateConfig: HibernateConfiguration by lazy { transaction { - HibernateConfiguration(schemas, databaseConfig, attributeConverters, jdbcUrl) + HibernateConfiguration(schemas, databaseConfig, attributeConverters, jdbcUrl, cacheFactory) } } diff --git a/node-api/src/main/kotlin/net/corda/nodeapi/internal/persistence/HibernateConfiguration.kt b/node-api/src/main/kotlin/net/corda/nodeapi/internal/persistence/HibernateConfiguration.kt index 5afeb3fe8d..ae371ca88e 100644 --- a/node-api/src/main/kotlin/net/corda/nodeapi/internal/persistence/HibernateConfiguration.kt +++ b/node-api/src/main/kotlin/net/corda/nodeapi/internal/persistence/HibernateConfiguration.kt @@ -1,7 +1,7 @@ package net.corda.nodeapi.internal.persistence import com.github.benmanes.caffeine.cache.Caffeine -import net.corda.core.internal.buildNamed +import net.corda.core.internal.NamedCacheFactory import net.corda.core.internal.castIfPossible import net.corda.core.schemas.MappedSchema import net.corda.core.utilities.contextLogger @@ -31,6 +31,7 @@ class HibernateConfiguration( private val databaseConfig: DatabaseConfig, private val attributeConverters: Collection>, private val jdbcUrl: String, + cacheFactory: NamedCacheFactory, val cordappClassLoader: ClassLoader? = null ) { companion object { @@ -58,7 +59,7 @@ class HibernateConfiguration( } } - private val sessionFactories = Caffeine.newBuilder().maximumSize(databaseConfig.mappedSchemaCacheSize).buildNamed, SessionFactory>("HibernateConfiguration_sessionFactories") + private val sessionFactories = cacheFactory.buildNamed, SessionFactory>(Caffeine.newBuilder(), "HibernateConfiguration_sessionFactories") val sessionFactoryForRegisteredSchemas = schemas.let { logger.info("Init HibernateConfiguration for schemas: $it") diff --git a/node/src/integration-test/kotlin/net/corda/node/services/messaging/ArtemisMessagingTest.kt b/node/src/integration-test/kotlin/net/corda/node/services/messaging/ArtemisMessagingTest.kt index 4a427940cf..04b5e2e973 100644 --- a/node/src/integration-test/kotlin/net/corda/node/services/messaging/ArtemisMessagingTest.kt +++ b/node/src/integration-test/kotlin/net/corda/node/services/messaging/ArtemisMessagingTest.kt @@ -7,7 +7,6 @@ import net.corda.core.crypto.generateKeyPair import net.corda.core.internal.div import net.corda.core.utilities.NetworkHostAndPort import net.corda.core.utilities.seconds -import net.corda.node.internal.configureDatabase import net.corda.node.services.config.FlowTimeoutConfiguration import net.corda.node.services.config.NodeConfiguration import net.corda.node.services.config.configureWithDevSSLCertificate @@ -21,11 +20,12 @@ import net.corda.testing.core.MAX_MESSAGE_SIZE import net.corda.testing.core.SerializationEnvironmentRule import net.corda.testing.driver.PortAllocation import net.corda.testing.internal.LogHelper +import net.corda.testing.internal.TestingNamedCacheFactory +import net.corda.testing.internal.configureDatabase import net.corda.testing.internal.rigorousMock import net.corda.testing.internal.stubs.CertificateStoreStubs import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import net.corda.testing.node.internal.MOCK_VERSION_INFO -import net.corda.testing.internal.TestingNamedCacheFactory import org.apache.activemq.artemis.api.core.ActiveMQConnectionTimedOutException import org.assertj.core.api.Assertions.assertThat import org.assertj.core.api.Assertions.assertThatThrownBy diff --git a/node/src/integration-test/kotlin/net/corda/node/services/network/PersistentNetworkMapCacheTest.kt b/node/src/integration-test/kotlin/net/corda/node/services/network/PersistentNetworkMapCacheTest.kt index 8f35b7a543..175b7530dd 100644 --- a/node/src/integration-test/kotlin/net/corda/node/services/network/PersistentNetworkMapCacheTest.kt +++ b/node/src/integration-test/kotlin/net/corda/node/services/network/PersistentNetworkMapCacheTest.kt @@ -2,14 +2,14 @@ package net.corda.node.services.network import net.corda.core.node.NodeInfo import net.corda.core.utilities.NetworkHostAndPort -import net.corda.node.internal.configureDatabase import net.corda.node.internal.schemas.NodeInfoSchemaV1 import net.corda.node.services.identity.InMemoryIdentityService import net.corda.nodeapi.internal.DEV_ROOT_CA import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.testing.core.* -import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import net.corda.testing.internal.TestingNamedCacheFactory +import net.corda.testing.internal.configureDatabase +import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import org.assertj.core.api.Assertions.assertThat import org.assertj.core.api.Assertions.assertThatIllegalArgumentException import org.junit.After diff --git a/node/src/integration-test/kotlin/net/corda/node/services/rpc/ArtemisRpcTests.kt b/node/src/integration-test/kotlin/net/corda/node/services/rpc/ArtemisRpcTests.kt index c1e3cc935b..75a2838286 100644 --- a/node/src/integration-test/kotlin/net/corda/node/services/rpc/ArtemisRpcTests.kt +++ b/node/src/integration-test/kotlin/net/corda/node/services/rpc/ArtemisRpcTests.kt @@ -23,6 +23,8 @@ import net.corda.nodeapi.internal.config.MutualSslConfiguration import net.corda.nodeapi.internal.config.User import net.corda.testing.core.SerializationEnvironmentRule import net.corda.testing.driver.PortAllocation +import net.corda.testing.internal.TestingNamedCacheFactory +import net.corda.testing.internal.fromUserList import net.corda.testing.internal.p2pSslOptions import org.apache.activemq.artemis.api.core.ActiveMQConnectionTimedOutException import org.apache.activemq.artemis.api.core.management.ActiveMQServerControl @@ -128,7 +130,7 @@ class ArtemisRpcTests { private fun InternalRPCMessagingClient.start(ops: OPS, securityManager: RPCSecurityManager, brokerControl: ActiveMQServerControl) { apply { - init(ops, securityManager) + init(ops, securityManager, TestingNamedCacheFactory()) start(brokerControl) } } diff --git a/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt b/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt index 141c92778b..950e2df204 100644 --- a/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt +++ b/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt @@ -17,6 +17,7 @@ import net.corda.core.identity.CordaX500Name import net.corda.core.identity.Party import net.corda.core.identity.PartyAndCertificate import net.corda.core.internal.FlowStateMachine +import net.corda.core.internal.NamedCacheFactory import net.corda.core.internal.VisibleForTesting import net.corda.core.internal.concurrent.map import net.corda.core.internal.concurrent.openFuture @@ -115,7 +116,7 @@ import net.corda.core.crypto.generateKeyPair as cryptoGenerateKeyPair // TODO Log warning if this node is a notary but not one of the ones specified in the network parameters, both for core and custom abstract class AbstractNode(val configuration: NodeConfiguration, val platformClock: CordaClock, - cacheFactoryPrototype: NamedCacheFactory, + cacheFactoryPrototype: BindableNamedCacheFactory, protected val versionInfo: VersionInfo, protected val serverThread: AffinityExecutor.ServiceAffinityExecutor, private val busyNodeLatch: ReusableLatch = ReusableLatch()) : SingletonSerializeAsToken() { @@ -150,8 +151,8 @@ abstract class AbstractNode(val configuration: NodeConfiguration, identityService::wellKnownPartyFromX500Name, identityService::wellKnownPartyFromAnonymous, schemaService, - configuration.dataSourceProperties - ) + configuration.dataSourceProperties, + cacheFactory) init { // TODO Break cyclic dependency identityService.database = database @@ -169,7 +170,7 @@ abstract class AbstractNode(val configuration: NodeConfiguration, val servicesForResolution = ServicesForResolutionImpl(identityService, attachments, cordappProvider, transactionStorage) @Suppress("LeakingThis") val vaultService = makeVaultService(keyManagementService, servicesForResolution, database).tokenize() - val nodeProperties = NodePropertiesPersistentStore(StubbedNodeUniqueIdProvider::value, database) + val nodeProperties = NodePropertiesPersistentStore(StubbedNodeUniqueIdProvider::value, database, cacheFactory) val flowLogicRefFactory = FlowLogicRefFactoryImpl(cordappLoader.appClassLoader) val networkMapUpdater = NetworkMapUpdater( networkMapCache, @@ -185,7 +186,7 @@ abstract class AbstractNode(val configuration: NodeConfiguration, ).closeOnStop() @Suppress("LeakingThis") val transactionVerifierService = InMemoryTransactionVerifierService(transactionVerifierWorkerCount).tokenize() - val contractUpgradeService = ContractUpgradeServiceImpl().tokenize() + val contractUpgradeService = ContractUpgradeServiceImpl(cacheFactory).tokenize() val auditService = DummyAuditService().tokenize() @Suppress("LeakingThis") protected val network: MessagingService = makeMessagingService().tokenize() @@ -999,23 +1000,12 @@ class FlowStarterImpl(private val smm: StateMachineManager, private val flowLogi class ConfigurationException(message: String) : CordaException(message) -// TODO This is no longer used by AbstractNode and can be moved elsewhere -fun configureDatabase(hikariProperties: Properties, - databaseConfig: DatabaseConfig, - wellKnownPartyFromX500Name: (CordaX500Name) -> Party?, - wellKnownPartyFromAnonymous: (AbstractParty) -> Party?, - schemaService: SchemaService = NodeSchemaService(), - internalSchemas: Set = NodeSchemaService().internalSchemas()): CordaPersistence { - val persistence = createCordaPersistence(databaseConfig, wellKnownPartyFromX500Name, wellKnownPartyFromAnonymous, schemaService, hikariProperties) - persistence.startHikariPool(hikariProperties, databaseConfig, internalSchemas) - return persistence -} - fun createCordaPersistence(databaseConfig: DatabaseConfig, wellKnownPartyFromX500Name: (CordaX500Name) -> Party?, wellKnownPartyFromAnonymous: (AbstractParty) -> Party?, schemaService: SchemaService, - hikariProperties: Properties): CordaPersistence { + hikariProperties: Properties, + cacheFactory: NamedCacheFactory): CordaPersistence { // Register the AbstractPartyDescriptor so Hibernate doesn't warn when encountering AbstractParty. Unfortunately // Hibernate warns about not being able to find a descriptor if we don't provide one, but won't use it by default // so we end up providing both descriptor and converter. We should re-examine this in later versions to see if @@ -1023,7 +1013,7 @@ fun createCordaPersistence(databaseConfig: DatabaseConfig, JavaTypeDescriptorRegistry.INSTANCE.addDescriptor(AbstractPartyDescriptor(wellKnownPartyFromX500Name, wellKnownPartyFromAnonymous)) val attributeConverters = listOf(AbstractPartyToX500NameAsStringConverter(wellKnownPartyFromX500Name, wellKnownPartyFromAnonymous)) val jdbcUrl = hikariProperties.getProperty("dataSource.url", "") - return CordaPersistence(databaseConfig, schemaService.schemaOptions.keys, jdbcUrl, attributeConverters) + return CordaPersistence(databaseConfig, schemaService.schemaOptions.keys, jdbcUrl, cacheFactory, attributeConverters) } fun CordaPersistence.startHikariPool(hikariProperties: Properties, databaseConfig: DatabaseConfig, schemas: Set, metricRegistry: MetricRegistry? = null) { diff --git a/node/src/main/kotlin/net/corda/node/internal/Node.kt b/node/src/main/kotlin/net/corda/node/internal/Node.kt index 0af843dd5b..c6cad69913 100644 --- a/node/src/main/kotlin/net/corda/node/internal/Node.kt +++ b/node/src/main/kotlin/net/corda/node/internal/Node.kt @@ -85,7 +85,7 @@ class NodeWithInfo(val node: Node, val info: NodeInfo) { open class Node(configuration: NodeConfiguration, versionInfo: VersionInfo, private val initialiseSerialization: Boolean = true, - cacheFactoryPrototype: NamedCacheFactory = DefaultNamedCacheFactory() + cacheFactoryPrototype: BindableNamedCacheFactory = DefaultNamedCacheFactory() ) : AbstractNode( configuration, createClock(configuration), @@ -223,7 +223,7 @@ open class Node(configuration: NodeConfiguration, val securityManagerConfig = configuration.security?.authService ?: SecurityConfiguration.AuthService.fromUsers(configuration.rpcUsers) - val securityManager = with(RPCSecurityManagerImpl(securityManagerConfig)) { + val securityManager = with(RPCSecurityManagerImpl(securityManagerConfig, cacheFactory)) { if (configuration.shouldStartLocalShell()) RPCSecurityManagerWithAdditionalUser(this, User(INTERNAL_SHELL_USER, INTERNAL_SHELL_USER, setOf(Permissions.all()))) else this } @@ -267,7 +267,7 @@ open class Node(configuration: NodeConfiguration, // Start up the MQ clients. internalRpcMessagingClient?.run { closeOnStop() - init(rpcOps, securityManager) + init(rpcOps, securityManager, cacheFactory) } network.closeOnStop() network.start( diff --git a/node/src/main/kotlin/net/corda/node/internal/security/RPCSecurityManagerImpl.kt b/node/src/main/kotlin/net/corda/node/internal/security/RPCSecurityManagerImpl.kt index bd7fcb9f72..5186e234b7 100644 --- a/node/src/main/kotlin/net/corda/node/internal/security/RPCSecurityManagerImpl.kt +++ b/node/src/main/kotlin/net/corda/node/internal/security/RPCSecurityManagerImpl.kt @@ -4,14 +4,13 @@ package net.corda.node.internal.security import com.github.benmanes.caffeine.cache.Cache import com.github.benmanes.caffeine.cache.Caffeine import com.google.common.primitives.Ints -import net.corda.core.context.AuthServiceId -import net.corda.core.internal.buildNamed +import net.corda.core.internal.NamedCacheFactory import net.corda.core.internal.uncheckedCast import net.corda.core.utilities.loggerFor import net.corda.node.internal.DataSourceFactory +import net.corda.node.services.config.AuthDataSourceType import net.corda.node.services.config.PasswordEncryption import net.corda.node.services.config.SecurityConfiguration -import net.corda.node.services.config.AuthDataSourceType import net.corda.nodeapi.internal.config.User import org.apache.shiro.authc.* import org.apache.shiro.authc.credential.PasswordMatcher @@ -28,22 +27,22 @@ import org.apache.shiro.realm.jdbc.JdbcRealm import org.apache.shiro.subject.PrincipalCollection import org.apache.shiro.subject.SimplePrincipalCollection import java.io.Closeable -import javax.security.auth.login.FailedLoginException import java.util.concurrent.ConcurrentHashMap -import java.util.concurrent.TimeUnit +import javax.security.auth.login.FailedLoginException + private typealias AuthServiceConfig = SecurityConfiguration.AuthService /** * Default implementation of [RPCSecurityManager] adapting * [org.apache.shiro.mgt.SecurityManager] */ -class RPCSecurityManagerImpl(config: AuthServiceConfig) : RPCSecurityManager { +class RPCSecurityManagerImpl(config: AuthServiceConfig, cacheFactory: NamedCacheFactory) : RPCSecurityManager { override val id = config.id private val manager: DefaultSecurityManager init { - manager = buildImpl(config) + manager = buildImpl(config, cacheFactory) } @Throws(FailedLoginException::class) @@ -75,14 +74,8 @@ class RPCSecurityManagerImpl(config: AuthServiceConfig) : RPCSecurityManager { private val logger = loggerFor() - /** - * Instantiate RPCSecurityManager initialised with users data from a list of [User] - */ - fun fromUserList(id: AuthServiceId, users: List) = - RPCSecurityManagerImpl(AuthServiceConfig.fromUsers(users).copy(id = id)) - // Build internal Shiro securityManager instance - private fun buildImpl(config: AuthServiceConfig): DefaultSecurityManager { + private fun buildImpl(config: AuthServiceConfig, cacheFactory: NamedCacheFactory): DefaultSecurityManager { val realm = when (config.dataSource.type) { AuthDataSourceType.DB -> { logger.info("Constructing DB-backed security data source: ${config.dataSource.connection}") @@ -98,7 +91,8 @@ class RPCSecurityManagerImpl(config: AuthServiceConfig) : RPCSecurityManager { it.cacheManager = config.options?.cache?.let { CaffeineCacheManager( timeToLiveSeconds = it.expireAfterSecs, - maxSize = it.maxEntries) + maxSize = it.maxEntries, + cacheFactory = cacheFactory) } } } @@ -294,7 +288,8 @@ private fun Cache.toShiroCache() = object : ShiroCache * cache implementation in [com.github.benmanes.caffeine.cache.Cache] */ private class CaffeineCacheManager(val maxSize: Long, - val timeToLiveSeconds: Long) : CacheManager { + val timeToLiveSeconds: Long, + val cacheFactory: NamedCacheFactory) : CacheManager { private val instances = ConcurrentHashMap>() @@ -306,11 +301,7 @@ private class CaffeineCacheManager(val maxSize: Long, private fun buildCache(name: String): ShiroCache { logger.info("Constructing cache '$name' with maximumSize=$maxSize, TTL=${timeToLiveSeconds}s") - return Caffeine.newBuilder() - .expireAfterWrite(timeToLiveSeconds, TimeUnit.SECONDS) - .maximumSize(maxSize) - .buildNamed("RPCSecurityManagerShiroCache_$name") - .toShiroCache() + return cacheFactory.buildNamed(Caffeine.newBuilder(), "RPCSecurityManagerShiroCache_$name").toShiroCache() } companion object { diff --git a/node/src/main/kotlin/net/corda/node/services/api/ServiceHubInternal.kt b/node/src/main/kotlin/net/corda/node/services/api/ServiceHubInternal.kt index 8d5583544f..7ccd6db8b9 100644 --- a/node/src/main/kotlin/net/corda/node/services/api/ServiceHubInternal.kt +++ b/node/src/main/kotlin/net/corda/node/services/api/ServiceHubInternal.kt @@ -6,6 +6,7 @@ import net.corda.core.crypto.SecureHash import net.corda.core.flows.FlowLogic import net.corda.core.flows.StateMachineRunId import net.corda.core.internal.FlowStateMachine +import net.corda.core.internal.NamedCacheFactory import net.corda.core.internal.concurrent.OpenFuture import net.corda.core.messaging.DataFeed import net.corda.core.messaging.StateMachineTransactionMapping @@ -25,7 +26,6 @@ import net.corda.node.services.network.NetworkMapUpdater import net.corda.node.services.persistence.AttachmentStorageInternal import net.corda.node.services.statemachine.ExternalEvent import net.corda.node.services.statemachine.FlowStateMachineImpl -import net.corda.node.utilities.NamedCacheFactory import net.corda.nodeapi.internal.persistence.CordaPersistence import java.security.PublicKey diff --git a/node/src/main/kotlin/net/corda/node/services/identity/PersistentIdentityService.kt b/node/src/main/kotlin/net/corda/node/services/identity/PersistentIdentityService.kt index 46f29b1ffd..43de48c6b7 100644 --- a/node/src/main/kotlin/net/corda/node/services/identity/PersistentIdentityService.kt +++ b/node/src/main/kotlin/net/corda/node/services/identity/PersistentIdentityService.kt @@ -2,6 +2,7 @@ package net.corda.node.services.identity import net.corda.core.crypto.SecureHash import net.corda.core.identity.* +import net.corda.core.internal.NamedCacheFactory import net.corda.core.internal.hash import net.corda.core.node.services.UnknownAnonymousPartyException import net.corda.core.serialization.SingletonSerializeAsToken @@ -10,7 +11,6 @@ import net.corda.core.utilities.contextLogger import net.corda.core.utilities.debug import net.corda.node.services.api.IdentityServiceInternal import net.corda.node.utilities.AppendOnlyPersistentMap -import net.corda.node.utilities.NamedCacheFactory import net.corda.nodeapi.internal.crypto.X509CertificateFactory import net.corda.nodeapi.internal.crypto.x509Certificates import net.corda.nodeapi.internal.persistence.CordaPersistence diff --git a/node/src/main/kotlin/net/corda/node/services/keys/PersistentKeyManagementService.kt b/node/src/main/kotlin/net/corda/node/services/keys/PersistentKeyManagementService.kt index 835a751c73..574ca82aca 100644 --- a/node/src/main/kotlin/net/corda/node/services/keys/PersistentKeyManagementService.kt +++ b/node/src/main/kotlin/net/corda/node/services/keys/PersistentKeyManagementService.kt @@ -2,11 +2,11 @@ package net.corda.node.services.keys import net.corda.core.crypto.* import net.corda.core.identity.PartyAndCertificate +import net.corda.core.internal.NamedCacheFactory import net.corda.core.serialization.SingletonSerializeAsToken import net.corda.core.utilities.MAX_HASH_HEX_SIZE import net.corda.node.services.identity.PersistentIdentityService import net.corda.node.utilities.AppendOnlyPersistentMap -import net.corda.node.utilities.NamedCacheFactory import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.NODE_DATABASE_PREFIX import org.apache.commons.lang.ArrayUtils.EMPTY_BYTE_ARRAY diff --git a/node/src/main/kotlin/net/corda/node/services/messaging/InternalRPCMessagingClient.kt b/node/src/main/kotlin/net/corda/node/services/messaging/InternalRPCMessagingClient.kt index a484261ba5..1390e14236 100644 --- a/node/src/main/kotlin/net/corda/node/services/messaging/InternalRPCMessagingClient.kt +++ b/node/src/main/kotlin/net/corda/node/services/messaging/InternalRPCMessagingClient.kt @@ -1,6 +1,7 @@ package net.corda.node.services.messaging import net.corda.core.identity.CordaX500Name +import net.corda.core.internal.NamedCacheFactory import net.corda.core.messaging.RPCOps import net.corda.core.serialization.SingletonSerializeAsToken import net.corda.core.serialization.internal.nodeSerializationEnv @@ -20,7 +21,7 @@ class InternalRPCMessagingClient(val sslConfig: MutualSslConfiguration, val serv private var locator: ServerLocator? = null private var rpcServer: RPCServer? = null - fun init(rpcOps: RPCOps, securityManager: RPCSecurityManager) = synchronized(this) { + fun init(rpcOps: RPCOps, securityManager: RPCSecurityManager, cacheFactory: NamedCacheFactory) = synchronized(this) { val tcpTransport = ArtemisTcpTransport.rpcInternalClientTcpTransport(serverAddress, sslConfig) locator = ActiveMQClient.createServerLocatorWithoutHA(tcpTransport).apply { @@ -32,7 +33,7 @@ class InternalRPCMessagingClient(val sslConfig: MutualSslConfiguration, val serv isUseGlobalPools = nodeSerializationEnv != null } - rpcServer = RPCServer(rpcOps, NODE_RPC_USER, NODE_RPC_USER, locator!!, securityManager, nodeName, rpcServerConfiguration) + rpcServer = RPCServer(rpcOps, NODE_RPC_USER, NODE_RPC_USER, locator!!, securityManager, nodeName, rpcServerConfiguration, cacheFactory) } fun start(serverControl: ActiveMQServerControl) = synchronized(this) { diff --git a/node/src/main/kotlin/net/corda/node/services/messaging/P2PMessageDeduplicator.kt b/node/src/main/kotlin/net/corda/node/services/messaging/P2PMessageDeduplicator.kt index b8c29d8955..a5c2975e2a 100644 --- a/node/src/main/kotlin/net/corda/node/services/messaging/P2PMessageDeduplicator.kt +++ b/node/src/main/kotlin/net/corda/node/services/messaging/P2PMessageDeduplicator.kt @@ -2,9 +2,9 @@ package net.corda.node.services.messaging import net.corda.core.crypto.SecureHash import net.corda.core.identity.CordaX500Name +import net.corda.core.internal.NamedCacheFactory import net.corda.node.services.statemachine.DeduplicationId import net.corda.node.utilities.AppendOnlyPersistentMap -import net.corda.node.utilities.NamedCacheFactory import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.NODE_DATABASE_PREFIX import java.time.Instant diff --git a/node/src/main/kotlin/net/corda/node/services/messaging/P2PMessagingClient.kt b/node/src/main/kotlin/net/corda/node/services/messaging/P2PMessagingClient.kt index 13b96c61b0..b2f54fc909 100644 --- a/node/src/main/kotlin/net/corda/node/services/messaging/P2PMessagingClient.kt +++ b/node/src/main/kotlin/net/corda/node/services/messaging/P2PMessagingClient.kt @@ -4,6 +4,7 @@ import co.paralleluniverse.fibers.Suspendable import com.codahale.metrics.MetricRegistry import net.corda.core.crypto.toStringShort import net.corda.core.identity.CordaX500Name +import net.corda.core.internal.NamedCacheFactory import net.corda.core.internal.ThreadBox import net.corda.core.messaging.CordaRPCOps import net.corda.core.messaging.MessageRecipients @@ -27,7 +28,6 @@ import net.corda.node.services.statemachine.DeduplicationId import net.corda.node.services.statemachine.ExternalEvent import net.corda.node.services.statemachine.SenderDeduplicationId import net.corda.node.utilities.AffinityExecutor -import net.corda.node.utilities.NamedCacheFactory import net.corda.nodeapi.internal.ArtemisMessagingComponent import net.corda.nodeapi.internal.ArtemisMessagingComponent.* import net.corda.nodeapi.internal.ArtemisMessagingComponent.Companion.BRIDGE_CONTROL diff --git a/node/src/main/kotlin/net/corda/node/services/messaging/RPCServer.kt b/node/src/main/kotlin/net/corda/node/services/messaging/RPCServer.kt index 7c9ca38e93..d8e144e17b 100644 --- a/node/src/main/kotlin/net/corda/node/services/messaging/RPCServer.kt +++ b/node/src/main/kotlin/net/corda/node/services/messaging/RPCServer.kt @@ -13,7 +13,7 @@ import net.corda.core.context.Trace import net.corda.core.context.Trace.InvocationId import net.corda.core.identity.CordaX500Name import net.corda.core.internal.LifeCycle -import net.corda.core.internal.buildNamed +import net.corda.core.internal.NamedCacheFactory import net.corda.core.messaging.RPCOps import net.corda.core.serialization.SerializationContext import net.corda.core.serialization.SerializationDefaults @@ -33,13 +33,8 @@ import net.corda.nodeapi.internal.persistence.contextDatabase import net.corda.nodeapi.internal.persistence.contextDatabaseOrNull import org.apache.activemq.artemis.api.core.Message import org.apache.activemq.artemis.api.core.SimpleString +import org.apache.activemq.artemis.api.core.client.* import org.apache.activemq.artemis.api.core.client.ActiveMQClient.DEFAULT_ACK_BATCH_SIZE -import org.apache.activemq.artemis.api.core.client.ClientConsumer -import org.apache.activemq.artemis.api.core.client.ClientMessage -import org.apache.activemq.artemis.api.core.client.ClientProducer -import org.apache.activemq.artemis.api.core.client.ClientSession -import org.apache.activemq.artemis.api.core.client.ClientSessionFactory -import org.apache.activemq.artemis.api.core.client.ServerLocator import org.apache.activemq.artemis.api.core.management.ActiveMQServerControl import org.apache.activemq.artemis.api.core.management.CoreNotificationType import org.apache.activemq.artemis.api.core.management.ManagementHelper @@ -49,12 +44,7 @@ import java.lang.reflect.InvocationTargetException import java.lang.reflect.Method import java.time.Duration import java.util.* -import java.util.concurrent.ConcurrentHashMap -import java.util.concurrent.Executors -import java.util.concurrent.LinkedBlockingQueue -import java.util.concurrent.ScheduledExecutorService -import java.util.concurrent.ScheduledFuture -import java.util.concurrent.TimeUnit +import java.util.concurrent.* import kotlin.concurrent.thread private typealias ObservableSubscriptionMap = Cache @@ -91,7 +81,8 @@ class RPCServer( private val serverLocator: ServerLocator, private val securityManager: RPCSecurityManager, private val nodeLegalName: CordaX500Name, - private val rpcConfiguration: RPCServerConfiguration + private val rpcConfiguration: RPCServerConfiguration, + private val cacheFactory: NamedCacheFactory ) { private companion object { private val log = contextLogger() @@ -136,7 +127,7 @@ class RPCServer( private val responseMessageBuffer = ConcurrentHashMap() private val sendJobQueue = LinkedBlockingQueue() - private val deduplicationChecker = DeduplicationChecker(rpcConfiguration.deduplicationCacheExpiry) + private val deduplicationChecker = DeduplicationChecker(rpcConfiguration.deduplicationCacheExpiry, cacheFactory = cacheFactory) private var deduplicationIdentity: String? = null init { @@ -154,7 +145,7 @@ class RPCServer( log.debug { "Unsubscribing from Observable with id $key because of $cause" } value!!.subscription.unsubscribe() } - return Caffeine.newBuilder().removalListener(onObservableRemove).executor(SameThreadExecutor.getExecutor()).buildNamed("RPCServer_observableSubscription") + return cacheFactory.buildNamed(Caffeine.newBuilder().removalListener(onObservableRemove).executor(SameThreadExecutor.getExecutor()), "RPCServer_observableSubscription") } fun start(activeMqServerControl: ActiveMQServerControl) { diff --git a/node/src/main/kotlin/net/corda/node/services/network/PersistentNetworkMapCache.kt b/node/src/main/kotlin/net/corda/node/services/network/PersistentNetworkMapCache.kt index f873120c32..4eea7af890 100644 --- a/node/src/main/kotlin/net/corda/node/services/network/PersistentNetworkMapCache.kt +++ b/node/src/main/kotlin/net/corda/node/services/network/PersistentNetworkMapCache.kt @@ -6,6 +6,7 @@ import net.corda.core.identity.AbstractParty import net.corda.core.identity.CordaX500Name import net.corda.core.identity.Party import net.corda.core.identity.PartyAndCertificate +import net.corda.core.internal.NamedCacheFactory import net.corda.core.internal.bufferUntilSubscribed import net.corda.core.internal.concurrent.OpenFuture import net.corda.core.internal.concurrent.openFuture @@ -23,7 +24,6 @@ import net.corda.core.utilities.contextLogger import net.corda.core.utilities.debug import net.corda.node.internal.schemas.NodeInfoSchemaV1 import net.corda.node.services.api.NetworkMapCacheInternal -import net.corda.node.utilities.NamedCacheFactory import net.corda.node.utilities.NonInvalidatingCache import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.bufferUntilDatabaseCommit diff --git a/node/src/main/kotlin/net/corda/node/services/persistence/DBTransactionStorage.kt b/node/src/main/kotlin/net/corda/node/services/persistence/DBTransactionStorage.kt index 034e91a2cf..2330d42571 100644 --- a/node/src/main/kotlin/net/corda/node/services/persistence/DBTransactionStorage.kt +++ b/node/src/main/kotlin/net/corda/node/services/persistence/DBTransactionStorage.kt @@ -3,6 +3,7 @@ package net.corda.node.services.persistence import net.corda.core.concurrent.CordaFuture import net.corda.core.crypto.SecureHash import net.corda.core.crypto.TransactionSignature +import net.corda.core.internal.NamedCacheFactory import net.corda.core.internal.ThreadBox import net.corda.core.internal.VisibleForTesting import net.corda.core.internal.bufferUntilSubscribed @@ -15,7 +16,6 @@ import net.corda.core.transactions.SignedTransaction import net.corda.node.services.api.WritableTransactionStorage import net.corda.node.services.statemachine.FlowStateMachineImpl import net.corda.node.utilities.AppendOnlyPersistentMapBase -import net.corda.node.utilities.NamedCacheFactory import net.corda.node.utilities.WeightBasedAppendOnlyPersistentMap import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.NODE_DATABASE_PREFIX diff --git a/node/src/main/kotlin/net/corda/node/services/persistence/NodeAttachmentService.kt b/node/src/main/kotlin/net/corda/node/services/persistence/NodeAttachmentService.kt index ca96ef1ad5..31f937f62b 100644 --- a/node/src/main/kotlin/net/corda/node/services/persistence/NodeAttachmentService.kt +++ b/node/src/main/kotlin/net/corda/node/services/persistence/NodeAttachmentService.kt @@ -19,7 +19,6 @@ import net.corda.core.node.services.vault.AttachmentSort import net.corda.core.serialization.* import net.corda.core.utilities.contextLogger import net.corda.node.services.vault.HibernateAttachmentQueryCriteriaParser -import net.corda.node.utilities.NamedCacheFactory import net.corda.node.utilities.NonInvalidatingCache import net.corda.node.utilities.NonInvalidatingWeightBasedCache import net.corda.nodeapi.exceptions.DuplicateAttachmentException diff --git a/node/src/main/kotlin/net/corda/node/services/persistence/NodePropertiesPersistentStore.kt b/node/src/main/kotlin/net/corda/node/services/persistence/NodePropertiesPersistentStore.kt index 2c48dfd6f9..386415f7c9 100644 --- a/node/src/main/kotlin/net/corda/node/services/persistence/NodePropertiesPersistentStore.kt +++ b/node/src/main/kotlin/net/corda/node/services/persistence/NodePropertiesPersistentStore.kt @@ -1,5 +1,6 @@ package net.corda.node.services.persistence +import net.corda.core.internal.NamedCacheFactory import net.corda.core.utilities.contextLogger import net.corda.core.utilities.debug import net.corda.node.services.api.NodePropertiesStore @@ -17,12 +18,12 @@ import javax.persistence.Table /** * Simple node properties key value store in DB. */ -class NodePropertiesPersistentStore(readPhysicalNodeId: () -> String, database: CordaPersistence) : NodePropertiesStore { +class NodePropertiesPersistentStore(readPhysicalNodeId: () -> String, database: CordaPersistence, cacheFactory: NamedCacheFactory) : NodePropertiesStore { private companion object { val logger = contextLogger() } - override val flowsDrainingMode = FlowsDrainingModeOperationsImpl(readPhysicalNodeId, database, logger) + override val flowsDrainingMode = FlowsDrainingModeOperationsImpl(readPhysicalNodeId, database, logger, cacheFactory) fun start() { flowsDrainingMode.map.preload() @@ -40,7 +41,7 @@ class NodePropertiesPersistentStore(readPhysicalNodeId: () -> String, database: ) } -class FlowsDrainingModeOperationsImpl(readPhysicalNodeId: () -> String, private val persistence: CordaPersistence, logger: Logger) : FlowsDrainingModeOperations { +class FlowsDrainingModeOperationsImpl(readPhysicalNodeId: () -> String, private val persistence: CordaPersistence, logger: Logger, cacheFactory: NamedCacheFactory) : FlowsDrainingModeOperations { private val nodeSpecificFlowsExecutionModeKey = "${readPhysicalNodeId()}_flowsExecutionMode" init { @@ -52,7 +53,8 @@ class FlowsDrainingModeOperationsImpl(readPhysicalNodeId: () -> String, private { key -> key }, { entity -> entity.key to entity.value!! }, NodePropertiesPersistentStore::DBNodeProperty, - NodePropertiesPersistentStore.DBNodeProperty::class.java + NodePropertiesPersistentStore.DBNodeProperty::class.java, + cacheFactory ) override val values = PublishSubject.create>()!! diff --git a/node/src/main/kotlin/net/corda/node/services/transactions/PersistentUniquenessProvider.kt b/node/src/main/kotlin/net/corda/node/services/transactions/PersistentUniquenessProvider.kt index b6c8ebf437..122e10d3c3 100644 --- a/node/src/main/kotlin/net/corda/node/services/transactions/PersistentUniquenessProvider.kt +++ b/node/src/main/kotlin/net/corda/node/services/transactions/PersistentUniquenessProvider.kt @@ -9,6 +9,7 @@ import net.corda.core.flows.NotarisationRequestSignature import net.corda.core.flows.NotaryError import net.corda.core.flows.StateConsumptionDetails import net.corda.core.identity.Party +import net.corda.core.internal.NamedCacheFactory import net.corda.core.internal.concurrent.OpenFuture import net.corda.core.internal.concurrent.openFuture import net.corda.core.internal.notary.AsyncUniquenessProvider @@ -22,7 +23,6 @@ import net.corda.core.serialization.serialize import net.corda.core.utilities.contextLogger import net.corda.core.utilities.debug import net.corda.node.utilities.AppendOnlyPersistentMap -import net.corda.node.utilities.NamedCacheFactory import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.NODE_DATABASE_PREFIX import net.corda.nodeapi.internal.persistence.currentDBSession diff --git a/node/src/main/kotlin/net/corda/node/services/upgrade/ContractUpgradeServiceImpl.kt b/node/src/main/kotlin/net/corda/node/services/upgrade/ContractUpgradeServiceImpl.kt index e6924dd3c7..9aa5739a61 100644 --- a/node/src/main/kotlin/net/corda/node/services/upgrade/ContractUpgradeServiceImpl.kt +++ b/node/src/main/kotlin/net/corda/node/services/upgrade/ContractUpgradeServiceImpl.kt @@ -2,6 +2,7 @@ package net.corda.node.services.upgrade import net.corda.core.contracts.StateRef import net.corda.core.contracts.UpgradedContract +import net.corda.core.internal.NamedCacheFactory import net.corda.core.node.services.ContractUpgradeService import net.corda.core.serialization.SingletonSerializeAsToken import net.corda.node.utilities.PersistentMap @@ -11,7 +12,7 @@ import javax.persistence.Entity import javax.persistence.Id import javax.persistence.Table -class ContractUpgradeServiceImpl : ContractUpgradeService, SingletonSerializeAsToken() { +class ContractUpgradeServiceImpl(cacheFactory: NamedCacheFactory) : ContractUpgradeService, SingletonSerializeAsToken() { @Entity @Table(name = "${NODE_DATABASE_PREFIX}contract_upgrades") @@ -26,7 +27,7 @@ class ContractUpgradeServiceImpl : ContractUpgradeService, SingletonSerializeAsT ) private companion object { - fun createContractUpgradesMap(): PersistentMap { + fun createContractUpgradesMap(cacheFactory: NamedCacheFactory): PersistentMap { return PersistentMap( "ContractUpgradeService_upgrades", toPersistentEntityKey = { it }, @@ -37,12 +38,13 @@ class ContractUpgradeServiceImpl : ContractUpgradeService, SingletonSerializeAsT upgradedContractClassName = value } }, - persistentEntityClass = DBContractUpgrade::class.java + persistentEntityClass = DBContractUpgrade::class.java, + cacheFactory = cacheFactory ) } } - private val authorisedUpgrade = createContractUpgradesMap() + private val authorisedUpgrade = createContractUpgradesMap(cacheFactory) fun start() { authorisedUpgrade.preload() diff --git a/node/src/main/kotlin/net/corda/node/utilities/AppendOnlyPersistentMap.kt b/node/src/main/kotlin/net/corda/node/utilities/AppendOnlyPersistentMap.kt index 81f080e425..532a71f497 100644 --- a/node/src/main/kotlin/net/corda/node/utilities/AppendOnlyPersistentMap.kt +++ b/node/src/main/kotlin/net/corda/node/utilities/AppendOnlyPersistentMap.kt @@ -2,6 +2,7 @@ package net.corda.node.utilities import com.github.benmanes.caffeine.cache.LoadingCache import com.github.benmanes.caffeine.cache.Weigher +import net.corda.core.internal.NamedCacheFactory import net.corda.core.utilities.contextLogger import net.corda.nodeapi.internal.persistence.DatabaseTransaction import net.corda.nodeapi.internal.persistence.contextTransaction @@ -12,7 +13,6 @@ import java.util.concurrent.ConcurrentHashMap import java.util.concurrent.atomic.AtomicBoolean import java.util.concurrent.atomic.AtomicReference - /** * Implements a caching layer on top of an *append-only* table accessed via Hibernate mapping. Note that if the same key is [set] twice, * typically this will result in a duplicate insert if this is racing with another transaction. The flow framework will then retry. diff --git a/node/src/main/kotlin/net/corda/node/utilities/NodeNamedCache.kt b/node/src/main/kotlin/net/corda/node/utilities/NodeNamedCache.kt index d11c9667c2..5c963ac80d 100644 --- a/node/src/main/kotlin/net/corda/node/utilities/NodeNamedCache.kt +++ b/node/src/main/kotlin/net/corda/node/utilities/NodeNamedCache.kt @@ -5,50 +5,77 @@ import com.github.benmanes.caffeine.cache.Cache import com.github.benmanes.caffeine.cache.CacheLoader import com.github.benmanes.caffeine.cache.Caffeine import com.github.benmanes.caffeine.cache.LoadingCache -import net.corda.core.internal.buildNamed +import net.corda.core.internal.NamedCacheFactory import net.corda.core.serialization.SerializeAsToken import net.corda.core.serialization.SingletonSerializeAsToken import net.corda.node.services.config.NodeConfiguration +import java.util.concurrent.TimeUnit /** - * Allow passing metrics and config to caching implementations. + * Allow passing metrics and config to caching implementations. This is needs to be distinct from [NamedCacheFactory] + * to avoid deterministic serialization from seeing metrics and config on method signatures. */ -interface NamedCacheFactory : SerializeAsToken { +interface BindableNamedCacheFactory : NamedCacheFactory, SerializeAsToken { /** * Build a new cache factory of the same type that incorporates metrics. */ - fun bindWithMetrics(metricRegistry: MetricRegistry): NamedCacheFactory + fun bindWithMetrics(metricRegistry: MetricRegistry): BindableNamedCacheFactory /** * Build a new cache factory of the same type that incorporates the associated configuration. */ - fun bindWithConfig(nodeConfiguration: NodeConfiguration): NamedCacheFactory - - fun buildNamed(caffeine: Caffeine, name: String): Cache - fun buildNamed(caffeine: Caffeine, name: String, loader: CacheLoader): LoadingCache + fun bindWithConfig(nodeConfiguration: NodeConfiguration): BindableNamedCacheFactory } -class DefaultNamedCacheFactory private constructor(private val metricRegistry: MetricRegistry?, private val nodeConfiguration: NodeConfiguration?) : NamedCacheFactory, SingletonSerializeAsToken() { +open class DefaultNamedCacheFactory private constructor(private val metricRegistry: MetricRegistry?, private val nodeConfiguration: NodeConfiguration?) : BindableNamedCacheFactory, SingletonSerializeAsToken() { constructor() : this(null, null) - override fun bindWithMetrics(metricRegistry: MetricRegistry): NamedCacheFactory = DefaultNamedCacheFactory(metricRegistry, this.nodeConfiguration) - override fun bindWithConfig(nodeConfiguration: NodeConfiguration): NamedCacheFactory = DefaultNamedCacheFactory(this.metricRegistry, nodeConfiguration) + override fun bindWithMetrics(metricRegistry: MetricRegistry): BindableNamedCacheFactory = DefaultNamedCacheFactory(metricRegistry, this.nodeConfiguration) + override fun bindWithConfig(nodeConfiguration: NodeConfiguration): BindableNamedCacheFactory = DefaultNamedCacheFactory(this.metricRegistry, nodeConfiguration) - override fun buildNamed(caffeine: Caffeine, name: String): Cache { + protected fun configuredForNamed(caffeine: Caffeine, name: String): Caffeine { + return with(nodeConfiguration!!) { + when { + name.startsWith("RPCSecurityManagerShiroCache_") -> with(security?.authService?.options?.cache!!) { caffeine.maximumSize(maxEntries).expireAfterWrite(expireAfterSecs, TimeUnit.SECONDS) } + name == "RPCServer_observableSubscription" -> caffeine + name == "RpcClientProxyHandler_rpcObservable" -> caffeine + name == "SerializationScheme_attachmentClassloader" -> caffeine + name == "HibernateConfiguration_sessionFactories" -> caffeine.maximumSize(database.mappedSchemaCacheSize) + name == "DBTransactionStorage_transactions" -> caffeine.maximumWeight(transactionCacheSizeBytes) + name == "NodeAttachmentService_attachmentContent" -> caffeine.maximumWeight(attachmentContentCacheSizeBytes) + name == "NodeAttachmentService_attachmentPresence" -> caffeine.maximumSize(attachmentCacheBound) + name == "PersistentIdentityService_partyByKey" -> caffeine.maximumSize(defaultCacheSize) + name == "PersistentIdentityService_partyByName" -> caffeine.maximumSize(defaultCacheSize) + name == "PersistentNetworkMap_nodesByKey" -> caffeine.maximumSize(defaultCacheSize) + name == "PersistentNetworkMap_idByLegalName" -> caffeine.maximumSize(defaultCacheSize) + name == "PersistentKeyManagementService_keys" -> caffeine.maximumSize(defaultCacheSize) + name == "FlowDrainingMode_nodeProperties" -> caffeine.maximumSize(defaultCacheSize) + name == "ContractUpgradeService_upgrades" -> caffeine.maximumSize(defaultCacheSize) + name == "PersistentUniquenessProvider_transactions" -> caffeine.maximumSize(defaultCacheSize) + name == "P2PMessageDeduplicator_processedMessages" -> caffeine.maximumSize(defaultCacheSize) + name == "DeduplicationChecker_watermark" -> caffeine + name == "BFTNonValidatingNotaryService_transactions" -> caffeine.maximumSize(defaultCacheSize) + name == "RaftUniquenessProvider_transactions" -> caffeine.maximumSize(defaultCacheSize) + else -> throw IllegalArgumentException("Unexpected cache name $name. Did you add a new cache?") + } + } + } + + protected fun checkState(name: String) { + checkCacheName(name) checkNotNull(metricRegistry) checkNotNull(nodeConfiguration) - return caffeine.maximumSize(1024).buildNamed(name) + } + + override fun buildNamed(caffeine: Caffeine, name: String): Cache { + checkState(name) + return configuredForNamed(caffeine, name).build() } override fun buildNamed(caffeine: Caffeine, name: String, loader: CacheLoader): LoadingCache { - checkNotNull(metricRegistry) - checkNotNull(nodeConfiguration) - val configuredCaffeine = when (name) { - "DBTransactionStorage_transactions" -> caffeine.maximumWeight(nodeConfiguration!!.transactionCacheSizeBytes) - "NodeAttachmentService_attachmentContent" -> caffeine.maximumWeight(nodeConfiguration!!.attachmentContentCacheSizeBytes) - "NodeAttachmentService_attachmentPresence" -> caffeine.maximumSize(nodeConfiguration!!.attachmentCacheBound) - else -> caffeine.maximumSize(1024) - } - return configuredCaffeine.buildNamed(name, loader) + checkState(name) + return configuredForNamed(caffeine, name).build(loader) } + + protected val defaultCacheSize = 1024L } \ No newline at end of file diff --git a/node/src/main/kotlin/net/corda/node/utilities/NonInvalidatingCache.kt b/node/src/main/kotlin/net/corda/node/utilities/NonInvalidatingCache.kt index 2cf4904282..25688233c2 100644 --- a/node/src/main/kotlin/net/corda/node/utilities/NonInvalidatingCache.kt +++ b/node/src/main/kotlin/net/corda/node/utilities/NonInvalidatingCache.kt @@ -4,6 +4,7 @@ import com.github.benmanes.caffeine.cache.CacheLoader import com.github.benmanes.caffeine.cache.Caffeine import com.github.benmanes.caffeine.cache.LoadingCache import com.github.benmanes.caffeine.cache.Weigher +import net.corda.core.internal.NamedCacheFactory class NonInvalidatingCache private constructor( val cache: LoadingCache diff --git a/node/src/main/kotlin/net/corda/node/utilities/NonInvalidatingUnboundCache.kt b/node/src/main/kotlin/net/corda/node/utilities/NonInvalidatingUnboundCache.kt index f16ec74f8f..634189a7b7 100644 --- a/node/src/main/kotlin/net/corda/node/utilities/NonInvalidatingUnboundCache.kt +++ b/node/src/main/kotlin/net/corda/node/utilities/NonInvalidatingUnboundCache.kt @@ -5,21 +5,21 @@ import com.github.benmanes.caffeine.cache.CacheLoader import com.github.benmanes.caffeine.cache.Caffeine import com.github.benmanes.caffeine.cache.LoadingCache import com.github.benmanes.caffeine.cache.RemovalListener -import net.corda.core.internal.buildNamed +import net.corda.core.internal.NamedCacheFactory class NonInvalidatingUnboundCache private constructor( val cache: LoadingCache ) : LoadingCache by cache { - constructor(name: String, loadFunction: (K) -> V, removalListener: RemovalListener = RemovalListener { _, _, _ -> }, + constructor(name: String, cacheFactory: NamedCacheFactory, loadFunction: (K) -> V, removalListener: RemovalListener = RemovalListener { _, _, _ -> }, keysToPreload: () -> Iterable = { emptyList() }) : - this(buildCache(name, loadFunction, removalListener, keysToPreload)) + this(buildCache(name, cacheFactory, loadFunction, removalListener, keysToPreload)) private companion object { - private fun buildCache(name: String, loadFunction: (K) -> V, removalListener: RemovalListener, + private fun buildCache(name: String, cacheFactory: NamedCacheFactory, loadFunction: (K) -> V, removalListener: RemovalListener, keysToPreload: () -> Iterable): LoadingCache { val builder = Caffeine.newBuilder().removalListener(removalListener).executor(SameThreadExecutor.getExecutor()) - return builder.buildNamed(name, NonInvalidatingCacheLoader(loadFunction)).apply { + return cacheFactory.buildNamed(builder, name, NonInvalidatingCacheLoader(loadFunction)).apply { getAll(keysToPreload()) } } diff --git a/node/src/main/kotlin/net/corda/node/utilities/PersistentMap.kt b/node/src/main/kotlin/net/corda/node/utilities/PersistentMap.kt index 2f04ea7bbe..a006ee9883 100644 --- a/node/src/main/kotlin/net/corda/node/utilities/PersistentMap.kt +++ b/node/src/main/kotlin/net/corda/node/utilities/PersistentMap.kt @@ -2,6 +2,7 @@ package net.corda.node.utilities import com.github.benmanes.caffeine.cache.RemovalCause import com.github.benmanes.caffeine.cache.RemovalListener +import net.corda.core.internal.NamedCacheFactory import net.corda.core.utilities.contextLogger import net.corda.nodeapi.internal.persistence.currentDBSession import java.util.* @@ -14,7 +15,8 @@ class PersistentMap( val toPersistentEntityKey: (K) -> EK, val fromPersistentEntity: (E) -> Pair, val toPersistentEntity: (key: K, value: V) -> E, - val persistentEntityClass: Class + val persistentEntityClass: Class, + cacheFactory: NamedCacheFactory ) : MutableMap, AbstractMap() { private companion object { @@ -24,7 +26,8 @@ class PersistentMap( private val cache = NonInvalidatingUnboundCache( name, loadFunction = { key -> Optional.ofNullable(loadValue(key)) }, - removalListener = ExplicitRemoval(toPersistentEntityKey, persistentEntityClass) + removalListener = ExplicitRemoval(toPersistentEntityKey, persistentEntityClass), + cacheFactory = cacheFactory ) /** Preload to allow [all] to take data only from the cache (cache is unbound) */ diff --git a/node/src/test/kotlin/net/corda/node/CordaRPCOpsImplTest.kt b/node/src/test/kotlin/net/corda/node/CordaRPCOpsImplTest.kt index 8c8c794378..8d25b1ac66 100644 --- a/node/src/test/kotlin/net/corda/node/CordaRPCOpsImplTest.kt +++ b/node/src/test/kotlin/net/corda/node/CordaRPCOpsImplTest.kt @@ -38,10 +38,11 @@ import net.corda.testing.core.ALICE_NAME import net.corda.testing.core.expect import net.corda.testing.core.expectEvents import net.corda.testing.core.sequence -import net.corda.testing.node.internal.cordappsForPackages +import net.corda.testing.internal.fromUserList import net.corda.testing.node.internal.InternalMockNetwork import net.corda.testing.node.internal.InternalMockNodeParameters import net.corda.testing.node.internal.TestStartedNode +import net.corda.testing.node.internal.cordappsForPackages import net.corda.testing.node.testActor import org.apache.commons.io.IOUtils import org.assertj.core.api.Assertions.* diff --git a/node/src/test/kotlin/net/corda/node/internal/AbstractNodeTests.kt b/node/src/test/kotlin/net/corda/node/internal/AbstractNodeTests.kt index 7e92e706b6..d4e6a24186 100644 --- a/node/src/test/kotlin/net/corda/node/internal/AbstractNodeTests.kt +++ b/node/src/test/kotlin/net/corda/node/internal/AbstractNodeTests.kt @@ -7,7 +7,7 @@ import net.corda.core.utilities.contextLogger import net.corda.core.utilities.getOrThrow import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.testing.common.internal.relaxedThoroughness -import net.corda.testing.internal.rigorousMock +import net.corda.testing.internal.configureDatabase import net.corda.testing.node.internal.ProcessUtilities.startJavaProcess import org.junit.Rule import org.junit.Test diff --git a/node/src/test/kotlin/net/corda/node/internal/NodeTest.kt b/node/src/test/kotlin/net/corda/node/internal/NodeTest.kt index 6134aa348f..48a266ce31 100644 --- a/node/src/test/kotlin/net/corda/node/internal/NodeTest.kt +++ b/node/src/test/kotlin/net/corda/node/internal/NodeTest.kt @@ -18,6 +18,7 @@ import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.testing.core.ALICE_NAME import net.corda.testing.core.SerializationEnvironmentRule +import net.corda.testing.internal.configureDatabase import net.corda.testing.internal.createNodeInfoAndSigned import net.corda.testing.internal.rigorousMock import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties diff --git a/node/src/test/kotlin/net/corda/node/services/RPCSecurityManagerTest.kt b/node/src/test/kotlin/net/corda/node/services/RPCSecurityManagerTest.kt index f64d72ea87..64ed341993 100644 --- a/node/src/test/kotlin/net/corda/node/services/RPCSecurityManagerTest.kt +++ b/node/src/test/kotlin/net/corda/node/services/RPCSecurityManagerTest.kt @@ -9,6 +9,8 @@ import net.corda.node.internal.security.tryAuthenticate import net.corda.node.services.Permissions.Companion.invokeRpc import net.corda.node.services.config.SecurityConfiguration import net.corda.nodeapi.internal.config.User +import net.corda.testing.internal.TestingNamedCacheFactory +import net.corda.testing.internal.fromUserList import org.assertj.core.api.Assertions.assertThatThrownBy import org.junit.Test import javax.security.auth.login.FailedLoginException @@ -134,7 +136,7 @@ class RPCSecurityManagerTest { private fun checkUserActions(permissions: Set, permitted: Set>) { val user = User(username = "user", password = "password", permissions = permissions) - val userRealms = RPCSecurityManagerImpl(SecurityConfiguration.AuthService.fromUsers(listOf(user))) + val userRealms = RPCSecurityManagerImpl(SecurityConfiguration.AuthService.fromUsers(listOf(user)), TestingNamedCacheFactory()) val disabled = allActions.filter { !permitted.contains(listOf(it)) } for (subject in listOf( userRealms.authenticate("user", Password("password")), diff --git a/node/src/test/kotlin/net/corda/node/services/events/NodeSchedulerServiceTest.kt b/node/src/test/kotlin/net/corda/node/services/events/NodeSchedulerServiceTest.kt index 7cd1955776..b0279be977 100644 --- a/node/src/test/kotlin/net/corda/node/services/events/NodeSchedulerServiceTest.kt +++ b/node/src/test/kotlin/net/corda/node/services/events/NodeSchedulerServiceTest.kt @@ -10,13 +10,13 @@ import net.corda.core.internal.FlowStateMachine import net.corda.core.internal.concurrent.openFuture import net.corda.core.node.ServicesForResolution import net.corda.core.utilities.days -import net.corda.node.internal.configureDatabase import net.corda.node.services.api.FlowStarter import net.corda.node.services.api.NodePropertiesStore import net.corda.node.services.messaging.DeduplicationHandler import net.corda.node.services.statemachine.ExternalEvent import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig +import net.corda.testing.internal.configureDatabase import net.corda.testing.internal.doLookup import net.corda.testing.internal.rigorousMock import net.corda.testing.internal.spectator diff --git a/node/src/test/kotlin/net/corda/node/services/events/PersistentScheduledFlowRepositoryTest.kt b/node/src/test/kotlin/net/corda/node/services/events/PersistentScheduledFlowRepositoryTest.kt index f25588ea74..d478dc473d 100644 --- a/node/src/test/kotlin/net/corda/node/services/events/PersistentScheduledFlowRepositoryTest.kt +++ b/node/src/test/kotlin/net/corda/node/services/events/PersistentScheduledFlowRepositoryTest.kt @@ -4,9 +4,8 @@ import net.corda.core.contracts.ScheduledStateRef import net.corda.core.contracts.StateRef import net.corda.core.crypto.SecureHash import net.corda.core.utilities.days -import net.corda.node.internal.configureDatabase import net.corda.nodeapi.internal.persistence.DatabaseConfig -import net.corda.testing.internal.rigorousMock +import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices import org.junit.Test import java.time.Instant diff --git a/node/src/test/kotlin/net/corda/node/services/identity/PersistentIdentityServiceTests.kt b/node/src/test/kotlin/net/corda/node/services/identity/PersistentIdentityServiceTests.kt index 186cfb13cc..bf8a16880f 100644 --- a/node/src/test/kotlin/net/corda/node/services/identity/PersistentIdentityServiceTests.kt +++ b/node/src/test/kotlin/net/corda/node/services/identity/PersistentIdentityServiceTests.kt @@ -7,7 +7,6 @@ import net.corda.core.identity.CordaX500Name import net.corda.core.identity.Party import net.corda.core.identity.PartyAndCertificate import net.corda.core.node.services.UnknownAnonymousPartyException -import net.corda.node.internal.configureDatabase import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.nodeapi.internal.crypto.CertificateType import net.corda.nodeapi.internal.crypto.X509Utilities @@ -17,6 +16,7 @@ import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.testing.core.* import net.corda.testing.internal.DEV_INTERMEDIATE_CA import net.corda.testing.internal.DEV_ROOT_CA +import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import net.corda.testing.node.makeTestIdentityService import org.junit.After diff --git a/node/src/test/kotlin/net/corda/node/services/persistence/AppendOnlyPersistentMapTest.kt b/node/src/test/kotlin/net/corda/node/services/persistence/AppendOnlyPersistentMapTest.kt index b52d8f39a8..5e36bd301c 100644 --- a/node/src/test/kotlin/net/corda/node/services/persistence/AppendOnlyPersistentMapTest.kt +++ b/node/src/test/kotlin/net/corda/node/services/persistence/AppendOnlyPersistentMapTest.kt @@ -2,11 +2,11 @@ package net.corda.node.services.persistence import net.corda.core.schemas.MappedSchema import net.corda.core.utilities.loggerFor -import net.corda.node.internal.configureDatabase import net.corda.node.services.schema.NodeSchemaService import net.corda.node.utilities.AppendOnlyPersistentMap import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.nodeapi.internal.persistence.DatabaseConfig +import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import org.junit.After import org.junit.Assert.* diff --git a/node/src/test/kotlin/net/corda/node/services/persistence/DBCheckpointStorageTests.kt b/node/src/test/kotlin/net/corda/node/services/persistence/DBCheckpointStorageTests.kt index 7fd4072d38..12782348c2 100644 --- a/node/src/test/kotlin/net/corda/node/services/persistence/DBCheckpointStorageTests.kt +++ b/node/src/test/kotlin/net/corda/node/services/persistence/DBCheckpointStorageTests.kt @@ -3,16 +3,15 @@ package net.corda.node.services.persistence import net.corda.core.context.InvocationContext import net.corda.core.flows.FlowLogic import net.corda.core.flows.StateMachineRunId -import net.corda.core.serialization.internal.CheckpointSerializationDefaults import net.corda.core.serialization.SerializedBytes +import net.corda.core.serialization.internal.CheckpointSerializationDefaults import net.corda.core.serialization.internal.checkpointSerialize import net.corda.node.internal.CheckpointIncompatibleException import net.corda.node.internal.CheckpointVerifier -import net.corda.node.internal.configureDatabase import net.corda.node.services.api.CheckpointStorage import net.corda.node.services.statemachine.Checkpoint -import net.corda.node.services.statemachine.SubFlowVersion import net.corda.node.services.statemachine.FlowStart +import net.corda.node.services.statemachine.SubFlowVersion import net.corda.node.services.transactions.PersistentUniquenessProvider import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig @@ -20,6 +19,7 @@ import net.corda.testing.core.ALICE_NAME import net.corda.testing.core.SerializationEnvironmentRule import net.corda.testing.core.TestIdentity import net.corda.testing.internal.LogHelper +import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import org.assertj.core.api.Assertions diff --git a/node/src/test/kotlin/net/corda/node/services/persistence/DBTransactionStorageTests.kt b/node/src/test/kotlin/net/corda/node/services/persistence/DBTransactionStorageTests.kt index 44c92b6630..00b9356d21 100644 --- a/node/src/test/kotlin/net/corda/node/services/persistence/DBTransactionStorageTests.kt +++ b/node/src/test/kotlin/net/corda/node/services/persistence/DBTransactionStorageTests.kt @@ -7,13 +7,13 @@ import net.corda.core.crypto.SignatureMetadata import net.corda.core.crypto.TransactionSignature import net.corda.core.toFuture import net.corda.core.transactions.SignedTransaction -import net.corda.node.internal.configureDatabase import net.corda.node.services.transactions.PersistentUniquenessProvider import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.testing.core.* import net.corda.testing.internal.LogHelper +import net.corda.testing.internal.configureDatabase import net.corda.testing.internal.createWireTransaction import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import org.assertj.core.api.Assertions.assertThat diff --git a/node/src/test/kotlin/net/corda/node/services/persistence/HibernateConfigurationTest.kt b/node/src/test/kotlin/net/corda/node/services/persistence/HibernateConfigurationTest.kt index 72071192cd..726d6b1cb6 100644 --- a/node/src/test/kotlin/net/corda/node/services/persistence/HibernateConfigurationTest.kt +++ b/node/src/test/kotlin/net/corda/node/services/persistence/HibernateConfigurationTest.kt @@ -29,7 +29,6 @@ import net.corda.finance.schemas.test.SampleCashSchemaV1 import net.corda.finance.schemas.test.SampleCashSchemaV2 import net.corda.finance.schemas.test.SampleCashSchemaV3 import net.corda.finance.utils.sumCash -import net.corda.node.internal.configureDatabase import net.corda.node.services.api.IdentityServiceInternal import net.corda.node.services.api.WritableTransactionStorage import net.corda.node.services.schema.ContractStateAndRef @@ -41,6 +40,7 @@ import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.nodeapi.internal.persistence.HibernateConfiguration import net.corda.testing.core.* +import net.corda.testing.internal.configureDatabase import net.corda.testing.internal.rigorousMock import net.corda.testing.internal.vault.DummyDealStateSchemaV1 import net.corda.testing.internal.vault.DummyLinearStateSchemaV1 diff --git a/node/src/test/kotlin/net/corda/node/services/persistence/NodeAttachmentServiceTest.kt b/node/src/test/kotlin/net/corda/node/services/persistence/NodeAttachmentServiceTest.kt index e4f6549e48..255f0c323f 100644 --- a/node/src/test/kotlin/net/corda/node/services/persistence/NodeAttachmentServiceTest.kt +++ b/node/src/test/kotlin/net/corda/node/services/persistence/NodeAttachmentServiceTest.kt @@ -13,12 +13,12 @@ import net.corda.core.node.services.vault.AttachmentSort import net.corda.core.node.services.vault.Builder import net.corda.core.node.services.vault.Sort import net.corda.core.utilities.getOrThrow -import net.corda.node.internal.configureDatabase import net.corda.node.services.transactions.PersistentUniquenessProvider import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.testing.internal.LogHelper +import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import net.corda.testing.node.internal.InternalMockNetwork import net.corda.testing.node.internal.startFlow diff --git a/node/src/test/kotlin/net/corda/node/services/persistence/TransactionCallbackTest.kt b/node/src/test/kotlin/net/corda/node/services/persistence/TransactionCallbackTest.kt index 7231c3afed..e6c4e8fdf2 100644 --- a/node/src/test/kotlin/net/corda/node/services/persistence/TransactionCallbackTest.kt +++ b/node/src/test/kotlin/net/corda/node/services/persistence/TransactionCallbackTest.kt @@ -1,7 +1,7 @@ package net.corda.node.services.persistence -import net.corda.node.internal.configureDatabase import net.corda.nodeapi.internal.persistence.DatabaseConfig +import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import org.junit.After import org.junit.Test diff --git a/node/src/test/kotlin/net/corda/node/services/schema/PersistentStateServiceTests.kt b/node/src/test/kotlin/net/corda/node/services/schema/PersistentStateServiceTests.kt index f1abf5e618..9fac402e43 100644 --- a/node/src/test/kotlin/net/corda/node/services/schema/PersistentStateServiceTests.kt +++ b/node/src/test/kotlin/net/corda/node/services/schema/PersistentStateServiceTests.kt @@ -7,24 +7,21 @@ import net.corda.core.contracts.TransactionState import net.corda.core.crypto.SecureHash import net.corda.core.identity.AbstractParty import net.corda.core.identity.CordaX500Name -import net.corda.core.node.services.Vault import net.corda.core.schemas.MappedSchema import net.corda.core.schemas.PersistentState import net.corda.core.schemas.QueryableState import net.corda.node.services.api.SchemaService -import net.corda.node.internal.configureDatabase import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.nodeapi.internal.persistence.currentDBSession -import net.corda.testing.internal.LogHelper -import net.corda.testing.core.TestIdentity import net.corda.testing.contracts.DummyContract +import net.corda.testing.core.TestIdentity +import net.corda.testing.internal.LogHelper +import net.corda.testing.internal.configureDatabase import net.corda.testing.internal.rigorousMock import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import org.junit.After import org.junit.Before -import org.junit.Ignore import org.junit.Test -import rx.subjects.PublishSubject import kotlin.test.assertEquals class PersistentStateServiceTests { diff --git a/node/src/test/kotlin/net/corda/node/services/transactions/PersistentUniquenessProviderTests.kt b/node/src/test/kotlin/net/corda/node/services/transactions/PersistentUniquenessProviderTests.kt index 65401708d4..3c63c71847 100644 --- a/node/src/test/kotlin/net/corda/node/services/transactions/PersistentUniquenessProviderTests.kt +++ b/node/src/test/kotlin/net/corda/node/services/transactions/PersistentUniquenessProviderTests.kt @@ -8,7 +8,6 @@ import net.corda.core.flows.NotarisationRequestSignature import net.corda.core.flows.NotaryError import net.corda.core.identity.CordaX500Name import net.corda.core.internal.notary.NotaryInternalException -import net.corda.node.internal.configureDatabase import net.corda.node.services.schema.NodeSchemaService import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.nodeapi.internal.persistence.CordaPersistence @@ -17,6 +16,7 @@ import net.corda.testing.core.SerializationEnvironmentRule import net.corda.testing.core.TestIdentity import net.corda.testing.core.generateStateRef import net.corda.testing.internal.LogHelper +import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import org.junit.After import org.junit.Before diff --git a/node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt b/node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt index d7f890a7ab..7f55504ee9 100644 --- a/node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt +++ b/node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt @@ -24,13 +24,13 @@ import net.corda.finance.schemas.CashSchemaV1.PersistentCashState import net.corda.finance.schemas.CommercialPaperSchemaV1 import net.corda.finance.schemas.test.SampleCashSchemaV2 import net.corda.finance.schemas.test.SampleCashSchemaV3 -import net.corda.node.internal.configureDatabase import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.nodeapi.internal.persistence.DatabaseTransaction import net.corda.testing.core.* import net.corda.testing.internal.TEST_TX_TIME import net.corda.testing.internal.chooseIdentity +import net.corda.testing.internal.configureDatabase import net.corda.testing.internal.rigorousMock import net.corda.testing.internal.vault.* import net.corda.testing.node.MockServices diff --git a/node/src/test/kotlin/net/corda/node/utilities/ObservablesTests.kt b/node/src/test/kotlin/net/corda/node/utilities/ObservablesTests.kt index d319c0b1d1..a28619a4d1 100644 --- a/node/src/test/kotlin/net/corda/node/utilities/ObservablesTests.kt +++ b/node/src/test/kotlin/net/corda/node/utilities/ObservablesTests.kt @@ -3,8 +3,8 @@ package net.corda.node.utilities import com.google.common.util.concurrent.SettableFuture import net.corda.core.internal.bufferUntilSubscribed import net.corda.core.internal.tee -import net.corda.node.internal.configureDatabase import net.corda.nodeapi.internal.persistence.* +import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import org.assertj.core.api.Assertions.assertThat import org.junit.After diff --git a/node/src/test/kotlin/net/corda/node/utilities/PersistentMapTests.kt b/node/src/test/kotlin/net/corda/node/utilities/PersistentMapTests.kt index baaf99a34f..69212e7369 100644 --- a/node/src/test/kotlin/net/corda/node/utilities/PersistentMapTests.kt +++ b/node/src/test/kotlin/net/corda/node/utilities/PersistentMapTests.kt @@ -1,9 +1,10 @@ package net.corda.node.utilities import net.corda.core.crypto.SecureHash -import net.corda.node.internal.configureDatabase import net.corda.node.services.upgrade.ContractUpgradeServiceImpl import net.corda.nodeapi.internal.persistence.DatabaseConfig +import net.corda.testing.internal.TestingNamedCacheFactory +import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices import org.junit.Test import kotlin.test.assertEquals @@ -25,7 +26,8 @@ class PersistentMapTests { upgradedContractClassName = value } }, - persistentEntityClass = ContractUpgradeServiceImpl.DBContractUpgrade::class.java + persistentEntityClass = ContractUpgradeServiceImpl.DBContractUpgrade::class.java, + cacheFactory = TestingNamedCacheFactory() ).apply { preload() } } diff --git a/samples/irs-demo/cordapp/src/test/kotlin/net/corda/irs/api/NodeInterestRatesTest.kt b/samples/irs-demo/cordapp/src/test/kotlin/net/corda/irs/api/NodeInterestRatesTest.kt index 4b2e01d4e4..09946bef03 100644 --- a/samples/irs-demo/cordapp/src/test/kotlin/net/corda/irs/api/NodeInterestRatesTest.kt +++ b/samples/irs-demo/cordapp/src/test/kotlin/net/corda/irs/api/NodeInterestRatesTest.kt @@ -10,13 +10,14 @@ import net.corda.finance.DOLLARS import net.corda.finance.contracts.Fix import net.corda.finance.contracts.asset.CASH import net.corda.finance.contracts.asset.Cash -import net.corda.node.internal.configureDatabase import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.testing.core.* +import net.corda.testing.internal.configureDatabase import net.corda.testing.internal.rigorousMock -import net.corda.testing.node.* +import net.corda.testing.node.MockServices import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties +import net.corda.testing.node.createMockCordaService import org.junit.After import org.junit.Assert.* import org.junit.Before diff --git a/serialization/src/main/kotlin/net/corda/serialization/internal/SerializationScheme.kt b/serialization/src/main/kotlin/net/corda/serialization/internal/SerializationScheme.kt index e94e2b95b4..9c78e01926 100644 --- a/serialization/src/main/kotlin/net/corda/serialization/internal/SerializationScheme.kt +++ b/serialization/src/main/kotlin/net/corda/serialization/internal/SerializationScheme.kt @@ -6,7 +6,6 @@ import net.corda.core.DeleteForDJVM import net.corda.core.KeepForDJVM import net.corda.core.contracts.Attachment import net.corda.core.crypto.SecureHash -import net.corda.core.internal.buildNamed import net.corda.core.internal.copyBytes import net.corda.core.serialization.* import net.corda.core.utilities.ByteSequence @@ -77,7 +76,7 @@ data class SerializationContextImpl @JvmOverloads constructor(override val prefe */ @DeleteForDJVM internal class AttachmentsClassLoaderBuilder(private val properties: Map, private val deserializationClassLoader: ClassLoader) { - private val cache: Cache, AttachmentsClassLoader> = Caffeine.newBuilder().weakValues().maximumSize(1024).buildNamed("SerializationScheme_attachmentClassloader") + private val cache: Cache, AttachmentsClassLoader> = Caffeine.newBuilder().weakValues().maximumSize(1024).build() fun build(attachmentHashes: List): AttachmentsClassLoader? { val serializationContext = properties[serializationContextKey] as? SerializeAsTokenContext ?: return null // Some tests don't set one. diff --git a/testing/node-driver/src/main/kotlin/net/corda/testing/node/MockServices.kt b/testing/node-driver/src/main/kotlin/net/corda/testing/node/MockServices.kt index e78d0c12a3..9d7dfa7cf6 100644 --- a/testing/node-driver/src/main/kotlin/net/corda/testing/node/MockServices.kt +++ b/testing/node-driver/src/main/kotlin/net/corda/testing/node/MockServices.kt @@ -21,7 +21,6 @@ import net.corda.core.utilities.NetworkHostAndPort import net.corda.node.VersionInfo import net.corda.node.cordapp.CordappLoader import net.corda.node.internal.ServicesForResolutionImpl -import net.corda.node.internal.configureDatabase import net.corda.node.internal.cordapp.JarScanningCordappLoader import net.corda.node.services.api.* import net.corda.node.services.identity.InMemoryIdentityService @@ -34,6 +33,7 @@ import net.corda.testing.common.internal.testNetworkParameters import net.corda.testing.core.TestIdentity import net.corda.testing.internal.DEV_ROOT_CA import net.corda.testing.internal.MockCordappProvider +import net.corda.testing.internal.configureDatabase import net.corda.testing.node.internal.* import net.corda.testing.services.MockAttachmentStorage import java.security.KeyPair diff --git a/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/RPCDriver.kt b/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/RPCDriver.kt index 38109bf114..d9c1cc3a2f 100644 --- a/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/RPCDriver.kt +++ b/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/RPCDriver.kt @@ -24,12 +24,14 @@ import net.corda.node.services.messaging.RPCServerConfiguration import net.corda.nodeapi.RPCApi import net.corda.nodeapi.internal.ArtemisTcpTransport import net.corda.serialization.internal.AMQP_RPC_CLIENT_CONTEXT -import net.corda.testing.node.TestCordapp import net.corda.testing.common.internal.testNetworkParameters import net.corda.testing.core.MAX_MESSAGE_SIZE import net.corda.testing.driver.JmxPolicy import net.corda.testing.driver.PortAllocation +import net.corda.testing.internal.TestingNamedCacheFactory +import net.corda.testing.internal.fromUserList import net.corda.testing.node.NotarySpec +import net.corda.testing.node.TestCordapp import net.corda.testing.node.User import net.corda.testing.node.internal.DriverDSLImpl.Companion.cordappsInCurrentAndAdditionalPackages import org.apache.activemq.artemis.api.core.SimpleString @@ -485,7 +487,8 @@ data class RPCDriverDSL( locator, rpcSecurityManager, nodeLegalName, - configuration + configuration, + TestingNamedCacheFactory() ) driverDSL.shutdownManager.registerShutdown { rpcServer.close(queueDrainTimeout) diff --git a/testing/test-utils/src/main/kotlin/net/corda/testing/internal/InternalTestUtils.kt b/testing/test-utils/src/main/kotlin/net/corda/testing/internal/InternalTestUtils.kt index 4ddca172af..8f82cf3792 100644 --- a/testing/test-utils/src/main/kotlin/net/corda/testing/internal/InternalTestUtils.kt +++ b/testing/test-utils/src/main/kotlin/net/corda/testing/internal/InternalTestUtils.kt @@ -1,26 +1,42 @@ package net.corda.testing.internal +import net.corda.core.context.AuthServiceId import net.corda.core.contracts.* import net.corda.core.crypto.Crypto import net.corda.core.crypto.Crypto.generateKeyPair import net.corda.core.crypto.SecureHash +import net.corda.core.identity.AbstractParty import net.corda.core.identity.CordaX500Name import net.corda.core.identity.Party import net.corda.core.identity.PartyAndCertificate +import net.corda.core.internal.NamedCacheFactory import net.corda.core.node.NodeInfo +import net.corda.core.schemas.MappedSchema import net.corda.core.transactions.WireTransaction import net.corda.core.utilities.loggerFor +import net.corda.node.internal.createCordaPersistence +import net.corda.node.internal.security.RPCSecurityManagerImpl +import net.corda.node.internal.startHikariPool +import net.corda.node.services.api.SchemaService +import net.corda.node.services.config.SecurityConfiguration +import net.corda.node.services.schema.NodeSchemaService import net.corda.nodeapi.BrokerRpcSslOptions import net.corda.nodeapi.internal.config.MutualSslConfiguration -import net.corda.nodeapi.internal.registerDevP2pCertificates +import net.corda.nodeapi.internal.config.User import net.corda.nodeapi.internal.createDevNodeCa -import net.corda.nodeapi.internal.crypto.* +import net.corda.nodeapi.internal.crypto.CertificateAndKeyPair +import net.corda.nodeapi.internal.crypto.CertificateType +import net.corda.nodeapi.internal.crypto.X509Utilities import net.corda.nodeapi.internal.loadDevCaTrustStore +import net.corda.nodeapi.internal.persistence.CordaPersistence +import net.corda.nodeapi.internal.persistence.DatabaseConfig +import net.corda.nodeapi.internal.registerDevP2pCertificates import net.corda.serialization.internal.amqp.AMQP_ENABLED import net.corda.testing.internal.stubs.CertificateStoreStubs import java.nio.file.Files import java.nio.file.Path import java.security.KeyPair +import java.util.* import javax.security.auth.x500.X500Principal @Suppress("unused") @@ -136,3 +152,24 @@ fun createWireTransaction(inputs: List, val componentGroups = WireTransaction.createComponentGroups(inputs, outputs, commands, attachments, notary, timeWindow) return WireTransaction(componentGroups, privacySalt) } + +/** + * Instantiate RPCSecurityManager initialised with users data from a list of [User] + */ +fun RPCSecurityManagerImpl.Companion.fromUserList(id: AuthServiceId, users: List) = + RPCSecurityManagerImpl(SecurityConfiguration.AuthService.fromUsers(users).copy(id = id), TestingNamedCacheFactory()) + +/** + * Convenience method for configuring a database for some tests. + */ +fun configureDatabase(hikariProperties: Properties, + databaseConfig: DatabaseConfig, + wellKnownPartyFromX500Name: (CordaX500Name) -> Party?, + wellKnownPartyFromAnonymous: (AbstractParty) -> Party?, + schemaService: SchemaService = NodeSchemaService(), + internalSchemas: Set = NodeSchemaService().internalSchemas(), + cacheFactory: NamedCacheFactory = TestingNamedCacheFactory()): CordaPersistence { + val persistence = createCordaPersistence(databaseConfig, wellKnownPartyFromX500Name, wellKnownPartyFromAnonymous, schemaService, hikariProperties, cacheFactory) + persistence.startHikariPool(hikariProperties, databaseConfig, internalSchemas) + return persistence +} \ No newline at end of file diff --git a/testing/test-utils/src/main/kotlin/net/corda/testing/internal/TestingNamedCacheFactory.kt b/testing/test-utils/src/main/kotlin/net/corda/testing/internal/TestingNamedCacheFactory.kt index 7908d5dd27..6802fd042e 100644 --- a/testing/test-utils/src/main/kotlin/net/corda/testing/internal/TestingNamedCacheFactory.kt +++ b/testing/test-utils/src/main/kotlin/net/corda/testing/internal/TestingNamedCacheFactory.kt @@ -5,21 +5,20 @@ import com.github.benmanes.caffeine.cache.Cache import com.github.benmanes.caffeine.cache.CacheLoader import com.github.benmanes.caffeine.cache.Caffeine import com.github.benmanes.caffeine.cache.LoadingCache -import net.corda.core.internal.buildNamed import net.corda.core.serialization.SingletonSerializeAsToken import net.corda.node.services.config.MB import net.corda.node.services.config.NodeConfiguration -import net.corda.node.utilities.NamedCacheFactory +import net.corda.node.utilities.BindableNamedCacheFactory -class TestingNamedCacheFactory private constructor(private val sizeOverride: Long, private val metricRegistry: MetricRegistry?, private val nodeConfiguration: NodeConfiguration?) : NamedCacheFactory, SingletonSerializeAsToken() { +class TestingNamedCacheFactory private constructor(private val sizeOverride: Long, private val metricRegistry: MetricRegistry?, private val nodeConfiguration: NodeConfiguration?) : BindableNamedCacheFactory, SingletonSerializeAsToken() { constructor(sizeOverride: Long = 1024) : this(sizeOverride, null, null) - override fun bindWithMetrics(metricRegistry: MetricRegistry): NamedCacheFactory = TestingNamedCacheFactory(sizeOverride, metricRegistry, this.nodeConfiguration) - override fun bindWithConfig(nodeConfiguration: NodeConfiguration): NamedCacheFactory = TestingNamedCacheFactory(sizeOverride, this.metricRegistry, nodeConfiguration) + override fun bindWithMetrics(metricRegistry: MetricRegistry): BindableNamedCacheFactory = TestingNamedCacheFactory(sizeOverride, metricRegistry, this.nodeConfiguration) + override fun bindWithConfig(nodeConfiguration: NodeConfiguration): BindableNamedCacheFactory = TestingNamedCacheFactory(sizeOverride, this.metricRegistry, nodeConfiguration) override fun buildNamed(caffeine: Caffeine, name: String): Cache { // Does not check metricRegistry or nodeConfiguration, because for tests we don't care. - return caffeine.maximumSize(sizeOverride).buildNamed(name) + return caffeine.maximumSize(sizeOverride).build() } override fun buildNamed(caffeine: Caffeine, name: String, loader: CacheLoader): LoadingCache { @@ -29,6 +28,6 @@ class TestingNamedCacheFactory private constructor(private val sizeOverride: Lon "NodeAttachmentService_attachmentContent" -> caffeine.maximumWeight(1.MB) else -> caffeine.maximumSize(sizeOverride) } - return configuredCaffeine.buildNamed(name, loader) + return configuredCaffeine.build(loader) } } \ No newline at end of file diff --git a/tools/explorer/src/main/kotlin/net/corda/explorer/identicon/IdenticonRenderer.kt b/tools/explorer/src/main/kotlin/net/corda/explorer/identicon/IdenticonRenderer.kt index db17fc7749..97a8549688 100644 --- a/tools/explorer/src/main/kotlin/net/corda/explorer/identicon/IdenticonRenderer.kt +++ b/tools/explorer/src/main/kotlin/net/corda/explorer/identicon/IdenticonRenderer.kt @@ -14,7 +14,6 @@ import javafx.scene.image.WritableImage import javafx.scene.paint.Color import javafx.scene.text.TextAlignment import net.corda.core.crypto.SecureHash -import net.corda.core.internal.buildNamed /** * (The MIT License) @@ -76,7 +75,7 @@ object IdenticonRenderer { private const val renderingSize = 30.0 - private val cache = Caffeine.newBuilder().buildNamed("IdentIconRenderer_image", CacheLoader { key -> + private val cache = Caffeine.newBuilder().build(CacheLoader { key -> key.let { render(key.hashCode(), renderingSize) } }) From e1b86d0422bc681376bb942d861a56415373daa8 Mon Sep 17 00:00:00 2001 From: "rick.parker" Date: Fri, 19 Oct 2018 08:05:15 +0100 Subject: [PATCH 3/8] Fix up errors --- .../raft/RaftTransactionCommitLogTests.kt | 2 +- .../kotlin/net/corda/rpcWorker/RpcWorker.kt | 8 ++-- .../corda/rpcWorker/RpcWorkerServiceHub.kt | 9 +++-- .../internal/crypto/X509UtilitiesTest.kt | 5 ++- .../messaging/ArtemisMessagingTest.kt | 4 -- .../network/PersistentNetworkMapCacheTest.kt | 8 +--- .../net/corda/node/internal/AbstractNode.kt | 5 +-- .../kotlin/net/corda/node/internal/Node.kt | 3 +- .../messaging/InternalRPCMessagingClient.kt | 4 +- .../services/messaging/P2PMessagingClient.kt | 1 + .../node/services/messaging/RPCServer.kt | 5 ++- .../persistence/DBTransactionStorage.kt | 2 +- .../node/services/vault/NodeVaultService.kt | 6 +-- .../node/utilities/CaffeineStatsCounter.kt | 7 ++-- .../utilities/EnterpriseNamedCacheFactory.kt | 39 ++++++++++--------- .../corda/node/utilities/NodeNamedCache.kt | 6 +-- .../PersistentIdentityServiceTests.kt | 2 +- .../AppendOnlyPersistentMapTest.kt | 2 +- .../persistence/DBTransactionStorageTests.kt | 2 +- .../persistence/NodeAttachmentServiceTest.kt | 2 +- .../persistence/RunOnceServiceTest.kt | 2 +- .../persistence/SchemaMigrationTest.kt | 5 ++- .../schema/PersistentStateServiceTests.kt | 1 - .../PersistentUniquenessProviderTests.kt | 2 +- .../node/services/vault/VaultQueryTests.kt | 2 +- .../EnterpriseMockNamedCachedFactory.kt | 13 +++---- .../node/internal/InternalMockNetwork.kt | 6 +-- .../testing/internal/InternalTestUtils.kt | 9 +++-- 28 files changed, 78 insertions(+), 84 deletions(-) diff --git a/experimental/notary-raft/src/test/kotlin/net/corda/notary/raft/RaftTransactionCommitLogTests.kt b/experimental/notary-raft/src/test/kotlin/net/corda/notary/raft/RaftTransactionCommitLogTests.kt index 3dab18eefd..eb3b614474 100644 --- a/experimental/notary-raft/src/test/kotlin/net/corda/notary/raft/RaftTransactionCommitLogTests.kt +++ b/experimental/notary-raft/src/test/kotlin/net/corda/notary/raft/RaftTransactionCommitLogTests.kt @@ -16,13 +16,13 @@ import net.corda.core.internal.concurrent.transpose import net.corda.core.utilities.NetworkHostAndPort import net.corda.core.utilities.getOrThrow import net.corda.node.services.schema.NodeSchemaService -import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.testing.core.ALICE_NAME import net.corda.testing.core.SerializationEnvironmentRule import net.corda.testing.driver.PortAllocation import net.corda.testing.internal.LogHelper +import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.testing.internal.configureDatabase import net.corda.testing.node.internal.makeInternalTestDataSourceProperties import org.hamcrest.Matchers.instanceOf diff --git a/experimental/rpc-worker/src/main/kotlin/net/corda/rpcWorker/RpcWorker.kt b/experimental/rpc-worker/src/main/kotlin/net/corda/rpcWorker/RpcWorker.kt index 503b3e97f6..9ac12f69ee 100644 --- a/experimental/rpc-worker/src/main/kotlin/net/corda/rpcWorker/RpcWorker.kt +++ b/experimental/rpc-worker/src/main/kotlin/net/corda/rpcWorker/RpcWorker.kt @@ -15,6 +15,7 @@ import net.corda.node.services.messaging.InternalRPCMessagingClient import net.corda.node.services.messaging.RPCOpsRouting import net.corda.node.services.messaging.RPCServerConfiguration import net.corda.node.services.rpc.ArtemisRpcBroker +import net.corda.node.utilities.DefaultNamedCacheFactory import net.corda.nodeapi.internal.config.User import net.corda.nodeapi.internal.crypto.X509Utilities import org.apache.activemq.artemis.api.core.management.ActiveMQServerControl @@ -100,7 +101,8 @@ class Main : Runnable { private fun createRpcWorkerBroker(config: NodeConfiguration, maxMessageSize: Int): ArtemisBroker { val rpcOptions = config.rpcOptions - val securityManager = RPCSecurityManagerImpl(SecurityConfiguration.AuthService.fromUsers(config.rpcUsers)) + // TODO: wire this up to an EnterpriseCacheFactory + val securityManager = RPCSecurityManagerImpl(SecurityConfiguration.AuthService.fromUsers(config.rpcUsers), DefaultNamedCacheFactory()) val broker = if (rpcOptions.useSsl) { ArtemisRpcBroker.withSsl(config.p2pSslOptions, rpcOptions.address, rpcOptions.adminAddress, rpcOptions.sslConfig!!, securityManager, maxMessageSize, false, config.baseDirectory / "artemis", false) } else { @@ -127,7 +129,7 @@ class RpcWorker(private val serverControl: ActiveMQServerControl, private val rp val rpcServerConfiguration = RPCServerConfiguration.DEFAULT.copy( rpcThreadPoolSize = rpcWorkerConfig.enterpriseConfiguration.tuning.rpcThreadPoolSize ) - val securityManager = RPCSecurityManagerImpl(SecurityConfiguration.AuthService.fromUsers(rpcWorkerConfig.rpcUsers)) + val securityManager = RPCSecurityManagerImpl(SecurityConfiguration.AuthService.fromUsers(rpcWorkerConfig.rpcUsers), rpcWorkerServiceHubs[0].cacheFactory) val nodeName = if(rpcWorkerServiceHubs.size == 1) { rpcWorkerServiceHubs.single().configuration.myLegalName @@ -140,7 +142,7 @@ class RpcWorker(private val serverControl: ActiveMQServerControl, private val rp val rpcOpsMap = rpcWorkerServiceHubs.map { Pair(it.myInfo.legalIdentities.single().name, it.rpcOps) }.toMap() - internalRpcMessagingClient.init(RPCOpsRouting(rpcOpsMap), securityManager) + internalRpcMessagingClient.init(RPCOpsRouting(rpcOpsMap), securityManager, rpcWorkerServiceHubs[0].cacheFactory) internalRpcMessagingClient.start(serverControl) runOnStop += { internalRpcMessagingClient.stop() } diff --git a/experimental/rpc-worker/src/main/kotlin/net/corda/rpcWorker/RpcWorkerServiceHub.kt b/experimental/rpc-worker/src/main/kotlin/net/corda/rpcWorker/RpcWorkerServiceHub.kt index f72fdc896d..0c1a5dbe19 100644 --- a/experimental/rpc-worker/src/main/kotlin/net/corda/rpcWorker/RpcWorkerServiceHub.kt +++ b/experimental/rpc-worker/src/main/kotlin/net/corda/rpcWorker/RpcWorkerServiceHub.kt @@ -14,7 +14,9 @@ import net.corda.core.node.services.ContractUpgradeService import net.corda.core.node.services.TransactionVerifierService import net.corda.core.serialization.SerializeAsToken import net.corda.core.serialization.SingletonSerializeAsToken -import net.corda.core.serialization.internal.* +import net.corda.core.serialization.internal.SerializationEnvironment +import net.corda.core.serialization.internal.effectiveSerializationEnv +import net.corda.core.serialization.internal.nodeSerializationEnv import net.corda.core.utilities.contextLogger import net.corda.node.CordaClock import net.corda.node.SimpleClock @@ -81,7 +83,8 @@ class RpcWorkerServiceHub(override val configuration: NodeConfiguration, overrid configuration.database, identityService::wellKnownPartyFromX500Name, identityService::wellKnownPartyFromAnonymous, - schemaService + schemaService, + cacheFactory ) init { @@ -102,7 +105,7 @@ class RpcWorkerServiceHub(override val configuration: NodeConfiguration, overrid private val servicesForResolution = ServicesForResolutionImpl(identityService, attachments, cordappProvider, validatedTransactions) @Suppress("LeakingThis") override val vaultService = NodeVaultService(clock, keyManagementService, servicesForResolution, database, schemaService, cacheFactory) - override val nodeProperties = NodePropertiesPersistentStore(StubbedNodeUniqueIdProvider::value, database) + override val nodeProperties = NodePropertiesPersistentStore(StubbedNodeUniqueIdProvider::value, database, cacheFactory) override val monitoringService = MonitoringService(metricRegistry) override val networkMapUpdater = NetworkMapUpdater( networkMapCache, diff --git a/node-api/src/test/kotlin/net/corda/nodeapi/internal/crypto/X509UtilitiesTest.kt b/node-api/src/test/kotlin/net/corda/nodeapi/internal/crypto/X509UtilitiesTest.kt index 0a3d770d33..e319ed71e8 100644 --- a/node-api/src/test/kotlin/net/corda/nodeapi/internal/crypto/X509UtilitiesTest.kt +++ b/node-api/src/test/kotlin/net/corda/nodeapi/internal/crypto/X509UtilitiesTest.kt @@ -5,7 +5,6 @@ import io.netty.handler.ssl.ClientAuth import io.netty.handler.ssl.SslContextBuilder import io.netty.handler.ssl.SslProvider import net.corda.core.crypto.Crypto -import net.corda.core.crypto.* import net.corda.core.crypto.Crypto.COMPOSITE_KEY import net.corda.core.crypto.Crypto.ECDSA_SECP256K1_SHA256 import net.corda.core.crypto.Crypto.ECDSA_SECP256R1_SHA256 @@ -13,6 +12,8 @@ import net.corda.core.crypto.Crypto.EDDSA_ED25519_SHA512 import net.corda.core.crypto.Crypto.RSA_SHA256 import net.corda.core.crypto.Crypto.SPHINCS256_SHA256 import net.corda.core.crypto.Crypto.generateKeyPair +import net.corda.core.crypto.SignatureScheme +import net.corda.core.crypto.newSecureRandom import net.corda.core.identity.CordaX500Name import net.corda.core.internal.div import net.corda.core.serialization.SerializationContext @@ -400,7 +401,7 @@ class X509UtilitiesTest { val clientHandler = NettyTestHandler { _, msg -> assertEquals("Hello", NettyTestHandler.readString(msg)) } NettyTestServer(sslServerContext, serverHandler, portAllocation.nextPort()).use { server -> server.start() - NettyTestClient(sslClientContext, InetAddress.getLocalHost().canonicalHostName, server.port, clientHandler).use { client -> + NettyTestClient(sslClientContext, "localhost", server.port, clientHandler).use { client -> client.start() clientHandler.writeString("Hello") diff --git a/node/src/integration-test/kotlin/net/corda/node/services/messaging/ArtemisMessagingTest.kt b/node/src/integration-test/kotlin/net/corda/node/services/messaging/ArtemisMessagingTest.kt index 9f72cc6976..9e2b93f8b2 100644 --- a/node/src/integration-test/kotlin/net/corda/node/services/messaging/ArtemisMessagingTest.kt +++ b/node/src/integration-test/kotlin/net/corda/node/services/messaging/ArtemisMessagingTest.kt @@ -9,9 +9,6 @@ import net.corda.core.internal.div import net.corda.core.utilities.NetworkHostAndPort import net.corda.core.utilities.seconds import net.corda.node.services.config.* -import net.corda.node.services.config.FlowTimeoutConfiguration -import net.corda.node.services.config.NodeConfiguration -import net.corda.node.services.config.configureWithDevSSLCertificate import net.corda.node.services.network.PersistentNetworkMapCache import net.corda.node.services.transactions.PersistentUniquenessProvider import net.corda.node.utilities.AffinityExecutor.ServiceAffinityExecutor @@ -30,7 +27,6 @@ import net.corda.testing.node.internal.MOCK_VERSION_INFO import net.corda.testing.node.internal.makeInternalTestDataSourceProperties import org.apache.activemq.artemis.api.core.Message.HDR_VALIDATED_USER import org.apache.activemq.artemis.api.core.SimpleString -import org.apache.activemq.artemis.api.core.ActiveMQConnectionTimedOutException import org.assertj.core.api.Assertions.assertThat import org.assertj.core.api.Assertions.assertThatThrownBy import org.junit.After diff --git a/node/src/integration-test/kotlin/net/corda/node/services/network/PersistentNetworkMapCacheTest.kt b/node/src/integration-test/kotlin/net/corda/node/services/network/PersistentNetworkMapCacheTest.kt index 5acacd95d7..9a99c3ccfa 100644 --- a/node/src/integration-test/kotlin/net/corda/node/services/network/PersistentNetworkMapCacheTest.kt +++ b/node/src/integration-test/kotlin/net/corda/node/services/network/PersistentNetworkMapCacheTest.kt @@ -8,12 +8,8 @@ import net.corda.nodeapi.internal.DEV_ROOT_CA import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.testing.core.* -import net.corda.testing.internal.IntegrationTest -import net.corda.testing.internal.IntegrationTestSchemas -import net.corda.testing.internal.toDatabaseSchemaName +import net.corda.testing.internal.* import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties -import net.corda.testing.internal.TestingNamedCacheFactory -import net.corda.testing.internal.configureDatabase import net.corda.testing.node.internal.makeTestDatabaseProperties import org.assertj.core.api.Assertions.assertThat import org.assertj.core.api.Assertions.assertThatIllegalArgumentException @@ -36,7 +32,7 @@ class PersistentNetworkMapCacheTest : IntegrationTest() { private var portCounter = 1000 //Enterprise only - objects created in the setup method, below initialized with dummy values to avoid need for nullable type declaration - private var database = CordaPersistence(DatabaseConfig(), emptySet()) + private var database = CordaPersistence(DatabaseConfig(), emptySet(), TestingNamedCacheFactory()) private var charlieNetMapCache = PersistentNetworkMapCache(TestingNamedCacheFactory(), database, InMemoryIdentityService(trustRoot = DEV_ROOT_CA.certificate)) @Before() diff --git a/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt b/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt index 089417cd38..43f0ad18f9 100644 --- a/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt +++ b/node/src/main/kotlin/net/corda/node/internal/AbstractNode.kt @@ -154,7 +154,6 @@ abstract class AbstractNode(val configuration: NodeConfiguration, identityService::wellKnownPartyFromX500Name, identityService::wellKnownPartyFromAnonymous, schemaService, - configuration.dataSourceProperties, cacheFactory) init { // TODO Break cyclic dependency @@ -1042,7 +1041,6 @@ fun createCordaPersistence(databaseConfig: DatabaseConfig, wellKnownPartyFromX500Name: (CordaX500Name) -> Party?, wellKnownPartyFromAnonymous: (AbstractParty) -> Party?, schemaService: SchemaService, - hikariProperties: Properties, cacheFactory: NamedCacheFactory): CordaPersistence { // Register the AbstractPartyDescriptor so Hibernate doesn't warn when encountering AbstractParty. Unfortunately // Hibernate warns about not being able to find a descriptor if we don't provide one, but won't use it by default @@ -1050,8 +1048,7 @@ fun createCordaPersistence(databaseConfig: DatabaseConfig, // either Hibernate can be convinced to stop warning, use the descriptor by default, or something else. JavaTypeDescriptorRegistry.INSTANCE.addDescriptor(AbstractPartyDescriptor(wellKnownPartyFromX500Name, wellKnownPartyFromAnonymous)) val attributeConverters = listOf(AbstractPartyToX500NameAsStringConverter(wellKnownPartyFromX500Name, wellKnownPartyFromAnonymous)) - val jdbcUrl = hikariProperties.getProperty("dataSource.url", "") - return CordaPersistence(databaseConfig, schemaService.schemaOptions.keys, jdbcUrl, cacheFactory, attributeConverters) + return CordaPersistence(databaseConfig, schemaService.schemaOptions.keys, cacheFactory, attributeConverters) } fun CordaPersistence.startHikariPool(hikariProperties: Properties, databaseConfig: DatabaseConfig, schemas: Set, metricRegistry: MetricRegistry? = null) { diff --git a/node/src/main/kotlin/net/corda/node/internal/Node.kt b/node/src/main/kotlin/net/corda/node/internal/Node.kt index a162df01c3..f2b7553d8d 100644 --- a/node/src/main/kotlin/net/corda/node/internal/Node.kt +++ b/node/src/main/kotlin/net/corda/node/internal/Node.kt @@ -44,6 +44,7 @@ import net.corda.node.services.config.* import net.corda.node.services.messaging.* import net.corda.node.services.rpc.ArtemisRpcBroker import net.corda.node.utilities.* +import net.corda.node.utilities.profiling.getTracingConfig import net.corda.nodeapi.internal.ArtemisMessagingClient import net.corda.nodeapi.internal.ArtemisMessagingComponent.Companion.INTERNAL_SHELL_USER import net.corda.nodeapi.internal.ShutdownHook @@ -86,7 +87,7 @@ class NodeWithInfo(val node: Node, val info: NodeInfo) { open class Node(configuration: NodeConfiguration, versionInfo: VersionInfo, private val initialiseSerialization: Boolean = true, - cacheFactoryPrototype: BindableNamedCacheFactory = DefaultNamedCacheFactory() + cacheFactoryPrototype: BindableNamedCacheFactory = EnterpriseNamedCacheFactory(configuration.enterpriseConfiguration.getTracingConfig()) ) : AbstractNode( configuration, createClock(configuration), diff --git a/node/src/main/kotlin/net/corda/node/services/messaging/InternalRPCMessagingClient.kt b/node/src/main/kotlin/net/corda/node/services/messaging/InternalRPCMessagingClient.kt index 4d74735552..544744d557 100644 --- a/node/src/main/kotlin/net/corda/node/services/messaging/InternalRPCMessagingClient.kt +++ b/node/src/main/kotlin/net/corda/node/services/messaging/InternalRPCMessagingClient.kt @@ -21,8 +21,8 @@ class InternalRPCMessagingClient(val sslConfig: MutualSslConfigura private var locator: ServerLocator? = null private var rpcServer: RPCServer? = null - fun init(rpcOps: OPS, securityManager: RPCSecurityManager) { - init(RPCOpsRouting.singleton(nodeName, rpcOps), securityManager) + fun init(rpcOps: OPS, securityManager: RPCSecurityManager, cacheFactory: NamedCacheFactory) { + init(RPCOpsRouting.singleton(nodeName, rpcOps), securityManager, cacheFactory) } fun init(rpcOpsRouting: RPCOpsRouting, securityManager: RPCSecurityManager, cacheFactory: NamedCacheFactory) = synchronized(this) { diff --git a/node/src/main/kotlin/net/corda/node/services/messaging/P2PMessagingClient.kt b/node/src/main/kotlin/net/corda/node/services/messaging/P2PMessagingClient.kt index efe8733fe7..8ec46dbaf9 100644 --- a/node/src/main/kotlin/net/corda/node/services/messaging/P2PMessagingClient.kt +++ b/node/src/main/kotlin/net/corda/node/services/messaging/P2PMessagingClient.kt @@ -5,6 +5,7 @@ import com.codahale.metrics.Clock import com.codahale.metrics.MetricRegistry import net.corda.core.crypto.toStringShort import net.corda.core.identity.CordaX500Name +import net.corda.core.identity.Party import net.corda.core.internal.NamedCacheFactory import net.corda.core.internal.ThreadBox import net.corda.core.messaging.CordaRPCOps diff --git a/node/src/main/kotlin/net/corda/node/services/messaging/RPCServer.kt b/node/src/main/kotlin/net/corda/node/services/messaging/RPCServer.kt index eeaad47dc6..5f06b38572 100644 --- a/node/src/main/kotlin/net/corda/node/services/messaging/RPCServer.kt +++ b/node/src/main/kotlin/net/corda/node/services/messaging/RPCServer.kt @@ -159,8 +159,9 @@ class RPCServer( serverLocator: ServerLocator, securityManager: RPCSecurityManager, nodeLegalName: CordaX500Name, - rpcConfiguration: RPCServerConfiguration - ) : this(RPCOpsRouting.singleton(nodeLegalName, ops), rpcServerUsername, rpcServerPassword, serverLocator, securityManager, nodeLegalName, rpcConfiguration) + rpcConfiguration: RPCServerConfiguration, + cacheFactory: NamedCacheFactory + ) : this(RPCOpsRouting.singleton(nodeLegalName, ops), rpcServerUsername, rpcServerPassword, serverLocator, securityManager, nodeLegalName, rpcConfiguration, cacheFactory) init { // It is assumed that all the identities have the same type of RPCOps associated with them. diff --git a/node/src/main/kotlin/net/corda/node/services/persistence/DBTransactionStorage.kt b/node/src/main/kotlin/net/corda/node/services/persistence/DBTransactionStorage.kt index 1ad1983631..308af5b356 100644 --- a/node/src/main/kotlin/net/corda/node/services/persistence/DBTransactionStorage.kt +++ b/node/src/main/kotlin/net/corda/node/services/persistence/DBTransactionStorage.kt @@ -3,8 +3,8 @@ package net.corda.node.services.persistence import net.corda.core.concurrent.CordaFuture import net.corda.core.crypto.SecureHash import net.corda.core.crypto.TransactionSignature -import net.corda.core.internal.NamedCacheFactory import net.corda.core.internal.ConcurrentBox +import net.corda.core.internal.NamedCacheFactory import net.corda.core.internal.VisibleForTesting import net.corda.core.internal.bufferUntilSubscribed import net.corda.core.internal.concurrent.doneFuture diff --git a/node/src/main/kotlin/net/corda/node/services/vault/NodeVaultService.kt b/node/src/main/kotlin/net/corda/node/services/vault/NodeVaultService.kt index ff5cc3de40..6f15c3466c 100644 --- a/node/src/main/kotlin/net/corda/node/services/vault/NodeVaultService.kt +++ b/node/src/main/kotlin/net/corda/node/services/vault/NodeVaultService.kt @@ -13,21 +13,17 @@ import net.corda.core.node.StatesToRecord import net.corda.core.node.services.KeyManagementService import net.corda.core.node.services.StatesNotAvailableException import net.corda.core.node.services.Vault +import net.corda.core.node.services.Vault.ConstraintInfo.Companion.constraintInfo import net.corda.core.node.services.VaultQueryException import net.corda.core.node.services.vault.* -import net.corda.core.node.services.Vault.ConstraintInfo.Companion.constraintInfo import net.corda.core.schemas.PersistentStateRef -import net.corda.core.serialization.SerializationDefaults.STORAGE_CONTEXT import net.corda.core.serialization.SingletonSerializeAsToken -import net.corda.core.serialization.deserialize -import net.corda.core.serialization.serialize import net.corda.core.transactions.* import net.corda.core.utilities.* import net.corda.node.services.api.SchemaService import net.corda.node.services.api.VaultServiceInternal import net.corda.node.services.schema.PersistentStateService import net.corda.node.services.statemachine.FlowStateMachineImpl -import net.corda.node.utilities.NamedCacheFactory import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.bufferUntilDatabaseCommit import net.corda.nodeapi.internal.persistence.currentDBSession diff --git a/node/src/main/kotlin/net/corda/node/utilities/CaffeineStatsCounter.kt b/node/src/main/kotlin/net/corda/node/utilities/CaffeineStatsCounter.kt index a0e33d84ff..3b97407047 100644 --- a/node/src/main/kotlin/net/corda/node/utilities/CaffeineStatsCounter.kt +++ b/node/src/main/kotlin/net/corda/node/utilities/CaffeineStatsCounter.kt @@ -10,16 +10,15 @@ import com.github.benmanes.caffeine.cache.Caffeine import com.github.benmanes.caffeine.cache.LoadingCache import com.github.benmanes.caffeine.cache.stats.CacheStats import com.github.benmanes.caffeine.cache.stats.StatsCounter -import net.corda.core.internal.buildNamed import java.util.concurrent.TimeUnit import java.util.function.Supplier /** - * Helper to export statistics to JMX and finally build the cache. + * Helpers to export statistics to JMX and finally build the cache. */ -fun Caffeine.buildNamed(registry: MetricRegistry, metricsPrefix: String): Cache = this.addMetrics(registry, metricsPrefix).buildNamed(metricsPrefix).addExtraMetrics(registry, metricsPrefix) +fun Caffeine.buildNamed(registry: MetricRegistry, metricsPrefix: String): Cache = this.addMetrics(registry, metricsPrefix).build().addExtraMetrics(registry, metricsPrefix) -fun Caffeine.buildNamed(registry: MetricRegistry, metricsPrefix: String, loader: CacheLoader): LoadingCache = this.recordStats(CaffeineStatsCounter.supplier(registry, "Caches/$metricsPrefix")).buildNamed(metricsPrefix, loader).addExtraMetrics(registry, metricsPrefix) +fun Caffeine.buildNamed(registry: MetricRegistry, metricsPrefix: String, loader: CacheLoader): LoadingCache = this.addMetrics(registry, metricsPrefix).build(loader).addExtraMetrics(registry, metricsPrefix) private fun Caffeine.addMetrics(registry: MetricRegistry, metricsPrefix: String): Caffeine = this.recordStats(CaffeineStatsCounter.supplier(registry, "Caches/$metricsPrefix")) private fun > C.addExtraMetrics(registry: MetricRegistry, metricsPrefix: String): C = this.apply { diff --git a/node/src/main/kotlin/net/corda/node/utilities/EnterpriseNamedCacheFactory.kt b/node/src/main/kotlin/net/corda/node/utilities/EnterpriseNamedCacheFactory.kt index d0295bbe2a..a62d75f903 100644 --- a/node/src/main/kotlin/net/corda/node/utilities/EnterpriseNamedCacheFactory.kt +++ b/node/src/main/kotlin/net/corda/node/utilities/EnterpriseNamedCacheFactory.kt @@ -7,18 +7,17 @@ import com.github.benmanes.caffeine.cache.Caffeine import com.github.benmanes.caffeine.cache.LoadingCache import net.corda.core.internal.profiling.CacheTracing import net.corda.core.internal.profiling.CacheTracing.Companion.wrap -import net.corda.core.serialization.SingletonSerializeAsToken import net.corda.node.services.config.KB import net.corda.node.services.config.MB import net.corda.node.services.config.NodeConfiguration import java.util.concurrent.TimeUnit -class EnterpriseNamedCacheFactory private constructor(private val tracingConfig: CacheTracing.CacheTracingConfig, private val metricRegistry: MetricRegistry?, private val nodeConfiguration: NodeConfiguration?) : NamedCacheFactory, SingletonSerializeAsToken() { +class EnterpriseNamedCacheFactory private constructor(private val tracingConfig: CacheTracing.CacheTracingConfig, private val metricRegistry: MetricRegistry?, private val nodeConfiguration: NodeConfiguration?) : DefaultNamedCacheFactory(metricRegistry, nodeConfiguration) { constructor(tracingConfig: CacheTracing.CacheTracingConfig) : this(tracingConfig, null, null) - override fun bindWithMetrics(metricRegistry: MetricRegistry): NamedCacheFactory = EnterpriseNamedCacheFactory(tracingConfig, metricRegistry, this.nodeConfiguration) - override fun bindWithConfig(nodeConfiguration: NodeConfiguration): NamedCacheFactory = EnterpriseNamedCacheFactory(tracingConfig, this.metricRegistry, nodeConfiguration) + override fun bindWithMetrics(metricRegistry: MetricRegistry): BindableNamedCacheFactory = EnterpriseNamedCacheFactory(tracingConfig, metricRegistry, this.nodeConfiguration) + override fun bindWithConfig(nodeConfiguration: NodeConfiguration): BindableNamedCacheFactory = EnterpriseNamedCacheFactory(tracingConfig, this.metricRegistry, nodeConfiguration) // Scale most caches off the transaction cache size. private fun defaultBound(nodeConfiguration: NodeConfiguration): Long = nodeConfiguration.transactionCacheSizeBytes / 8.KB @@ -29,23 +28,25 @@ class EnterpriseNamedCacheFactory private constructor(private val tracingConfig: // This results in a minium of 1024 entries as per OS, but then grows linearly with attachment cache size. private fun defaultAttachmentPresenceCacheBound(nodeConfiguration: NodeConfiguration): Long = defaultAttachmentCacheBound(nodeConfiguration) / 10.KB - override fun buildNamed(caffeine: Caffeine, name: String): Cache { - checkNotNull(metricRegistry) - checkNotNull(nodeConfiguration) - return wrap(caffeine.maximumSize(defaultBound(nodeConfiguration!!)).buildNamed(metricRegistry!!, name), tracingConfig, name) + override val defaultCacheSize = if (nodeConfiguration == null) super.defaultCacheSize else defaultAttachmentCacheBound(nodeConfiguration) + + override fun configuredForNamed(caffeine: Caffeine, name: String): Caffeine { + return with(nodeConfiguration!!) { + when { + name == "NodeVaultService_producedStates" -> caffeine.maximumSize(defaultCacheSize) + name == "NodeAttachmentService_attachmentContent" -> caffeine.maximumWeight(defaultAttachmentCacheBound(this)) + name == "NodeAttachmentService_attachmentPresence" -> caffeine.maximumSize(defaultAttachmentPresenceCacheBound(this)) + name == "P2PMessageDeduplicator_senderUUIDSeqNoHWM" -> caffeine.expireAfterAccess(7, TimeUnit.DAYS) + else -> super.configuredForNamed(caffeine, name) + } + } + } + + override fun buildNamed(caffeine: Caffeine, name: String): Cache { + return wrap(configuredForNamed(caffeine, name).buildNamed(metricRegistry!!, name), tracingConfig, name) } - // TODO: allow a config file override for any named cache. override fun buildNamed(caffeine: Caffeine, name: String, loader: CacheLoader): LoadingCache { - checkNotNull(metricRegistry) - checkNotNull(nodeConfiguration) - val configuredCaffeine = when (name) { - "DBTransactionStorage_transactions" -> caffeine.maximumWeight(nodeConfiguration!!.transactionCacheSizeBytes) - "NodeAttachmentService_attachmentContent" -> caffeine.maximumWeight(defaultAttachmentCacheBound(nodeConfiguration!!)) - "NodeAttachmentService_attachmentPresence" -> caffeine.maximumSize(defaultAttachmentPresenceCacheBound(nodeConfiguration!!)) - "P2PMessageDeduplicator_senderUUIDSeqNoHWM" -> caffeine.expireAfterAccess(7, TimeUnit.DAYS) - else -> caffeine.maximumSize(defaultBound(nodeConfiguration!!)) - } - return wrap(configuredCaffeine.buildNamed(metricRegistry!!, name, loader), tracingConfig, name) + return wrap(configuredForNamed(caffeine, name).buildNamed(metricRegistry!!, name, loader), tracingConfig, name) } } \ No newline at end of file diff --git a/node/src/main/kotlin/net/corda/node/utilities/NodeNamedCache.kt b/node/src/main/kotlin/net/corda/node/utilities/NodeNamedCache.kt index 5c963ac80d..5c2b9a1241 100644 --- a/node/src/main/kotlin/net/corda/node/utilities/NodeNamedCache.kt +++ b/node/src/main/kotlin/net/corda/node/utilities/NodeNamedCache.kt @@ -27,13 +27,13 @@ interface BindableNamedCacheFactory : NamedCacheFactory, SerializeAsToken { fun bindWithConfig(nodeConfiguration: NodeConfiguration): BindableNamedCacheFactory } -open class DefaultNamedCacheFactory private constructor(private val metricRegistry: MetricRegistry?, private val nodeConfiguration: NodeConfiguration?) : BindableNamedCacheFactory, SingletonSerializeAsToken() { +open class DefaultNamedCacheFactory protected constructor(private val metricRegistry: MetricRegistry?, private val nodeConfiguration: NodeConfiguration?) : BindableNamedCacheFactory, SingletonSerializeAsToken() { constructor() : this(null, null) override fun bindWithMetrics(metricRegistry: MetricRegistry): BindableNamedCacheFactory = DefaultNamedCacheFactory(metricRegistry, this.nodeConfiguration) override fun bindWithConfig(nodeConfiguration: NodeConfiguration): BindableNamedCacheFactory = DefaultNamedCacheFactory(this.metricRegistry, nodeConfiguration) - protected fun configuredForNamed(caffeine: Caffeine, name: String): Caffeine { + open protected fun configuredForNamed(caffeine: Caffeine, name: String): Caffeine { return with(nodeConfiguration!!) { when { name.startsWith("RPCSecurityManagerShiroCache_") -> with(security?.authService?.options?.cache!!) { caffeine.maximumSize(maxEntries).expireAfterWrite(expireAfterSecs, TimeUnit.SECONDS) } @@ -77,5 +77,5 @@ open class DefaultNamedCacheFactory private constructor(private val metricRegist return configuredForNamed(caffeine, name).build(loader) } - protected val defaultCacheSize = 1024L + open protected val defaultCacheSize = 1024L } \ No newline at end of file diff --git a/node/src/test/kotlin/net/corda/node/services/identity/PersistentIdentityServiceTests.kt b/node/src/test/kotlin/net/corda/node/services/identity/PersistentIdentityServiceTests.kt index 0ec80de4f0..719e1fd380 100644 --- a/node/src/test/kotlin/net/corda/node/services/identity/PersistentIdentityServiceTests.kt +++ b/node/src/test/kotlin/net/corda/node/services/identity/PersistentIdentityServiceTests.kt @@ -7,7 +7,6 @@ import net.corda.core.identity.CordaX500Name import net.corda.core.identity.Party import net.corda.core.identity.PartyAndCertificate import net.corda.core.node.services.UnknownAnonymousPartyException -import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.nodeapi.internal.crypto.CertificateType import net.corda.nodeapi.internal.crypto.X509Utilities import net.corda.nodeapi.internal.crypto.x509Certificates @@ -16,6 +15,7 @@ import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.testing.core.* import net.corda.testing.internal.DEV_INTERMEDIATE_CA import net.corda.testing.internal.DEV_ROOT_CA +import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import net.corda.testing.node.makeTestIdentityService diff --git a/node/src/test/kotlin/net/corda/node/services/persistence/AppendOnlyPersistentMapTest.kt b/node/src/test/kotlin/net/corda/node/services/persistence/AppendOnlyPersistentMapTest.kt index 5e36bd301c..0dea08f1b6 100644 --- a/node/src/test/kotlin/net/corda/node/services/persistence/AppendOnlyPersistentMapTest.kt +++ b/node/src/test/kotlin/net/corda/node/services/persistence/AppendOnlyPersistentMapTest.kt @@ -4,8 +4,8 @@ import net.corda.core.schemas.MappedSchema import net.corda.core.utilities.loggerFor import net.corda.node.services.schema.NodeSchemaService import net.corda.node.utilities.AppendOnlyPersistentMap -import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.nodeapi.internal.persistence.DatabaseConfig +import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import org.junit.After diff --git a/node/src/test/kotlin/net/corda/node/services/persistence/DBTransactionStorageTests.kt b/node/src/test/kotlin/net/corda/node/services/persistence/DBTransactionStorageTests.kt index 65518db845..66ac3198c3 100644 --- a/node/src/test/kotlin/net/corda/node/services/persistence/DBTransactionStorageTests.kt +++ b/node/src/test/kotlin/net/corda/node/services/persistence/DBTransactionStorageTests.kt @@ -8,11 +8,11 @@ import net.corda.core.crypto.TransactionSignature import net.corda.core.toFuture import net.corda.core.transactions.SignedTransaction import net.corda.node.services.transactions.PersistentUniquenessProvider -import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.testing.core.* import net.corda.testing.internal.LogHelper +import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.testing.internal.configureDatabase import net.corda.testing.internal.createWireTransaction import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties diff --git a/node/src/test/kotlin/net/corda/node/services/persistence/NodeAttachmentServiceTest.kt b/node/src/test/kotlin/net/corda/node/services/persistence/NodeAttachmentServiceTest.kt index 0784bfc9df..a34053f8ae 100644 --- a/node/src/test/kotlin/net/corda/node/services/persistence/NodeAttachmentServiceTest.kt +++ b/node/src/test/kotlin/net/corda/node/services/persistence/NodeAttachmentServiceTest.kt @@ -14,10 +14,10 @@ import net.corda.core.node.services.vault.Builder import net.corda.core.node.services.vault.Sort import net.corda.core.utilities.getOrThrow import net.corda.node.services.transactions.PersistentUniquenessProvider -import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.testing.internal.LogHelper +import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import net.corda.testing.node.internal.InternalMockNetwork diff --git a/node/src/test/kotlin/net/corda/node/services/persistence/RunOnceServiceTest.kt b/node/src/test/kotlin/net/corda/node/services/persistence/RunOnceServiceTest.kt index 614f0f5ea4..1cc10972e2 100644 --- a/node/src/test/kotlin/net/corda/node/services/persistence/RunOnceServiceTest.kt +++ b/node/src/test/kotlin/net/corda/node/services/persistence/RunOnceServiceTest.kt @@ -4,8 +4,8 @@ import com.nhaarman.mockito_kotlin.any import com.nhaarman.mockito_kotlin.mock import com.nhaarman.mockito_kotlin.verify import com.nhaarman.mockito_kotlin.whenever -import net.corda.node.internal.configureDatabase import net.corda.nodeapi.internal.persistence.CordaPersistence +import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import net.corda.testing.node.internal.makeTestDatabaseProperties import org.junit.After diff --git a/node/src/test/kotlin/net/corda/node/services/persistence/SchemaMigrationTest.kt b/node/src/test/kotlin/net/corda/node/services/persistence/SchemaMigrationTest.kt index f775621d18..6dea181d19 100644 --- a/node/src/test/kotlin/net/corda/node/services/persistence/SchemaMigrationTest.kt +++ b/node/src/test/kotlin/net/corda/node/services/persistence/SchemaMigrationTest.kt @@ -12,6 +12,7 @@ import net.corda.node.services.schema.NodeSchemaService import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.nodeapi.internal.persistence.SchemaMigration +import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.testing.node.MockServices import org.apache.commons.io.FileUtils import org.assertj.core.api.Assertions.assertThat @@ -19,18 +20,18 @@ import org.junit.Test import java.lang.reflect.Method import java.math.BigInteger import java.net.URL -import javax.persistence.* import java.net.URLClassLoader import java.nio.file.Files import java.nio.file.Path import java.util.* +import javax.persistence.* class SchemaMigrationTest { private fun configureDatabase(hikariProperties: Properties, databaseConfig: DatabaseConfig, schemaService: NodeSchemaService = NodeSchemaService()): CordaPersistence = - createCordaPersistence(databaseConfig, { null }, { null }, schemaService) + createCordaPersistence(databaseConfig, { null }, { null }, schemaService, TestingNamedCacheFactory()) .apply { startHikariPool(hikariProperties, databaseConfig, schemaService.schemaOptions.keys) } @Test diff --git a/node/src/test/kotlin/net/corda/node/services/schema/PersistentStateServiceTests.kt b/node/src/test/kotlin/net/corda/node/services/schema/PersistentStateServiceTests.kt index a3b142b55f..37e8a21255 100644 --- a/node/src/test/kotlin/net/corda/node/services/schema/PersistentStateServiceTests.kt +++ b/node/src/test/kotlin/net/corda/node/services/schema/PersistentStateServiceTests.kt @@ -17,7 +17,6 @@ import net.corda.testing.contracts.DummyContract import net.corda.testing.core.TestIdentity import net.corda.testing.internal.LogHelper import net.corda.testing.internal.configureDatabase -import net.corda.testing.internal.rigorousMock import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import org.junit.After import org.junit.Before diff --git a/node/src/test/kotlin/net/corda/node/services/transactions/PersistentUniquenessProviderTests.kt b/node/src/test/kotlin/net/corda/node/services/transactions/PersistentUniquenessProviderTests.kt index 067f6ac7eb..b8e63f4b13 100644 --- a/node/src/test/kotlin/net/corda/node/services/transactions/PersistentUniquenessProviderTests.kt +++ b/node/src/test/kotlin/net/corda/node/services/transactions/PersistentUniquenessProviderTests.kt @@ -9,13 +9,13 @@ import net.corda.core.flows.NotaryError import net.corda.core.identity.CordaX500Name import net.corda.core.internal.notary.NotaryInternalException import net.corda.node.services.schema.NodeSchemaService -import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig import net.corda.testing.core.SerializationEnvironmentRule import net.corda.testing.core.TestIdentity import net.corda.testing.core.generateStateRef import net.corda.testing.internal.LogHelper +import net.corda.testing.internal.TestingNamedCacheFactory import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import org.junit.After diff --git a/node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt b/node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt index 2e976fa9af..cd98fe5476 100644 --- a/node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt +++ b/node/src/test/kotlin/net/corda/node/services/vault/VaultQueryTests.kt @@ -7,9 +7,9 @@ import net.corda.core.identity.CordaX500Name import net.corda.core.identity.Party import net.corda.core.internal.packageName import net.corda.core.node.services.* +import net.corda.core.node.services.Vault.ConstraintInfo.Type.* import net.corda.core.node.services.vault.* import net.corda.core.node.services.vault.QueryCriteria.* -import net.corda.core.node.services.Vault.ConstraintInfo.Type.* import net.corda.core.transactions.LedgerTransaction import net.corda.core.transactions.TransactionBuilder import net.corda.core.utilities.* diff --git a/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/EnterpriseMockNamedCachedFactory.kt b/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/EnterpriseMockNamedCachedFactory.kt index c6d989af37..9468624e7e 100644 --- a/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/EnterpriseMockNamedCachedFactory.kt +++ b/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/EnterpriseMockNamedCachedFactory.kt @@ -5,22 +5,21 @@ import com.github.benmanes.caffeine.cache.Cache import com.github.benmanes.caffeine.cache.CacheLoader import com.github.benmanes.caffeine.cache.Caffeine import com.github.benmanes.caffeine.cache.LoadingCache -import net.corda.core.internal.buildNamed import net.corda.core.serialization.SingletonSerializeAsToken import net.corda.node.services.config.MB import net.corda.node.services.config.NodeConfiguration -import net.corda.node.utilities.NamedCacheFactory +import net.corda.node.utilities.BindableNamedCacheFactory import java.util.concurrent.TimeUnit -class EnterpriseMockNamedCachedFactory(private val sizeOverride: Long, private val metricRegistry: MetricRegistry?, private val nodeConfiguration: NodeConfiguration?) : NamedCacheFactory, SingletonSerializeAsToken() { +class EnterpriseMockNamedCachedFactory(private val sizeOverride: Long, private val metricRegistry: MetricRegistry?, private val nodeConfiguration: NodeConfiguration?) : BindableNamedCacheFactory, SingletonSerializeAsToken() { constructor(sizeOverride: Long = 1024) : this(sizeOverride, null, null) - override fun bindWithMetrics(metricRegistry: MetricRegistry): NamedCacheFactory = EnterpriseMockNamedCachedFactory(sizeOverride, metricRegistry, this.nodeConfiguration) - override fun bindWithConfig(nodeConfiguration: NodeConfiguration): NamedCacheFactory = EnterpriseMockNamedCachedFactory(sizeOverride, this.metricRegistry, nodeConfiguration) + override fun bindWithMetrics(metricRegistry: MetricRegistry): BindableNamedCacheFactory = EnterpriseMockNamedCachedFactory(sizeOverride, metricRegistry, this.nodeConfiguration) + override fun bindWithConfig(nodeConfiguration: NodeConfiguration): BindableNamedCacheFactory = EnterpriseMockNamedCachedFactory(sizeOverride, this.metricRegistry, nodeConfiguration) override fun buildNamed(caffeine: Caffeine, name: String): Cache { // Does not check metricRegistry or nodeConfiguration, because for tests we don't care. - return caffeine.maximumSize(sizeOverride).buildNamed(name) + return caffeine.maximumSize(sizeOverride).build() } override fun buildNamed(caffeine: Caffeine, name: String, loader: CacheLoader): LoadingCache { @@ -31,6 +30,6 @@ class EnterpriseMockNamedCachedFactory(private val sizeOverride: Long, private v "P2PMessageDeduplicator_senderUUIDSeqNoHWM" -> caffeine.expireAfterAccess(1, TimeUnit.HOURS) else -> caffeine.maximumSize(sizeOverride) } - return configuredCaffeine.buildNamed(name, loader) + return configuredCaffeine.build(loader) } } diff --git a/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/InternalMockNetwork.kt b/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/InternalMockNetwork.kt index 34b6027eba..8efbc54931 100644 --- a/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/InternalMockNetwork.kt +++ b/testing/node-driver/src/main/kotlin/net/corda/testing/node/internal/InternalMockNetwork.kt @@ -42,13 +42,13 @@ import net.corda.node.services.messaging.MessagingService import net.corda.node.services.persistence.NodeAttachmentService import net.corda.node.services.statemachine.StateMachineManager import net.corda.node.utilities.AffinityExecutor.ServiceAffinityExecutor -import net.corda.node.utilities.DefaultNamedCacheFactory +import net.corda.node.utilities.EnterpriseNamedCacheFactory +import net.corda.node.utilities.profiling.getTracingConfig import net.corda.nodeapi.internal.DevIdentityGenerator import net.corda.nodeapi.internal.config.User import net.corda.nodeapi.internal.network.NetworkParametersCopier import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig -import net.corda.testing.node.TestCordapp import net.corda.testing.common.internal.testNetworkParameters import net.corda.testing.internal.rigorousMock import net.corda.testing.internal.setGlobalSerialization @@ -269,7 +269,7 @@ open class InternalMockNetwork(defaultParameters: MockNetworkParameters = MockNe open class MockNode(args: MockNodeArgs) : AbstractNode( args.config, TestClock(Clock.systemUTC()), - DefaultNamedCacheFactory(), + EnterpriseNamedCacheFactory(args.config.enterpriseConfiguration.getTracingConfig()), args.version, args.network.getServerThread(args.id), args.network.busyLatch diff --git a/testing/test-utils/src/main/kotlin/net/corda/testing/internal/InternalTestUtils.kt b/testing/test-utils/src/main/kotlin/net/corda/testing/internal/InternalTestUtils.kt index 8f82cf3792..e3e7b0ae6d 100644 --- a/testing/test-utils/src/main/kotlin/net/corda/testing/internal/InternalTestUtils.kt +++ b/testing/test-utils/src/main/kotlin/net/corda/testing/internal/InternalTestUtils.kt @@ -11,7 +11,6 @@ import net.corda.core.identity.Party import net.corda.core.identity.PartyAndCertificate import net.corda.core.internal.NamedCacheFactory import net.corda.core.node.NodeInfo -import net.corda.core.schemas.MappedSchema import net.corda.core.transactions.WireTransaction import net.corda.core.utilities.loggerFor import net.corda.node.internal.createCordaPersistence @@ -30,6 +29,7 @@ import net.corda.nodeapi.internal.crypto.X509Utilities import net.corda.nodeapi.internal.loadDevCaTrustStore import net.corda.nodeapi.internal.persistence.CordaPersistence import net.corda.nodeapi.internal.persistence.DatabaseConfig +import net.corda.nodeapi.internal.persistence.isH2Database import net.corda.nodeapi.internal.registerDevP2pCertificates import net.corda.serialization.internal.amqp.AMQP_ENABLED import net.corda.testing.internal.stubs.CertificateStoreStubs @@ -167,9 +167,10 @@ fun configureDatabase(hikariProperties: Properties, wellKnownPartyFromX500Name: (CordaX500Name) -> Party?, wellKnownPartyFromAnonymous: (AbstractParty) -> Party?, schemaService: SchemaService = NodeSchemaService(), - internalSchemas: Set = NodeSchemaService().internalSchemas(), cacheFactory: NamedCacheFactory = TestingNamedCacheFactory()): CordaPersistence { - val persistence = createCordaPersistence(databaseConfig, wellKnownPartyFromX500Name, wellKnownPartyFromAnonymous, schemaService, hikariProperties, cacheFactory) - persistence.startHikariPool(hikariProperties, databaseConfig, internalSchemas) + val isH2Database = isH2Database(hikariProperties.getProperty("dataSource.url", "")) + val schemas = if (isH2Database) NodeSchemaService().internalSchemas() else schemaService.schemaOptions.keys + val persistence = createCordaPersistence(databaseConfig, wellKnownPartyFromX500Name, wellKnownPartyFromAnonymous, schemaService, cacheFactory) + persistence.startHikariPool(hikariProperties, databaseConfig, schemas) return persistence } \ No newline at end of file From dcf2e7b7ea7f7f19988b969f81726fb7d82b50f5 Mon Sep 17 00:00:00 2001 From: "rick.parker" Date: Fri, 19 Oct 2018 10:39:40 +0100 Subject: [PATCH 4/8] Fix up errors and revert changelog.rst --- docs/source/changelog.rst | 115 +++--------------- .../corda/flowworker/FlowWorkerServiceHub.kt | 7 +- 2 files changed, 18 insertions(+), 104 deletions(-) diff --git a/docs/source/changelog.rst b/docs/source/changelog.rst index 46fe5c0b43..ba16ef9b20 100644 --- a/docs/source/changelog.rst +++ b/docs/source/changelog.rst @@ -26,6 +26,8 @@ Unreleased * Removed experimental feature ``CordformDefinition`` +* Vault query fix: support query by parent classes of Contract State classes (see https://github.com/corda/corda/issues/3714) + * Added ``registerResponderFlow`` method to ``StartedMockNode``, to support isolated testing of responder flow behaviour. * "app", "rpc", "p2p" and "unknown" are no longer allowed as uploader values when importing attachments. These are used @@ -62,6 +64,9 @@ Unreleased interfaces that will have unimplemented methods. This is useful, for example, for object viewers. This can be turned on with ``SerializationContext.withLenientCarpenter``. +* Introduced a grace period before the initial node registration fails if the node cannot connect to the Doorman. + It retries 10 times with a 1 minute interval in between each try. At the moment this is not configurable. + * Added a ``FlowMonitor`` to log information about flows that have been waiting for IO more than a configurable threshold. * H2 database changes: @@ -175,7 +180,7 @@ Unreleased * Added public support for creating ``CordaRPCClient`` using SSL. For this to work the node needs to provide client applications a certificate to be added to a truststore. See :doc:`tutorial-clientrpc-api` -* The node RPC broker opens 2 endpoints that are configured with ``address`` and ``adminAddress``. RPC Clients would connect +*The node RPC broker opens 2 endpoints that are configured with ``address`` and ``adminAddress``. RPC Clients would connect to the address, while the node will connect to the adminAddress. Previously if ssl was enabled for RPC the ``adminAddress`` was equal to ``address``. @@ -229,73 +234,6 @@ Unreleased normal state when it occurs in an input or output position. *This feature is only available on Corda networks running with a minimum platform version of 4.* -Version 3.3 ------------ - -* Vault query fix: support query by parent classes of Contract State classes (see https://github.com/corda/corda/issues/3714) - -* Fixed an issue preventing Shell from returning control to the user when CTRL+C is pressed in the terminal. - -* Fixed a problem that sometimes prevented nodes from starting in presence of custom state types in the database without a corresponding type from installed CorDapps. - -* Introduced a grace period before the initial node registration fails if the node cannot connect to the Doorman. - It retries 10 times with a 1 minute interval in between each try. At the moment this is not configurable. - -* Fixed an error thrown by NodeVaultService upon recording a transaction with a number of inputs greater than the default page size. - -* Changes to the JSON/YAML serialisation format from ``JacksonSupport``, which also applies to the node shell: - - * ``Instant`` and ``Date`` objects are serialised as ISO-8601 formatted strings rather than timestamps - * ``PublicKey`` objects are serialised and looked up according to their Base58 encoded string - * ``Party`` objects can be deserialised by looking up their public key, in addition to their name - * ``NodeInfo`` objects are serialised as an object and can be looked up using the same mechanism as ``Party`` - * ``NetworkHostAndPort`` serialised according to its ``toString()`` - * ``PartyAndCertificate`` is serialised as the name - * ``SerializedBytes`` is serialised by materialising the bytes into the object it represents, and then serialising that - object into YAML/JSON - * ``X509Certificate`` is serialised as an object with key fields such as ``issuer``, ``publicKey``, ``serialNumber``, etc. - The encoded bytes are also serialised into the ``encoded`` field. This can be used to deserialise an ``X509Certificate`` - back. - * ``CertPath`` objects are serialised as a list of ``X509Certificate`` objects. - -* ``fullParties`` boolean parameter added to ``JacksonSupport.createDefaultMapper`` and ``createNonRpcMapper``. If ``true`` - then ``Party`` objects are serialised as JSON objects with the ``name`` and ``owningKey`` fields. For ``PartyAndCertificate`` - the ``certPath`` is serialised. - -* Several members of ``JacksonSupport`` have been deprecated to highlight that they are internal and not to be used - -* ``ServiceHub`` and ``CordaRPCOps`` can now safely be used from multiple threads without incurring in database transaction problems. - -* Fixed an issue preventing out of process nodes started by the ``Driver`` from logging to file. - -* The Vault Criteria API has been extended to take a more precise specification of which class contains a field. This primarily impacts Java users; Kotlin users need take no action. The old methods have been deprecated but still work - the new methods avoid bugs that can occur when JPA schemas inherit from each other. - -* Removed -xmx VM argument from Explorer's Capsule setup. This helps avoiding out of memory errors. - -* Node will now gracefully fail to start if one of the required ports is already in use. - -* Fixed incorrect exception handling in ``NodeVaultService._query()``. - -* Avoided a memory leak deriving from incorrect MappedSchema caching strategy. - -* Fix CORDA-1403 where a property of a class that implemented a generic interface could not be deserialised in - a factory without a serialiser as the subtype check for the class instance failed. Fix is to compare the raw - type. - -* Fix CORDA-1229. Setter-based serialization was broken with generic types when the property was stored - as the raw type, List for example. - -.. _changelog_v3.2: - -Version 3.2 ------------ - -* Doorman and NetworkMap URLs can now be configured individually rather than being assumed to be - the same server. Current ``compatibilityZoneURL`` configurations remain valid. See both :doc:`corda-configuration-file` - and :doc:`permissioning` for details. - -* Table name with a typo changed from ``NODE_ATTCHMENTS_CONTRACTS`` to ``NODE_ATTACHMENTS_CONTRACTS``. - .. _changelog_v3.1: Version 3.1 @@ -304,7 +242,7 @@ Version 3.1 * Update the fast-classpath-scanner dependent library version from 2.0.21 to 2.12.3 .. note:: Whilst this is not the latest version of this library, that being 2.18.1 at time of writing, versions -later than 2.12.3 (including 2.12.4) exhibit a different issue. + later than 2.12.3 (including 2.12.4) exhibit a different issue. * Added `database.hibernateDialect` node configuration option @@ -342,18 +280,15 @@ Corda Enterprise 3.0 Developer Preview where they all share a common identity. ``NetworkMapCache.getNodeByLegalName`` has been tightened to throw if more than one node with the legal name is found. -* Due to a security risk, the `conflict` property has been removed from `NotaryError.Conflict` error object. It has been replaced - with `consumedStates` instead. The new property no longer specifies the original requesting party and transaction id for - a consumed state. Instead, only the hash of the transaction id is revealed. For more details why this change had to be - made please refer to the release notes. +* Per CorDapp configuration is now exposed. ``CordappContext`` now exposes a ``CordappConfig`` object that is populated + at CorDapp context creation time from a file source during runtime. -* Added ``NetworkMapCache.getNodesByLegalName`` for querying nodes belonging to a distributed service such as a notary cluster - where they all share a common identity. ``NetworkMapCache.getNodeByLegalName`` has been tightened to throw if more than - one node with the legal name is found. +* Introduced Flow Draining mode, in which a node continues executing existing flows, but does not start new. This is to + support graceful node shutdown/restarts. In particular, when this mode is on, new flows through RPC will be rejected, + scheduled flows will be ignored, and initial session messages will not be consumed. This will ensure that the number of + checkpoints will strictly diminish with time, allowing for a clean shutdown. -* Introduced Flow Draining mode, in which a node continues executing existing flows, but does not start new. This is to support graceful node shutdown/restarts. - In particular, when this mode is on, new flows through RPC will be rejected, scheduled flows will be ignored, and initial session messages will not be consumed. - This will ensure that the number of checkpoints will strictly diminish with time, allowing for a clean shutdown. +* Make the serialisation finger-printer a pluggable entity rather than hard wiring into the factory * Removed blacklisted word checks in Corda X.500 name to allow "Server" or "Node" to be use as part of the legal name. @@ -376,28 +311,6 @@ Corda Enterprise 3.0 Developer Preview * JPA Mapping annotations for States extending ``CommonSchemaV1.LinearState`` and ``CommonSchemaV1.FungibleState`` on the `participants` collection need to be moved to the actual class. This allows to properly specify the unique table name per a collection. See: DummyDealStateSchemaV1.PersistentDummyDealState -* Database schema changes - an H2 database instance of Corda 1.0 and 2.0 cannot be reused for Corda 3.0, listed changes for Vault and Finance module: - - * ``NODE_TRANSACTIONS``: - column ``"TRANSACTION”`` renamed to ``TRANSACTION_VALUE``, serialization format of BLOB stored in the column has changed to AMQP - * ``VAULT_STATES``: - column ``CONTRACT_STATE`` removed - * ``VAULT_FUNGIBLE_STATES``: - column ``ISSUER_REFERENCE`` renamed to ``ISSUER_REF`` and the field size increased - * ``"VAULTSCHEMAV1$VAULTFUNGIBLESTATES_PARTICIPANTS"``: - table renamed to ``VAULT_FUNGIBLE_STATES_PARTS``, - column ``"VAULTSCHEMAV1$VAULTFUNGIBLESTATES_OUTPUT_INDEX"`` renamed to ``OUTPUT_INDEX``, - column ``"VAULTSCHEMAV1$VAULTFUNGIBLESTATES_TRANSACTION_ID"`` renamed to ``TRANSACTION_ID`` - * ``VAULT_LINEAR_STATES``: - type of column ``"UUID"`` changed from ``VARBINARY`` to ``VARCHAR(255)`` - select varbinary column as ``CAST("UUID" AS UUID)`` to get UUID in varchar format - * ``"VAULTSCHEMAV1$VAULTLINEARSTATES_PARTICIPANTS"``: - table renamed to ``VAULT_LINEAR_STATES_PARTS``, - column ``"VAULTSCHEMAV1$VAULTLINEARSTATES_OUTPUT_INDEX"`` renamed to ``OUTPUT_INDEX``, - column ``"VAULTSCHEMAV1$VAULTLINEARSTATES_TRANSACTION_ID"`` renamed to ``TRANSACTION_ID`` - * ``contract_cash_states``: - columns storing Base58 representation of the serialised public key (e.g. ``issuer_key``) were changed to store Base58 representation of SHA-256 of public key prefixed with `DL` - * ``contract_cp_states``: - table renamed to ``cp_states``, column changes as for ``contract_cash_states`` * JPA Mapping annotations for States extending ``CommonSchemaV1.LinearState`` and ``CommonSchemaV1.FungibleState`` on the `participants` collection need to be moved to the actual State class. This allows developers to properly specify diff --git a/experimental/flow-worker/src/main/kotlin/net/corda/flowworker/FlowWorkerServiceHub.kt b/experimental/flow-worker/src/main/kotlin/net/corda/flowworker/FlowWorkerServiceHub.kt index 43238f2d9e..d40a2949bc 100644 --- a/experimental/flow-worker/src/main/kotlin/net/corda/flowworker/FlowWorkerServiceHub.kt +++ b/experimental/flow-worker/src/main/kotlin/net/corda/flowworker/FlowWorkerServiceHub.kt @@ -123,7 +123,8 @@ class FlowWorkerServiceHub(override val configuration: NodeConfiguration, overri configuration.database, identityService::wellKnownPartyFromX500Name, identityService::wellKnownPartyFromAnonymous, - schemaService + schemaService, + cacheFactory ) init { @@ -142,7 +143,7 @@ class FlowWorkerServiceHub(override val configuration: NodeConfiguration, overri private val servicesForResolution = ServicesForResolutionImpl(identityService, attachments, cordappProvider, validatedTransactions) @Suppress("LeakingThis") override val vaultService = NodeVaultService(clock, keyManagementService, servicesForResolution, database, schemaService, cacheFactory).tokenize() - override val nodeProperties = NodePropertiesPersistentStore(StubbedNodeUniqueIdProvider::value, database) + override val nodeProperties = NodePropertiesPersistentStore(StubbedNodeUniqueIdProvider::value, database, cacheFactory) val flowLogicRefFactory = FlowLogicRefFactoryImpl(cordappLoader.appClassLoader) override val monitoringService = MonitoringService(metricRegistry).tokenize() @@ -163,7 +164,7 @@ class FlowWorkerServiceHub(override val configuration: NodeConfiguration, overri private val transactionVerifierWorkerCount = 4 @Suppress("LeakingThis") override val transactionVerifierService = InMemoryTransactionVerifierService(transactionVerifierWorkerCount).tokenize() - override val contractUpgradeService = ContractUpgradeServiceImpl().tokenize() + override val contractUpgradeService = ContractUpgradeServiceImpl(cacheFactory).tokenize() override val auditService = DummyAuditService().tokenize() @Suppress("LeakingThis") From 3092dd64b4c9e1959ad008b02b63abf831ea70ec Mon Sep 17 00:00:00 2001 From: "rick.parker" Date: Fri, 19 Oct 2018 11:08:30 +0100 Subject: [PATCH 5/8] Accidental change whilst trying to get tests to pass locally. --- .../net/corda/nodeapi/internal/crypto/X509UtilitiesTest.kt | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/node-api/src/test/kotlin/net/corda/nodeapi/internal/crypto/X509UtilitiesTest.kt b/node-api/src/test/kotlin/net/corda/nodeapi/internal/crypto/X509UtilitiesTest.kt index e319ed71e8..0a3d770d33 100644 --- a/node-api/src/test/kotlin/net/corda/nodeapi/internal/crypto/X509UtilitiesTest.kt +++ b/node-api/src/test/kotlin/net/corda/nodeapi/internal/crypto/X509UtilitiesTest.kt @@ -5,6 +5,7 @@ import io.netty.handler.ssl.ClientAuth import io.netty.handler.ssl.SslContextBuilder import io.netty.handler.ssl.SslProvider import net.corda.core.crypto.Crypto +import net.corda.core.crypto.* import net.corda.core.crypto.Crypto.COMPOSITE_KEY import net.corda.core.crypto.Crypto.ECDSA_SECP256K1_SHA256 import net.corda.core.crypto.Crypto.ECDSA_SECP256R1_SHA256 @@ -12,8 +13,6 @@ import net.corda.core.crypto.Crypto.EDDSA_ED25519_SHA512 import net.corda.core.crypto.Crypto.RSA_SHA256 import net.corda.core.crypto.Crypto.SPHINCS256_SHA256 import net.corda.core.crypto.Crypto.generateKeyPair -import net.corda.core.crypto.SignatureScheme -import net.corda.core.crypto.newSecureRandom import net.corda.core.identity.CordaX500Name import net.corda.core.internal.div import net.corda.core.serialization.SerializationContext @@ -401,7 +400,7 @@ class X509UtilitiesTest { val clientHandler = NettyTestHandler { _, msg -> assertEquals("Hello", NettyTestHandler.readString(msg)) } NettyTestServer(sslServerContext, serverHandler, portAllocation.nextPort()).use { server -> server.start() - NettyTestClient(sslClientContext, "localhost", server.port, clientHandler).use { client -> + NettyTestClient(sslClientContext, InetAddress.getLocalHost().canonicalHostName, server.port, clientHandler).use { client -> client.start() clientHandler.writeString("Hello") From c16dbfd82b3b8691e30a2a2171648a4d50d8d2ee Mon Sep 17 00:00:00 2001 From: "rick.parker" Date: Fri, 19 Oct 2018 11:47:52 +0100 Subject: [PATCH 6/8] Compilation error fix. --- .../kotlin/net/corda/notary/jpa/JPAUniquenessProviderTests.kt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/notary/jpa/src/test/kotlin/net/corda/notary/jpa/JPAUniquenessProviderTests.kt b/notary/jpa/src/test/kotlin/net/corda/notary/jpa/JPAUniquenessProviderTests.kt index 068a2c416d..4fa10a59b1 100644 --- a/notary/jpa/src/test/kotlin/net/corda/notary/jpa/JPAUniquenessProviderTests.kt +++ b/notary/jpa/src/test/kotlin/net/corda/notary/jpa/JPAUniquenessProviderTests.kt @@ -8,7 +8,6 @@ import net.corda.core.flows.NotarisationRequestSignature import net.corda.core.flows.NotaryError import net.corda.core.identity.CordaX500Name import net.corda.core.internal.notary.NotaryInternalException -import net.corda.node.internal.configureDatabase import net.corda.node.services.config.NotaryConfig import net.corda.node.services.schema.NodeSchemaService import net.corda.notary.jpa.JPAUniquenessProvider.Companion.decodeStateRef @@ -19,6 +18,7 @@ import net.corda.testing.core.SerializationEnvironmentRule import net.corda.testing.core.TestIdentity import net.corda.testing.core.generateStateRef import net.corda.testing.internal.LogHelper +import net.corda.testing.internal.configureDatabase import net.corda.testing.node.MockServices.Companion.makeTestDataSourceProperties import org.junit.After import org.junit.Before From eef4c5792b83dfaebc45f995a3033a8ad90446ea Mon Sep 17 00:00:00 2001 From: "rick.parker" Date: Fri, 19 Oct 2018 14:19:50 +0100 Subject: [PATCH 7/8] Compilation error fix. --- .../kotlin/net/corda/rpcWorker/RpcFlowWorkerDriver.kt | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/experimental/rpc-worker/src/integration-test/kotlin/net/corda/rpcWorker/RpcFlowWorkerDriver.kt b/experimental/rpc-worker/src/integration-test/kotlin/net/corda/rpcWorker/RpcFlowWorkerDriver.kt index 5d07d4e73c..be3ea0938d 100644 --- a/experimental/rpc-worker/src/integration-test/kotlin/net/corda/rpcWorker/RpcFlowWorkerDriver.kt +++ b/experimental/rpc-worker/src/integration-test/kotlin/net/corda/rpcWorker/RpcFlowWorkerDriver.kt @@ -32,6 +32,8 @@ import net.corda.node.services.config.* import net.corda.node.services.messaging.ArtemisMessagingServer import net.corda.node.services.network.NodeInfoWatcher import net.corda.node.services.rpc.ArtemisRpcBroker +import net.corda.node.utilities.EnterpriseNamedCacheFactory +import net.corda.node.utilities.profiling.getTracingConfig import net.corda.nodeapi.internal.NodeInfoAndSigned import net.corda.nodeapi.internal.config.User import net.corda.nodeapi.internal.config.toConfig @@ -221,7 +223,7 @@ data class RpcFlowWorkerDriverDSL(private val driverDSL: DriverDSLImpl) : Intern private fun createRpcWorkerBroker(config: NodeConfiguration): ArtemisBroker { val rpcOptions = config.rpcOptions - val securityManager = RPCSecurityManagerImpl(SecurityConfiguration.AuthService.fromUsers(config.rpcUsers)) + val securityManager = RPCSecurityManagerImpl(SecurityConfiguration.AuthService.fromUsers(config.rpcUsers), EnterpriseNamedCacheFactory(config.enterpriseConfiguration.getTracingConfig())) val broker = if (rpcOptions.useSsl) { ArtemisRpcBroker.withSsl(config.p2pSslOptions, rpcOptions.address, rpcOptions.adminAddress, rpcOptions.sslConfig!!, securityManager, Node.MAX_RPC_MESSAGE_SIZE, false, config.baseDirectory / "artemis", false) From 3dcd8bc84ccc3fdcdb111b1c70d7dd75aaf7e4e6 Mon Sep 17 00:00:00 2001 From: "rick.parker" Date: Fri, 19 Oct 2018 15:27:35 +0100 Subject: [PATCH 8/8] Fix TODO --- .../src/main/kotlin/net/corda/rpcWorker/RpcWorker.kt | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/experimental/rpc-worker/src/main/kotlin/net/corda/rpcWorker/RpcWorker.kt b/experimental/rpc-worker/src/main/kotlin/net/corda/rpcWorker/RpcWorker.kt index 9ac12f69ee..0ddf23fb71 100644 --- a/experimental/rpc-worker/src/main/kotlin/net/corda/rpcWorker/RpcWorker.kt +++ b/experimental/rpc-worker/src/main/kotlin/net/corda/rpcWorker/RpcWorker.kt @@ -15,7 +15,8 @@ import net.corda.node.services.messaging.InternalRPCMessagingClient import net.corda.node.services.messaging.RPCOpsRouting import net.corda.node.services.messaging.RPCServerConfiguration import net.corda.node.services.rpc.ArtemisRpcBroker -import net.corda.node.utilities.DefaultNamedCacheFactory +import net.corda.node.utilities.EnterpriseNamedCacheFactory +import net.corda.node.utilities.profiling.getTracingConfig import net.corda.nodeapi.internal.config.User import net.corda.nodeapi.internal.crypto.X509Utilities import org.apache.activemq.artemis.api.core.management.ActiveMQServerControl @@ -101,8 +102,7 @@ class Main : Runnable { private fun createRpcWorkerBroker(config: NodeConfiguration, maxMessageSize: Int): ArtemisBroker { val rpcOptions = config.rpcOptions - // TODO: wire this up to an EnterpriseCacheFactory - val securityManager = RPCSecurityManagerImpl(SecurityConfiguration.AuthService.fromUsers(config.rpcUsers), DefaultNamedCacheFactory()) + val securityManager = RPCSecurityManagerImpl(SecurityConfiguration.AuthService.fromUsers(config.rpcUsers), EnterpriseNamedCacheFactory(config.enterpriseConfiguration.getTracingConfig())) val broker = if (rpcOptions.useSsl) { ArtemisRpcBroker.withSsl(config.p2pSslOptions, rpcOptions.address, rpcOptions.adminAddress, rpcOptions.sslConfig!!, securityManager, maxMessageSize, false, config.baseDirectory / "artemis", false) } else {