Commit Graph

7664 Commits

Author SHA1 Message Date
Mike Hearn
d5f6d90b37 TWP: Address review comments 2019-08-22 19:05:06 +02:00
Mike Hearn
46a305602b Attempt to make a sentence about constraints easier to understand. 2019-08-22 19:05:06 +02:00
Mike Hearn
32450631a7 Move some sections into a "future work" section. 2019-08-22 19:05:06 +02:00
Mike Hearn
cc389f2a9c Read through the rest of the paper with minor tweaks. 2019-08-22 19:05:06 +02:00
Mike Hearn
59f839d80a Add remark about flow deniability. 2019-08-22 19:05:06 +02:00
Mike Hearn
d3c9679389 Rewrap file to a column limit that should fit in the GitHub diff viewer. 2019-08-22 19:05:06 +02:00
Mike Hearn
bf5cb51236 Add discussion of state pointers and the tokens SDK. (#5243)
* Add discussion of state pointers and the tokens SDK.

Fix ugly hyperlink boxes.

Comment out "market infrastructure" section for now as it was never implemented. It will come back in a later PR that creates a "future work" section.

* Address review comments.
2019-08-22 19:05:06 +02:00
Mike Hearn
a2c5cd1947 Tech white paper refresh, part 1 (#5233)
Tech white paper refresh, part 1.

In part 1:

* A new section is added on package namespace ownership and the no-overlap rule.
* The spelling of "serialize" is standardized on the US spelling used by the code, and add some content on serialization to the docs.
* Make a variety of smaller edits intended to make it read better.
* Spelling fixes.
* The discussion of C-I is temporarily removed, pending later re-addition in a new privacy section.
* Reference states are described.
* More TODOs are added to help me keep track of things that are needed.
* The discussion of time and clock sync is updated.
* The discussion of identity lookups is removed.
2019-08-22 19:05:06 +02:00
Mike Hearn
c7cb6ef725 Re-organise a part of the versioning discussion into a new toctree section. 2019-08-22 19:00:51 +02:00
Mike Hearn
178814b2d9 Improve the PDF by giving the book its own short intro page instead of
reusing the HTML intro, which doesn't make sense due to HTML-only markup
like videos.
2019-08-22 19:00:51 +02:00
Dan Newton
8fd5b7e489 DOCS - Correct links to checkpoint-tooling.html (#5387) 2019-08-22 16:30:02 +01:00
carolynequinn
b13f00e0ee Update UAT.md (#5382) 2019-08-21 11:13:48 +01:00
James Higgs
d1b9e95cd5 [ENT-4024] Use string for the status column in the transaction table (#5379) 2019-08-21 10:50:08 +01:00
Shams Asari
57d0d8c10c
Fixed headings in UAT.md (#5378) 2019-08-20 17:45:56 +01:00
Katelyn Baker
171c2be2db NOTICK - Typo in comment 2019-08-19 22:40:10 +01:00
Katelyn Baker
19adbffffb NOTICK - Break up the Property Reader Class into multiple files
As having them all live in a single PropertyReader header / source
file was getting far too big. Additionally, sue some of our aliased
types to shorted lines (sVec, sList, uPtr etc)
2019-08-19 22:37:22 +01:00
Katelyn Baker
eb8eaba1d7 NOTICK - Fix namespace allocation for C++ Serialiser
Now things are working for the most part, move the readers into their
own internal namespace. Also start creating an actual separation of the
interface and the implementation with some Interface classes at the
non internal level.
2019-08-19 20:13:04 +01:00
James Higgs
101d978050 [CORDA-3149] Update cache to check node identity keys in identity table (#5371) 2019-08-19 20:03:51 +01:00
carolynequinn
abd3a2db52 Update UAT.md (#5321) 2019-08-19 10:59:09 +01:00
Florian Friemel
76e2c696f2 [ENT-3809] Move purejavacomm dependency to libs. (#5269) 2019-08-19 10:19:54 +01:00
Asher
8d098fe845 fix wrong index in readme (#5315) 2019-08-19 10:16:54 +01:00
Katelyn Baker
81e9e8a842 NOTICK - Small tidy up based on lint suggestions
Move functions into .cxx files and out of the inline implementation in
the headers, std::move constructor params.
2019-08-17 16:19:02 +01:00
Katelyn Baker
3c3f5f5e04 NOTICK - Move CompositeFactory into amqp::internal namespace 2019-08-17 16:11:58 +01:00
Katelyn Baker
2ba8676013 NOTICK - Remove commented out code 2019-08-17 09:53:08 +01:00
Katelyn Baker
cfa657a1a0 NOTICK - CPP project gitignore 2019-08-16 12:58:23 +01:00
Shams Asari
69d1b4e4e5
CORDA-3138: Removed InMemoryTransactionsResolver as it's not needed and other resolution cleanup (#5370) 2019-08-16 12:34:25 +01:00
James Higgs
ee1aa9fea1 [CORDA-3082] Update app upgrade notes to document source incompatibility (#5344) 2019-08-16 10:53:31 +01:00
Katelyn Baker
913ba1d46e
NOTICK - Initial work on a non JVM (C++) serialiser (#5368)
Currently, it's more of a blob inspector than a full-blown
implementation, but this code does have the ability to understand a Corda
serialized blob and build a set of deserialisers to handle a blob. It
does this in the same way the native java implementation works by
interrogating the types (as per the definition in the envelope) and
building recursive type readers that are able to pull the relevant
information out of the byte stream.

Lots of future work to undertake, but as a starting point its a little
bit useful with the scope it could be made much more useful with some
work.
2019-08-15 21:16:20 +01:00
Dan Newton
a477d59c15 CORDA-3094 improvements to checkpoint dumper (#5324)
- Handle errors in Jackson and checkpoint deserialisation. A file
  notifying the user that the checkpoint dump failed is created when
  errors occur.
- Handle message deserialisation errors. A string placeholder is used
  if an error occurs.
- Add more information about subflows (include their `FlowLogic`)
- Increase clarity in checkpoint output field names

* CORDA-3904 Add `flowCallStackSummary`

Add `flowCallStackSummary` to the output which contains the same content
as `flowCallStack` minus each subflow's `FlowLogic`. The `FlowLogic`
contains a ton of info which is normally repeated in each subflow.
Adding the summary gives an overview of the steps the flow executed
and which step it is currently on.

The `suspendedOn` field is put underneath the summary and the original
call stack is moved below the suspended info.

This puts the most useful information towards the top of the json file.
2019-08-14 14:08:58 +01:00
James Higgs
72ac722451 [CORDA-3130] Add a cache for looking up external UUIDs from public keys (#5357) 2019-08-14 13:24:56 +01:00
Dimos Raptis
132244c437 [DOCS] - Clarify behaviour of hospital in unhandled errors (#5366) 2019-08-14 11:24:43 +01:00
Ryan Fowler
98329444f5 Revert "ENT-3658, ENT-3660: Add timeouts and hospital flow handling to CryptoService (#5364)
This reverts commit 6df142bf

* Clean some code to better merge between versions
2019-08-13 11:42:03 +01:00
szymonsztuka
4c7604b5c3 ENT-3965 Improved welcome message for Standalone Shell - bye command to exit shell only, docs clarifications gracefulShutdown/shutdown needs 'run' as other commands (#5363) 2019-08-12 13:22:15 +01:00
Viktor Kolomeyko
6db3ded032 CORDA-3139: Cater for port already bound scenario during port allocation (#5361)
* CORDA-3139: Cater for port already bound scenario during port allocation

Also moved `SharedMemoryIncremental` into a separate file as it getting bigger
and improved readability of logic and added some logging.

* CORDA-3139: Fix the unit test

* CORDA-3139: Improve logging when failing

* CORDA-3139: Improve stability of the test
2019-08-12 10:38:15 +01:00
James Brown
b8e278680d CORDA-3120 Update owasp scanner (#5335) 2019-08-08 16:05:54 +01:00
Dan Newton
fc265ee472 CORDA-3018 Whitelisting attachments by public key - relax signer restrictions (#5358)
* CORDA-3018 Allow loading an untrusted contract jar if another attachment exists that was signed with the same keys and uploaded by a trusted uploader

`TransactionUtils.isAttachmentTrusted` requirements have been relaxed
to allow an untrusted attachment to be loaded as long as another
attachment exists that is signed by the same keys and was uploaded
by a trusted uploader.

The requirement of containing the same contract classes has been
removed. Therefore the contents of the existing trusted attachment
no longer matters.

* CORDA-3018 Allow a subset/intersection of signers in `isAttachmentTrusted`

Allow a subset/intersection of signers to satisfy the signer
requirements of `isAttachmentTrusted`. This allows an "untrusted"
attachment that is signed by one or more keys to be "trusted" as long
as another trusted attachment already exists that is signed by at least
one of the "untrusted" attachments signers.

A cache of trusted and untrusted public keys is now held (replacing the
previous cache of `List<PublicKey>`.

Tests have been added to `NodeAttachmentServiceTest` to confirm that
an attachment query using an `EQUAL` statement will actually return
attachments that are signed by any of the keys passed into the query.

Confirming this allowed an `EQUAL` query to satisfy the search that
had to be done as part of this change.

`MockAttachmentStorage`'s query criteria was updated to better match
the real `NodeAttachmentService` implementation.

* CORDA-3018 Update cache name and kdoc on `isAttachmentTrusted`

* CORDA-3018 Verify that chains of trust do not occur

* CORDA-3018 Switch keys around to improve chain of trust tests
2019-08-08 09:33:45 +01:00
James Higgs
44428b6048 ENT-3801 Backport to OS (#5355)
* [ENT-3801] Store transactions in the database during transaction resolution (#2305)

* ENT-3801: Store downloaded txns as part of the backchain resolution into the db rather than the checkpoint

It's very inefficient to store the downloaded backchain in the checkpoint as more of it downloaded. Instead, if a threshold is reached (which currently defaults at 0) then the backchain is stored in the transactions table as unverified. A new is_verified column has been added to track this. Initially testing on the OS codebase has been very promising but unfortunately this current code is not quite ready. I had to quickly port it to ENT as this is meant to be an ENT-only optimisation.

To that effect, there is a TransactionResolver abstraction with two implementations: an in-memory one which has the old behaviour, and which will be the behaviour for OS, and a db one.

DBTransactionStorage hasn't been fully updated and I had to comment out the optimistic path for now.

Most of these changes will need to be ported to OS to keep the merge conflicts in check, but obviously not DbTransactionsResolver and the "is_verified" changes in DBTransactionStorage. DBTransactionStorage does have other refactoring which will make sense to port though.

* [ENT-3801] Start work on allowing modifications in AppendOnlyPersistentMap

* [ENT-3801] Add transaction resolver tests

* [ENT-3801] Adjust suspendable annotations

* [ENT-3801] Fix the ResolveTransactionFlow tests

* [ENT-3801] Update ResolveTransactionsFlow tests

* [ENT-3801] Add a liquibase migration script for isVerified

* [ENT-3801] Ensure the migration runs in the correct place

* [ENT-3801] Handle resolution of already present transactions

* [ENT-3801] Fix compile error in performance test app

* [ENT-3801] Logging and comment updates, plus a test case

* [ENT-3801] Add a notary change resolution test

* [ENT-3801] Add a contract upgrade transaction test

* [ENT-3801] Change new column to be a character based status

* [ENT-3801] Migration script type change

* [ENT-3801] Address first round of review comments

* [ENT-3801] Update variable names in AppendOnlyPersistentMap

* [ENT-3801] Another variable name clarification

* [ENT-3801] Fix missing name changes

* [ENT-3801] Make the signature list immutable when constructing cache value

* [ENT-3801] Add a locking strategy for unverified transactions

* [ENT-3801] Address tidying up review comments

* [ENT-3801] First attempt at ensuring locks are released after commit

* [ENT-3801] Remove references to old cache name

* [ENT-3801] Update locking logic

* [ENT-3801] Fix potential deadlock with read/write transaction locks

* [ENT-3801] Remove read locks, and ensure minimal extra suspends

* [ENT-3801] Fix build issues in tests

* [ENT-3801] Use the correct clock when calculating sleep durations

* [ENT-3801] Add a pessimism flag for writing verified transactions

* [ENT-3801] Change logging statement to debug

(cherry picked from commit 8ab6a55e17)

* [NOTICK] Fix up imports for some changed files

* [NOTICK] Fix transaction resolution tests

* [NOTICK] Reinstate the DBTransactionsResolver

* [NOTICK] Add the topological sort back to recordTransactions

* [NOTICK] Adjust test case to remove dependency on query ordering

* [NOTICK] Make test code match that in ENT
2019-08-07 16:49:49 +01:00
carolynequinn
39094f1918 Update index.md 2019-08-07 14:17:15 +02:00
Joel Dudley
68cfb1d2ea
Removes reference to future functionality. 2019-08-06 17:34:59 +01:00
Viktor Kolomeyko
b60ab70440 CORDA-2617: Add failover listeners to terminate node process (#5337)
* CORDA-2617: Add failover listeners to terminate node process

This is a backport of changes done in Corda Enterprise.

It will be triggered in case of:
a) Loss of connectivity to in-built Artemis for Bridge Control;
b) Loss of connectivity to in-built Artemis for P2P connectivity.

Note on merge to CE: Disregard these changes and take whatever CE already has.

* CORDA-2617: Update documentation on stability of Corda Node

* CORDA-2617: Documentation update after discussion with @mnesbit
2019-08-06 11:28:16 +01:00
Stefano Franz
fa75711647 CORDA-3055 - Parallel node info download (#5097)
* parallelize download of nodeInfos

* actually call new list based addNodes method

* address review comments
fix NetworkMapUpdaterTest

* ensure threadpools are shutdown after network-map download is completed

* use NamedThreadFactory instead of re-implementing it.

* fix imports after rebase

* address review comments

* remove extra whitespace
2019-08-02 13:54:18 +01:00
Stefan Iliev
7f3eca44dd Ignore RPCStability tests. (#5351)
* Added Ignore on repeatedly failing tests to start getting builds through.

* Added ignore reason and link to jira ticket.
2019-08-02 13:51:26 +01:00
Chris Rankin
813ff39eaa CORDA-3127: Move evaluationDependsOn() from core to core-tests. (#5349)
* CORDA-3127: Move evaluationDependsOn() from core to core-tests.

* Increase JVM memory for node unit tests to 3G.
2019-08-02 09:05:19 +01:00
Dan Newton
9166755954 Add constants for the open source and samples repos branch names (#5263)
* Add constants for the open source and samples repos branch names

Open source branch name - `openSourceBranchName`
Sample repos branch name - `openSourceSamplesBranchName`

* Add base path constants to the open source and samples repos

Fully replace base path to open source codebase - `openSourceBranch`
Fully replace base path to open source samples - `openSourceSamplesBranch`

These can be accessed in the docs using `os_branch` and `os_samples_branch`

Correct usages in the docs to use these constants.

* Correct links
2019-08-02 09:00:46 +01:00
josecoll
7f79a856af
CORDA-3121 - Fix incorrect rendering of Independent Foundation URL (in HTML) (#5336)
* Fix incorrect rendering of Independent Foundation URL (in HTML)
2019-08-01 16:05:19 +01:00
josecoll
99e802c5c5 CORDA-3071 - Checkpoint agent tool (#5295)
* Initial commit based on experimental kryo hook agent.

* WIP

* Added documentation.

* Additional improvements and documentation following more testing.

* Added field level instrumentation + basic type handlers for String, byteArray, charArray, primitive types.

* Working version (without array type handling)

* Missing build.gradle file.

* Handle display of Arrays and String.
Pruning output to avoid repetition (by loop depth, object count).

* Added configurable StackDepth (for display purposes) and filter out ProgressTracker stacks.

* Further array handling (Object arrays, 2D, 3D), improved display and general code cleanup.

* Various fixes and improvements following demo to RP.

* Clean-up

* Further clean-up

* Set checkpoint id before deserialization.

* Update documentation

* Final clean-up.

* Minor documentation fixes.

* Updates following PR review feedback.

* Add changelog entry.
2019-08-01 09:40:52 +01:00
szymonsztuka
eff762a5c3 CORDA-3012 More information in log warning for Cordapps missing advised JAR manifest file entries. (#5325) 2019-07-31 15:30:00 +01:00
Chris Rankin
d1289ec379 CORDA-2871: Restore CompositeKey support to core-deterministic. (#5333)
* CORDA-2871: Restore CompositeKey support to core-deterministic.

* Delete empty CordaSecurityProviderKt class from api-current.
2019-07-30 17:53:33 +01:00
Roger Willis
36c756eeb6 Added accounts design doc. (#5193)
* Added accounts design doc.

* Fixed typos in accounts design docs
2019-07-30 16:17:04 +00:00
Dan Newton
34368bf0d5 CORDA-2216 Restrict extended key usage of certificate types (#5294)
* CORDA-2216 Restrict extended key usage of certificate types

- Remove `anyExtendedKeyUsage` from all certificates
- Restrict to server and client auth for all certificates except
  `ROOT_CA` which has had all extended key usages removed

* CORDA-2216 Update certificate types

- Remove server authentication from all certificates except for TLS
- Add client authentication to all certificates except for ROOT
2019-07-30 15:32:12 +01:00