Design Review Process (#94)

Design Review Process V1.0

docs/source/design/README.md

@@ -0,0 +1,40 @@
+![Corda](https://www.corda.net/wp-content/uploads/2016/11/fg005_corda_b.png)
+
+<a href="https://ci-master.corda.r3cev.com/viewType.html?buildTypeId=CordaEnterprise_Build&tab=buildTypeStatusDiv"><img src="https://ci.corda.r3cev.com/app/rest/builds/buildType:Corda_CordaBuild/statusIcon"/></a>
+
+# Design Documentation
+
+This directory should be used to version control Corda design documents.
+
+These should be written in [Markdown](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet) (a design template is provided for general guidance) and follow the design review process outlined below. It is recommended you use a Markdown editor such as [Typora](https://typora.io/), or an appropriate plugin for your favourite editor (eg. [Sublime Markdown editing theme](http://plaintext-productivity.net/2-04-how-to-set-up-sublime-text-for-markdown-editing.html)).
+
+## Design Review Process
+
+Please see the [design review process](./designReviewProcess.md).
+
+* Feature request submission
+* High level design
+* Review / approve gate
+* Technical design
+* Review / approve gate
+* Plan, prototype, implement, QA 
+
+## Design Template
+
+Please copy this [directory](./designTemplate) to a new location under `/docs/source/design` (use a meaningful short descriptive directory name) and use the [Design Template](./designTemplate/design.md) contained within to guide writing your Design Proposal. Whilst the section headings may be treated as placeholders for guidance, you are expected to be able to answer any questions related to pertinent section headings (where relevant to your design) at the design review stage. Use the [Design Decision Template](./designTemplate/decisions/decision.md)  (as many times as needed) to record the pros and cons, and justification of any design decision recommendations where multiple options are available. These should be directly referenced from the *Design Decisions* section of the main design document.
+
+The design document may be completed in one or two iterations, by completing the following main two sections individually or singularly:
+
+* High level design   
+  Where a feature requirement is specified at a high level, and multiple design solutions are possible, this section should be completed and circulated for review prior to completing the detailed technical design.
+  High level designs will often benefit from a formal meeting and discussion review amongst stakeholders to reach consensus on the preferred way to proceed. The design author will then incorporate all meeting outcome decisions back into a revision for final GitHub PR approval.   
+* Technical design 
+  The technical design will consist of implementation specific details which require a deeper understanding of the Corda software stack, such as public API's and services, libraries, and associated middleware infrastructure (messaging,security, database persistence, serialization) used to realize these.
+  Technical designs should lead directly to a GitHub PR review process.
+
+Once a design is approved using the GitHub PR process, please commit the PR to the GitHub repository with a meaningful version identifier (eg. my super design document - **V1.0**)
+
+## Design Repository
+
+All design documents will be version controlled under github under the directory `/docs/source/design`.
+For designs that relate to Enterprise-only features (and that may contain proprietary IP), these should be stored under the [Enterprise Github repository](https://github.com/corda/enterprise). All other public designs should be stored under the [Open Source Github repository](https://github.com/corda/corda). 

docs/source/design/designReviewProcess.md

@@ -0,0 +1,106 @@
+
+# Overview
+
+The Corda Design Review process defines a means of editing, storing, collaborating, reviewing and approving Corda documentation in a consistent, structured, easily accessible and open manner.   
+
+# Background
+
+Historically, Corda design documentation has been produced in an ad hoc fashion to include:
+* Multiple sources and formats of storage
+    * Internal ([Tech/Arch technical discussion](https://r3-cev.atlassian.net/wiki/spaces/AR/pages/2588746/Internal+Technical+Discussion)) and External ([AWG design documents](https://r3-cev.atlassian.net/wiki/spaces/AWG/pages/56623418/Design+Documents)) facing wiki(s)
+    * [Public github wiki](https://github.com/corda/corda/wiki)
+    * [Discourse posts](https://discourse.corda.net/c/corda-discussion)
+* Multiple authored versions of same design with differing coverage
+    * Elaboration and/or additions to scope
+    * Differing opinions, proposals, suggestions.
+* Unstructured prose (no consistency in format and structure)
+* Lack of versioning (wiki documents typically evolve without versioned references)
+* Lack of traceability (audit) to original requirement(s)
+* Undefined review and approval process, leading to misunderstandings and open interpretations at time of implementation by platform development team
+* Lack of proposed implementation plan (time, resources, effort).
+* Often missing stakeholder collaboration and review in the feedback cycle. 
+
+# Process
+
+This process specifies:
+
+1. Usage of a design template to include:
+    * Versioning: design documents can be referenced at a point in time, and evolve from such.
+    * Review and approval history: incorporating relevant stakeholders from R3 (Platform, Product Management, Services) and 
+      other relevant review groups (community, partners, customers, key collaborators) as deemed appropriate to the request. Ensure design 
+      meets the requirements and is realizable within a proposed implementation timeframe.
+    * Consistent structure and headings: top level headings should be preserved, second level headings provide guidance on 
+      content to include, and may be omitted where not relevant.
+    * The design template includes both High Level (conceptual, logical) and Technical (implementation specific) sections.
+    * Design decisions are clearly identified with pros/cons of proposed options, and agreed recommendation.  
+
+2. Document review and approval by relevant stakeholders and impacted parties to include R3 organisational units, such as Platform Engineering, Product Management and Services (where relevant), and key stakeholders, to include customers, partners, key collaborators, and community leaders. 
+    * Product owner (originator of requirements)
+    * Design Approval Board (DAB)
+    * Platform Development technical lead (and/or nominated developer(s))
+    * Project Technical Lead / Solution Architect (if originating from an R3 Technical Services project)
+    * Other identified stakeholders (community leaders, partners, customers, key collaborators)
+
+3. Planning: allocation to Corda (open source) or Enterprise project JIRA epic(s) (and/or set of stories) and prioritisation within Product Backlog for future implementation within a Development Team Sprint.
+
+4. Document repository locations, according to whether the design is related to Open Source or Enterprise (internal only).
+  The recommended repository source is GitHub, and documents should be stored in [Markdown](https://en.wikipedia.org/wiki/Markdown).
+  The collaboration and review process should follow the standard [GitHub Pull Request](https://confluence.atlassian.com/bitbucket/work-with-pull-requests-223220593.html) mechanism.
+    * [Enterprise Github repository](https://github.com/corda/enterprise)
+    * [Open Source Github repository](https://github.com/corda/corda) 
+
+The following diagram illustrates the process flow:
+
+![Design Review Process](./designReviewProcess.png)
+
+# Review Groups
+Design documents should include all relevant stakeholders in their distribution (mostly as PR reviewers in github).  This will often vary and depend on the origin of the Feature Request, particularly for high level business requirements. Technical Design Documents will tend to include a small set of stakeholders (Design Approval Board, Platform Development, DevOps). Final approval authority lays with at least one member of the Design Approval Board (DAB) or nominated delegate(s).
+
+Design Approval Board (DAB)	
+* Richard G Brown (CTO)
+* James Carlyle (Chief Engineer)
+* Mike Hearn (Lead Platform Engineer)
+* Mark Oldfield (Lead Platform Architect)
+* Jonathan Sartin (Information Security manager)
+* Select external key contributors (directly involved in design process)
+
+Other review groups inlcude:
+
+* Product Management 
+
+* Developer Relations 
+
+* Platform Development Team Leads
+
+  (may nominate team members as design leads)
+
+* DevOps 
+
+* Services – Project (Incubation & Acceleration)
+
+  * Nominated project leads
+
+  Services – Technical (Consulting)
+  * Nominated solution architects
+
+* External
+
+  * AWG (general)
+  * Consortium members
+  * ISV, SI, Partners
+  * Customers
+  * Key collaborators
+
+# Applicability and Timing
+
+This process should be applied to any major feature request gathered by the product management team or lead technologists that has been entered in the product backlog as a requirement, and has been prioritized for imminent execution.
+
+Publication and distribution of a design document from initial review to full approval will vary on a case by case basis. 
+
+In general,
+ * High Level designs may require a longer approval cycle as they may need to host a formal review meeting with the DAB in attendance, 
+   and will typically have larger stakeholder audiences (potentially including external reviewers), thus leading to multiple iterations of revision.
+   In either case the High Level design must be raised as a GitHub PR and obtain formal approval by reviewers.  
+ * Technical designs are anticipated to go through a shorter cycle, with immediate feedback via the GitHub PR workflow. 
+   Once approved, a Technical Design should be decomposed into a set of implementable Epic/Stories for prioritization and 
+   scheduling as part of Development team(s) delivery cycle(s).

docs/source/design/designTemplate/decisions/decision.md

@@ -0,0 +1,39 @@
+![Corda](https://www.corda.net/wp-content/uploads/2016/11/fg005_corda_b.png)
+
+--------------------------------------------
+Design Decision: <Description heading>
+============================================
+
+## Background / Context
+
+Short outline of decision point.
+
+## Options Analysis
+
+### A. <Option summary>
+
+#### Advantages
+
+1.    ​
+2.    ​
+
+#### Disadvantages
+
+1.    ​
+2.    ​
+
+### B. <Option summary>
+
+#### Advantages
+
+1. ​
+2. ​
+
+#### Disadvantages
+
+1. ​
+2. ​
+
+## Recommendation and justification
+
+Proceed with Option <A or B or ... > 

docs/source/design/designTemplate/design.md

@@ -0,0 +1,242 @@
+![Corda](https://www.corda.net/wp-content/uploads/2016/11/fg005_corda_b.png)
+
+# Design Template
+
+Please read the [Design Review Process](../designReviewProcess.md) before completing a design.
+
+This design template should be used for capturing new Corda feature requests that have been raised as JIRA requirements stories by the product management team. The design may be completed in two stages depending on the complexity and scope of the new feature.
+
+1. High-level: conceptual designs based on business requirements and/or technical vision. Without detailing implementation, this level of design should position the overall solution within the Corda architecture from a logical perspective (independent from code implementation). It should illustrate and walk through the use case scenarios intended to be satisfied by this new feature request. The design should consider non-functional aspects of the system such as performance, scalability, high availability, security, and operational aspects such as management and monitoring.
+
+This section of the document should go through a formal review process (eg. presentation of design at meeting and subsequent PR review workflow)
+
+2. Technical: implementable designs with reference to Corda code. This level of design should focus on API specifications, service definitions, public library additions, data models and schemas, code modularity, configuration, execution and deployment of the new feature. It should also list any new software libraries, frameworks or development approaches to be adopted. The technical design should also consider all aspects of the test lifecycle (unit, integration, smoke tests, performance).
+
+This section of the document should be raised as a PR for development team review.
+
+An outcome of the Design Document should be an implementation plan that defines JIRA stories and tasks to be completed to produce shippable, demonstrable, executable code.
+
+Please complete and/or remove section headings as appropriate to the design being proposed. These are provided as guidance and to structure the design in a consistent and coherent manner.
+
+DOCUMENT MANAGEMENT
+---
+
+Design documents should follow the standard GitHub version management and pull request (PR) review workflow mechanism.
+
+## Document Control
+
+| Title                |                                          |
+| -------------------- | ---------------------------------------- |
+| Date                 |                                          |
+| Author               |                                          |
+| Distribution         | (see review groups in design review process) |
+| Corda target version | (enterprise, open source and enterprise) |
+| JIRA reference       | (reference to primary Feature Request JIRA story outlining requirements) |
+
+## Approvals
+
+#### Document Sign-off
+
+| Author            |                                          |
+| ----------------- | ---------------------------------------- |
+| Reviewer(s)       | (GitHub PR reviewers)                    |
+| Final approver(s) | (GitHub PR approver(s) from Design Approval Board) |
+
+#### Design Decisions
+
+| Description                              | Recommendation  | Approval*               |
+| ---------------------------------------- | --------------- | ----------------------- |
+| [Design Decision 1](decisions/decision.md) | Selected option | (Design Approval Board) |
+| [Design Decision 2](decisions/decision.md) | Selected option | (Design Approval Board) |
+| [Design Decision 3](decisions/decision.md) | Selected option | (Design Approval Board) |
+
+\* only required for formal Design Approval Board meetings.
+
+## Document History 
+
+To be managed by GitHub revision control 
+(please use meaningful identifiers when committing a PR approved design to GitHub - eg. my super design V1.0)
+
+HIGH LEVEL DESIGN
+---
+
+## Overview
+
+General overall of design proposal (goal, objectives, simple outline)
+
+## Background
+
+Description of existing solution (if any) and/or rationale for requirement.
+
+* Reference(s) to discussions held elsewhere (slack, wiki, etc).
+* Definitions, acronyms and abbreviations 
+
+## Scope
+
+* Goals
+* Non-goals (eg. out of scope)
+* Reference(s) to similar or related work
+
+## Timeline
+
+* Is this a short, medium or long-term solution?
+* Outline timeline expectations
+
+    Eg1. required for Customer Project X by end of Qy'2049)
+
+    Eg2. required to release Enterprise Vx.y (reference roadmap)
+
+* Where short-term design, is this evolvable / extensible or stop-gap (eg. potentially throwaway)?
+
+## Requirements
+
+* Reference(s) to any of following:
+
+    * Captured Product Backlog JIRA entry
+
+    * Internal White Paper feature item and/or visionary feature
+
+    * Project related requirement (POC, RFP, Pilot, Prototype) from
+
+        * Internal Incubator / Accelerator project
+
+        * Direct from Customer, ISV, SI, Partner
+* Use Cases 
+* Assumptions
+
+## Design Decisions
+
+List of design decisions identified in defining the target solution:
+(for each item, please complete the attached [Design Decision template](decisions/decision.md))
+
+| Heading (link to completed Decision document using template) | Recommendation |
+| ---------------------------------------- | -------------- |
+| [Design Decision 1](decisions/decision.md) | Option A       |
+| [Design Decision 2](decisions/decision.md) | TBD*           |
+| [Design Decision 3](decisions/decision.md) | Option B       |
+
+It is reasonable to expect decisions to be challenged prior to any formal review and approval. 
+*In certain scenarios the Design Decision itself may solicit a recommendation from reviewers.
+
+## Target Solution
+
+* Illustrate any business process with diagrams
+
+    * Business Process Flow (or formal BPMN 2.0), swimlane activity
+
+    * UML:  activity, state, sequence
+
+* Illustrate operational solutions with deployment diagrams
+
+    * Network
+
+* Validation matrix (against requirements)
+
+    * Role, requirement, how design satisfies requirement
+
+* Sample walk through (against Use Cases)
+
+* Implications
+
+    * Technical
+    * Operational
+    * Security
+
+* Adherence to existing industry standards or approaches
+* List any standards to be followed / adopted
+* Outstanding issues
+
+## Complementary solutions
+
+Other solutions that provide similar functionality and/or overlap with the proposed.
+Where overlap with existing solution(s), describe how this design fits in and complements the current state. 
+
+## Final recommendation
+
+* Proposed solution (if more than one option presented)
+* Proceed direct to implementation
+* Proceed to Technical Design stage
+* Proposed Platform Technical team(s) to implement design (if not already decided)
+
+TECHNICAL DESIGN
+---
+
+## Interfaces
+
+* Public APIs impact
+* Internal APIs impacted
+* Modules impacted
+
+    * Illustrate with Software Component diagrams
+
+## Functional
+
+* UI requirements
+
+    * Illustrate with UI Mockups and/or Wireframes
+
+* (Subsystem) Components descriptions and interactions)
+
+    Consider and list existing impacted components and services within Corda:
+
+    * Doorman
+    * Network Map
+    * Public API's (ServiceHub, RPCOps)
+    * Vault
+    * Notaries
+    * Identity services
+    * Flow framework
+    * Attachments
+    * Core data structures, libraries or utilities
+    * Testing frameworks
+    * Pluggable infrastructure: DBs, Message Brokers, LDAP
+
+* Data model & serialization impact and changes required
+
+    * Illustrate with ERD diagrams
+
+* Infrastructure services: persistence (schemas), messaging
+
+## Non-Functional
+
+* Performance
+* Scalability
+* High Availability
+
+## Operational
+
+* Deployment
+
+    * Versioning
+
+* Maintenance
+
+    * Upgradability, migration
+
+* Management
+
+    * Audit, alerting, monitoring, backup/recovery, archiving
+
+## Security
+
+* Data privacy
+* Authentication
+* Access control
+
+## Software Development Tools and Programming Standards to be adopted.
+
+* languages
+* frameworks
+* 3rd party libraries
+* architectural / design patterns
+* supporting tools
+
+## Testability
+
+* Unit
+* Integration
+* Smoke
+* Non-functional (performance)
+
+APPENDICES
+---