Overview ======== The Behave Cucumber Scenarios defined under these sub-directories exercise Corda Enterprise and OS Corda distributions to include: - Open Source master - Corda Enterprise master - Released versions of Open Source from V3.0 onwards - Released versions of Corda Enterprise from V3.0 DEV PREVIEW 3 onwards **Compatibility** scenarios exercise: - mixed networks: combinations of OS and Corda Enterprise nodes configured within an R3 Network - mixed-versioned nodes: combinations of different versions of OS (master, V3.0, V3.1, etc) and Corda Enterprise (master, V3.0 DP, V3.0 GA) nodes within an R3 Network. - mixed-versioned corDapps: combinations of nodes running different versions of CorDapps (both valid and invalid upgrade mixes). - mixed-services: ability to continue transacting upon notaries, oracles, doorman upgrades & changes. **Functional** scenarios exercise the key behaviours of Corda Enterprise components and node configurations to include: - basic cash management functions (issuance, transfer, redemption) - vault usage of different database providers (H2, sql-server, postgreSQL) - doorman certificate issuance and usage - compatibility zone network parameter registration and updates - network map registration and updates - notary registration and updates NOTE: - the CTS framework uses R3's internal artifactory repositories to setup/configure officially released versions of Corda: 1. https://ci-artifactory.corda.r3cev.com/artifactory/corda-releases 2. https://ci-artifactory.corda.r3cev.com/artifactory/r3-corda-releases - Scenarios specifying `master` will use the latest code checked into the respective Open Source and Enterprise repositories: 1. https://github.com/corda/corda 2. https://github.com/corda/enterprise/ Further goals of this project are to: - automatically generate and register Docker images with target Azure Test Environments (eg. Functional/QA, CTS) - effect execution of scenarios in specified Target Environments - use Before and After hooks to pre-configure Target Environments to allow batch running of multiple scenarios (eg. a given feature will specify the environment once for one or many scenarios) QA setup and usage instructions =============================== Setup ----- Set up the staging area which will hold all the distributions of Corda referenced in the test scenarios: - specify the staging root directory: ```bash $export STAGING_ROOT=$HOME/staging ``` - if any of your tests are referencing a master version of [Corda OS](https://github.com/corda/corda), checkout the latest version of the Open Source repo and run: ```bash $ cd experimental/behave $ ./prepare.sh ``` You should now see the following artifacts in your staging area: ```bash $ ls -lR /Users/home/staging/corda/corda-master total 285440 drwxr-xr-x 3 user staff 96 4 Jun 16:11 apps -rw-r--r-- 1 user staff 55439705 4 Jun 16:11 corda.jar -rw-r--r-- 1 user staff 90699598 4 Jun 16:11 network-bootstrapper.jar /Users/home/staging/corda/corda-master/apps: total 4688 -rw------- 1 user staff 2396954 4 Jun 16:11 corda-finance-4.0-SNAPSHOT.jar ``` - similarly, if any of your tests are referencing a master version of [Corda Enterprise](https://github.com/corda/enterprise), checkout the latest version of the Enterprise repo and run: ```bash $ cd experimental/behave $ ./prepare.sh ``` You should now see the following artifacts in your staging area: ```bash $ ls -lR /Users/home/staging/corda/r3corda-master/ total 503264 drwxr-xr-x 6 user staff 192 4 Jun 16:39 apps -rw-r--r-- 1 user staff 6239281 4 Jun 16:39 corda-rpcProxy.jar -rw-r--r-- 1 user staff 66441262 4 Jun 16:38 corda.jar -rw-r--r-- 1 user staff 71987460 4 Jun 16:39 dbmigration.jar -rw-r--r-- 1 user staff 112985324 4 Jun 16:39 network-bootstrapper.jar -rwxr-xr-x 1 user staff 1224 4 Jun 16:39 startRPCproxy.sh /Users/home/staging/corda/r3corda-master/apps: total 82248 -rw------- 1 user staff 2418144 4 Jun 16:38 corda-finance-R3.CORDA-3.0-SNAPSHOT.jar ``` Notes: - all versions of Corda published to the Artifactory release repositories are automatically downloaded and configured on the fly as part of a test scenario. - Docker must be running for SQL Server, Oracle and PostrgeSQL tests. Usage ----- Checkout the latest version of the Enterprise repo and run: ```bash $cd experimental/behave ../../gradlew behaveJar ```` Change to the QA testing directory containing the scenario scripts and run: ```bash $cd ../../test/qa $export BEHAVE_JAR=$(ls ../../../experimental/behave/build/libs/corda-behave-*.jar | tail -n1) ``` To verify the tool is ready to be used run: ```bash $ java -DSTAGING_ROOT=${STAGING_ROOT} -jar ${BEHAVE_JAR} ``` You should now see the following output providing details on how to run the tool: ```bash Missing required option(s) [path] Usage: ScenarioRunner [options] --path Examples: ScenarioRunner -path ScenarioRunner -path /.feature ScenarioRunner -path /.feature:3:9 ScenarioRunner -path --plugin html --tags @qa ScenarioRunner -path --plugin html --tags @compatibility Please refer to the Cucumber documentation https://cucumber.io/docs/reference/jvm for more info. Option (* = required) Description --------------------- ----------- -d --glue [location of additional step (default: net.corda.behave.scenarios) definitions, hooks and plugins] * --path --plugin [register additional plugins (default: pretty) (see https://cucumber. io/docs/reference/jvm)] --tags [only run scenarios marked as @] ``` Note: passing in a *-d* option will perform a dry run only (validates the syntax of the scenario but does not execute the code) Scenario scripts ---------------- There are currently two sets of scripts: 1. Functional ```bash # the Cucumber behave test scenario definitions themselves $ls -l functional/resources/features/functional.feature # Unix script to easily run these $ls -l functional/resources/scripts/run-functional.sh ``` 2. Interoperability & Compatibility ```bash # the Cucumber behave test scenario definitions themselves $ls -l compatibility/resources/features/interoperability.feature # Unix script to easily run these $ls -l compatibility/resources/scripts/run-interoperability.sh ``` Note: the complete suite of Compatibility test scenarios have note yet been fully implemented. You can now run the above test scripts in a number of different ways: * as a complete suite called a *feature set*: ```bash # run all scenarios in the functional.feature file $java -DSTAGING_ROOT=${STAGING_ROOT} -jar ${BEHAVE_JAR} -path functional/resources/features/functional.feature ``` ```bash # run all scenarios in the interoperability.feature file $java -DSTAGING_ROOT=${STAGING_ROOT} -jar ${BEHAVE_JAR} -path compatibility/resources/features/interoperability.feature ``` * as a set of tagged scenarios within a feature file: ```bash # run all scenarios tagged with qa in the interoperability.feature file $java -DSTAGING_ROOT=${STAGING_ROOT} -jar ${BEHAVE_JAR} -path compatibility/resources/features/interoperability.feature:@qa ``` * as an individual scenario for all example configurations: ```bash # run the scenario defined on line 5 of the interoperability.feature file $java -DSTAGING_ROOT=${STAGING_ROOT} -jar ${BEHAVE_JAR} -path compatibility/resources/features/interoperability.feature:5 # by default the above will run as many times as there are parameterised variable definitions Examples: | R3-Corda-Node-Version | Corda-Node-Version | Currency | | r3-master | corda-master | GBP | | corda-3.0 | corda-3.1 | GBP | | R3.CORDA-3.0.0-DEV-PREVIEW-3 | corda-3.0 | GBP | | R3.CORDA-3.0.0-DEV-PREVIEW-3 | corda-3.1 | GBP | ``` * as an individual scenario for a single example configuration you will need to comment out the other example line items: ```bash # run the scenario defined on line 5 of the interoperability.feature file $java -DSTAGING_ROOT=${STAGING_ROOT} -jar ${BEHAVE_JAR} -path compatibility/resources/features/interoperability.feature:5 # by default the above will the scenario using *corda-3.0* and *corda-3.1* node versions only: Examples: | R3-Corda-Node-Version | Corda-Node-Version | Currency | # | r3-master | corda-master | GBP | | corda-3.0 | corda-3.1 | GBP | # | R3.CORDA-3.0.0-DEV-PREVIEW-3 | corda-3.0 | GBP | # | R3.CORDA-3.0.0-DEV-PREVIEW-3 | corda-3.1 | GBP | ```