Add example job pipelines (#778)

README.md

@@ -5,7 +5,7 @@
 Project OneFuzz enables continuous developer-driven fuzzing to proactively
 harden software prior to release.  With a [single 
 command](docs/getting-started.md#launching-a-job), which can be [baked into
-CICD](https://github.com/microsoft/onefuzz-samples), developers can launch
+CICD](contrib/onefuzz-job-github-actions/README.md), developers can launch
 fuzz jobs from a few virtual machines to thousands of cores.
 
 ## Build Status

contrib/README.md

@@ -0,0 +1,6 @@
+# Supplemental Contributions related to OneFuzz
+
+* [Deploying fuzzing jobs in your Azure Devops Pipelines](onefuzz-job-azure-devops-pipeline)
+* [Deploying fuzzing jobs in your Github Actions workflow](onefuzz-job-github-actions)
+* [Keep your OneFuzz deployment up-to-date using Azure Devops pipelines](deploy-onefuzz-via-azure-devops)
+* [Demo web application that forwards OneFuzz webhook events to Microsoft Teams](webhook-teams-service)

contrib/onefuzz-job-azure-devops-pipeline/README.md

@@ -0,0 +1,24 @@
+# Deploying OneFuzz jobs via ADO Pipelines
+
+This pipeline uses an [AAD Service Principal](https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals) to authenticate to Onefuzz.
+
+To create work items upon finding crashes, this pipeline uses a [Azure Devops Personal Access Token](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate) to report any crashes found during fuzzing as [Azure Devops Work Items](../../docs/notifications/ado.md).
+
+## Configuration
+
+This example uses the [Azure Devops Variable Group](https://docs.microsoft.com/en-us/azure/devops/pipelines/library/variable-groups), named `onefuzz-config`, which can be shared across multiple pipelines.   The following variables are defined in `onefuzz-config`:
+* `endpoint`: The Onefuzz Instance.  This should be the URL of the instance, such as `https://onefuzz-playground.azurewebsites.net`.
+* `client_id`: The Client ID of the [service principal]((https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals).
+* `client_secret`: The Client Secret of the [service principal](https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals).
+* `ado_pat`: The [Azure Devops Personal Access Token](https://docs.microsoft.com/en-us/azure/devops/organizations/accounts/use-personal-access-tokens-to-authenticate).  This should be a "secret" variable.
+
+This example uses [Azure Devops Runtime parameters](https://docs.microsoft.com/en-us/azure/devops/pipelines/process/runtime-parameters), which are specific to this pipeline.  The following parameters are defined in this pipeline:
+* `onefuzz_project`: The name of your project.  As an example, "Browser".  Unless otherwise specified, this defaults to `sample`.
+* `onefuzz_target`: The name of your target.  As an example, "jpg-parser".  Unless otherwise specified, this defaults to `sample`.
+* `onefuzz_pool`: The name of the fuzzing [Pool](../../docs/terminology.md#pool) to use.  Unless otherwise specified, this defaults to `linux`.
+
+### Azure Devops Configuration
+In the [notification configuration](ado-work-items.json), there are a few items that are hard-coded that you should update for your instance:
+* Replace `INSERT_YOUR_ORG_HERE` with the name of your Azure Devops organization.
+* Replace `INSERT_YOUR_PROJECT_HERE` with the name of your Azure Devops project.
+* Replace `OneFuzz-Ado-Integration` with the Area Path for your work items to be filed.

contrib/onefuzz-job-azure-devops-pipeline/ado-work-items.json

@@ -0,0 +1,25 @@
+{
+  "config": {
+    "base_url": "https://dev.azure.com/INSERT_YOUR_ORG_HERE",
+    "auth_token": "INSERT_PAT_HERE",
+    "project": "INSERT_YOUR_PROJECT_HERE",
+    "type": "Bug",
+    "unique_fields": [
+      "System.Title",
+      "System.AreaPath"
+    ],
+    "comment": "<br> This is my call stack: <ul> {% for item in report.call_stack %} <li> {{ item }} </li> {% endfor %} </ul>  Repro Command: <br> <pre> {{ repro_cmd }} </pre>",
+    "ado_fields": {
+      "System.AreaPath": "OneFuzz-Ado-Integration",
+      "System.Title": "{{report.task_id}}"
+    },
+    "on_duplicate": {
+      "increment": [],
+      "comment": "DUP {{report.input_sha256}} <br> Repro Command: <br> <pre> {{ repro_cmd }} </pre> ",
+      "set_state": {
+        "Resolved": "Active"
+      },
+      "ado_fields": {}
+    }
+  }
+}

contrib/onefuzz-job-azure-devops-pipeline/azure-pipelines.yml

@@ -0,0 +1,42 @@
+trigger:
+- main
+parameters:
+- name: onefuzz_project
+  displayName: OneFuzz Project
+  type: string
+  default: sample
+- name: onefuzz_target
+  displayName: OneFuzz Target Name
+  type: string
+  default: sample
+- name: onefuzz_pool
+  displayName: Fuzzing Pool to use
+  type: string
+  default: linux
+jobs:
+- job: Example
+  pool:
+    vmImage: ubuntu-latest
+  variables:
+  - group: onefuzz-config
+  steps:
+  - bash: |
+      set -ex
+      make
+    displayName: Build
+  - task: UsePythonVersion@0
+    inputs:
+      versionSpec: '3.7'
+      addToPath: true
+      architecture: x64
+    displayName: setup python env
+  - bash: |
+      set -ex
+      pip install onefuzz
+      sed -i s/INSERT_PAT_HERE/$(ado_pat)/ ado-work-items.json
+      onefuzz config --endpoint $(endpoint) --client_id $(client_id) --client_secret $(client_secret)
+      onefuzz template libfuzzer basic ${{ parameters.onefuzz_project }} ${{ parameters.onefuzz_target }} $(Build.SourceVersion) ${{ parameters.onefuzz_pool }} --target_exe fuzz.exe --colocate_all_tasks --duration 1 --vm_count 1 --notification_config @./ado-work-items.json
+    displayName: run onefuzz
+    env:
+      client_secret: client_secret
+      ado_pat: ado_pat

contrib/onefuzz-job-github-actions/README.md

@@ -0,0 +1,23 @@
+# Deploying OneFuzz jobs via GitHub Actions
+
+This pipeline uses an [AAD Service Principal](https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals) to authenticate to Onefuzz.
+
+To create work items upon finding crashes, this pipeline uses a [GitHub Personal Access Token](https://github.com/settings/tokens) to report any crashes found during fuzzing as [GitHub Issues](../../docs/notifications/github.md).
+
+## Configuration
+This example uses [Encrypted Secrets](https://docs.github.com/en/actions/reference/encrypted-secrets) to configure the workflow:
+* `onefuzz_endpoint`: The Onefuzz Instance.  This should be the URL for the instance, such as `https://onefuzz-playground.azurewebsites.net`.
+* `onefuzz_client_id`: The Client ID for the [service principal]((https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals).
+* `onefuzz_client_secret`: The Client Secret for the [service principal](https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals).
+* `onefuzz_pat`: The [GitHub Personal Access Token](https://github.com/settings/tokens).
+
+This example uses environment variables to configure the workflow:
+* `ONEFUZZ_PROJECT`:The name of your project.  As an example, "Browser".
+* `ONEFUZZ_NAME`: The name of your target application.  As an example, "jpg-parser".
+* `ONEFUZZ_POOL`:The name of the fuzzing [Pool](../../docs/terminology.md#pool) to use.  As an example, `linux`.
+
+### GitHub Issues Configuration
+In the [notification configuration](github-issues.json), there are a few items that are hard-coded that you should update for your instance:
+* Replace `INSERT_YOUR_USERNAME_HERE` with the name of your GitHub username used to file issues.
+* Replace `organization` with the name of your GitHub organization to file issues.
+* Replace `repository` with the name of your GitHub repository to file issues.

contrib/onefuzz-job-github-actions/github-issues.json

@@ -0,0 +1,23 @@
+{
+   "config": {
+      "auth": {
+         "user": "INSERT_YOUR_USERNAME_HERE",
+         "personal_access_token": "INSERT_YOUR_PERSONAL_ACCESS_TOKEN_HERE"
+      },
+      "organization": "contoso",
+      "repository": "sample-project",
+      "title": "{{ report.executable }} - {{report.crash_site}}",
+      "body": "## Files\n\n* input: [{{ report.input_blob.name }}]({{ input_url }})\n* exe: [{{ report.executable }}]( {{ target_url }})\n* report: [{{ report_filename }}]({{ report_url }})\n\n## Repro\n\n `{{ repro_cmd }}`\n\n## Call Stack\n\n```{% for item in report.call_stack %}{{ item }}\n{% endfor %}```\n\n## ASAN Log\n\n```{{ report.asan_log }}```",
+      "unique_search": {
+         "field_match": ["title"],
+         "string": "{{ report.executable }}"
+      },
+      "assignees": [],
+      "labels": ["bug", "{{ report.crash_type }}"],
+      "on_duplicate": {
+         "comment": "Duplicate found.\n\n* input: [{{ report.input_blob.name }}]({{ input_url }})\n* exe: [{{ report.executable }}]( {{ target_url }})\n* report: [{{ report_filename }}]({{ report_url }})",
+         "labels": ["{{ report.crash_type }}"],
+         "reopen": true
+      }
+   }
+}

contrib/onefuzz-job-github-actions/github-pipeline.yml

@@ -0,0 +1,36 @@
+
+
+name: Onefuzz Sample Pipeline
+
+on:
+  push:
+    branches: [ main ]
+
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: build
+        run: |
+          set -ex
+          make
+      - name: Setup Python
+        uses: actions/setup-python@v2
+        with:
+          python-version: 3.7 
+      - name: submit onefuzz job
+        env:
+          ONEFUZZ_ENDPOINT: ${{ secrets.onefuzz_endpoint }}
+          ONEFUZZ_CLIENT_ID: ${{ secrets.onefuzz_client_id }}
+          ONEFUZZ_CLIENT_SECRET: ${{ secrets.onefuzz_client_secret }}
+          ONEFUZZ_PAT: ${{ secrets.onefuzz_pat }}
+          ONEFUZZ_PROJECT: sample
+          ONEFUZZ_NAME: sample
+          ONEFUZZ_POOL: linux
+        run: |
+          set -ex
+          pip install onefuzz
+          sed -i s/INSERT_YOUR_PERSONAL_ACCESS_TOKEN_HERE/${ONEFUZZ_PAT}/ github-issues.json
+          onefuzz config --endpoint $ONEFUZZ_ENDPOINT --client_id $ONEFUZZ_CLIENT_ID --client_secret $ONEFUZZ_CLIENT_SECRET
+          onefuzz template libfuzzer basic $ONEFUZZ_PROJECT $ONEFUZZ_NAME $GITHUB_SHA $ONEFUZZ_POOL --target_exe fuzz.exe --colocate_all_tasks --duration 1 --vm_count 1 --notification_config @./github-issues.json