onefuzz/docs/managing-templates.md

Managing Declarative Job Templates

Declarative Job Templates, currently a preview feature, allow a user to define a reusable fuzzing pipeline as a template. Once saved, any user of the OneFuzz instance can create fuzzing jobs based on the templates.

This is a walk-through guide for updating an existing job template, though the process is similar for creating templates from scratch.

This process demonstrates adding Microsoft Teams notifications for new unique crash reports to an existing libfuzzer_linux job template and saving it as libfuzzer_with_teams.

Using the CLI & modifying JSON

1. Enable support for declarative templates.

   onefuzz config --enable_feature job_templates
   

2. List available templates.

   onefuzz job_templates list
   

3. Save a copy of the template locally.

   onefuzz job_templates manage get libfuzzer_linux > libfuzzer_linux.json
   

4. With your preferred text editor, add the following to the notifications list:

   {
       "container_type": "unique_reports", 
       "notification": {
           "config": {
               "url": "https://contoso.com/webhook-url-here"
           }
       }
   }
   

5. Upload the template.

   onefuzz job_templates manage upload libfuzzer_with_teams @./libfuzzer_linux.json
   

   NOTE: Using @./filename allows specifying read the contents of a file, rather than specifying JSON on the command line.

Using the SDK

1. Enable support for declarative templates.

   onefuzz config --enable_feature job_templates
   

2. Run the following python

   from onefuzztypes.job_templates import JobTemplateNotification
   from onefuzztypes.models import NotificationConfig, TeamsTemplate
   from onefuzztypes.enums import ContainerType
   from onefuzz.api import Onefuzz
   
   o = Onefuzz()
   template = o.job_templates.manage.get("libfuzzer_linux")
   template.notifications.append(
       JobTemplateNotification(
           container_type=ContainerType.unique_reports,
           notification=NotificationConfig(
               config=TeamsTemplate(url="https://contoso.com/webhook-url-here")
           ),
       )
   )
   o.job_templates.manage.upload("libfuzzer_with_teams", template)
   

Using the updated template

The OneFuzz SDK caches the list of Declarative Job Templates and will automatically refresh the templates every 24 hours. As shown below, users can refresh the declarative job template cache on demand.

If an existing template is changed without requiring new user input via form fields, using the template can happen transparently.

If you create a new template or update an existing template that changes the user interaction, users will need to refresh their template cache to make use of the change.

Now let's make use of our new template.

1. Update our template cache to make sure we have the latest libfuzzer_with_teams template

   $ onefuzz job_templates refresh
   WARNING:onefuzz:job_templates are a preview-feature and may change in an upcoming release
   INFO:onefuzz:refreshing job template cache
   INFO:onefuzz:updated template definition: libfuzzer_linux
   INFO:onefuzz:updated template definition: libfuzzer_with_teams
   INFO:onefuzz:updated template definition: afl_windows
   INFO:onefuzz:updated template definition: afl_linux
   INFO:onefuzz:updated template definition: libfuzzer_windows
   $
   

2. Launch our job

   $ onefuzz job_templates submit libfuzzer_with_teams example-project example-target build-1 linux --target_exe ./fuzz.exe
   WARNING:onefuzz:job_templates are a preview-feature and may change in an upcoming release
   INFO:onefuzz:creating container: oft-inputs-88dfb15b9ab758b88b122508d4648687
   INFO:onefuzz:creating container: oft-readonly-inputs-88dfb15b9ab758b88b122508d4648687
   INFO:onefuzz:creating container: oft-no-repro-88dfb15b9ab758b88b122508d4648687
   INFO:onefuzz:creating container: oft-crashes-88dfb15b9ab758b88b122508d4648687
   INFO:onefuzz:creating container: oft-reports-88dfb15b9ab758b88b122508d4648687
   INFO:onefuzz:creating container: oft-unique-reports-88dfb15b9ab758b88b122508d4648687
   INFO:onefuzz:creating container: oft-setup-fde90db8a8e65b4e8b7518f9d1350036
   INFO:onefuzz:uploading ./fuzz.exe to oft-setup-fde90db8a8e65b4e8b7518f9d1350036
   INFO:onefuzz:creating container: oft-coverage-88dfb15b9ab758b88b122508d4648687
   {
       "config": {
           "build": "build-1",
           "duration": 24,
           "name": "example-target",
           "project": "example-project"
       },
       "job_id": "d3259dfe-fdad-45a0-bf90-a381b8dc1ee8",
       "state": "init"
   }
   $ 
   

3. Verify a notification was set up for the unique reports container

   $ onefuzz notifications list --query "[?container == 'oft-unique-reports-88dfb15b9ab758b88b122508d4648687']"
   [
       {
           "config": {
               "url": "***"
           },
           "container": "oft-unique-reports-88dfb15b9ab758b88b122508d4648687",
           "notification_id": "0e0c10a1-78ef-4f65-be56-d3ba0788fcb5"
       }
   ]
   

Adding a required field

In many cases, we want users of the template to provide more detail, such as data that can be used to tailor the notifications based on the target at hand. This is accomplished via a form fields.

Let's make a new template that enables notifications via Azure Devops Work Items, where we require the user to specify the Area Path for the work items that OneFuzz will create.

This example will demonstrate setting the following:

• Project via a project name specified during job creation.
• Area Path via a new required field.
• Iteration Path via a new optional field, with a default value in the template.

1. Enable support for declarative templates. This is required because declarative templates are not yet stabilized.

   onefuzz config --enable_feature job_templates
   

2. Save a copy of the template locally.

   onefuzz job_templates manage get libfuzzer_linux > libfuzzer_linux_ado_areapath.json
   

3. With your preferred text editor, add the following to the notifications list:

   {
       "container_type": "unique_reports",
       "notification": {
           "config": {
               "base_url": "https://dev.azure.com/<your-org-name-here>",
               "auth_token": "ADO_AUTH_TOKEN",
               "type": "Bug",
               "project": "{{ job.project }}",
               "ado_fields": {
                   "System.AreaPath": "{{ task.tags['area_path'] }}",
                   "Microsoft.VSTS.Scheduling.StoryPoints": "1",
                   "System.IterationPath": "{{ task.tags['iteration_path'] }}",
                   "System.Title": "{{ report.crash_site }} - {{ report.executable }}",
                   "Microsoft.VSTS.TCM.ReproSteps": "This is my call stack: <ul> {% for item in report.call_stack %} <li> {{ item }} </li> {% endfor %} </ul>"
               },
               "comment": "This is my comment. {{ report.input_sha256 }} {{ input_url }} <br> <pre>{{ repro_cmd }}</pre>",
               "unique_fields": ["System.Title", "System.AreaPath"],
               "on_duplicate": {
                   "comment": "Another <a href='{{ input_url }}'>POC</a> was found in <a href='{{ target_url }}'>target</a>. <br> <pre>{{ repro_cmd }}</pre>",
                   "set_state": { "Resolved": "Active" },
                   "ado_fields": {
                       "System.IterationPath": "{{ task.tags['iteration_path'] }}"
                   },
                   "increment": ["Microsoft.VSTS.Scheduling.StoryPoints"]
               }
           }
       }
   }
   

   NOTE: The use of task.tags throughout the template. The next steps will define how values get assigned to this dictionary.

4. With your preferred text editor, add the following to user fields to the end of the user_fields list.
   1. A required field that specifies the tag name area_path.

      {
          "help": "Area path for reported crashes",
          "locations": [
              {
                  "op": "add",
                  "path": "/tasks/1/tags/area_path"
              }
          ],
          "name": "area_path",
          "required": true,
          "type": "Str"
      }
      

   2. An optional field that specifies the iteration_path

      {
          "default": "Iteration\\Path\\Default\\Here",
          "help": "Iteration path for reported crashes",
          "locations": [
              {
                  "op": "add",
                  "path": "/tasks/1/tags/iteration_path"
              }
          ],
          "name": "iteration_path",
          "required": false,
          "type": "Str"
      }
      

   NOTE: Each of these fields use the jsonpatch path to add a tag to the second task (by array index), which is the crash_report task. Check out form fields for more information.

5. Upload the template.

   onefuzz job_templates manage upload libfuzzer_linux_ado_areapath @./libfuzzer_linux_ado_areapath.json
   

   NOTE: Using @./filename allows specifying read the contents of a file, rather than specifying JSON on the command line.

6. Refresh the template cache.

   onefuzz job_templates refresh
   

Using --help, we can see the new optional and required arguments.

$ onefuzz job_templates submit libfuzzer_linux_ado_areapath --help
usage: onefuzz job_templates submit libfuzzer_linux_ado_areapath [-h] [-v] [--format {json,raw}] [--query QUERY]
    [--target_exe TARGET_EXE] [--duration DURATION]
    [--target_workers TARGET_WORKERS] [--vm_count VM_COUNT]
    [--target_options [TARGET_OPTIONS [TARGET_OPTIONS ...]]]
    [--target_env str=str [str=str ...]] [--reboot_after_setup]
    [--check_retry_count CHECK_RETRY_COUNT]
    [--target_timeout TARGET_TIMEOUT] [--mytags str=str [str=str ...]]
    [--iteration_path ITERATION_PATH] [--setup_dir SETUP_DIR]
    [--inputs_dir INPUTS_DIR] [--readonly_inputs_dir READONLY_INPUTS_DIR]
    [--container_names ContainerType=str [ContainerType=str ...]]
    [--parameters JobTemplateRequestParameters]
    project name build pool_name area_path

The new field, area_path is now a required argument and iteration_path is an optional argument.

Let's create a job using this template and verify the tags are as we expect. Since the parameters we added only modify the libfuzzer_crash_report task, we'll search just for the that task.

$ onefuzz job_templates submit libfuzzer_linux_ado_areapath myproject myname build1 linux My\\Iteration\\Path
WARNING:onefuzz:job_templates are a preview-feature and may change in an upcoming release
INFO:onefuzz:creating container: oft-setup-cf346c84f2df551381d9991a5efbd030
INFO:onefuzz:uploading fuzz.exe to oft-setup-cf346c84f2df551381d9991a5efbd030
INFO:onefuzz:creating container: oft-unique-reports-71bdf0b1ce175a2796954f8d62f5d3d0
INFO:onefuzz:creating container: oft-reports-71bdf0b1ce175a2796954f8d62f5d3d0
INFO:onefuzz:creating container: oft-coverage-71bdf0b1ce175a2796954f8d62f5d3d0
INFO:onefuzz:creating container: oft-no-repro-71bdf0b1ce175a2796954f8d62f5d3d0
INFO:onefuzz:creating container: oft-inputs-71bdf0b1ce175a2796954f8d62f5d3d0
INFO:onefuzz:creating container: oft-crashes-71bdf0b1ce175a2796954f8d62f5d3d0
INFO:onefuzz:creating container: oft-readonly-inputs-71bdf0b1ce175a2796954f8d62f5d3d0
{
    "config": {
        "build": "build1",
        "duration": 24,
        "name": "myname",
        "project": "myproject"
    },
    "job_id": "1eaa9a23-fcdb-400a-a26c-3ebe712fde53",
    "state": "init"
}
$ onefuzz jobs tasks list 1eaa9a23-fcdb-400a-a26c-3ebe712fde53 --query "[?config.task.type == 'libfuzzer_crash_report'].config.tags"
[
    {
        "area_path": "My\\Iteration\\Path",
        "iteration_path": "Iteration\\Path\\Default\\Here"
    }
]
$

As we can see, the tag area_path is the value we specified, and because we didn't specify iteration_path, it uses the default from the template.