Update the filter rule format and implementation to be simpler and user-predictable. In particular, we remove an accidental dependence of rule application on hash map iteration order.
* Documents `crashes_account` and `crashes_container`
* Adds `reports_dir` and support for `unique_reports`, `reports`, and `no_repro` containers to the generic analysis task
* Adds `microsoft_telemetry_key` and `instance_telemetry_key` to generic supervisor, generator, and analysis tasks
* renames `telemetry_key` to `microsoft_telemetry_key`
* renames `instrumentation_key` to `instance_telemetry_key`
* renames `can_share` to `can_share_with_microsoft`
* renames the `applicationinsights-rs` instances to `internal` and `microsoft` respective of the keys used during construction.
This clarifies the underlying use of Application Insights keys and uses struct tuple to ensure the keys are used correctly via rust's type checker.
This expands the documentation to demonstrate how nested lists are handled in practice, as well as to provide examples for how to deal with the nested list expansion.
RS5-Pro is no longer updated in the Azure Marketplace. In order to ensure the Windows 10 VMs are regularly updated, we need to switch the default image to 20H2-Pro, which is regularly maintained.
Updates the following libraries in the service:
* azure-core
* azure-functions
* azure-identity
* azure-keyvault-keys
* azure-keyvault-secrets
* azure-mgmt-compute
* azure-mgmt-core
* azure-mgmt-loganalytics
* azure-mgmt-network
* azure-mgmt-resource
* azure-mgmt-storage
* azure-mgmt-subscription
* azure-storage-blob
* azure-storage-queue
* pydantic
* requests
* jsonpatch
Removes the following libraries in the service:
* azure-cli-core
* azure-cli-nspkg
* azure-mgmt-cosmosdb
* azure-servicebus
Updates the following libraries in the CLI:
* requests
* semver
* asciimatics
* pydantic
* tenacity
Updates the following libraries in onefuzztypes:
* pydantic
The primary "legacy" libraries are [azure-graphrbac](https://pypi.org/project/azure-graphrbac/) and azure-cosmosdb-table. The former has not been updated to use azure-identity yet. The later is being rewritten as [azure-data-tables](https://pypi.org/project/azure-data-tables/), but is still in early beta.
## Summary of the Pull Request
Adds a new placeholder {setup_dir} for the setup directory
## PR Checklist
* [x] Applies to work item: #221
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/onefuzz) and sign the CLI.
* [x] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
## Info on Pull Request
_What does this include?_
## Validation Steps Performed
_How does someone test & validate?_
Enables co-locating multiple tasks in a given work-set.
Tasks are bucketed by the following:
* OS
* job id
* setup container
* VM SKU & image (used in pre-1.0 style tasks)
* pool name (used in 1.0+ style tasks)
* if the task needs rebooting after the task setup script executes.
Additionally, a task will end up in a unique bucket if any of the following are true:
* The task is set to run on more than one VM
* The task is missing the `task.config.colocate` flag (all tasks created prior to this functionality) or the value is False
This updates the libfuzzer template to make use of colocation. Users can specify co-locating all of the tasks *or* co-locating the secondary tasks.
When running libfuzzer in 'fuzzing' mode, we expect the following on exit.
If the exit code is zero, crashing input isn't required. This happens if the user specifies '-runs=N'
If the exit code is non-zero, then crashes are expected. In practice, there are two causes to non-zero exits.
1. If the binary can't execute for some reason, like a missing prerequisite
2. If the binary _can_ execute, sometimes the sanitizers are put in such a bad place that they are unable to record the input that caused the crash.
This PR enables handling these two non-zero exit cases.
1. Optionally verify the libfuzzer target loads appropriately using `target_exe -help=1`. This allows failing faster in the common issues, such a missing prerequisite library.
2. Optionally allow non-zero exits without crashes to be a warning, rather than a task failure.
Add a job_template example that demonstrates customization of the arguments to the job.
This example demonstrates setting the Area and Iteration paths for Azure Devops work items.
This PR makes user information from JWT tokens available as part of a Task.
Included changes:
* Renamed `verify_token` to `call_if_agent`, since this function is specific to agent token verification
* Renames `is_authorized` to `is_agent`, since this function checks if the token is an agent
* Adds support for unmanaged nodes in `is_agent` (see #133 for information)
* Saves the user information from the JWT token on task create as part of `TaskConfig`
Note, `TaskConfig` is what is provided to notification templates. This enables Github issues and ADO work items to tie back to the user that created the task.
Note, while `upn` _usually_ means email for AAD user tokens. If we were going to make use of the email address, we should perform a graph lookup based on the `oid`, but we're not.
