This helps keep nodes on scalesets that use `latest` OS image SKUs reasonably up-to-date with OS patches without disrupting running fuzzing tasks with patch reboot cycles.
In combination with the already-merged #416, this PR closes#414.
This expands the documentation to demonstrate how nested lists are handled in practice, as well as to provide examples for how to deal with the nested list expansion.
RS5-Pro is no longer updated in the Azure Marketplace. In order to ensure the Windows 10 VMs are regularly updated, we need to switch the default image to 20H2-Pro, which is regularly maintained.
Updates the following libraries in the service:
* azure-core
* azure-functions
* azure-identity
* azure-keyvault-keys
* azure-keyvault-secrets
* azure-mgmt-compute
* azure-mgmt-core
* azure-mgmt-loganalytics
* azure-mgmt-network
* azure-mgmt-resource
* azure-mgmt-storage
* azure-mgmt-subscription
* azure-storage-blob
* azure-storage-queue
* pydantic
* requests
* jsonpatch
Removes the following libraries in the service:
* azure-cli-core
* azure-cli-nspkg
* azure-mgmt-cosmosdb
* azure-servicebus
Updates the following libraries in the CLI:
* requests
* semver
* asciimatics
* pydantic
* tenacity
Updates the following libraries in onefuzztypes:
* pydantic
The primary "legacy" libraries are [azure-graphrbac](https://pypi.org/project/azure-graphrbac/) and azure-cosmosdb-table. The former has not been updated to use azure-identity yet. The later is being rewritten as [azure-data-tables](https://pypi.org/project/azure-data-tables/), but is still in early beta.
Mitigate the deployment issue related to the conditional access policy.
The registration logic is updated to use the old rbac python library when possible.
The deployment will print some manual step for operations that cannot be automated
- Switch to using `goblin` for both ELF and PE parsing
- Refactor block entry point recovery, with better documentation
- Fix a broken example binary
Co-authored-by: bmc-msft <41130664+bmc-msft@users.noreply.github.com>
This PR adds an experimental "local" mode for the agent, starting with `libfuzzer`. For tasks that poll a queue, in local mode, they just monitor a directory for new files.
Supported commands:
* libfuzzer-fuzz (models the `libfuzzer-fuzz` task)
* libfuzzer-coverage (models the `libfuzzer-coverage` task)
* libfuzzer-crash-report (models the `libfuzzer-crash-report` task)
* libfuzzer (models the `libfuzzer basic` job template, running libfuzzer-fuzz and libfuzzer-crash-report tasks concurrently, where any files that show up in `crashes_dir` are automatically turned into reports, and optionally runs the coverage task which runs the coverage data exporter for each file that shows up in `inputs_dir`).
Under the hood, there are a handful of changes required to the rest of the system to enable this feature.
1. `SyncedDir` URLs are now optional. In local mode, these no longer make sense. (We've discussed moving management of `SyncedDirs` to the Supervisor. This is tangential to that effort.)
2. `InputPoller` uses a `tempdir` rather than abusing `task_id` for temporary directory naming.
3. Moved the `agent` to only use a single tokio runtime, rather than one for each of the subcommands.
4. Sets the default log level to `info`. (RUST_LOG can still be used as is).
Note, this removes the `onefuzz-agent debug` commands for the tasks that are now exposed via `onefuzz-agent local`, as these provide a more featureful version of the debug tasks.
## Summary of the Pull Request
Adds a new placeholder {setup_dir} for the setup directory
## PR Checklist
* [x] Applies to work item: #221
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/onefuzz) and sign the CLI.
* [x] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
## Info on Pull Request
_What does this include?_
## Validation Steps Performed
_How does someone test & validate?_
- Depend on `memmap2`, a maintained fork of the abandoned `memmap` crate
- Revert #364, which temporarily suppressed the relevant `cargo-audit` CI error
Closes#363.
## Summary of the Pull Request
_What is this about?_
## PR Checklist
* [ ] Applies to work item: #xxx
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/onefuzz) and sign the CLI.
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
## Info on Pull Request
_What does this include?_
## Validation Steps Performed
_How does someone test & validate?_
The underlying impact is that nodes must re-register on a more frequent basis.
Nodes find out they are out-of-date is during registration and immediately prior to starting a new set of work. Requiring nodes re-register on a shortened cycle provides more opportunities for nodes to get re-imaged.
Additionally, this addresses an issue handling the SAS URL expiry in a more clean fashion in the supervisor.
