In `Scaleset.cleanup_nodes`, nodes that are no longer part of the scaleset should get deleted. Without filtering the list, the nodes could get re-saved to the Node table later on.
Adds a random initial jitter the size of the heartbeat periodicity to prevent heartbeats storming the service when we launch 3000 nodes roughly at the same time.
Fixes#386
Adds the following:
1. Serializes a workset to disk during setup.
2. Upon deserializing a RebootContext, delete the file from disk (We support rebooting once and only once)
3. Check if a workset exists with a RebootContext
1. If True, continuing processing
2. if False, mark the tasks & node as "Done" with appropriate errors via:
1. send WorkerEvent::Done events for each of the tasks in the work set
2. send StateUpdateEvent::Done for the node
Add a job_template example that demonstrates customization of the arguments to the job.
This example demonstrates setting the Area and Iteration paths for Azure Devops work items.
1. Merge 'create' and 'update' to a single 'save' operation.
2. Allow fetching a single template.
This enables the following workflow:
```
$ onefuzz job_templates manage get libfuzzer_linux > template.json
$ <... update template as desired ...>
$ onefuzz job_templates manage save libfuzzer_linux @./template.json
$
```
Over the weekend, pyOpenssl 20.0 was released. This causes an incompatible library issue during deployment.
Prior to this change, deployment would generate the following error
```
ERROR: pyopenssl 20.0.0 has requirement cryptography>=3.2, but you'll have cryptography 2.9.2 which is incompatible.
```
When running automated deployments, 'tools' were not being properly replaced with the updated versions if the deployment was created _prior_ to the original instance deployment.
## Summary of the Pull Request
Originally, the yml file printed out a semi-generalized _onefuzz config --endpoint_ comman. This command did have a specified _--authority_ and so it used the Microsoft id by default. To enable users to work with OneFuzz on tenants other than the standard Microsoft tenant, we have added a _--authority_ parameter that is printed out at the end of the deployment.
## PR Checklist
* [ ] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx
## Info on Pull Request
Changes to the yml file.
## Validation Steps Performed
We have made this change to our local automation repository and tested an automated deployment pipeline with this change.
Starting earlier today, I saw roughly 1 in 3 deployments fail with the error `Azure.Functions.Cli.Common.CliException: Timed out waiting for SCM to update the Environment Settings`. Redeploying the application resolves the issue. New builds and past releases alike hit this exception.
According to https://github.com/Azure/azure-functions-core-tools/issues/1863, function app deployments may fail due to timeouts related to cold-start.
This PR executes the deploy in a loop with a delay in the case of failure.
This PR fixes two issues:
- First, in MSVC compiled binaries both the LLVM _and_ MSVC symbols are
present, but only the MSVC symbols have correct values. For example:
```
0:000> cdb: Reading initial command '.scriptload DumpCountersOld.js ; !dumpcounters "cov" ; q'
JavaScript script successfully loaded from 'DumpCountersOld.js'
[+] not disabling sympath
INFO: Seed: 58715679
INFO: Loaded 1 modules (3968 inline 8-bit counters): 3968 [00007FF70DB4B000, 00007FF70DB4BF80), # XXX Note
xxx.exe: Running 1 inputs 1 time(s) each.
Running: inp
[+] processing xxx.exe
[+] using LLVM 10 symbols - 0x7ff70db72b00:0x7ff70db72b08 # XXX These are wrong
```
This means the order we search for the coverage symbols is important.
- Secondly, this enables support for MSVC 8bit counter coverage.
## Validation Steps Performed
Running any recent MSVC compiled libfuzzer target should fail to actually collect coverage, instead just returning the 8 null bytes described in the linked issue.
We need to move to supporting data sharding.
One of the steps towards that is stop passing around `account_id`, rather we need to specify the type of storage we need.
This script moves more of the run-time actions to setup-time. This is important for running fuzzing within docker containers, such that installing llvm & gdb is done as part of the container, rather than on each launch.
## Summary of the Pull Request
This enables feature flags for the SDK, which enables gating access to preview features to those that have specifically asked for them. This is intended to be used within #266.
Note, this change also moves to using a `pydantic` model for the config, rather than hand-crafted JSON dicts.
