Add a snapshot-based test to our coverage implementation. This adds coverage for a bunch of code which did not previously have it, and allows us to easily see that coverage is being generated as expected.
A fix was made to the recording code to remove the use of `TargetAllowList` as it was easy to misconfigure `CoverageRecorder` when using it. The use of `TargetAllowList` is now only a container struct in our generic coverage task.
* Look for azcopy.exe in AZCOPY if it's a directory
* Make file check in AZCOPY dir more concise
Co-authored-by: George Pollard <gpollard@microsoft.com>
* Add logic to support searching for azcopy on linux
---------
Co-authored-by: George Pollard <gpollard@microsoft.com>
* Add support for regression reports to 'repro get_files'
* Make get_files logic more clear
* Format api.py with black
* Change crash_info default values for mypy
* Set crash_info['input_blob_container'] as a Container instead of string
* Add logic for handling a missing 'original_crash_test_result' or missing 'crash_report's
* Adjust code format in get_files
* Update logic to comply with mypy
* Fix missing dict index operation
Noticed a few things while doing #3304, so fixing them in another PR:
- Add `CreateNewContainer` which will always create a new container instead of looking for one first.
- InstanceId was not being cached properly (the Lazy would be recreated upon every request) — my fault!
- Add `GetBlob` beside `GetBlogWithTags` for when we don't need the tags
- Make `GetBlogWithTags` do both fetches in parallel
This reverts commit 25f125748d.
The change broke the libfuzzer dotnet template (I could have sworn I validated first with check-pr but…)
Two things didn't work:
1. the path to the LibFuzzerDotNetLoader needed to change; I attempted to fix this in #3324
2. minidumps weren't being captured, for an unknown reason. I haven't figured out why yet, thus I'm going to simply revert the change so we have this working for the next release.
When `LibFuzzerDotNetLoader` loads DLLs that have dependencies, they are not currently resolved correctly, as sibling DLLs in the same directory are not considered. In order to do this, we need to provide a custom `AssemblyLoadContext`. This PR does so, based on the [dotnet core example code](https://learn.microsoft.com/en-us/dotnet/core/tutorials/creating-app-with-plugin-support).
Validated with custom fuzzer implementation; we don't yet have a full test for this in check-pr.
Closes#3326.
Closes#3156.
From now on, we will store additional information about containers in a table (`ContainerInformation`). This is populated when a container is created, or lazily when looking for a container we have not yet stored in the table.
Reads from this table are also cached in memory for a short period of time (currently 10 minutes).
The table is currently used to speed up lookups for containers across storage accounts; we round-robin creation of containers across multiple storage accounts to spread load, so when we read from a container we need to find what storage account it lives in.
The `ContainerInformation` table could also be used in future to store information about access control (see, e.g. #1419).
See #3316. This helps diagnosing problems when running the service.
Read all configuration variables in the service upon startup to detect problems as early as possible. This also allows us to remove a bunch of nullability annotations and makes the config easier to use, as well as enforcing stronger types like `Url` and `ResourceIdentifier`.
* Trim LibFuzzerDotNetLoader
* Trim more, update invocations
* Undo trimming
* Also reduce size of SharpFuzz
* Undo Sharpfuzz changes; they break it for some reason
Starts to address #3158.
Rather than storing the UPN and then removing it later, do not store it in the first place.
I found one problem while working on this which was that `UserInfo` was not actually stored on `Job`, so address that as well. This was because properties which are not constructor parameters are not stored by the ORM, so I've also removed other examples from `Task` and `Node` and have them only on the response types.
---
Also started to simplify the ORM roundtripping tests; remove the custom equality comparison and instead simply compare JSON-serialized values. Rely on the FsCheck automatic generation of values where it works, rather than writing a custom Arbitrary which does the same thing.
Many jobs uploading output into the same artifact causes spurious failures unless the job which consumes the artifact depends upon all of the previous jobs. So, split the uploads into separate artifacts so that the CLI job (which is the main one that fails) can depend upon only a subset of the jobs.
Also add caching to the AFL/AFLPP builds.
