Commit Graph

2027 Commits

Author SHA1 Message Date
938176b700 Use FluentAssertions.Analyzers (#3314) 2023-07-20 09:07:39 -07:00
b9911206ac Bump notify from 5.1.0 to 6.0.1 in /src/agent (#3195)
* Bump notify from 5.1.0 to 6.0.1 in /src/agent

Bumps [notify](https://github.com/notify-rs/notify) from 5.1.0 to 6.0.1.
- [Release notes](https://github.com/notify-rs/notify/releases)
- [Changelog](https://github.com/notify-rs/notify/blob/main/CHANGELOG.md)
- [Commits](https://github.com/notify-rs/notify/compare/notify-5.1.0...notify-6.0.1)

---
updated-dependencies:
- dependency-name: notify
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update for new notify API which has renames

* Add tests for expected behaviour

* Disable crossbeam support as recommended for use with Tokio

* Fix Windows behaviour

* Pre-filter events

* Log when ignoring unknown rename event

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-07-19 23:57:46 +00:00
06fb0d3fd0 Kanan/test template fix (#3308)
* Fix test_template logic

* Rewrite test_template logic to make it's intention more clear
2023-07-19 15:53:42 -07:00
9bc1ab3c55 Bump Microsoft.Azure.Functions.Worker from 1.14.1 to 1.18.0 in /src/ApiService (#3291)
* Bump Microsoft.Azure.Functions.Worker in /src/ApiService

Bumps [Microsoft.Azure.Functions.Worker](https://github.com/Azure/azure-functions-dotnet-worker) from 1.14.1 to 1.18.0.
- [Release notes](https://github.com/Azure/azure-functions-dotnet-worker/releases)
- [Changelog](https://github.com/Azure/azure-functions-dotnet-worker/blob/1.18.0/release_notes.md)
- [Commits](https://github.com/Azure/azure-functions-dotnet-worker/compare/1.14.1...1.18.0)

---
updated-dependencies:
- dependency-name: Microsoft.Azure.Functions.Worker
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* dotnet restore

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-07-18 22:15:28 +00:00
84d01153be Release-8.5.0 (#3288)
* Release-8.5.0
2023-07-18 19:45:14 +00:00
25f125748d Update packaging of LibFuzzerDotNetLoader (#3248)
* Trim LibFuzzerDotNetLoader

* Trim more, update invocations

* Undo trimming

* Also reduce size of SharpFuzz

* Undo Sharpfuzz changes; they break it for some reason
2023-07-18 04:16:44 +00:00
0bac434163 UPN changes/fixes (#3289)
Starts to address #3158.

Rather than storing the UPN and then removing it later, do not store it in the first place.

I found one problem while working on this which was that `UserInfo` was not actually stored on `Job`, so address that as well. This was because properties which are not constructor parameters are not stored by the ORM, so I've also removed other examples from `Task` and `Node` and have them only on the response types.

---

Also started to simplify the ORM roundtripping tests; remove the custom equality comparison and instead simply compare JSON-serialized values. Rely on the FsCheck automatic generation of values where it works, rather than writing a custom Arbitrary which does the same thing.
2023-07-17 22:09:41 +00:00
5d05cdd403 Fix some CI flakiness by separating more artifacts (#3250)
Many jobs uploading output into the same artifact causes spurious failures unless the job which consumes the artifact depends upon all of the previous jobs. So, split the uploads into separate artifacts so that the CLI job (which is the main one that fails) can depend upon only a subset of the jobs.

Also add caching to the AFL/AFLPP builds.
2023-07-17 21:58:25 +00:00
4fa55f3ebd Bump insta from 1.30.0 to 1.31.0 in /src/agent (#3297)
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.30.0 to 1.31.0.
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.30.0...1.31.0)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-17 21:25:35 +00:00
5583d0ddd5 Bump Microsoft.Rest.ClientRuntime (#3301) 2023-07-17 21:03:41 +00:00
9a792cf7ba Use debuginfo cache (#3280)
* Use debuginfo cache

* PR comments
2023-07-17 20:45:41 +00:00
a658fc9389 Update README.md (#3279) 2023-07-17 13:56:37 +00:00
f958cb7916 Trim System.Title if length is > 128 (#3284)
* Trim System.Title if length is > 128

* Format Ado.cs

* Add System.Title to Repro Steps if it needs to be trimmed

* Use Convert.ToHexString() instead of an Aggregate

* Add the full length System.Title to the Scriban template instead of inserting it into repro steps

* Upgrade transient dependency for component governance
2023-07-15 01:37:45 +00:00
930bb3f0b7 allow tasks environment variables to be set (#3294)
* allow tasks environment variables to be set

* build fix

* build fix

* build fix

* another fix

* ajusting the analysis paramters

* make tools dir optional in the analysis task

* bug fix
2023-07-14 17:13:15 -07:00
7a92dab7c2 Rust 1.71 (#3292) 2023-07-13 22:00:05 +00:00
e07a55b411 Set up .local file before invoking LibFuzzer (#3269)
Closes #3180.
2023-07-13 21:31:40 +00:00
8a12f95f23 Basic codecov config (#3290)
Don't count examples or test code towards coverage.
2023-07-13 06:01:24 -07:00
0226db281d Make WebhookMessageEventGrid compatible with the event grid format (#3286)
* Make WebhookMessageEventGrid compatible with the event grid format

* build fix

* use webhookmessage in event grid
2023-07-12 01:12:41 +00:00
06b0d158f8 Add --target_timeout flag for qemu_user template command (#3277) 2023-07-11 13:50:29 -07:00
d5b2a33125 Bump pretty_assertions from 1.3.0 to 1.4.0 in /src/agent (#3278)
Bumps [pretty_assertions](https://github.com/rust-pretty-assertions/rust-pretty-assertions) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/rust-pretty-assertions/rust-pretty-assertions/releases)
- [Changelog](https://github.com/rust-pretty-assertions/rust-pretty-assertions/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-pretty-assertions/rust-pretty-assertions/compare/v1.3.0...v1.4.0)

---
updated-dependencies:
- dependency-name: pretty_assertions
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-10 22:12:38 +00:00
41f754e29b Bump regex from 1.8.1 to 1.9.1 in /src/agent (#3281)
Bumps [regex](https://github.com/rust-lang/regex) from 1.8.1 to 1.9.1.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.8.1...1.9.1)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-11 09:57:17 +12:00
da967ace0a Add events retention policy (#3186)
* Start event retention policy

* Add a retention policy to events and enforce the policy

* Add feature flags

* Tests

* .

* .

* Tests work

* fmt

* Fix gen

* Validated todo

* fmt

* PR comments
2023-07-07 14:54:40 +00:00
f503097282 Bump memmap2 from 0.5.3 to 0.7.1 in /src/agent (#3258)
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.3 to 0.7.1.
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.3...v0.7.1)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-06 03:39:04 +00:00
4bb3be3e53 Bump clap from 4.1.6 to 4.3.11 in /src/proxy-manager (#3273)
Bumps [clap](https://github.com/clap-rs/clap) from 4.1.6 to 4.3.11.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.1.6...v4.3.11)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-07-06 03:21:13 +00:00
9be7b9e007 Add tool to check source allowlists (#3246)
* Add tool to check source allowlists

* Format to pass build

---------

Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-07-06 02:32:52 +00:00
e2b34a0dae Resolving Regression Bugs - File Bugs on regression_report and properly reset state on duplicate. (#3263)
* Resolving Regression Bugs.

* lint

* Fixing type.

* Remove log line.

* Switchin to log error.
2023-07-05 14:57:01 -07:00
2bb627c8a3 Bump azure_core from 0.12.0 to 0.13.0 in /src/agent (#3257)
* Bump azure_core from 0.12.0 to 0.13.0 in /src/agent

Bumps [azure_core](https://github.com/azure/azure-sdk-for-rust) from 0.12.0 to 0.13.0.
- [Release notes](https://github.com/azure/azure-sdk-for-rust/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-rust/blob/main/HISTORY.md)
- [Commits](https://github.com/azure/azure-sdk-for-rust/commits)

---
updated-dependencies:
- dependency-name: azure_core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update all azure crates

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-07-05 20:57:28 +00:00
63a66e8248 Updated Threat Model. (#3215) 2023-07-05 13:29:05 -07:00
cd42b5132d Update dependencies to get rid of old windows-sys crates (#3271) 2023-07-04 18:12:56 -07:00
a22f13ef4d Get rid of yanked hermit-abi versions (#3270)
* Get rid of yanked hermit-abi versions

* Update proxy-manager packages
2023-07-05 11:55:06 +12:00
b5c7a0c2e0 Fix timestamp (#3237)
`TimeStamp` should be cased `Timestamp` to match the Python API.
2023-07-03 21:17:49 +00:00
c61aa0af68 Increase test timeout (#3261) 2023-06-29 14:35:51 -07:00
0daa84d70a Bump FsCheck from 2.16.4 to 2.16.5 in /src/ApiService (#3254)
* Bump FsCheck from 2.16.4 to 2.16.5 in /src/ApiService

Bumps [FsCheck](https://github.com/fsharp/FsCheck) from 2.16.4 to 2.16.5.
- [Release notes](https://github.com/fsharp/FsCheck/releases)
- [Changelog](https://github.com/fscheck/FsCheck/blob/master/FsCheck%20Release%20Notes.md)
- [Commits](https://github.com/fsharp/FsCheck/compare/2.16.4...2.16.5)

---
updated-dependencies:
- dependency-name: FsCheck
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* dotnet restore

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-06-29 21:03:31 +00:00
6a7c7ba38a Update container functional test (#3251) 2023-06-29 19:11:41 +00:00
db7ea4b2eb switch to default HttpCompletion, which is ResponseRead (#3259)
Co-authored-by: stas <statis@microsoft.com>
2023-06-29 10:31:02 -07:00
2cbc45fe96 Precache debuginfo analysis for target exe in coverage example (#3225)
* Precache debuginfo analysis for target exe

* Format

* Pre-cache via absolute path

* Add timer logging
2023-06-29 13:46:21 +00:00
99ae2e55e0 Remove .debug_pubnames and .debug_pubtypes sections (#3252) 2023-06-29 09:27:24 -04:00
e1923b4e89 Cache integration test artifacts (#3249)
We build integration test executables every time even when they rarely change; so only build them if they change.
2023-06-29 02:10:41 +00:00
c30fc0d7de Shrink published Rust debug info (#3247)
I noticed these were getting very big and impacting our CI time due to copying huge artifacts. Presumably this would be slowing down copying in the live environment as well.

Two changes have been made:
- Use `debug=1` instead of `debug=true` (equivalent to `debug=2`); this should be sufficient for our needs
- On Linux, compress debug information after building
 
| Binary | Before | with `debug=1` | after compression |
|--|--:|--:|--:|
| onefuzz-agent (Linux) | 170 MB | 83 MB | 30 MB |
| onefuzz-task (Linux) | 284 MB  | 134 MB | 46 MB |
| onefuzz_agent.pdb (Windows) | 89 MB | 42 MB | — |
| onefuzz_task.pdb (Windows) | 150 MB | 63 MB | — |
| onefuzz-deployment.zip | 364 MB | 286 MB | 285 MB |

Overall the compressed `release-artifacts` reduced from 374 → 297 MB.

Build speed improvements (latest build on `main` vs this PR):

| Step | Before | After |
|--|--:|--:|
| agent upload-artifact | 26s/32s/45s | 16s/15s/20s |
| package download-artifact | 1m 57s | 26 s |
| package upload-artifact | 2m 8s | 1m 35s |
2023-06-28 18:45:22 -07:00
042fc24011 Bump Azure.ResourceManager.Resources from 1.3.0 to 1.6.0 in /src/ApiService (#3199)
* Bump Azure.ResourceManager.Resources in /src/ApiService

Bumps [Azure.ResourceManager.Resources](https://github.com/Azure/azure-sdk-for-net) from 1.3.0 to 1.6.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-net/releases)
- [Commits](https://github.com/Azure/azure-sdk-for-net/compare/Azure.ResourceManager.Resources_1.3.0...Azure.ResourceManager.Resources_1.6.0)

---
updated-dependencies:
- dependency-name: Azure.ResourceManager.Resources
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* dotnet restore

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-06-29 00:02:53 +00:00
e6c2ba7c88 Bump tokio from 1.28.0 to 1.29.0 in /src/proxy-manager (#3239)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.28.0 to 1.29.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.28.0...tokio-1.29.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 23:05:48 +00:00
f4774c2f44 Bump tokio from 1.28.0 to 1.29.0 in /src/agent (#3240)
* Bump tokio from 1.28.0 to 1.29.0 in /src/agent

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.28.0 to 1.29.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.28.0...tokio-1.29.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update dependencies list for ARM64

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-06-28 22:40:20 +00:00
9d9df7c6cb Change how Git hash is appended to .NET version string (#3244) 2023-06-28 21:20:19 +00:00
b236514274 Bump Moq from 4.17.2 to 4.18.4 in /src/ApiService (#3203)
Bumps [Moq](https://github.com/moq/moq4) from 4.17.2 to 4.18.4.
- [Release notes](https://github.com/moq/moq4/releases)
- [Changelog](https://github.com/moq/moq4/blob/main/CHANGELOG.md)
- [Commits](https://github.com/moq/moq4/compare/v4.17.2...v4.18.4)

---
updated-dependencies:
- dependency-name: Moq
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 20:55:15 +00:00
f80386f4d9 Updated keyvault access policy (#3109)
* updated keyvault access policy for function app compatibility
2023-06-28 13:37:01 -07:00
9483abc7ad Improve ado validation problem reporting and resiliency (#3222)
* Start adding retries

* Use Polly for retry, better error messages

* Forgot enums.py

* Handle one more exception type
2023-06-28 19:44:51 +00:00
85c3e10991 Correlate cli to service (#3137)
* Start event retention policy

* .

* Correlate telemetry from cli to service and out

* Traces end to end

* Linting

* .

* Fix build failures

* Trying to fix python dependency error

* .

* Lets let pip figure it out

* .

* Modified the wrong file

* .

* .

* .

* .

* .

* .

* This is the one

* fix lints?

* I _love_ python

* ...

* Undo some unnecessary changes

* Works again

* PR comments
2023-06-28 18:47:21 +00:00
b3fa826698 Cleanup (#3241) 2023-06-28 17:24:22 +00:00
f70246a1a6 log exception as "error" since we are retrying anyways (#3238)
* log exception as "error" since we are retrying anyways

* log exception when sending of webhook runs out of attempts

---------

Co-authored-by: stas <statis@microsoft.com>
2023-06-28 08:17:31 -07:00
ab2a2b7851 Add optional condition when updating/re-opening Work Items (#3227)
* Add unless to adotemplate.on_duplicate

* docs

* Tests

* fmt

* Extend capability to support multiple exclusion cases

* Remove unnecessary comment
2023-06-28 13:32:13 +00:00