Commit Graph

2157 Commits

Author SHA1 Message Date
41f754e29b Bump regex from 1.8.1 to 1.9.1 in /src/agent (#3281)
Bumps [regex](https://github.com/rust-lang/regex) from 1.8.1 to 1.9.1.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.8.1...1.9.1)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-11 09:57:17 +12:00
da967ace0a Add events retention policy (#3186)
* Start event retention policy

* Add a retention policy to events and enforce the policy

* Add feature flags

* Tests

* .

* .

* Tests work

* fmt

* Fix gen

* Validated todo

* fmt

* PR comments
2023-07-07 14:54:40 +00:00
f503097282 Bump memmap2 from 0.5.3 to 0.7.1 in /src/agent (#3258)
Bumps [memmap2](https://github.com/RazrFalcon/memmap2-rs) from 0.5.3 to 0.7.1.
- [Changelog](https://github.com/RazrFalcon/memmap2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/RazrFalcon/memmap2-rs/compare/v0.5.3...v0.7.1)

---
updated-dependencies:
- dependency-name: memmap2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-07-06 03:39:04 +00:00
4bb3be3e53 Bump clap from 4.1.6 to 4.3.11 in /src/proxy-manager (#3273)
Bumps [clap](https://github.com/clap-rs/clap) from 4.1.6 to 4.3.11.
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.1.6...v4.3.11)

---
updated-dependencies:
- dependency-name: clap
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-07-06 03:21:13 +00:00
9be7b9e007 Add tool to check source allowlists (#3246)
* Add tool to check source allowlists

* Format to pass build

---------

Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-07-06 02:32:52 +00:00
e2b34a0dae Resolving Regression Bugs - File Bugs on regression_report and properly reset state on duplicate. (#3263)
* Resolving Regression Bugs.

* lint

* Fixing type.

* Remove log line.

* Switchin to log error.
2023-07-05 14:57:01 -07:00
2bb627c8a3 Bump azure_core from 0.12.0 to 0.13.0 in /src/agent (#3257)
* Bump azure_core from 0.12.0 to 0.13.0 in /src/agent

Bumps [azure_core](https://github.com/azure/azure-sdk-for-rust) from 0.12.0 to 0.13.0.
- [Release notes](https://github.com/azure/azure-sdk-for-rust/releases)
- [Changelog](https://github.com/Azure/azure-sdk-for-rust/blob/main/HISTORY.md)
- [Commits](https://github.com/azure/azure-sdk-for-rust/commits)

---
updated-dependencies:
- dependency-name: azure_core
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update all azure crates

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-07-05 20:57:28 +00:00
63a66e8248 Updated Threat Model. (#3215) 2023-07-05 13:29:05 -07:00
cd42b5132d Update dependencies to get rid of old windows-sys crates (#3271) 2023-07-04 18:12:56 -07:00
a22f13ef4d Get rid of yanked hermit-abi versions (#3270)
* Get rid of yanked hermit-abi versions

* Update proxy-manager packages
2023-07-05 11:55:06 +12:00
b5c7a0c2e0 Fix timestamp (#3237)
`TimeStamp` should be cased `Timestamp` to match the Python API.
2023-07-03 21:17:49 +00:00
c61aa0af68 Increase test timeout (#3261) 2023-06-29 14:35:51 -07:00
0daa84d70a Bump FsCheck from 2.16.4 to 2.16.5 in /src/ApiService (#3254)
* Bump FsCheck from 2.16.4 to 2.16.5 in /src/ApiService

Bumps [FsCheck](https://github.com/fsharp/FsCheck) from 2.16.4 to 2.16.5.
- [Release notes](https://github.com/fsharp/FsCheck/releases)
- [Changelog](https://github.com/fscheck/FsCheck/blob/master/FsCheck%20Release%20Notes.md)
- [Commits](https://github.com/fsharp/FsCheck/compare/2.16.4...2.16.5)

---
updated-dependencies:
- dependency-name: FsCheck
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

* dotnet restore

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-06-29 21:03:31 +00:00
6a7c7ba38a Update container functional test (#3251) 2023-06-29 19:11:41 +00:00
db7ea4b2eb switch to default HttpCompletion, which is ResponseRead (#3259)
Co-authored-by: stas <statis@microsoft.com>
2023-06-29 10:31:02 -07:00
2cbc45fe96 Precache debuginfo analysis for target exe in coverage example (#3225)
* Precache debuginfo analysis for target exe

* Format

* Pre-cache via absolute path

* Add timer logging
2023-06-29 13:46:21 +00:00
99ae2e55e0 Remove .debug_pubnames and .debug_pubtypes sections (#3252) 2023-06-29 09:27:24 -04:00
e1923b4e89 Cache integration test artifacts (#3249)
We build integration test executables every time even when they rarely change; so only build them if they change.
2023-06-29 02:10:41 +00:00
c30fc0d7de Shrink published Rust debug info (#3247)
I noticed these were getting very big and impacting our CI time due to copying huge artifacts. Presumably this would be slowing down copying in the live environment as well.

Two changes have been made:
- Use `debug=1` instead of `debug=true` (equivalent to `debug=2`); this should be sufficient for our needs
- On Linux, compress debug information after building
 
| Binary | Before | with `debug=1` | after compression |
|--|--:|--:|--:|
| onefuzz-agent (Linux) | 170 MB | 83 MB | 30 MB |
| onefuzz-task (Linux) | 284 MB  | 134 MB | 46 MB |
| onefuzz_agent.pdb (Windows) | 89 MB | 42 MB | — |
| onefuzz_task.pdb (Windows) | 150 MB | 63 MB | — |
| onefuzz-deployment.zip | 364 MB | 286 MB | 285 MB |

Overall the compressed `release-artifacts` reduced from 374 → 297 MB.

Build speed improvements (latest build on `main` vs this PR):

| Step | Before | After |
|--|--:|--:|
| agent upload-artifact | 26s/32s/45s | 16s/15s/20s |
| package download-artifact | 1m 57s | 26 s |
| package upload-artifact | 2m 8s | 1m 35s |
2023-06-28 18:45:22 -07:00
042fc24011 Bump Azure.ResourceManager.Resources from 1.3.0 to 1.6.0 in /src/ApiService (#3199)
* Bump Azure.ResourceManager.Resources in /src/ApiService

Bumps [Azure.ResourceManager.Resources](https://github.com/Azure/azure-sdk-for-net) from 1.3.0 to 1.6.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-net/releases)
- [Commits](https://github.com/Azure/azure-sdk-for-net/compare/Azure.ResourceManager.Resources_1.3.0...Azure.ResourceManager.Resources_1.6.0)

---
updated-dependencies:
- dependency-name: Azure.ResourceManager.Resources
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* dotnet restore

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-06-29 00:02:53 +00:00
e6c2ba7c88 Bump tokio from 1.28.0 to 1.29.0 in /src/proxy-manager (#3239)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.28.0 to 1.29.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.28.0...tokio-1.29.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 23:05:48 +00:00
f4774c2f44 Bump tokio from 1.28.0 to 1.29.0 in /src/agent (#3240)
* Bump tokio from 1.28.0 to 1.29.0 in /src/agent

Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.28.0 to 1.29.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.28.0...tokio-1.29.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update dependencies list for ARM64

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-06-28 22:40:20 +00:00
9d9df7c6cb Change how Git hash is appended to .NET version string (#3244) 2023-06-28 21:20:19 +00:00
b236514274 Bump Moq from 4.17.2 to 4.18.4 in /src/ApiService (#3203)
Bumps [Moq](https://github.com/moq/moq4) from 4.17.2 to 4.18.4.
- [Release notes](https://github.com/moq/moq4/releases)
- [Changelog](https://github.com/moq/moq4/blob/main/CHANGELOG.md)
- [Commits](https://github.com/moq/moq4/compare/v4.17.2...v4.18.4)

---
updated-dependencies:
- dependency-name: Moq
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 20:55:15 +00:00
f80386f4d9 Updated keyvault access policy (#3109)
* updated keyvault access policy for function app compatibility
2023-06-28 13:37:01 -07:00
9483abc7ad Improve ado validation problem reporting and resiliency (#3222)
* Start adding retries

* Use Polly for retry, better error messages

* Forgot enums.py

* Handle one more exception type
2023-06-28 19:44:51 +00:00
85c3e10991 Correlate cli to service (#3137)
* Start event retention policy

* .

* Correlate telemetry from cli to service and out

* Traces end to end

* Linting

* .

* Fix build failures

* Trying to fix python dependency error

* .

* Lets let pip figure it out

* .

* Modified the wrong file

* .

* .

* .

* .

* .

* .

* This is the one

* fix lints?

* I _love_ python

* ...

* Undo some unnecessary changes

* Works again

* PR comments
2023-06-28 18:47:21 +00:00
b3fa826698 Cleanup (#3241) 2023-06-28 17:24:22 +00:00
f70246a1a6 log exception as "error" since we are retrying anyways (#3238)
* log exception as "error" since we are retrying anyways

* log exception when sending of webhook runs out of attempts

---------

Co-authored-by: stas <statis@microsoft.com>
2023-06-28 08:17:31 -07:00
ab2a2b7851 Add optional condition when updating/re-opening Work Items (#3227)
* Add unless to adotemplate.on_duplicate

* docs

* Tests

* fmt

* Extend capability to support multiple exclusion cases

* Remove unnecessary comment
2023-06-28 13:32:13 +00:00
e8ba37b2e3 Bump uuid from 1.3.0 to 1.4.0 in /src/agent (#3231)
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.3.0...1.4.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 00:09:12 +00:00
ceacb3c101 Bump uuid from 1.3.0 to 1.4.0 in /src/proxy-manager (#3230)
Bumps [uuid](https://github.com/uuid-rs/uuid) from 1.3.0 to 1.4.0.
- [Release notes](https://github.com/uuid-rs/uuid/releases)
- [Commits](https://github.com/uuid-rs/uuid/compare/1.3.0...1.4.0)

---
updated-dependencies:
- dependency-name: uuid
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-27 23:46:40 +00:00
954ccce93b Bump insta from 1.29.0 to 1.30.0 in /src/agent (#3221)
Bumps [insta](https://github.com/mitsuhiko/insta) from 1.29.0 to 1.30.0.
- [Changelog](https://github.com/mitsuhiko/insta/blob/master/CHANGELOG.md)
- [Commits](https://github.com/mitsuhiko/insta/compare/1.29.0...1.30.0)

---
updated-dependencies:
- dependency-name: insta
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-27 23:26:17 +00:00
c6d546d4f5 include the task id in the prerequisite task failure message (#3219)
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-06-27 22:39:30 +00:00
7c0dc7b65e Better errors from Download: Make GetFileSasUrl nullable (#3229)
This allows us to generate 404s when someone attempts to download from a non-existent container. At the moment we generate a 500 which isn't useful, or very good looks-wise.
2023-06-27 22:24:32 +00:00
4b437533f0 Enable codecov for ARM64 (#3236) 2023-06-27 15:13:47 -07:00
44674e803a Release 8.4.0 (#3201)
* Release-8.4.0
2023-06-27 12:11:13 -07:00
512bb078b7 Downgrade error to warning when retrying because of message too large (#3224) 2023-06-26 17:38:25 +00:00
db1e489e55 Handle the case where the secret is not serialized as json (#3223) 2023-06-23 15:35:21 -04:00
a941020a11 Add cache for coverage locations and debuginfo (#3218)
* Add debuginfo cache, use in Linux recorder

* Use debuginfo cache on Windows

* Format

* Rely on `Copy` impl

* Deref

* Remove redundant clone
2023-06-23 11:32:14 -04:00
e603fa9c17 Document how to use the validation tools (#3212)
* Document how to use the validation tools

* address comment
2023-06-22 17:24:47 +00:00
7716a81b22 Creating CustomMetrics for Rust CustomEvents (#3188)
* Adding initial metric.

* Syntax.

* syntax.

* Trying something else.

* Playing around with new metric function:
:

* Trying new format

* Fixing arguments.

* Importing metrics

* Reverting to events

* Removing

* Adding.

* Changing to int.

* Changing back to float.

* Adding metric lines for all events.

* trying to set.

* Fixing.

* Adding copy.

* Was this a problem..

* Adding different.

* Solution for all.

* Another.

* removing

* Resolving.
2023-06-21 15:10:41 -07:00
fda68ac01b Apply allowlist to source conversion (#3208)
* Apply allowlist to source conversion

* Remove vestigial lifetime

* Remove redundant clone
2023-06-21 12:40:35 -07:00
a720dd1444 allow report to be sent when debugging (#3206) 2023-06-21 16:42:45 +00:00
c752114643 Bump strum from 0.24.0 to 0.25.0 in /src/agent (#3198)
Bumps [strum](https://github.com/Peternator7/strum) from 0.24.0 to 0.25.0.
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Peternator7/strum/commits)

---
updated-dependencies:
- dependency-name: strum
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-21 16:12:48 +00:00
f6e936d7a3 Bump ctrlc from 3.2.5 to 3.4.0 in /src/agent (#3146)
Bumps [ctrlc](https://github.com/Detegr/rust-ctrlc) from 3.2.5 to 3.4.0.
- [Release notes](https://github.com/Detegr/rust-ctrlc/releases)
- [Commits](https://github.com/Detegr/rust-ctrlc/compare/3.2.5...3.4.0)

---
updated-dependencies:
- dependency-name: ctrlc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-21 15:53:42 +00:00
c7f404c5ce Bump openssl crate (#3209) 2023-06-21 08:33:50 -07:00
a1e331a5a6 Require input marker in args when given corpus dir (#3205)
* Require input marker in args when given corpus dir

* Remove redundant borrow
2023-06-20 16:49:54 -04:00
f1af0ba190 Rename example to match docs (#3204)
* Fix coverage example docs

* Revert

* Rename example to match docs
2023-06-20 10:35:28 -07:00
fcd698c492 updated System.Security.Cryptography.Pkcs ver to 7.0.2 (#3200)
* updated System.Security.Cryptography.Pkcs ver to 7.0.2
2023-06-19 17:36:08 -07:00