Joe Ranweiler
a1e331a5a6
Require input marker in args when given corpus dir ( #3205 )
...
* Require input marker in args when given corpus dir
* Remove redundant borrow
2023-06-20 16:49:54 -04:00
Joe Ranweiler
f1af0ba190
Rename example to match docs ( #3204 )
...
* Fix coverage example docs
* Revert
* Rename example to match docs
2023-06-20 10:35:28 -07:00
Adam
fcd698c492
updated System.Security.Cryptography.Pkcs ver to 7.0.2 ( #3200 )
...
* updated System.Security.Cryptography.Pkcs ver to 7.0.2
2023-06-19 17:36:08 -07:00
George Pollard
28e36ef2c9
Rust build caching improvements & fixes ( #3197 )
...
Caches are getting too big and we are exceeding the 10GB limit, leading to cache churning.
1. Try to make the caches smaller by using `Swatinem/rust-cache`, which is smarter about what gets cached.
- After doing this it turns out we don't really need `sccache` any more, it has very little impact upon compile times as the cache hit ratio is low. So remove it, to reduce complexity of build and size of build caches.
2. Also fix artifact caching which had been broken by a version format change (4956cf5406fc6817c41928a4713b7da3e4bd130d).
2023-06-19 08:20:04 -07:00
dependabot[bot]
b1b7cf0d27
Bump Microsoft.NET.Test.Sdk from 17.1.0 to 17.6.2 in /src/ApiService ( #3163 )
...
* Bump Microsoft.NET.Test.Sdk from 17.1.0 to 17.6.2 in /src/ApiService
Bumps [Microsoft.NET.Test.Sdk](https://github.com/microsoft/vstest ) from 17.1.0 to 17.6.2.
- [Release notes](https://github.com/microsoft/vstest/releases )
- [Changelog](https://github.com/microsoft/vstest/blob/main/docs/releases.md )
- [Commits](https://github.com/microsoft/vstest/compare/v17.1.0...v17.6.2 )
---
updated-dependencies:
- dependency-name: Microsoft.NET.Test.Sdk
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* dotnet restore
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-06-19 02:18:39 +00:00
microsoft-github-policy-service[bot]
b3955ab4ae
Onboarding to GitOps.ResourceManagement ( #3193 )
...
* Add prIssueManagement.yml to onboard repo to GitOps.ResourceManagement as FabricBot replacement
* Deleting fabricbot.json
---------
Co-authored-by: microsoft-github-policy-service[bot] <77245923+microsoft-github-policy-service[bot]@users.noreply.github.com>
2023-06-18 23:12:01 +00:00
Cheick Keita
181545dfcc
Include debug info in the release binaries to improve backtraces and debuggability ( #3194 )
2023-06-15 15:30:11 -07:00
George Pollard
2e1891c7d8
Update Cargo.lock ( #3191 )
2023-06-15 19:51:07 +00:00
Noah McGregor Harper
229c03832d
Enable Custom Metrics. ( #3190 )
2023-06-15 09:52:10 -07:00
George Pollard
aa54a15427
Add extra_output container, rename extra container ( #3064 )
...
## Summary of the Pull Request
- **Breaking** (but as far as I know this feature is not yet in use): rename the `extra_container` to `extra_setup_container`.
- **Add**: the `extra_output_container`, which pushes its outputs continually.
- We may also want a type of container which both pushes & pulls? See discussion below.
- **Improved**: if `onefuzz-task` fails upon launch, we will log its output for diagnosis (might close #3113 )
---
Some thoughts for the future:
We might want to redesign the containers so that we have something like the following which is passed to the agent, and the agent doesn't need to know the specifics of the containers supplied:
```jsonc
{
// ...
"containers": {
"extra_setup_dir": {
"mode": "pull",
"container_name": "yyy",
},
"extra_output_dir": {
"mode": "push",
"continuous": true, // keep pushing while job is running
"container_name": "xxx"
}
}
}
```
At the moment the agent needs to know what each container is for, for each task type. A more generic and flexible method might be simpler overall.
2023-06-15 02:48:27 +00:00
Cheick Keita
630b083f64
removing agent traces from appinsight ( #3143 )
2023-06-15 00:47:41 +00:00
dependabot[bot]
0d6283f705
Bump quick-xml from 0.28.1 to 0.29.0 in /src/agent ( #3185 )
...
Bumps [quick-xml](https://github.com/tafia/quick-xml ) from 0.28.1 to 0.29.0.
- [Release notes](https://github.com/tafia/quick-xml/releases )
- [Changelog](https://github.com/tafia/quick-xml/blob/master/Changelog.md )
- [Commits](https://github.com/tafia/quick-xml/compare/v0.28.1...v0.29.0 )
---
updated-dependencies:
- dependency-name: quick-xml
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-14 21:35:04 +00:00
Cheick Keita
aff9ffe7e8
Adding a time out when closing the app insight channels ( #3181 )
2023-06-14 19:11:00 +00:00
dependabot[bot]
8cdbf55ea9
Bump tempfile from 3.5.0 to 3.6.0 in /src/agent ( #3169 )
...
Bumps [tempfile](https://github.com/Stebalien/tempfile ) from 3.5.0 to 3.6.0.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md )
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.5.0...v3.6.0 )
---
updated-dependencies:
- dependency-name: tempfile
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-13 21:28:17 +00:00
dependabot[bot]
363c2b0bf8
Bump iced-x86 from 1.18.0 to 1.19.0 in /src/agent ( #3153 )
...
Bumps [iced-x86](https://github.com/icedland/iced ) from 1.18.0 to 1.19.0.
- [Release notes](https://github.com/icedland/iced/releases )
- [Commits](https://github.com/icedland/iced/compare/v1.18.0...v1.19.0 )
---
updated-dependencies:
- dependency-name: iced-x86
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-13 21:02:18 +00:00
dependabot[bot]
8a2b2b4b60
Bump tempfile from 3.5.0 to 3.6.0 in /src/proxy-manager ( #3168 )
...
Bumps [tempfile](https://github.com/Stebalien/tempfile ) from 3.5.0 to 3.6.0.
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md )
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.5.0...v3.6.0 )
---
updated-dependencies:
- dependency-name: tempfile
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-13 20:40:07 +00:00
Joe Ranweiler
39673be256
Try to kill debuggee if Linux recording times out ( #3177 )
...
* Try to kill debuggee if Linux recording times out
* Add extra cleanup
* Fix Windows warnings
* Fix import
* Minimize mutex lock scope
* Remove redundant wait
* Remove unused import
2023-06-13 16:10:27 -04:00
Teo Voinea
8c904199da
Improve error reporting from scale-in protection modification ( #3184 )
2023-06-09 19:41:14 +00:00
Joe Ranweiler
a040aa21ed
Skip entire function if entry offset excluded ( #3172 )
...
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-06-09 19:18:29 +00:00
Adam
300a3e2d76
Release-8.3.0 ( #3159 )
...
* Release-8.3.0
2023-06-08 17:14:02 +00:00
George Pollard
15351a7b44
Bump Rust to 1.70 ( #3165 )
...
https://releases.rs/docs/1.70.0/
2023-06-08 08:53:51 -04:00
George Pollard
96456d0f7f
Make coverage-recording errors non-fatal ( #3166 )
...
When we record coverage, if any of the files fail (e.g. due to timeouts during coverage recording), then we fail the whole process.
Instead, make coverage recording best-effort: if any files fail then we continue to record coverage using the other files. A warning is printed that we can monitor for any ongoing problems.
Closes #3041
2023-06-07 22:58:30 +00:00
George Pollard
37aede810a
Fix install of dotnet in devcontainer ( #3176 )
...
* Install dotnet from Ubuntu package sources
* Update to more recent devcontainer format
2023-06-07 22:21:10 +00:00
Stas
f724741bd3
Port current implementation to ILogger ( #3173 )
...
* Port logging to ILogger
* addressing pr comments
* enable tracking telemetry
* solving merge conflicts
* if debug enable developer mode
* format
* resolving more merging issues
* add reference links to comments
---------
Co-authored-by: stas <statis@microsoft.com>
2023-06-07 12:11:47 -07:00
Kanan B
8e2e11fcd9
Downloading files to locally repro crashes ( #3160 )
...
Add `repro get_files` command
2023-06-07 08:17:51 -07:00
George Pollard
e448947abe
Move auth into middleware ( #3133 )
...
Closes #2098 .
This cleans up the authentication a bit; after this change we have two stages in the middleware pipeline:
- `AuthenticationMiddleware` reads the JWT token (it does not validate it, this is done by the Azure Functions service) and stores it in `FunctionContext.Items["ONEFUZZ_USER_INFO"]`
- `AuthorizationMiddleware` checks the user info against the `[Authorize]` attribute to see if the user has the required permissions
- Functions can read the user info from the `FunctionContext` if needed
The authorize attribute can be `[Authorize(Allow.User)]` or `Allow.Agent` or `Allow.Admin`. The `Admin` case is new and allows this to be declaratively specified rather than being checked in code. We have several functions which could be changed to use this (e.g. Pool POST/DELETE/PATCH, Scaleset POST/DELETE/PATCH), but I have only changed one so far (JinjaToScriban).
One of the benefits here is that this simplifies the test code a lot: we can set the desired user info directly onto our `(Test)FunctionContext` rather than having to supply a fake that pretends to parse the token from the HTTP request. This will also have benefits when running the service locally for testing purposes (refer to internal issue).
The other benefit is the ability to programmatically read the required authentication for each function, which may help with Swagger generation.
2023-06-07 13:57:22 +12:00
Cheick Keita
b44cff5236
Store authentication info in keyvault ( #3127 )
...
* Store authentication info in keyvault
* fix tests
* fix tests
* fix test
* fix build
* test fix
* more fix
* format
* fix test
* fix test
* build
* cleanup
* build fix
* test fix
* catch exception when secret does not exist
* more cleanup
* fix tests
* cleanup
* address comments
* more null check
2023-06-06 13:49:00 -07:00
George Pollard
9aa2519e90
Don’t validate error codes on client side ( #3131 )
...
* Don’t validate error codes on client side
* Update docs
* Format
* Format
* Format
2023-06-05 23:32:44 +00:00
dependabot[bot]
6c2dbbf67b
Bump url from 2.3.0 to 2.4.0 in /src/proxy-manager ( #3152 )
...
Bumps [url](https://github.com/servo/rust-url ) from 2.3.0 to 2.4.0.
- [Release notes](https://github.com/servo/rust-url/releases )
- [Commits](https://github.com/servo/rust-url/compare/v2.3.0...v2.4.0 )
---
updated-dependencies:
- dependency-name: url
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-05 22:29:36 +00:00
dependabot[bot]
db59cd367d
Bump url from 2.3.0 to 2.4.0 in /src/agent ( #3151 )
...
Bumps [url](https://github.com/servo/rust-url ) from 2.3.0 to 2.4.0.
- [Release notes](https://github.com/servo/rust-url/releases )
- [Commits](https://github.com/servo/rust-url/compare/v2.3.0...v2.4.0 )
---
updated-dependencies:
- dependency-name: url
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-06 10:01:39 +12:00
Cheick Keita
c76cb9b44f
switch tui to ratatui ( #3155 )
2023-06-05 17:18:17 +00:00
Cheick Keita
bada352699
Include a reason when a task has never started ( #3148 )
...
* Include a reason to mark a task as failed
* mark dependent task cancelled when the task is cancelled
* cleanup
* build fix
2023-06-04 20:39:29 -07:00
Cheick Keita
5d8f2452d8
remove dependency on the users crate ( #3150 )
2023-06-02 15:43:27 -07:00
Teo Voinea
0f6073b46d
Fix bug for scale-in protection ( #3144 )
...
* Fix bug for scale-in protection
* Update metric name
2023-05-31 12:05:58 +00:00
Adam
98007be1c2
Update requests version in CLI requirements to fix CVE ( #3145 )
...
* updated requests version in CLI requirements to fix CVE
2023-05-30 17:45:38 -07:00
dependabot[bot]
c09daac757
Bump requests in /contrib/deploy-onefuzz-via-azure-devops ( #3120 )
2023-05-30 18:56:00 +00:00
dependabot[bot]
15747c122d
Bump FluentAssertions from 6.10.0 to 6.11.0 in /src/ApiService ( #3140 )
...
Bumps [FluentAssertions](https://github.com/fluentassertions/fluentassertions ) from 6.10.0 to 6.11.0.
- [Release notes](https://github.com/fluentassertions/fluentassertions/releases )
- [Changelog](https://github.com/fluentassertions/fluentassertions/blob/develop/AcceptApiChanges.ps1 )
- [Commits](https://github.com/fluentassertions/fluentassertions/compare/6.10.0...6.11.0 )
---
updated-dependencies:
- dependency-name: FluentAssertions
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-29 21:58:56 +00:00
dependabot[bot]
b63425419e
Bump Azure.ResourceManager from 1.3.1 to 1.6.0 in /src/ApiService ( #3106 )
...
* Bump Azure.ResourceManager from 1.3.1 to 1.6.0 in /src/ApiService
Bumps [Azure.ResourceManager](https://github.com/Azure/azure-sdk-for-net ) from 1.3.1 to 1.6.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-net/releases )
- [Commits](https://github.com/Azure/azure-sdk-for-net/compare/Azure.ResourceManager_1.3.1...Azure.ResourceManager_1.6.0 )
---
updated-dependencies:
- dependency-name: Azure.ResourceManager
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
* dotnet restore
---------
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-05-29 13:03:23 +12:00
George Pollard
18e99cdd2f
Use --locked when installing Rust pre-reqs ( #3139 )
2023-05-29 11:05:09 +12:00
dependabot[bot]
a390822124
Bump path-absolutize from 3.0.11 to 3.1.0 in /src/agent ( #3088 )
...
Bumps [path-absolutize](https://github.com/magiclen/path-absolutize ) from 3.0.11 to 3.1.0.
- [Commits](https://github.com/magiclen/path-absolutize/compare/v3.0.11...v3.1.0 )
---
updated-dependencies:
- dependency-name: path-absolutize
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-25 23:47:08 +00:00
dependabot[bot]
223a90c593
Bump clap from 4.2.4 to 4.3.0 in /src/agent ( #3118 )
...
Bumps [clap](https://github.com/clap-rs/clap ) from 4.2.4 to 4.3.0.
- [Release notes](https://github.com/clap-rs/clap/releases )
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md )
- [Commits](https://github.com/clap-rs/clap/compare/v4.2.4...clap_complete-v4.3.0 )
---
updated-dependencies:
- dependency-name: clap
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-05-25 23:25:15 +00:00
Noah McGregor Harper
5fc5659056
Release 8.2.0 ( #3126 )
...
* Release 8.2.0
* Updating.
* Updated
2023-05-25 12:36:39 -07:00
George Pollard
be56814e43
Bump azure_* crates to 0.12 ( #3132 )
2023-05-25 09:30:16 -04:00
Cheick Keita
dc0e7f76b3
fix logic to set check_fuzzer_help ( #3130 )
2023-05-24 09:22:27 -07:00
Cheick Keita
213fc2471f
fix comment explaining why a validator is needed on the templates ( #3125 )
2023-05-24 15:47:47 +00:00
George Pollard
66b990bb7e
Bring ErrorCode enums into sync ( #3129 )
...
ErrorCode enums on Python & C# side had gotten out of sync. This can cause the CLI to fail to parse responses.
2023-05-24 08:41:41 -04:00
George Pollard
5048e6d50c
Allow use of readonly_inputs for qemu_user template ( #3116 )
...
* Allow use of readonly_inputs for qemu_user template
* formatting fix
2023-05-19 12:33:49 +12:00
George Pollard
dc66164cda
Increase lock wait timeout for qemu_user setup ( #3114 )
2023-05-17 20:55:47 +00:00
George Pollard
2f478d6c0b
Expand valid scaleset names ( #3045 )
...
Scaleset names are now permitted to be any (valid) strings, instead of only GUIDs. When we generate a scaleset name it is now based upon the pool name; for example the pool `pool` might get a scaleset named `pool-3b24ba211cad4b078655914754485838`.
This should be backwards-compatible since GUIDs are [already serialized to table storage as strings](dddcfa4949/src/ApiService/ApiService/onefuzzlib/orm/EntityConverter.cs (L190-L191)
2023-05-17 09:58:58 +12:00
Noah McGregor Harper
d84b72b5fd
Initial Custom Metrics - Node & Task Heartbeat ( #3082 )
...
* Refactor logging to use and new function queue
* Testing setup of custom metric.
* Changing host.json
* Updating log interface.
* changes.
* Fix encoding.
* Updating.
* Updating tests.
* Adding metrics to program
* Pushing latest changes.
* Update interface references.
* Removing string type.
* Add string back.
* Getting additional data for task heartbeat.
* Removing additional fields.
* Removing containers.
* Cleaning up.
* Adding feature flag.
* Adding bicep changes.
* Fixing tests.
* Fixing test metrics.
* Removing most of tests.
* Telemetry Refact Round 2.
* Updated metrics.
* Remove custom metric function.
* Syncing events.cs
* Making optional.
* Using events as metric dimensions.
* Fixing ORM tests.
* Remove metric records.
* Removing bad test.
* Remove testmetrics.'
* Adding test back.
* Improving custom dimensions serialization.
* Update src/ApiService/ApiService/onefuzzlib/Metrics.cs
Co-authored-by: Cheick Keita <kcheick@gmail.com>
* Reverting change.
---------
Co-authored-by: Cheick Keita <kcheick@gmail.com>
2023-05-16 13:17:28 -07:00
