1882 Commits

Author SHA1 Message Date
Adam
3adb2fee36
Update integration test pool size (#2935)
* Upping VMSS count from 10 -> 20 on default tests
2023-03-23 16:26:46 -07:00
Cheick Keita
2a8dca21c1
Fix WS-2023-0045 (#2931)
* Fix WS-2023-0045
Upgrade version of tempfile
remove_dir_all was imported by tempfile. The new version removed that dependency

* fix build
2023-03-23 11:17:08 -07:00
George Pollard
3e0d42006c
Remove xml-rs dependency (#2936) 2023-03-22 13:14:48 +13:00
Cheick Keita
cc08109e39
fix CVE-2023-0286 (#2933)
* fix CVE-2023-0286
removing dependency oncryptography==3.3.2

* update azure-cli-core and azure-cli to 2.46.0

* fix version handling in the server
2023-03-21 18:23:07 +00:00
George Pollard
658d2aa01f
Bump to Rust 1.68 (#2934) 2023-03-21 13:04:45 -04:00
Cheick Keita
4956cf5406
fix condition when generating the task config (#2925) 2023-03-17 20:44:30 +00:00
Adam
1f67494334
Deployment fix for --auto_create_cli_app flag bug (#2921)
* Update .gitignore

* re-add sync-fork.yml

deleted after merge from origin/main

* Update README.md

TEST

* Update README.md

* Update sync-fork.yml

bump ver to 1.8

* updated deploy.py and configuration.py

* cleanup

* formatting

* linter cleanup

* linter cleanup 2

* better logging

* last linter issue

* remove extra app

* Updating getting started docs for config refactor

* Update docs/getting-started.md

Co-authored-by: Noah McGregor Harper <74685766+nharper285@users.noreply.github.com>

* update getting-started.md doc for config refactor

* update getting-started.md doc for config refactor

---------

Co-authored-by: Noah McGregor Harper <74685766+nharper285@users.noreply.github.com>
2023-03-10 15:26:40 -08:00
Teo Voinea
49543cfa14
Ipc between agent and task (#2912)
* .

* It doesn't work yet but we're making progress

* Added graceful shutdown and tests

* Small fix

* Fix crate issues

* test fix

* Fix build

* make clippy happy

* The order changed

* Use timeout in kill

* Almost done shutting down ipc

* It should all work now

* Update deny.toml

* Fix warning
2023-03-10 17:21:18 +00:00
Noah McGregor Harper
a374939225
Fix onefuzz repro bug - Remove managed identity from CustomScriptExtension (#2920)
* Adding potential repro fix.

* Removing managed identity.

* Remove return statement.

* reverting changes.

* removing logging.

* Reverting changes.

* Adding back trace.

* Removing protected settings.

* Removing managed identited.

* Conditionally setting protected settings.
2023-03-10 09:11:12 -08:00
Teo Voinea
f00248fb98
Fix notification validation (#2914)
* Add new command

* Update remaining jinja templates and references to use scriban

* Add missing properties to render context when validating notification config
2023-03-07 20:30:32 +00:00
Noah McGregor Harper
e5dc7872ce
Add additional filter check for reports and regressions (#2911)
* Add additional filter check for reports and regressions.

* Change comparator option.
2023-03-07 10:13:30 -08:00
Teo Voinea
6f66fcb9f8
Revert "Create 2 way IPC connection between agent and task" (#2910)
* Revert "Create 2 way IPC connection between agent and task (#2886)"

This reverts commit 091c870be6d9813cfceb60d61932f09c35f9bb67.

* Temporarily allow vulnerability since a new one just came out

* Temporarily allow vulnerability

* Update proxy.sh

* Update agent.sh

* Update deny.toml
2023-03-06 16:09:29 -05:00
Noah McGregor Harper
5bfcc4e242
Port 'missing field' error fix to C# (#2905)
* Port  Fix to C#.

* Remove extra space.

* Adding correct condition.

---------

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>
2023-03-03 13:36:48 -08:00
Teo Voinea
091c870be6
Create 2 way IPC connection between agent and task (#2886)
* .

* It doesn't work yet but we're making progress

* Added graceful shutdown and tests

* Small fix

* Fix crate issues

* test fix

* Fix build

* make clippy happy

* The order changed

* Use timeout in kill
2023-03-03 14:08:34 -05:00
Noah McGregor Harper
8a7a0b0138
[Bug Fix] Create Wrapper Function To Handle GUID Table Properties (#2898)
* Investigating Proxy Issues.

* Creating and using wrapper function that converts GUIDs to strings.

* Remove log statements.

* Removing logging statements.

* Formatting imports.

* Removing more logging.

* Adding unit test for filter.

* Remove comment.
2023-03-02 12:57:55 -08:00
Noah McGregor Harper
e653f9b73e
Resolving Variable Reference Error (#2903) 2023-03-02 12:44:49 -08:00
Joe Ranweiler
f12319b359
Document coverage crate and tool (#2904) 2023-03-02 11:50:13 -08:00
Cheick Keita
71c82235c6
Update the error truncating logic to retrieve the last messages (#2896)
* Update the error truncating logic to retrieve the last messages instead of the for first ones

* format
2023-03-02 08:57:37 -08:00
Adam
503fc7cd9e
Update c# functional testing InfoResponse (#2894)
* Updating func test info 'properties' key to 'versions'
2023-03-01 10:22:55 -08:00
Noah McGregor Harper
ca7e07f027
Reverting client_id Name Change (#2889) 2023-02-27 16:01:57 -08:00
Cheick Keita
3d299ce51e
fix extra container intilization (#2887)
* fix extra container intilization

* fix extra url download

* fix extra dir parameter to the agent

* rename extra to extra_dir
2023-02-27 13:57:32 -08:00
George Pollard
a7eab4d973
Update SharpFuzz to a version supporting .net7.0, change .NET installation method (#2878)
* Update SharpFuzz to a version supporting .net7.0

* Two-digit version numbers are Channels, not Versions
2023-02-23 21:25:16 +00:00
Cheick Keita
b84896802c
Adding extra container to tasks (#2847)
* adding extra container to tasks

* setup expand

* build fix

* generate docs

* build fix

* build fix

* build fix

* format

* format

* build fix

* fix extra container references

* format

* Update "Needs Triage" label to the one we use. (#2845)

* Report extension errors (#2846)

Old failure message:
```
failed to launch extension
```

New failure message:

```
failed to launch extension(s): Errors for extension 'CustomScriptExtension':
:Error: ProvisioningState/failed/3 (Provisioning failed) - Failed to download all specified files. Exiting. Error Message: The remote server returned an error: (400) Bad Request.
```

* Sematically validate notification configs (#2850)

* Add new command

* Update remaining jinja templates and references to use scriban

* Add ado template validation

* Validate ado and github templates

* Remove unnecessary function

* Update src/ApiService/ApiService/OneFuzzTypes/Model.cs

Co-authored-by: Cheick Keita <kcheick@gmail.com>

---------

Co-authored-by: Cheick Keita <kcheick@gmail.com>

* adding extra container to integration tests

* adding doc

* update tests

* format

* build and clippy fix

* Update src/agent/onefuzz-task/src/tasks/report/generic.rs

Co-authored-by: Teo Voinea <58236992+tevoinea@users.noreply.github.com>

---------

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
Co-authored-by: Teo Voinea <58236992+tevoinea@users.noreply.github.com>
2023-02-23 19:08:01 +00:00
George Pollard
6cd53b1db6
Fix .NET integration tests (#2879) 2023-02-22 12:05:45 +13:00
George Pollard
cfcff716d1
Bump azure_* libs (#2876) 2023-02-21 09:13:14 -05:00
Joe Ranweiler
4caaf8fc32
Remove old libfuzzer dotnet template (#2875)
### Context
The original `libfuzzer dotnet` job template was a proof of concept that demonstrated how the `libfuzzer_fuzz` task could be used to express fuzzing via the (pre SharpFuzz 2.0) `libfuzzer-dotnet` tool. It (and its associated integration test) used a harness that linked an older version of SharpFuzz, and which is incompatible with LibFuzzerDotnetLoader (which requires SharpFuzz 2.0 or greater).

### Changes
- Rename `libfuzzer dotnet_dll` job template to `libfuzzer dotnet`, making it the _only_ `libfuzzer-dotnet` template
- Remove integration tests and docs for the old proof-of-concept job type

### Notice

This is a breaking change.

Closes #2874.
2023-02-21 10:54:36 +13:00
Joe Ranweiler
42c4f6204d
Remove rest of unused telemetry (#2863) 2023-02-20 06:21:29 -08:00
dependabot[bot]
9d3321b532
Bump cpp_demangle from 0.3.5 to 0.4.0 in /src/agent (#2544)
Bumps [cpp_demangle](https://github.com/gimli-rs/cpp_demangle) from 0.3.5 to 0.4.0.
- [Release notes](https://github.com/gimli-rs/cpp_demangle/releases)
- [Changelog](https://github.com/gimli-rs/cpp_demangle/blob/master/CHANGELOG.md)
- [Commits](https://github.com/gimli-rs/cpp_demangle/commits)

---
updated-dependencies:
- dependency-name: cpp_demangle
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-19 23:46:53 +00:00
dependabot[bot]
740e4c9745
Bump tui from 0.18.0 to 0.19.0, crossterm from 0.22.1 to 0.25 (#2274)
* Bump tui from 0.18.0 to 0.19.0, crossterm from 0.22.1 to 0.25

Dependabot couldn't find the original pull request head commit, 07b732fded52e9c960a880710489c31e753a7cba.

* Bump crossterm

---------

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-02-20 12:22:54 +13:00
George Pollard
2ac50624c5
Path to file has changed, update it (#2154) 2023-02-19 22:27:07 +00:00
dependabot[bot]
9abade2517
Bump regex from 1.6.0 to 1.7.1 in /src/agent (#2857)
Bumps [regex](https://github.com/rust-lang/regex) from 1.6.0 to 1.7.1.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.6.0...1.7.1)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-19 22:06:07 +00:00
George Pollard
cd18c6066d
Update clap to remove suppressions (#2856)
A Friday afternoon jaunt.

To be merged after #2855; does the remaining work to close #2277 (and closes off some things that Dependabot was trying to upgrade).

Tested by manually running the commands; we don't have good coverage for this kind of stuff. OTOH, most of these commands are for the experimental local fuzzing mode, which is not fully supported yet. I did specifically test the `onefuzz-task managed` command which is the one used in production.

## Details

- Bump `clap` to 4.1.6
  - Remove `structopt` as this is subsumed by clap now
- Bump `envlogger` to 0.10 (removes problematic dependency)
- Set `default-features=false` on `proc-maps` (removes a feature which is only needed to support FreeBSD), and bump it to 0.3

The main changes migrating `clap` are:

- `value_t!` is gone; now use `matches.get_one::<T>`. If `T` is not `String` then a parser must have been registered on the `Arg` when it was created, with `arg.value_parser(value_parser!(T))`.
- `Command::with_name` and `Arg::with_name` are now called `new`.
- `Command` and `Subcommand` were unified, and `App` is removed.
- `arg.takes_value(true)` is gone; it is the default. For flags use `arg.action(ArgAction::SetTrue)` and then retrieve the flag value with `matches.get_flag`.

This code would be simplified a lot by using the `clap::Parser` on structs, but that requires reworking the code significantly as we cannot dynamically add/remove arguments the way that this is currently done.

## Also found

Found one bug while manually testing the `onefuzz-task local` commands; see comment below.
2023-02-20 10:45:15 +13:00
George Pollard
1b07b7df25
Bump clap in proxy to 4.0 (#2855)
* Upgrade clap in proxy-manager
2023-02-19 20:19:51 +00:00
Joe Ranweiler
d5cff033d9
Remove function allowlist (#2859) 2023-02-17 20:19:38 -08:00
Noah McGregor Harper
1ac3fd4bed
Config Refactor Part 2 - Change Opt Param Names & Set File Expiry (#2835)
* Remove Old Optional Parameters and Hardcoded Values.

* Set file to expire.

* Adding expiry.

* test sleep

* Tested expiry.

* Set expirty to 24hrs.

* Syntax error.

* Formatting.

* Changing optional.

* Adding new params.

* Removing arguments.

* Removing arguments.

* Changing param names.

* Update params.
2023-02-17 16:12:17 -08:00
Joe Ranweiler
ddbc715b3f
Remove Z3 telemetry (#2860) 2023-02-17 14:32:52 -08:00
dependabot[bot]
b38ab5de9e
Bump arraydeque from 0.4.5 to 0.5.1 in /src/agent (#2849)
* Bump arraydeque from 0.4.5 to 0.5.1 in /src/agent

Bumps [arraydeque](https://github.com/andylokandy/arraydeque) from 0.4.5 to 0.5.1.
- [Release notes](https://github.com/andylokandy/arraydeque/releases)
- [Commits](https://github.com/andylokandy/arraydeque/commits)

---
updated-dependencies:
- dependency-name: arraydeque
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fixups

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-02-16 22:04:08 +00:00
Joe Ranweiler
ea90eb387b
Accept optional dir of coverage test inputs (#2853) 2023-02-16 12:59:48 -08:00
Teo Voinea
572671b58e
Sematically validate notification configs (#2850)
* Add new command

* Update remaining jinja templates and references to use scriban

* Add ado template validation

* Validate ado and github templates

* Remove unnecessary function

* Update src/ApiService/ApiService/OneFuzzTypes/Model.cs

Co-authored-by: Cheick Keita <kcheick@gmail.com>

---------

Co-authored-by: Cheick Keita <kcheick@gmail.com>
2023-02-16 18:08:30 +00:00
George Pollard
124f50ded6
Report extension errors (#2846)
Old failure message:
```
failed to launch extension
```

New failure message:

```
failed to launch extension(s): Errors for extension 'CustomScriptExtension':
:Error: ProvisioningState/failed/3 (Provisioning failed) - Failed to download all specified files. Exiting. Error Message: The remote server returned an error: (400) Bad Request.
```
2023-02-16 16:04:59 +13:00
George Pollard
f3524c37aa
Fix error reporting for creating a Scaleset under a missing Pool (#2844) 2023-02-15 22:32:21 +00:00
George Pollard
5ea0128727
Add --notification_config support for dotnet templates (#2842) 2023-02-16 10:54:58 +13:00
Joe Ranweiler
58f756de47
Switch over to new coverage task (#2741) 2023-02-15 12:41:19 -08:00
Teo Voinea
2d59a2a5ca
Support another case when adding AssignedTo to telemetry (#2829)
* Add new command

* Catch another case
2023-02-15 19:24:11 +00:00
Teo Voinea
32cdf8b738
Sets EnableScriban to false by default at deployment time (#2839) 2023-02-15 18:58:49 +00:00
Teo Voinea
e9f5a6a2e7
Validate scriban on new notifications (#2834)
* Add new command

* Enforce scriban at notification creation time

* fmt

* missed when merging
2023-02-15 13:16:01 -05:00
Joe Ranweiler
21374b36e6
Defer setting coverage breakpoints (#2832) 2023-02-14 10:52:08 -08:00
Joe Ranweiler
0c58a59011
Fix coverage example build (#2831) 2023-02-14 07:54:19 -08:00
Joe Ranweiler
bb82580a1f
Add function allowlist to coverage example exe (#2830) 2023-02-14 07:25:10 -08:00
Teo Voinea
307526fb64
[Breaking Change] Deploy with scriban only turned on (#2809)
* Deploy with scriban only turned on

* Improve description
2023-02-14 13:31:39 +00:00