2148 Commits

Author SHA1 Message Date
Noah McGregor Harper
5bfcc4e242
Port 'missing field' error fix to C# (#2905)
* Port  Fix to C#.

* Remove extra space.

* Adding correct condition.

---------

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>
2023-03-03 13:36:48 -08:00
Teo Voinea
091c870be6
Create 2 way IPC connection between agent and task (#2886)
* .

* It doesn't work yet but we're making progress

* Added graceful shutdown and tests

* Small fix

* Fix crate issues

* test fix

* Fix build

* make clippy happy

* The order changed

* Use timeout in kill
2023-03-03 14:08:34 -05:00
Noah McGregor Harper
8a7a0b0138
[Bug Fix] Create Wrapper Function To Handle GUID Table Properties (#2898)
* Investigating Proxy Issues.

* Creating and using wrapper function that converts GUIDs to strings.

* Remove log statements.

* Removing logging statements.

* Formatting imports.

* Removing more logging.

* Adding unit test for filter.

* Remove comment.
2023-03-02 12:57:55 -08:00
Noah McGregor Harper
e653f9b73e
Resolving Variable Reference Error (#2903) 2023-03-02 12:44:49 -08:00
Joe Ranweiler
f12319b359
Document coverage crate and tool (#2904) 2023-03-02 11:50:13 -08:00
Cheick Keita
71c82235c6
Update the error truncating logic to retrieve the last messages (#2896)
* Update the error truncating logic to retrieve the last messages instead of the for first ones

* format
2023-03-02 08:57:37 -08:00
Adam
503fc7cd9e
Update c# functional testing InfoResponse (#2894)
* Updating func test info 'properties' key to 'versions'
2023-03-01 10:22:55 -08:00
Noah McGregor Harper
ca7e07f027
Reverting client_id Name Change (#2889) 2023-02-27 16:01:57 -08:00
Cheick Keita
3d299ce51e
fix extra container intilization (#2887)
* fix extra container intilization

* fix extra url download

* fix extra dir parameter to the agent

* rename extra to extra_dir
2023-02-27 13:57:32 -08:00
George Pollard
a7eab4d973
Update SharpFuzz to a version supporting .net7.0, change .NET installation method (#2878)
* Update SharpFuzz to a version supporting .net7.0

* Two-digit version numbers are Channels, not Versions
2023-02-23 21:25:16 +00:00
Cheick Keita
b84896802c
Adding extra container to tasks (#2847)
* adding extra container to tasks

* setup expand

* build fix

* generate docs

* build fix

* build fix

* build fix

* format

* format

* build fix

* fix extra container references

* format

* Update "Needs Triage" label to the one we use. (#2845)

* Report extension errors (#2846)

Old failure message:
```
failed to launch extension
```

New failure message:

```
failed to launch extension(s): Errors for extension 'CustomScriptExtension':
:Error: ProvisioningState/failed/3 (Provisioning failed) - Failed to download all specified files. Exiting. Error Message: The remote server returned an error: (400) Bad Request.
```

* Sematically validate notification configs (#2850)

* Add new command

* Update remaining jinja templates and references to use scriban

* Add ado template validation

* Validate ado and github templates

* Remove unnecessary function

* Update src/ApiService/ApiService/OneFuzzTypes/Model.cs

Co-authored-by: Cheick Keita <kcheick@gmail.com>

---------

Co-authored-by: Cheick Keita <kcheick@gmail.com>

* adding extra container to integration tests

* adding doc

* update tests

* format

* build and clippy fix

* Update src/agent/onefuzz-task/src/tasks/report/generic.rs

Co-authored-by: Teo Voinea <58236992+tevoinea@users.noreply.github.com>

---------

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
Co-authored-by: Teo Voinea <58236992+tevoinea@users.noreply.github.com>
2023-02-23 19:08:01 +00:00
Marc Greisen
dfb0db87c1
Update the document name. (#2882) 2023-02-22 14:38:45 -08:00
George Pollard
6cd53b1db6
Fix .NET integration tests (#2879) 2023-02-22 12:05:45 +13:00
George Pollard
cfcff716d1
Bump azure_* libs (#2876) 2023-02-21 09:13:14 -05:00
Joe Ranweiler
4caaf8fc32
Remove old libfuzzer dotnet template (#2875)
### Context
The original `libfuzzer dotnet` job template was a proof of concept that demonstrated how the `libfuzzer_fuzz` task could be used to express fuzzing via the (pre SharpFuzz 2.0) `libfuzzer-dotnet` tool. It (and its associated integration test) used a harness that linked an older version of SharpFuzz, and which is incompatible with LibFuzzerDotnetLoader (which requires SharpFuzz 2.0 or greater).

### Changes
- Rename `libfuzzer dotnet_dll` job template to `libfuzzer dotnet`, making it the _only_ `libfuzzer-dotnet` template
- Remove integration tests and docs for the old proof-of-concept job type

### Notice

This is a breaking change.

Closes #2874.
2023-02-21 10:54:36 +13:00
Joe Ranweiler
42c4f6204d
Remove rest of unused telemetry (#2863) 2023-02-20 06:21:29 -08:00
dependabot[bot]
9d3321b532
Bump cpp_demangle from 0.3.5 to 0.4.0 in /src/agent (#2544)
Bumps [cpp_demangle](https://github.com/gimli-rs/cpp_demangle) from 0.3.5 to 0.4.0.
- [Release notes](https://github.com/gimli-rs/cpp_demangle/releases)
- [Changelog](https://github.com/gimli-rs/cpp_demangle/blob/master/CHANGELOG.md)
- [Commits](https://github.com/gimli-rs/cpp_demangle/commits)

---
updated-dependencies:
- dependency-name: cpp_demangle
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-19 23:46:53 +00:00
dependabot[bot]
740e4c9745
Bump tui from 0.18.0 to 0.19.0, crossterm from 0.22.1 to 0.25 (#2274)
* Bump tui from 0.18.0 to 0.19.0, crossterm from 0.22.1 to 0.25

Dependabot couldn't find the original pull request head commit, 07b732fded52e9c960a880710489c31e753a7cba.

* Bump crossterm

---------

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-02-20 12:22:54 +13:00
George Pollard
2ac50624c5
Path to file has changed, update it (#2154) 2023-02-19 22:27:07 +00:00
dependabot[bot]
9abade2517
Bump regex from 1.6.0 to 1.7.1 in /src/agent (#2857)
Bumps [regex](https://github.com/rust-lang/regex) from 1.6.0 to 1.7.1.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.6.0...1.7.1)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-19 22:06:07 +00:00
George Pollard
cd18c6066d
Update clap to remove suppressions (#2856)
A Friday afternoon jaunt.

To be merged after #2855; does the remaining work to close #2277 (and closes off some things that Dependabot was trying to upgrade).

Tested by manually running the commands; we don't have good coverage for this kind of stuff. OTOH, most of these commands are for the experimental local fuzzing mode, which is not fully supported yet. I did specifically test the `onefuzz-task managed` command which is the one used in production.

## Details

- Bump `clap` to 4.1.6
  - Remove `structopt` as this is subsumed by clap now
- Bump `envlogger` to 0.10 (removes problematic dependency)
- Set `default-features=false` on `proc-maps` (removes a feature which is only needed to support FreeBSD), and bump it to 0.3

The main changes migrating `clap` are:

- `value_t!` is gone; now use `matches.get_one::<T>`. If `T` is not `String` then a parser must have been registered on the `Arg` when it was created, with `arg.value_parser(value_parser!(T))`.
- `Command::with_name` and `Arg::with_name` are now called `new`.
- `Command` and `Subcommand` were unified, and `App` is removed.
- `arg.takes_value(true)` is gone; it is the default. For flags use `arg.action(ArgAction::SetTrue)` and then retrieve the flag value with `matches.get_flag`.

This code would be simplified a lot by using the `clap::Parser` on structs, but that requires reworking the code significantly as we cannot dynamically add/remove arguments the way that this is currently done.

## Also found

Found one bug while manually testing the `onefuzz-task local` commands; see comment below.
2023-02-20 10:45:15 +13:00
George Pollard
1b07b7df25
Bump clap in proxy to 4.0 (#2855)
* Upgrade clap in proxy-manager
2023-02-19 20:19:51 +00:00
Joe Ranweiler
d5cff033d9
Remove function allowlist (#2859) 2023-02-17 20:19:38 -08:00
Noah McGregor Harper
1ac3fd4bed
Config Refactor Part 2 - Change Opt Param Names & Set File Expiry (#2835)
* Remove Old Optional Parameters and Hardcoded Values.

* Set file to expire.

* Adding expiry.

* test sleep

* Tested expiry.

* Set expirty to 24hrs.

* Syntax error.

* Formatting.

* Changing optional.

* Adding new params.

* Removing arguments.

* Removing arguments.

* Changing param names.

* Update params.
2023-02-17 16:12:17 -08:00
Joe Ranweiler
ddbc715b3f
Remove Z3 telemetry (#2860) 2023-02-17 14:32:52 -08:00
Noah McGregor Harper
5061a3de29
Updating Files w/ New Flags (#2861) 2023-02-17 13:23:56 -08:00
dependabot[bot]
b38ab5de9e
Bump arraydeque from 0.4.5 to 0.5.1 in /src/agent (#2849)
* Bump arraydeque from 0.4.5 to 0.5.1 in /src/agent

Bumps [arraydeque](https://github.com/andylokandy/arraydeque) from 0.4.5 to 0.5.1.
- [Release notes](https://github.com/andylokandy/arraydeque/releases)
- [Commits](https://github.com/andylokandy/arraydeque/commits)

---
updated-dependencies:
- dependency-name: arraydeque
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Fixups

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2023-02-16 22:04:08 +00:00
Joe Ranweiler
ea90eb387b
Accept optional dir of coverage test inputs (#2853) 2023-02-16 12:59:48 -08:00
Teo Voinea
572671b58e
Sematically validate notification configs (#2850)
* Add new command

* Update remaining jinja templates and references to use scriban

* Add ado template validation

* Validate ado and github templates

* Remove unnecessary function

* Update src/ApiService/ApiService/OneFuzzTypes/Model.cs

Co-authored-by: Cheick Keita <kcheick@gmail.com>

---------

Co-authored-by: Cheick Keita <kcheick@gmail.com>
2023-02-16 18:08:30 +00:00
George Pollard
124f50ded6
Report extension errors (#2846)
Old failure message:
```
failed to launch extension
```

New failure message:

```
failed to launch extension(s): Errors for extension 'CustomScriptExtension':
:Error: ProvisioningState/failed/3 (Provisioning failed) - Failed to download all specified files. Exiting. Error Message: The remote server returned an error: (400) Bad Request.
```
2023-02-16 16:04:59 +13:00
Marc Greisen
e0e6981f75
Update "Needs Triage" label to the one we use. (#2845) 2023-02-15 23:53:40 +00:00
George Pollard
f3524c37aa
Fix error reporting for creating a Scaleset under a missing Pool (#2844) 2023-02-15 22:32:21 +00:00
George Pollard
5ea0128727
Add --notification_config support for dotnet templates (#2842) 2023-02-16 10:54:58 +13:00
Joe Ranweiler
58f756de47
Switch over to new coverage task (#2741) 2023-02-15 12:41:19 -08:00
Teo Voinea
2d59a2a5ca
Support another case when adding AssignedTo to telemetry (#2829)
* Add new command

* Catch another case
2023-02-15 19:24:11 +00:00
Teo Voinea
32cdf8b738
Sets EnableScriban to false by default at deployment time (#2839) 2023-02-15 18:58:49 +00:00
Marc Greisen
fbdba5d76b
Add the new fabricBot IssueResponder. For now only trigger to add the Needs Triage label. (#2822)
Update the Sync workflow so that it does not run when labels are added or removed.
2023-02-15 10:45:58 -08:00
Teo Voinea
08bb0ec355
Update remaining jinja docs (#2838)
* Add new command

* Update remaining jinja templates and references to use scriban
2023-02-15 18:25:40 +00:00
Teo Voinea
e9f5a6a2e7
Validate scriban on new notifications (#2834)
* Add new command

* Enforce scriban at notification creation time

* fmt

* missed when merging
2023-02-15 13:16:01 -05:00
Joe Ranweiler
21374b36e6
Defer setting coverage breakpoints (#2832) 2023-02-14 10:52:08 -08:00
Joe Ranweiler
0c58a59011
Fix coverage example build (#2831) 2023-02-14 07:54:19 -08:00
Joe Ranweiler
bb82580a1f
Add function allowlist to coverage example exe (#2830) 2023-02-14 07:25:10 -08:00
Teo Voinea
307526fb64
[Breaking Change] Deploy with scriban only turned on (#2809)
* Deploy with scriban only turned on

* Improve description
2023-02-14 13:31:39 +00:00
Noah McGregor Harper
0bc22a8073
Fixing bug in registration.py for creating CLI SP's. (#2828)
* Fixing bug in registration.py for creating CLI SP's.

* Formatting.
2023-02-10 15:11:35 -08:00
Joe Ranweiler
85fe434b3e
Remove stray print statement (#2823) 2023-02-10 14:31:34 -08:00
Adam
a6814f0745
Release-6.4.0 (#2799)
* Release-6.4.0
---------
Co-authored-by: Noah McGregor Harper <74685766+nharper285@users.noreply.github.com>
2023-02-10 06:42:00 -08:00
Teo Voinea
2e6eef7654
Add notification get command (#2818)
* Add new command

* lint
2023-02-09 11:30:40 -05:00
Teo Voinea
a13bb2feb3
Interact with migrations via cli (#2816)
* Interact with migrations via cli

* Move command under notifications
2023-02-08 22:02:40 +00:00
Cheick Keita
f13f52ab71
Deprecating the job template feature (#2798)
* deprecating the job template feature

* removing the code

* format
2023-02-08 19:21:03 +00:00
Cheick Keita
d732028201
Add unmanaged nodes integration tests (#2780)
* Add docker file to the runtime tools

* fixes

* bug fixes

* more bug fixes and added doc

* don;t overwrite the RUST_LOG env var

* integration test for unmanaged nodes

* add unamanged parameters to launch()

* add ing object_id

* more bug fixes

* bug fixes

* chmod on the linux files in docker

* format

* cleanup merge

* added test_unmanaged command

* cleanup

* use a single image for the docker compose
remove images after the test

* docs and formatting

* format

* format

* format and bug fixes

* using windows server

* fix linux container
make the base image a paramter on windows
use the windows server base image on  windows server

* format

* bug fix

* more fixes

* allow reboot

* more fixes

* added more logging around the service principal creation

* format

* more logging

* change restart policy

* fix multi tenant domain

* more fixes

* exit instead of reboot when running inside docker

* remove comment

* build fix

* try_exist instead of exist

* save the docker logs

* bug_fix

* adding timeout

* fix timeout logic

* adding a build profile

* make all agent depend on the first one

* remove profile

* another fix

* restart agent 1

* Update docs/unmnaged-nodes.md

Co-authored-by: Teo Voinea <58236992+tevoinea@users.noreply.github.com>

---------

Co-authored-by: Teo Voinea <58236992+tevoinea@users.noreply.github.com>
2023-02-08 11:07:19 -08:00