Commit Graph

54 Commits

Author SHA1 Message Date
58f756de47 Switch over to new coverage task (#2741) 2023-02-15 12:41:19 -08:00
c1f6dfc366 Validate scriban from cli (#2800)
* Add validate scriban endpoint to cli

* missed a file

* Lint -- I miss C#

* docs
2023-02-06 08:32:49 -05:00
c277cd1ec3 Update References to Ubuntu 18.04 to 20.04 (#2535)
* Update References to 18.04 to 20.04

* Fixing Ubuntu Image.

* Fix image.
2022-10-28 10:17:57 -07:00
ee0cbd70a0 Enable dotnet fuzzing (#2273)
Add a new CLI job template, `libfuzzer dotnet_dll`, and supporting server-side definitions.
2022-10-19 14:17:27 -07:00
3f35d81f4b Adding New Default Image Config Value to IC. (#2434)
* Adding New Default Image Config Value to IC.

* Removing forced image setting.

* Updating Webhook Events.

* Removing typo.

* Updating webhook_events again.

* Syncing webhook events.

* Fixing check for os type.

* Fixing import.

* PR Suggestions.

* Fix C# Model Typo.

* Removing other refs to images.

* Removing remaining refs to images outside of models.

* Removing hardcoded image values from tests.

* Update Default Proxy and Repro Images.

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>
2022-09-23 10:40:44 -07:00
f37224e8bb Add dotnet coverage task (#2062)
* checkpoint

* some more progress

* more progress

* More progress

* Now it's time to test it

* It works locally 🎉

* Attempting clean build

* fmt

* temporarily stub out macos

* missed a few

* please be the last one

* .

* .

* .

* noop change to unstuck actions

* .

* .

* Fix setup script

* Some fixes

* It works except for a race condition -- use a directory watcher to fix it

* It works end to end!

* Execute the commands using tokio's structs and timeout mechanism

* It works.... for real this time

* Undo timer changes

* Cleanup

* 🧹

* Fix import

* .

* PR comments

* Fix clippy

* Clippy whyyy

* Only check dotnet path once

* fmt

* Fix a couple more comments
2022-07-06 16:13:45 -04:00
52ccf05a29 Remove deprecated libfuzzer_coverage task (#2021)
- Remove the ability to create or execute a `libfuzzer_coverage` task
- Preserve the enum variant in `onefuzztypes` to prevent errors when deserializing old data
- Remove doc references to `libfuzzer_coverage`
2022-06-13 12:38:35 -07:00
9989189e60 Adding Node State to Node Heartbeat (#2024)
* Adding Node State to Node Heartbeat.

* Updating docs.

* Fixing webhook events.

* Formatting.

* Resetting type.

* Updting param.

* Setting to nodestate.
2022-06-13 10:13:57 -07:00
79cc5d54d3 Fix equire_admin_privileges Logic. (#2016) 2022-06-03 15:59:08 -07:00
b080f5b164 Make tool_version, onefuzz_version, tool_name optinal (#1940) 2022-05-13 20:12:22 +00:00
44059f20ca Adding Admin Checks to Node Operations. (#1779)
* Adding Admin Checks to Node Operations.

* Importing function.

* Changing naming convention.

* Fixing webhook events.

* Adding changes to scaleset init.
2022-04-27 11:31:43 -07:00
cb45c5685f add tool_name and onfuzz_version to CrashReport (#1635) 2022-04-18 23:56:07 +00:00
7add51fd3a Log redirection, service side (#1727)
* Setting the service side of the log management
- a log is created or reused when e create a job
- when scheduling the task we send the log location to the agent
The expected log structure looks liek
{fuzzContainer}/logs/{job_id}/{task_id}/{machine_id}/1.log

* regenerate doces

* including job_id in the container name

* regenerating docs
removing bad doc file
2022-03-29 18:47:20 +00:00
77dcd57b46 Add EventGrid compatible webhook format (#1640) 2022-02-11 16:39:19 -08:00
83e48e7e7b Adding new InstanceConfig value for VMSS & VM tags (#1560)
* Adding new instanceconfig value for tags.

* Removing bad import.

* Updating where tags are generated.

* Updating tag generation for scalesets.

* Updating tag generation in vm.

* Updating vm tag generation.

* Updating vm tag generation.

* Fixing extension.

* Fixing import.

* Fixing typing.

* Fixing get_vm calls.

* Fixing calls to get_vm.

* Fixing optional tag.
2022-01-05 13:16:03 -08:00
aa74550160 Group membership check (#1074) 2021-11-22 14:06:03 -08:00
cbe6ef8e40 Update NSGs after changes to instance config (#1385)
* Refactor set_admins into configuration.py and update deployment params with nsg_config

* Fixing arguments.

* Param takes in network config json

* Fixing Client in deploy

* removing import

* Adding onefuzztypes to reqs.txt

* Reverting to single list

* Removing imports.

* Retriggering build

* Setting specific pip version for local testing.

* Removing imports?

* More imports.

* Fixing formatting.

* Updating how to parse nsg param.

* Removing old logging statements.

* Fixing types.

* REmoving bad log

* Removing local pip version.

* Removing comments

* fixing

* Formatting

* Fixing .split()

* Adding NSG rule checks and type.

* Formatting.

* Formatting.

* Removing imports.

* Fixing formatting.

* Testing formatting.

* Retrigger?

* New InstanceConfigClient class.

* Retrigger.

* Cherry picked commit.

* Reformatting.

* Actually fixing formatting.

* Fixing table_service call.

* Fixing return statement and nsg_rule pass.

* Full config.

* Removing commented out code.

* Fixing logic.

* Adding wildcard check.

* Code for updating NSGs when instance_config updated.

* Updating argument to set_allowed_rules

* Updating model to no longer be optional.

* Fixing args for set_allowed_rules

* trying to fix calls to get_nsg

* Updating calls to nsg lib

* Fixing imports.

* Updating calls to set_allowed and creating constructor for NSGConfig type.

* Removing constructor and manually setting default ip

* Fixing models.

* Hopefully fixing docs.

* Fix set_allowed call

* Adding error handling for update config.

* Changing to error check.

* Fixing error call.

* Fixing imports.

* Adding empty() function on request.

* Removing empty function.

* Fixing files for update.

* Fixing nsg.py.

* Fixing imports.

* removing commented code.

Co-authored-by: nharper285 <nharper285@gmail.com>

Update configuration.py to check for 'block all' configuration. (#1394)

* Creating InstanceConfig Attributes for NSG Refactor (#1331)

* Updating instance_config

* Updating attribute names.

* Updating list factory.

* Updating config attributes.

Co-authored-by: nharper285 <nharper285@gmail.com>

* NSG deployment on a creation of new debug/repro proxy. (#1340)

Co-authored-by: stas <statis@microsoft.com>

* Build fix (#1374)

* Bump reqwest from 0.11.4 to 0.11.5 in /src/proxy-manager (#1336)

Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.4 to 0.11.5.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.4...v0.11.5)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump reqwest from 0.11.4 to 0.11.5 in /src/agent (#1335)

Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.4 to 0.11.5.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.4...v0.11.5)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Work around for newly-upgraded pip breaking pip-licenses (#1346)

* Work around for newly upgrdaded pip breaking pip-licenses  (can be reverted once https://github.com/raimon49/pip-licenses/issues/113 is fixed)

* Update .github/workflows/ci.yml

Co-authored-by: Joe Ranweiler <joe@lemma.co>

Co-authored-by: stas <statis@microsoft.com>
Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Bump iced-x86 from 1.14.0 to 1.15.0 in /src/agent (#1337)

Bumps [iced-x86](https://github.com/icedland/iced) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/icedland/iced/releases)
- [Commits](https://github.com/icedland/iced/compare/v1.14.0...v1.15.0)

---
updated-dependencies:
- dependency-name: iced-x86
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* revert pip breaking pip-licenses workaround (#1348)

Co-authored-by: stas <statis@microsoft.com>

* Bump thiserror from 1.0.29 to 1.0.30 in /src/proxy-manager (#1341)

Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.29 to 1.0.30.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.29...1.0.30)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump thiserror from 1.0.29 to 1.0.30 in /src/agent (#1342)

Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.29 to 1.0.30.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.29...1.0.30)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump strum from 0.21.0 to 0.22.0 in /src/agent (#1343)

Bumps [strum](https://github.com/Peternator7/strum) from 0.21.0 to 0.22.0.
- [Release notes](https://github.com/Peternator7/strum/releases)
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Peternator7/strum/commits)

---
updated-dependencies:
- dependency-name: strum
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump azure cli to 2.27.2 (#1355)

* Bump azure cli to 2.27.2

* fixing up add-corpus-storage-account script

Co-authored-by: stas <statis@microsoft.com>

* Bump azure-identity to 1.6.1 (#1356)

Co-authored-by: stas <statis@microsoft.com>

* Bump strum_macros from 0.21.1 to 0.22.0 in /src/agent (#1344)

Bumps [strum_macros](https://github.com/Peternator7/strum) from 0.21.1 to 0.22.0.
- [Release notes](https://github.com/Peternator7/strum/releases)
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Peternator7/strum/commits)

---
updated-dependencies:
- dependency-name: strum_macros
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marc Greisen <marc@greisen.org>

* Bump sysinfo from 0.20.4 to 0.20.5 in /src/agent (#1353)

Bumps [sysinfo](https://github.com/GuillaumeGomez/sysinfo) from 0.20.4 to 0.20.5.
- [Release notes](https://github.com/GuillaumeGomez/sysinfo/releases)
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GuillaumeGomez/sysinfo/commits)

---
updated-dependencies:
- dependency-name: sysinfo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Release 3.2.0 (#1361)

* Release 3.2.0

* Added python dependencies

* Update CHANGELOG.md

Co-authored-by: Cheick Keita <kcheick@gmail.com>

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Update grammar

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

Co-authored-by: Cheick Keita <kcheick@gmail.com>
Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Temporarily ignore non-actionable `cargo audit` errors (#1365)

* Azure DevOps notifications not appearing (#1370)

Co-authored-by: stas <statis@microsoft.com>

* Bump procfs from 0.10.1 to 0.11.0 in /src/agent (#1360)

Bumps [procfs](https://github.com/eminence/procfs) from 0.10.1 to 0.11.0.
- [Release notes](https://github.com/eminence/procfs/releases)
- [Commits](https://github.com/eminence/procfs/compare/v0.10.1...v0.11.0)

---
updated-dependencies:
- dependency-name: procfs
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marc Greisen <marc@greisen.org>

* Bump structopt from 0.3.23 to 0.3.25 in /src/agent (#1364)

Bumps [structopt](https://github.com/TeXitoi/structopt) from 0.3.23 to 0.3.25.
- [Release notes](https://github.com/TeXitoi/structopt/releases)
- [Changelog](https://github.com/TeXitoi/structopt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TeXitoi/structopt/compare/v0.3.23...v0.3.25)

---
updated-dependencies:
- dependency-name: structopt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump reqwest from 0.11.5 to 0.11.6 in /src/proxy-manager (#1367)

Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.5...v0.11.6)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: stas <statis@microsoft.com>
Co-authored-by: Joe Ranweiler <joe@lemma.co>
Co-authored-by: Marc Greisen <marc@greisen.org>
Co-authored-by: Cheick Keita <kcheick@gmail.com>
Co-authored-by: Joe Ranweiler <joranwei@microsoft.com>

* Delete NSG if no resources associated with it (#1358)

Co-authored-by: stas <statis@microsoft.com>

* Update NSGs after changes to instance config (#1385)

* Refactor set_admins into configuration.py and update deployment params with nsg_config

* Fixing arguments.

* Param takes in network config json

* Fixing Client in deploy

* removing import

* Adding onefuzztypes to reqs.txt

* Reverting to single list

* Removing imports.

* Retriggering build

* Setting specific pip version for local testing.

* Removing imports?

* More imports.

* Fixing formatting.

* Updating how to parse nsg param.

* Removing old logging statements.

* Fixing types.

* REmoving bad log

* Removing local pip version.

* Removing comments

* fixing

* Formatting

* Fixing .split()

* Adding NSG rule checks and type.

* Formatting.

* Formatting.

* Removing imports.

* Fixing formatting.

* Testing formatting.

* Retrigger?

* New InstanceConfigClient class.

* Retrigger.

* Cherry picked commit.

* Reformatting.

* Actually fixing formatting.

* Fixing table_service call.

* Fixing return statement and nsg_rule pass.

* Full config.

* Removing commented out code.

* Fixing logic.

* Adding wildcard check.

* Code for updating NSGs when instance_config updated.

* Updating argument to set_allowed_rules

* Updating model to no longer be optional.

* Fixing args for set_allowed_rules

* trying to fix calls to get_nsg

* Updating calls to nsg lib

* Fixing imports.

* Updating calls to set_allowed and creating constructor for NSGConfig type.

* Removing constructor and manually setting default ip

* Fixing models.

* Hopefully fixing docs.

* Fix set_allowed call

* Adding error handling for update config.

* Changing to error check.

* Fixing error call.

* Fixing imports.

* Adding empty() function on request.

* Removing empty function.

* Fixing files for update.

* Fixing nsg.py.

* Fixing imports.

* removing commented code.

Co-authored-by: nharper285 <nharper285@gmail.com>

* Aligning feature branch with main (#1389)

* Bump reqwest from 0.11.4 to 0.11.5 in /src/proxy-manager (#1336)

Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.4 to 0.11.5.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.4...v0.11.5)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump reqwest from 0.11.4 to 0.11.5 in /src/agent (#1335)

Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.4 to 0.11.5.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.4...v0.11.5)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Work around for newly-upgraded pip breaking pip-licenses (#1346)

* Work around for newly upgrdaded pip breaking pip-licenses  (can be reverted once https://github.com/raimon49/pip-licenses/issues/113 is fixed)

* Update .github/workflows/ci.yml

Co-authored-by: Joe Ranweiler <joe@lemma.co>

Co-authored-by: stas <statis@microsoft.com>
Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Bump iced-x86 from 1.14.0 to 1.15.0 in /src/agent (#1337)

Bumps [iced-x86](https://github.com/icedland/iced) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/icedland/iced/releases)
- [Commits](https://github.com/icedland/iced/compare/v1.14.0...v1.15.0)

---
updated-dependencies:
- dependency-name: iced-x86
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* revert pip breaking pip-licenses workaround (#1348)

Co-authored-by: stas <statis@microsoft.com>

* Bump thiserror from 1.0.29 to 1.0.30 in /src/proxy-manager (#1341)

Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.29 to 1.0.30.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.29...1.0.30)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump thiserror from 1.0.29 to 1.0.30 in /src/agent (#1342)

Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.29 to 1.0.30.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](https://github.com/dtolnay/thiserror/compare/1.0.29...1.0.30)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump strum from 0.21.0 to 0.22.0 in /src/agent (#1343)

Bumps [strum](https://github.com/Peternator7/strum) from 0.21.0 to 0.22.0.
- [Release notes](https://github.com/Peternator7/strum/releases)
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Peternator7/strum/commits)

---
updated-dependencies:
- dependency-name: strum
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump azure cli to 2.27.2 (#1355)

* Bump azure cli to 2.27.2

* fixing up add-corpus-storage-account script

Co-authored-by: stas <statis@microsoft.com>

* Bump azure-identity to 1.6.1 (#1356)

Co-authored-by: stas <statis@microsoft.com>

* Bump strum_macros from 0.21.1 to 0.22.0 in /src/agent (#1344)

Bumps [strum_macros](https://github.com/Peternator7/strum) from 0.21.1 to 0.22.0.
- [Release notes](https://github.com/Peternator7/strum/releases)
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Peternator7/strum/commits)

---
updated-dependencies:
- dependency-name: strum_macros
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marc Greisen <marc@greisen.org>

* Bump sysinfo from 0.20.4 to 0.20.5 in /src/agent (#1353)

Bumps [sysinfo](https://github.com/GuillaumeGomez/sysinfo) from 0.20.4 to 0.20.5.
- [Release notes](https://github.com/GuillaumeGomez/sysinfo/releases)
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GuillaumeGomez/sysinfo/commits)

---
updated-dependencies:
- dependency-name: sysinfo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Release 3.2.0 (#1361)

* Release 3.2.0

* Added python dependencies

* Update CHANGELOG.md

Co-authored-by: Cheick Keita <kcheick@gmail.com>

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Update grammar

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

Co-authored-by: Cheick Keita <kcheick@gmail.com>
Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Temporarily ignore non-actionable `cargo audit` errors (#1365)

* Azure DevOps notifications not appearing (#1370)

Co-authored-by: stas <statis@microsoft.com>

* Bump procfs from 0.10.1 to 0.11.0 in /src/agent (#1360)

Bumps [procfs](https://github.com/eminence/procfs) from 0.10.1 to 0.11.0.
- [Release notes](https://github.com/eminence/procfs/releases)
- [Commits](https://github.com/eminence/procfs/compare/v0.10.1...v0.11.0)

---
updated-dependencies:
- dependency-name: procfs
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marc Greisen <marc@greisen.org>

* Bump structopt from 0.3.23 to 0.3.25 in /src/agent (#1364)

Bumps [structopt](https://github.com/TeXitoi/structopt) from 0.3.23 to 0.3.25.
- [Release notes](https://github.com/TeXitoi/structopt/releases)
- [Changelog](https://github.com/TeXitoi/structopt/blob/master/CHANGELOG.md)
- [Commits](https://github.com/TeXitoi/structopt/compare/v0.3.23...v0.3.25)

---
updated-dependencies:
- dependency-name: structopt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump reqwest from 0.11.5 to 0.11.6 in /src/proxy-manager (#1367)

Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.5...v0.11.6)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump reqwest from 0.11.5 to 0.11.6 in /src/agent (#1368)

Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](https://github.com/seanmonstar/reqwest/compare/v0.11.5...v0.11.6)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump crossterm from 0.21.0 to 0.22.1 in /src/agent (#1369)

Bumps [crossterm](https://github.com/crossterm-rs/crossterm) from 0.21.0 to 0.22.1.
- [Release notes](https://github.com/crossterm-rs/crossterm/releases)
- [Changelog](https://github.com/crossterm-rs/crossterm/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crossterm-rs/crossterm/commits)

---
updated-dependencies:
- dependency-name: crossterm
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marc Greisen <marc@greisen.org>

* Fix validation of `target_exe` blob name (#1371)

* NSG Updated After CLI Update to Instance_Config (#1375)

* Creating InstanceConfig Attributes for NSG Refactor (#1331)

* Updating instance_config

* Updating attribute names.

* Updating list factory.

* Updating config attributes.

Co-authored-by: nharper285 <nharper285@gmail.com>

* NSG deployment on a creation of new debug/repro proxy. (#1340)

Co-authored-by: stas <statis@microsoft.com>

* Code for updating NSGs when instance_config updated.

* Updating argument to set_allowed_rules

* Temporarily ignore non-actionable `cargo audit` errors (#1365)

* Updating model to no longer be optional.

* Fixing args for set_allowed_rules

* trying to fix calls to get_nsg

* Updating calls to nsg lib

* Fixing imports.

* Updating calls to set_allowed and creating constructor for NSGConfig type.

* Removing constructor and manually setting default ip

* Fixing models.

* Hopefully fixing docs.

* Fix set_allowed call

* Adding error handling for update config.

* Changing to error check.

* Fixing error call.

* Fixing imports.

* Updating instanceconfig retrieval.

* Fixing imports.

* Adding empty() function on request.

* Fixing name of function.

* Removing empty function.

Co-authored-by: nharper285 <nharper285@gmail.com>
Co-authored-by: Stas <stishkin@live.com>
Co-authored-by: stas <statis@microsoft.com>
Co-authored-by: Joe Ranweiler <joranwei@microsoft.com>

* Revert "NSG Updated After CLI Update to Instance_Config (#1375)" (#1384)

This reverts commit 357bc4fcad.

* Bump backtrace from 0.3.61 to 0.3.62 in /src/agent (#1382)

Bumps [backtrace](https://github.com/rust-lang/backtrace-rs) from 0.3.61 to 0.3.62.
- [Release notes](https://github.com/rust-lang/backtrace-rs/releases)
- [Commits](https://github.com/rust-lang/backtrace-rs/compare/0.3.61...0.3.62)

---
updated-dependencies:
- dependency-name: backtrace
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marc Greisen <marc@greisen.org>

* Set compiler env vars to effect Win10 SDK downgrade (#1388)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: stas <statis@microsoft.com>
Co-authored-by: Joe Ranweiler <joe@lemma.co>
Co-authored-by: Marc Greisen <marc@greisen.org>
Co-authored-by: Cheick Keita <kcheick@gmail.com>
Co-authored-by: Joe Ranweiler <joranwei@microsoft.com>
Co-authored-by: Noah McGregor Harper <74685766+nharper285@users.noreply.github.com>
Co-authored-by: nharper285 <nharper285@gmail.com>

* Updating configuration.py to check for 'block all' config.

* Fixing error message.

* associate subnets with NSG (#1393)

* associate subnets with NSG

change NSG rule protocol to ANY

* subnet wait

* Improve NSG update logic

Co-authored-by: stas <statis@microsoft.com>

Co-authored-by: nharper285 <nharper285@gmail.com>
Co-authored-by: Stas <stishkin@live.com>
Co-authored-by: stas <statis@microsoft.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Joe Ranweiler <joe@lemma.co>
Co-authored-by: Marc Greisen <marc@greisen.org>
Co-authored-by: Cheick Keita <kcheick@gmail.com>
Co-authored-by: Joe Ranweiler <joranwei@microsoft.com>
2021-11-04 18:59:23 -07:00
9f4db320b4 Creating InstanceConfig Attributes for NSG Refactor (#1331)
* Updating instance_config

* Updating attribute names.

* Updating list factory.

* Updating config attributes.

Co-authored-by: nharper285 <nharper285@gmail.com>
2021-11-04 18:59:23 -07:00
98cd7c9c56 migrate to msgraph (#966)
* migrate to msgraph

* add subscription id to query_microsoft_graph

* migrating remaingin references

* formatting

* adding missing dependencies

* flake fix

* fix get_tenant_id

* cleanup

* formatting

* migrate application creation in deploy.py

* foramt

* mypy fix

* isort

* isort

* format

* bug fixes

* specify the correct signInAudience

* fix backing service principal creation
fix preauthorized application

* remove remaining references to graphrbac

* fix ms graph authentication

* formatting

* fix typo

* format

* deployment fix

* set implicitGrantSettings in the deployment

* format

* fix deployment

* fix graph authentication on the server

* use the current cli logged in account to retrive the backend token cache

* assign the the msgraph app role permissions to the web app during the deployment

* formatting

* fix build

* build fix

* fix bandit issue

* mypy fix

* isort

* deploy fixes

* formatting

* remove assign_app_permissions

* mypy fix

* build fix

* mypy fix

* format

* formatting

* flake fix

* remove webapp identity permission assignment

* remove unused reference to assign_app_role

* remove manual registration message

* fixing name and logging

* address PR coments

* address PR comments

* build fix

* lint

* lint

* mypy fix

* mypy fix

* formatting

* address PR comments

* linting

* lint

* remove ONEFUZZ_AAD_GROUP_ID check

* regenerate webhook_events.md

* change return type of query_microsoft_graph_list

* fix tenant_id

Co-authored-by: Marc Greisen <marc@greisen.org>
Co-authored-by: Stas <stishkin@live.com>
2021-10-22 11:59:05 -07:00
b238bfea03 Revert "NSG Updated After CLI Update to Instance_Config (#1375)" (#1384)
This reverts commit 357bc4fcad.
2021-10-21 12:51:20 -07:00
357bc4fcad NSG Updated After CLI Update to Instance_Config (#1375)
* Creating InstanceConfig Attributes for NSG Refactor (#1331)

* Updating instance_config

* Updating attribute names.

* Updating list factory.

* Updating config attributes.

Co-authored-by: nharper285 <nharper285@gmail.com>

* NSG deployment on a creation of new debug/repro proxy. (#1340)

Co-authored-by: stas <statis@microsoft.com>

* Code for updating NSGs when instance_config updated.

* Updating argument to set_allowed_rules

* Temporarily ignore non-actionable `cargo audit` errors (#1365)

* Updating model to no longer be optional.

* Fixing args for set_allowed_rules

* trying to fix calls to get_nsg

* Updating calls to nsg lib

* Fixing imports.

* Updating calls to set_allowed and creating constructor for NSGConfig type.

* Removing constructor and manually setting default ip

* Fixing models.

* Hopefully fixing docs.

* Fix set_allowed call

* Adding error handling for update config.

* Changing to error check.

* Fixing error call.

* Fixing imports.

* Updating instanceconfig retrieval.

* Fixing imports.

* Adding empty() function on request.

* Fixing name of function.

* Removing empty function.

Co-authored-by: nharper285 <nharper285@gmail.com>
Co-authored-by: Stas <stishkin@live.com>
Co-authored-by: stas <statis@microsoft.com>
Co-authored-by: Joe Ranweiler <joranwei@microsoft.com>
2021-10-21 11:54:09 -07:00
22b2d62e29 enable configurable virtual network ranges (#1268) 2021-09-27 18:01:32 +00:00
599c400fa0 Custom Extension Instance Configuration (#1184) 2021-09-24 12:27:39 -04:00
2a2844ae7a enable configuring proxy VM sku (#1128) 2021-08-23 16:04:59 +00:00
2fcb499888 Merge pull request from GHSA-q5vh-6whw-x745
* verify aad tenants, primarily needed in multi-tenant deployments

* add logging and fix trailing slash for issuer

* handle call_if* not supporting additional argument callbacks

* add logging

* include new datatype in webhook docs

* fix pytypes unit tests

Co-authored-by: Brian Caswell <bmc@shmoo.com>
2021-08-13 14:50:54 -04:00
5ba04d0cd2 use pydantic built-in size validators (#1048) 2021-07-07 17:07:05 -04:00
52f83b5b26 add EventScalesetResizeScheduled (#1047) 2021-07-07 14:15:26 -04:00
a43f693761 Provide minimized stack lines (#993)
Using the function name only method for minimization results in a
function with two different malloc related bugs to get bucketed
together.

This provides a minimized stack without function address of stack depth.

For entries without source information, such as:
`# 34 0xf19113f in ChromeMain+0x13f (C:\\clusterfuzz\\bot\\builds\\chrome-test-builds_media_win32-release\\revisions\\asan-win32-release-335593\\chrome_child.dll+0x113f)`
users will see this:
`ChromeMain+0x13f`

For entries with source information, such as:
`# 20 0x58bd3bb in v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) ../../v8/src/builtins/builtins-api.cc:137:5`
users will see this:
`v8::internal::Builtin_Impl_HandleApiCall(v8::internal::BuiltinArguments, v8::internal::Isolate*) builtins-api.cc:137:5`
2021-07-01 21:14:31 +00:00
29dda54b83 instance wide configuration (#1010)
TODO:
* [x] add setting initial set of admins during deployment
2021-06-30 21:13:58 +00:00
2c72bd590f Add generic coverage task (#763)
**Todo:**
- [x] Finalize format for coverage file(s)
- [x] Add service support
- [x] Integration test
- [x] Merge #926 
- [x] Merge #929
2021-06-03 23:36:00 +00:00
a4bb670fb2 add proxy_state_updated events (#908) 2021-05-21 12:47:54 -04:00
2f81c44f01 Refactoring proxy lifetime to only shutdown when proxy is out-of-date. (#839)
## Summary of the Pull Request

_What is this about?_
We'd like to refactor the proxy lifecycle to only delete when the proxy is out-of-date - i.e. when the proxy is older than 7 days or a mismatched version. I've changed two files, proxy.py and timer_daily\init.py to check for the version and timestamp before stopping a live proxy. 

## PR Checklist
* [ ] Applies to work item: #xxx
* [ ] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/onefuzz) and sign the CLI.
* [ ] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

## Info on Pull Request

_What does this include?_
Changes to two files: 
proxy.py: 
- get_or_create() edited to check if timestamp is >7 days.
- Created is_outdated() to check version and timestamp for out-of-date proxy. 
timer_daily/init.py
- Proxy check now includes is_outdated() before determining if a proxy should be shutdown. 

## Validation Steps Performed
Deploying test instance to determine if proxy lives past a single day.
2021-05-20 14:33:29 +00:00
cb5e786bcd add event for scaleset state updates (#882)
This moves all scaleset state updates through `Scaleset.set_state` and adds a new event EventScalesetStateUpdated.
2021-05-13 21:23:02 +00:00
b599aea5cd allow extra fields in Events (#805) 2021-04-14 13:09:21 -04:00
46b8bdccbc add TaskConfig to crash_reported and regression_reported events (#793)
resolves #757 and #758
2021-04-13 10:24:12 +00:00
3096f99e86 enable using ephemeral disks by default (#461) 2021-03-30 18:48:44 -04:00
e80b658a1a expose new minimized stack fields in the crash report model (#719) 2021-03-23 12:29:08 -04:00
516b1e000e expose minimized_stack_depth functionality in the CLI/API (#715) 2021-03-23 10:09:34 -04:00
6e60a8cf10 add regression testing tasks (#664) 2021-03-18 15:37:19 -04:00
0a3812d8bc Add job stopped task info (#648) 2021-03-09 10:06:06 -05:00
4489036d9f add node & task heartbeat events (#621)
This adds node & task heartbeats and makes the event data available as a structured data in the logs.
2021-03-02 22:04:39 +00:00
305c23a4d9 add instance information to webhooks (#577)
Fixes #574
2021-02-19 21:00:51 +00:00
4992b494f1 add task config to all task events (#580) 2021-02-19 14:10:48 -05:00
bdcab6eb08 handle tokens from x-ms-token-aad-id-token (#531) 2021-02-10 12:41:15 -05:00
a46f7b4193 expose supervisor tasks that are fully self-contained fuzzing tasks in the service (#474)
Exposes the functionality added in #454 to the service & CLI.

Fixes #439
2021-01-29 00:01:59 +00:00
165257e989 update python prereqs (#427)
Updates the following libraries in the service:
* azure-core
* azure-functions
* azure-identity
* azure-keyvault-keys
* azure-keyvault-secrets
* azure-mgmt-compute
* azure-mgmt-core
* azure-mgmt-loganalytics
* azure-mgmt-network
* azure-mgmt-resource
* azure-mgmt-storage
* azure-mgmt-subscription
* azure-storage-blob
* azure-storage-queue
* pydantic
* requests
* jsonpatch

Removes the following libraries in the service:
* azure-cli-core
* azure-cli-nspkg
* azure-mgmt-cosmosdb
* azure-servicebus

Updates the following libraries in the CLI:
* requests
* semver
* asciimatics
* pydantic
* tenacity

Updates the following libraries in onefuzztypes:
* pydantic

The primary "legacy" libraries are [azure-graphrbac](https://pypi.org/project/azure-graphrbac/) and azure-cosmosdb-table.  The former has not been updated to use azure-identity yet. The later is being rewritten as [azure-data-tables](https://pypi.org/project/azure-data-tables/), but is still in early beta.
2021-01-25 20:53:40 +00:00
7e6415b15f ensure consistency in doc generation by always sorting json keys (#449) 2021-01-21 15:28:11 +00:00
513d1f52c9 Unify Dashboard & Webhook events (#394)
This change unifies the previously adhoc SignalR events and Webhooks into a single event format.
2021-01-11 21:43:09 +00:00
c1a50f6f6c Colocate tasks (#402)
Enables co-locating multiple tasks in a given work-set.

Tasks are bucketed by the following:
* OS
* job id
* setup container
* VM SKU & image (used in pre-1.0 style tasks)
* pool name (used in 1.0+ style tasks)
* if the task needs rebooting after the task setup script executes.

Additionally, a task will end up in a unique bucket if any of the following are true:
* The task is set to run on more than one VM
* The task is missing the `task.config.colocate` flag (all tasks created prior to this functionality) or the value is False

This updates the libfuzzer template to make use of colocation.  Users can specify co-locating all of the tasks *or* co-locating the secondary tasks.
2021-01-06 13:49:15 +00:00
37f06bb324 handle libfuzzer fuzzing non-zero exits better (#381)
When running libfuzzer in 'fuzzing' mode, we expect the following on exit.

If the exit code is zero, crashing input isn't required.  This happens if the user specifies '-runs=N'

If the exit code is non-zero, then crashes are expected.  In practice, there are two causes to non-zero exits.
1. If the binary can't execute for some reason, like a missing prerequisite
2. If the binary _can_ execute, sometimes the sanitizers are put in such a bad place that they are unable to record the input that caused the crash.

This PR enables handling these two non-zero exit cases.

1. Optionally verify the libfuzzer target loads appropriately using `target_exe -help=1`.  This allows failing faster in the common issues, such a missing prerequisite library.
2. Optionally allow non-zero exits without crashes to be a warning, rather than a task failure.
2021-01-05 14:40:15 +00:00