Commit Graph

2050 Commits

Author SHA1 Message Date
a8d5ab7c5c Add binary coverage merging (#2724) 2023-01-04 10:11:40 -08:00
df40c99cb9 Use a custom Output type when recording coverage (#2723) 2022-12-22 12:01:51 -08:00
0d6a0d7d89 Escape periods when converting globs (#2721) 2022-12-22 10:14:53 -08:00
d51d4e0c7e Ignore benign recv hangup in timer (#2722) 2022-12-22 10:02:48 -08:00
07a0a926c6 Add deserialization compat functions (#2719) 2022-12-21 13:32:41 -08:00
04f83e0b9c Add lines key to v1 source coverage JSON (#2718) 2022-12-21 11:46:09 -08:00
4dbc9c099a Reduce mutation in the agent state machine (#2710)
* reduce unnecessary mutation in the agent

* cleanup and fix test

* cleanup

* reduce log some levels

* desambiguate self call in the coordinator
2022-12-21 19:20:09 +00:00
3a5101be9b Capture target output, allow Loader reuse (#2716) 2022-12-21 09:13:06 -08:00
a1060c119d Use FF to get node disposal strategy (#2713) 2022-12-21 14:46:50 +00:00
f7bd50853f Add key for block coverage, tests (#2715) 2022-12-20 15:01:25 -08:00
1726beda87 Bump rayon from 1.5.1 to 1.6.1 in /src/agent (#2690)
Bumps [rayon](https://github.com/rayon-rs/rayon) from 1.5.1 to 1.6.1.
- [Release notes](https://github.com/rayon-rs/rayon/releases)
- [Changelog](https://github.com/rayon-rs/rayon/blob/master/RELEASES.md)
- [Commits](https://github.com/rayon-rs/rayon/compare/v1.5.1...rayon-core-v1.6.1)

---
updated-dependencies:
- dependency-name: rayon
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-19 19:58:56 +00:00
188a7ca42e Rust 1.66 (#2704)
* Bump to Rust 1.66

* Another fix

* One more

* Lint
2022-12-19 10:04:04 -08:00
aa902d34d0 Add onefuzz-file-format (#2706)
* Add `onefuzz-file-format`

* Add license

* Add newline

* Lint
2022-12-19 09:13:56 +13:00
349604f03a Removing Remaining App Function References from Deployment Code. (#2682)
* Removing Remaining App Function References from Deployment Code.

* Update src/deployment/deploy.py

Co-authored-by: Adam <103067949+AdamL-Microsoft@users.noreply.github.com>

* Consolidating.

* Updating function  handles.

* Removing copy.

* Fixing.

* Formatting.

* Fixing array calls.

* Adding json settings back.

* Formatting.

Co-authored-by: Adam <103067949+AdamL-Microsoft@users.noreply.github.com>
2022-12-15 20:03:42 -08:00
0fb8bc4a86 Bug fixes and documentation (#2694) 2022-12-15 19:39:17 -08:00
ff923d28e7 Record coverage using debuggable-module (#2701) 2022-12-15 15:20:26 -08:00
054910e39b Adding Public Identifier to Events. (#2705)
* Adding Public Identifier to Events.

* Fixing.
2022-12-15 14:35:53 -08:00
abcfa4fc36 Bump tokio from 1.20.0 to 1.23.0 in /src/proxy-manager (#2674)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.20.0 to 1.23.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.20.0...tokio-1.23.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-15 16:36:40 +13:00
e2dc560bcb Add debuggable-module (#2700) 2022-12-14 10:53:54 -08:00
f6a70602de Bump certifi in /contrib/deploy-onefuzz-via-azure-devops (#2683)
Bumps [certifi](https://github.com/certifi/python-certifi) from 2022.5.18.1 to 2022.12.7.
- [Release notes](https://github.com/certifi/python-certifi/releases)
- [Commits](https://github.com/certifi/python-certifi/compare/2022.05.18.1...2022.12.07)

---
updated-dependencies:
- dependency-name: certifi
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-13 15:30:46 -08:00
31929a8850 Fixing build issue related to dotnet version 7.0.101 (#2698) 2022-12-13 21:50:37 +00:00
4c1adb6e96 rename client_id in pool to object_id (#2673)
* rename client_id in pool to object_id

* fix tests

* print out the content body when receiving an error response in the agent

* fix test

* Apply suggestions from code review

* Update src/ApiService/ApiService/Functions/AgentRegistration.cs

* format

* cleanup

* format

* address pr comment
2022-12-13 03:39:49 +00:00
3cf09c6a40 Include dotnet version in info response (#2693)
* Include dotnet info in version response

* Python code requires non-null
2022-12-13 15:13:36 +13:00
4af4f93167 Only warn on cannot delete nodes from scaleset warning (#2691) 2022-12-12 14:57:29 -05:00
fa3e6a4fdd Fix NullRef exception when getting a scaleset that does not exist (#2692)
* Fix NullRef exception when getting a scaleset that does not exist
Updated the ScalesetOperations.GetNodes to return a non nullable dictionary

* build fix
2022-12-12 10:00:44 -08:00
fdd05aeb78 Include missing fields in the report record (#2679)
* Include missing field  in the report

* update tests
2022-12-12 17:00:53 +00:00
a45bda9838 Allow multiple agents to run on the same machine (#2671)
* Allow multiple agents to run on the same machine
- Move the work directory for each agent into a machine specific folder
- added option on the command line to override the machine id and name
- added an option to remove the done lock when starting the agent

* add git_ignore

* fix tests

* clippy+ format

* fix linux build

* fix test

* cleanup

* cleanup

* fix reset_lock parsing

* clippy
2022-12-09 20:31:38 -08:00
fb03bd6408 Release 6.1.0 (#2687) 6.1.0 2022-12-09 13:41:59 -08:00
2a44900c71 Support Contains Words in WIQL (#2686) 2022-12-09 16:15:37 +00:00
f48546f542 Rename coverage to coverage-legacy (#2685) 2022-12-08 16:26:33 -08:00
11923d2b9d Remove unused coverage recorder from input-tester (#2681) 2022-12-08 15:15:10 -08:00
aad29295e1 Repro Create should fail if insert fails, add tests (#2678)
At the moment the result of the insert is ignored.
2022-12-08 21:31:40 +00:00
97910e4793 post JobCreated when job is created (#2677)
* post JobCreated when job is created
2022-12-08 12:22:54 -08:00
3f37fd2b58 Validate notification template (#2655)
* Need to keep working on making dummy values

* Create endpoing to validate scriban templates

* Fix dev experience with app config manager, allow scriban validate endpoint to override jinja converssion

* Use strict validation

Co-authored-by: Teo Voinea <Teodor.Voinea@microsoft.com>
2022-12-08 09:16:03 -05:00
b356c85df5 Update LLVM from v10 to v12 (#2617)
While I was investigating `llvm-symbolizer` issues, I found that we are installing LLVM-10 by default.

Now that we are using the Ubuntu 20.04 image as the default, use the latest LLVM version available for that release. (In addition, v12 is also available on 22.04, whereas v10 is not.)
2022-12-06 01:44:36 +00:00
2685cb6b3f Fix local fuzzing mode (#2669)
* repairing local fuzzing mode

* make clippy happy

* make the ui an option
2022-12-05 13:59:12 -08:00
7091c5460d Except on uppercase application names in deployments (#2665)
* Adding check to deploy.py for uppercase names in application names

Co-authored-by: Cheick Keita <kcheick@gmail.com>
2022-12-05 08:28:25 -08:00
59360103f1 Add support for feature flags (#2620)
* Foundation for feature flags

* Demo usage

* Use managed identity

* Add FFs using DI method

* missed a file

* Cleanup

* Forgot test context

* Default is off if the FF doesn't already exist

* Update src/ApiService/ApiService/Program.cs

Co-authored-by: George Pollard <porges@porg.es>

* force evaluate lock files

Co-authored-by: Teo Voinea <Teodor.Voinea@microsoft.com>
Co-authored-by: George Pollard <porges@porg.es>
2022-12-05 09:07:00 -05:00
af806a3263 Release 6.0.0 (#2657)
* Release 6.0.0

* Adding breakline.

* Respond to comment.

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Additional.

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Adding additional changes.

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* comment.

* Updated change log.

* Last minute changes.

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>
6.0.0
2022-12-02 13:53:02 -08:00
7fc6fc3668 Reverting Enable/Disable App Function Changes - Force -net App Function To Only Run Agent Functions (#2660)
* Testing app function.

* Addding functions back.

* Fixing name.

* updating to string.

* Fixing bicep.

* Fixing function names.

* Update src/ApiService/ApiService/Functions/ReproVmss.cs

Co-authored-by: George Pollard <gpollard@microsoft.com>

* Adding route specifier.

* Update src/ApiService/ApiService/Functions/NodeAddSshKey.cs

Co-authored-by: George Pollard <gpollard@microsoft.com>

* Update src/ApiService/ApiService/Functions/WebhookLogs.cs

Co-authored-by: George Pollard <gpollard@microsoft.com>

* Update src/ApiService/ApiService/Functions/WebhookPing.cs

Co-authored-by: George Pollard <gpollard@microsoft.com>

* Fixing function names.

* Increasing pool_size

* Adding logging statements.

* Adding check for transient failure.

* Adding check to else.

* Fix var

* Adding retry logic to dbg_lnx

* FIxing retry.

* Maybe fixed integration test.

* Formatting.

* Adding try and fail logic.

* Changing error.

* Refactoring logic.

* Fixing logic.

* Logic.

* Removing bad comparison.

* Fixing.

* All ready.

* Switching back to call.

* Add.

* Fixing

* Adding quit back.

* Reversing cli changes.

* Removing bad file.

* Formatting.

Co-authored-by: George Pollard <gpollard@microsoft.com>
2022-12-02 11:14:13 -08:00
d17d50d536 Remove reportgenerator (#2666) 2022-12-02 09:19:03 -05:00
4aec5ed5c5 CLI: Retry on connection errors while acquiring token (#2668)
While running tests we sometimes see token acquisition fail due to "Connection reset by peer". This should be retried by default.
2022-12-02 16:30:16 +13:00
be066bd368 Check extensions status before transitioning to Running (#2667)
This is a regression from Python to C# version: the C# code was not checking the result of the AddExtensions function, so would transition Repro VMs or Proxy VMs into Running state potentially before they are ready to be used.

This could cause test failures or confusing errors when running `create_and_connect`.
2022-12-02 00:40:27 +00:00
62647e0c6d pass the machine_identity from the agent to the task (#2662)
* pass the machine_identity from the agent to the task

* clippy fix

* removing default from MachineIdenitty

* fix tests

* more tests fixes

* format

* clippy fix

* another fix

* fix test

* Update src/agent/onefuzz-task/src/local/common.rs
2022-12-01 23:17:52 +00:00
38dfa668bc Standardize HTTP error results, better rejection message when parsing validated strings (#2663)
1. When parsing a `ValidatedString` from JSON and it fails, include a message about the expected format of the string.
   - Reworked the classes using C#11 features to reduce the amount of boilerplate needed to add a new validated string type.

2. Change to use [RFC7807](https://www.rfc-editor.org/rfc/rfc7807) format for HTTP error responses. At the moment we returned the `Error` type which was undocumented.
3. Update CLI to parse RFC7807 responses.

Old error looked like:

```console
$ onefuzz containers create AbCd
ERROR:cli:command failed: request did not succeed: HTTP 500 -
```

New error looks like:

```console
$ onefuzz containers create AbCd
ERROR:cli:command failed: request did not succeed (400: INVALID_REQUEST): Unable
to parse 'AbCd' as a Container: Container name must be 3-63 lowercase letters, numbers,
or non-consecutive hyphens (see: https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/resource-name-rules#microsoftstorage)
```

Closes #2661.
2022-12-02 10:33:14 +13:00
88e8c11a02 LLVM_SYMBOLIZER_PATH must be absolute (#2664) 2022-12-01 14:15:00 +13:00
510752094a Enable MSVC security features (#2575)
Enabling some native code features which are SDL requirements. (Specifically, the requirement "Executable binary files must be hardened to leverage available platform security mitigations".)

I attempted to use BinSkim to enforce the requirements but it has trouble with Rust binaries at present.
2022-11-30 19:58:13 +00:00
4cfc831635 Include a reason when telling a node it is not allowed to schedule work (#2643)
* include a reason when telling a node it is not allowed to schedule work

* address pr comment

* build fix

* Update src/deny.toml
2022-11-30 18:15:28 +00:00
ba9d6acbe2 Unpin pipenv version (#2659)
* updated contrib/deploy-onefuzz-via-azure-devops/tox.ini
* updated .github/workflows/ci.yml
2022-11-29 12:51:46 -08:00
461053cb43 Check Bicep files during CI (#2658) 2022-11-29 20:15:34 +00:00