Commit Graph

1767 Commits

Author SHA1 Message Date
3f37fd2b58 Validate notification template (#2655)
* Need to keep working on making dummy values

* Create endpoing to validate scriban templates

* Fix dev experience with app config manager, allow scriban validate endpoint to override jinja converssion

* Use strict validation

Co-authored-by: Teo Voinea <Teodor.Voinea@microsoft.com>
2022-12-08 09:16:03 -05:00
b356c85df5 Update LLVM from v10 to v12 (#2617)
While I was investigating `llvm-symbolizer` issues, I found that we are installing LLVM-10 by default.

Now that we are using the Ubuntu 20.04 image as the default, use the latest LLVM version available for that release. (In addition, v12 is also available on 22.04, whereas v10 is not.)
2022-12-06 01:44:36 +00:00
2685cb6b3f Fix local fuzzing mode (#2669)
* repairing local fuzzing mode

* make clippy happy

* make the ui an option
2022-12-05 13:59:12 -08:00
7091c5460d Except on uppercase application names in deployments (#2665)
* Adding check to deploy.py for uppercase names in application names

Co-authored-by: Cheick Keita <kcheick@gmail.com>
2022-12-05 08:28:25 -08:00
59360103f1 Add support for feature flags (#2620)
* Foundation for feature flags

* Demo usage

* Use managed identity

* Add FFs using DI method

* missed a file

* Cleanup

* Forgot test context

* Default is off if the FF doesn't already exist

* Update src/ApiService/ApiService/Program.cs

Co-authored-by: George Pollard <porges@porg.es>

* force evaluate lock files

Co-authored-by: Teo Voinea <Teodor.Voinea@microsoft.com>
Co-authored-by: George Pollard <porges@porg.es>
2022-12-05 09:07:00 -05:00
af806a3263 Release 6.0.0 (#2657)
* Release 6.0.0

* Adding breakline.

* Respond to comment.

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Additional.

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Adding additional changes.

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* comment.

* Updated change log.

* Last minute changes.

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>
6.0.0
2022-12-02 13:53:02 -08:00
7fc6fc3668 Reverting Enable/Disable App Function Changes - Force -net App Function To Only Run Agent Functions (#2660)
* Testing app function.

* Addding functions back.

* Fixing name.

* updating to string.

* Fixing bicep.

* Fixing function names.

* Update src/ApiService/ApiService/Functions/ReproVmss.cs

Co-authored-by: George Pollard <gpollard@microsoft.com>

* Adding route specifier.

* Update src/ApiService/ApiService/Functions/NodeAddSshKey.cs

Co-authored-by: George Pollard <gpollard@microsoft.com>

* Update src/ApiService/ApiService/Functions/WebhookLogs.cs

Co-authored-by: George Pollard <gpollard@microsoft.com>

* Update src/ApiService/ApiService/Functions/WebhookPing.cs

Co-authored-by: George Pollard <gpollard@microsoft.com>

* Fixing function names.

* Increasing pool_size

* Adding logging statements.

* Adding check for transient failure.

* Adding check to else.

* Fix var

* Adding retry logic to dbg_lnx

* FIxing retry.

* Maybe fixed integration test.

* Formatting.

* Adding try and fail logic.

* Changing error.

* Refactoring logic.

* Fixing logic.

* Logic.

* Removing bad comparison.

* Fixing.

* All ready.

* Switching back to call.

* Add.

* Fixing

* Adding quit back.

* Reversing cli changes.

* Removing bad file.

* Formatting.

Co-authored-by: George Pollard <gpollard@microsoft.com>
2022-12-02 11:14:13 -08:00
d17d50d536 Remove reportgenerator (#2666) 2022-12-02 09:19:03 -05:00
4aec5ed5c5 CLI: Retry on connection errors while acquiring token (#2668)
While running tests we sometimes see token acquisition fail due to "Connection reset by peer". This should be retried by default.
2022-12-02 16:30:16 +13:00
be066bd368 Check extensions status before transitioning to Running (#2667)
This is a regression from Python to C# version: the C# code was not checking the result of the AddExtensions function, so would transition Repro VMs or Proxy VMs into Running state potentially before they are ready to be used.

This could cause test failures or confusing errors when running `create_and_connect`.
2022-12-02 00:40:27 +00:00
62647e0c6d pass the machine_identity from the agent to the task (#2662)
* pass the machine_identity from the agent to the task

* clippy fix

* removing default from MachineIdenitty

* fix tests

* more tests fixes

* format

* clippy fix

* another fix

* fix test

* Update src/agent/onefuzz-task/src/local/common.rs
2022-12-01 23:17:52 +00:00
38dfa668bc Standardize HTTP error results, better rejection message when parsing validated strings (#2663)
1. When parsing a `ValidatedString` from JSON and it fails, include a message about the expected format of the string.
   - Reworked the classes using C#11 features to reduce the amount of boilerplate needed to add a new validated string type.

2. Change to use [RFC7807](https://www.rfc-editor.org/rfc/rfc7807) format for HTTP error responses. At the moment we returned the `Error` type which was undocumented.
3. Update CLI to parse RFC7807 responses.

Old error looked like:

```console
$ onefuzz containers create AbCd
ERROR:cli:command failed: request did not succeed: HTTP 500 -
```

New error looks like:

```console
$ onefuzz containers create AbCd
ERROR:cli:command failed: request did not succeed (400: INVALID_REQUEST): Unable
to parse 'AbCd' as a Container: Container name must be 3-63 lowercase letters, numbers,
or non-consecutive hyphens (see: https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/resource-name-rules#microsoftstorage)
```

Closes #2661.
2022-12-02 10:33:14 +13:00
88e8c11a02 LLVM_SYMBOLIZER_PATH must be absolute (#2664) 2022-12-01 14:15:00 +13:00
510752094a Enable MSVC security features (#2575)
Enabling some native code features which are SDL requirements. (Specifically, the requirement "Executable binary files must be hardened to leverage available platform security mitigations".)

I attempted to use BinSkim to enforce the requirements but it has trouble with Rust binaries at present.
2022-11-30 19:58:13 +00:00
4cfc831635 Include a reason when telling a node it is not allowed to schedule work (#2643)
* include a reason when telling a node it is not allowed to schedule work

* address pr comment

* build fix

* Update src/deny.toml
2022-11-30 18:15:28 +00:00
ba9d6acbe2 Unpin pipenv version (#2659)
* updated contrib/deploy-onefuzz-via-azure-devops/tox.ini
* updated .github/workflows/ci.yml
2022-11-29 12:51:46 -08:00
461053cb43 Check Bicep files during CI (#2658) 2022-11-29 20:15:34 +00:00
c4f1adeff6 Fixing .NET crash report no-repro (#2642)
The minidump file being generated did not have enough information for SOS to correctly locate the exception object. Switch to “full” minidumps instead.

- Fix a small bug with coverage reporting task ID as job ID.

- I had thought we needed .NET 6 installed as well to run SharpFuzz, but now I'm not so sure. In any case, installing both versions will not hurt (and may help users).
2022-11-28 20:54:41 +00:00
56bde9fbb0 Add a test for parse_sos_print_exception_output (#2648)
A very simple test, mostly to show the expected input format.
2022-11-28 18:52:35 +00:00
95abeeacef Improve logging around notification failures (#2653)
Co-authored-by: Teo Voinea <Teodor.Voinea@microsoft.com>
2022-11-25 09:04:42 -05:00
e0634a3365 Deploy update (#2650)
* Remove Python App Function Deployment Code and  Code.

* Updating yml and zip names.

* Fixing ci.yml.

* Typo.

* Format

* Trying to remove python service.

* Updating directories.

* Removing flag.

* Format.

* Fixng api-service-net ref.

* Re-add requirement.

* Fixing refs in bicep.

* Specifying version.

* Removing dotnet refs in integration tests.

* Updating role assignment naming convention.

* Adding ignore.

* Update src/deny.toml

Co-authored-by: George Pollard <porges@porg.es>

* Update version.

* Removing onefuzztypes dependency.

* Switch app service plan to windows.

* Update test version.

* Changing version.

* Returning version.

* Trying to add onefuzz types back.

* Force pipenv version.

* Fix.

* Syntax.

* Renaming.

* Trying different version.

* Removing build step.

* Fixing bicep parameter.

* Retrying run with older version.

* Trying pipenv with another version.

* Forcing pipenv version in tox.

* Adding pipenv fix and updating version.

Co-authored-by: George Pollard <porges@porg.es>
2022-11-23 17:48:30 -08:00
6c981f613d Making machine identity a parameter of the agent config (#2649)
* making machine identity a parameter

* build fix

* build fix

* format
2022-11-23 16:32:19 -08:00
ae827b2dda Fix invocation of functionapp in the deployment script (#2645) 2022-11-21 21:13:53 -08:00
abb7b4781e Add gitignore for C# artifacts (#2640) 2022-11-21 21:35:43 +00:00
1755683be4 More default devcontainer settings (#2641) 2022-11-21 21:06:44 +00:00
06849b29f0 Bump azure_* crates (#2637) 2022-11-21 20:38:44 +00:00
04d39a3f28 Setup cargo-deny (#2638)
Using [`cargo-deny`](https://embarkstudios.github.io/cargo-deny/) to ensure that disallowed dependencies removed in #2423 do not accidentally make their way back in.

`cargo-deny` subsumes the `cargo-audit` functionality, so switch to the `cargo-deny` version.

Setting this up required explicitly stating the license which was not in some of our `Cargo.toml` files.
2022-11-21 08:23:20 -05:00
894dcc62be Update to .NET 7 (#2615)
Update service code to .NET 7.0.

Also update the version installed onto agent nodes via setup scripts to 7.0.100.
2022-11-18 10:39:45 +13:00
1f46388e6d Bug fixes related to the unmanaged nodes (#2632)
* bug fixes related to the unmanaged nodes
- fix the token request in the client
- adding a is_unmnaged field to the agentConfig
- fix typo in the appId claim type
- fix typo in the UnmanagedNode claim value
- fix query PoolOperation.GetByClientId

* remove unused import

* build fix

* change unmanaged  field to managed

* Apply suggestions from code review

Co-authored-by: Teo Voinea <58236992+tevoinea@users.noreply.github.com>

Co-authored-by: Teo Voinea <58236992+tevoinea@users.noreply.github.com>
2022-11-17 20:51:52 +00:00
5bc3dac1ae Fix clippy failure (#2634) 2022-11-17 20:38:14 +00:00
21205198c9 Update OMI to 1.6.10.2 on Ubuntu VMs (#2629)
* updating OMI version
2022-11-16 15:37:31 -08:00
40c85b8c16 Make the --container_type parameter required (#2631)
* Make --container_type a required parameter (#2574)

* Fix targetUrl so it no longer contains full exe path

* Make --container_type a required parameter (#2574)
2022-11-16 11:38:42 -08:00
2a3f1e7241 Fix a logic bug in the notification hook (#2627)
* Fix a logic bug in the notification hook

* Format
2022-11-15 08:41:52 -08:00
07befc58b2 Fix SignalR client code not reading responses correctly (#2626)
The `Response` object must have the JSON loaded via `json()` (`backend.request` was updated but this callsite was missed).

Also, import the `Onefuzz` type so that this is type-checked to avoid the same problem in future.
2022-11-15 02:16:06 +00:00
e546f78cce Release 5.20.0 (#2614) 2022-11-15 00:44:01 +00:00
b41e61a784 Fix targetUrl so it no longer contains full exe path (#2625) 2022-11-14 14:13:05 -08:00
c5840eb69b Use broker or browser login instead of device flow (#2612)
Update CLI to attempt broker or browser-based authentication first; if you `Ctrl-C` to cancel it, you can fall back to device code login.

Also updated the MSAL dependency to latest version and pass `allow_broker=True` which will allow the use of Web Account Manager (WAM), if it is available.

Using browser auth requires the `http://localhost` redirect URI, and using the broker requires a special custom URI including the app ID (see code).
2022-11-14 20:13:36 +00:00
0caac2fc00 Only add "re-opened" comment if bug was reopened (#2623) 2022-11-15 08:51:59 +13:00
b4750ae52c Make Proxy TelemetryKey optional (#2619) 2022-11-14 03:18:36 +00:00
0b19af3381 Switch to quick-xml (#2611) 2022-11-14 12:45:05 +13:00
f5851d0764 Update the Authorization layer to allow unmanaged nodes (#2584)
* create the config_path folder if it does not exist

* Update the Authorization layer to allow unmanaged nodes

* check the role assignment

* fix merge

* build fix

* fix typo

* formatting

* formatting
2022-11-10 17:58:20 +00:00
ff85b808e9 Adding an optional retry_limit to onefuzz debug connect (#2609)
* adding an optional retry_limit to onefuzz debug connect

* make none the default retry limit

* fix logic

* fix default
2022-11-10 01:51:35 +00:00
c50a9daf20 Bump cpufeatures to 0.2.5 (#2608) 2022-11-09 20:33:20 +00:00
7b9fbae38d Ensure wheel updated when pipenv check invoked (#2593) 2022-11-09 10:19:36 -08:00
09e564f6fd Catching Model Exceptions During VMSS Update (#2607)
* Catching Update VMSS Exceptions.

* Adding additional exception handling.

* Updating wording.

* Adding handler for model error when catching scale protection exceptions.

* Fixing error messaging.
2022-11-08 15:20:49 -08:00
f20c482052 Adding Additional Information to Decommission Functionality. (#2605)
* Adding Additional Information to Decommission Function.

* Separating instance and machine ids
2022-11-08 10:04:18 -08:00
2a7a072333 Fixing status top bug (#2604) 2022-11-08 01:03:55 +00:00
7e9ff9cd11 Add endpoint to download the agent binaries (#2600)
* Add endpoint to download the agent binaries

* build fix

* fix types

* adding test

* format

* remove comments

* address pr comment

* build fix
2022-11-07 15:28:22 -08:00
f7dd265fdd Release-5.19.0 (#2599)
* Release-5.19.0

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>
2022-11-07 15:09:01 -08:00
1c23b87f83 Release 5.18.0 (#2582)
* Release 5.18.0

* Combining PRs.

* Typo.

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Update CHANGELOG.md

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>

* Updating changelog.

* Adding fix PR.

Co-authored-by: Marc Greisen <mgreisen@microsoft.com>
2022-11-03 10:09:18 -07:00