1894 Commits

Author SHA1 Message Date
George Pollard
d17d50d536
Remove reportgenerator (#2666) 2022-12-02 09:19:03 -05:00
George Pollard
4aec5ed5c5
CLI: Retry on connection errors while acquiring token (#2668)
While running tests we sometimes see token acquisition fail due to "Connection reset by peer". This should be retried by default.
2022-12-02 16:30:16 +13:00
George Pollard
be066bd368
Check extensions status before transitioning to Running (#2667)
This is a regression from Python to C# version: the C# code was not checking the result of the AddExtensions function, so would transition Repro VMs or Proxy VMs into Running state potentially before they are ready to be used.

This could cause test failures or confusing errors when running `create_and_connect`.
2022-12-02 00:40:27 +00:00
Cheick Keita
62647e0c6d
pass the machine_identity from the agent to the task (#2662)
* pass the machine_identity from the agent to the task

* clippy fix

* removing default from MachineIdenitty

* fix tests

* more tests fixes

* format

* clippy fix

* another fix

* fix test

* Update src/agent/onefuzz-task/src/local/common.rs
2022-12-01 23:17:52 +00:00
George Pollard
38dfa668bc
Standardize HTTP error results, better rejection message when parsing validated strings (#2663)
1. When parsing a `ValidatedString` from JSON and it fails, include a message about the expected format of the string.
   - Reworked the classes using C#11 features to reduce the amount of boilerplate needed to add a new validated string type.

2. Change to use [RFC7807](https://www.rfc-editor.org/rfc/rfc7807) format for HTTP error responses. At the moment we returned the `Error` type which was undocumented.
3. Update CLI to parse RFC7807 responses.

Old error looked like:

```console
$ onefuzz containers create AbCd
ERROR:cli:command failed: request did not succeed: HTTP 500 -
```

New error looks like:

```console
$ onefuzz containers create AbCd
ERROR:cli:command failed: request did not succeed (400: INVALID_REQUEST): Unable
to parse 'AbCd' as a Container: Container name must be 3-63 lowercase letters, numbers,
or non-consecutive hyphens (see: https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/resource-name-rules#microsoftstorage)
```

Closes #2661.
2022-12-02 10:33:14 +13:00
George Pollard
88e8c11a02
LLVM_SYMBOLIZER_PATH must be absolute (#2664) 2022-12-01 14:15:00 +13:00
George Pollard
510752094a
Enable MSVC security features (#2575)
Enabling some native code features which are SDL requirements. (Specifically, the requirement "Executable binary files must be hardened to leverage available platform security mitigations".)

I attempted to use BinSkim to enforce the requirements but it has trouble with Rust binaries at present.
2022-11-30 19:58:13 +00:00
Cheick Keita
4cfc831635
Include a reason when telling a node it is not allowed to schedule work (#2643)
* include a reason when telling a node it is not allowed to schedule work

* address pr comment

* build fix

* Update src/deny.toml
2022-11-30 18:15:28 +00:00
George Pollard
c4f1adeff6
Fixing .NET crash report no-repro (#2642)
The minidump file being generated did not have enough information for SOS to correctly locate the exception object. Switch to “full” minidumps instead.

- Fix a small bug with coverage reporting task ID as job ID.

- I had thought we needed .NET 6 installed as well to run SharpFuzz, but now I'm not so sure. In any case, installing both versions will not hurt (and may help users).
2022-11-28 20:54:41 +00:00
George Pollard
56bde9fbb0
Add a test for parse_sos_print_exception_output (#2648)
A very simple test, mostly to show the expected input format.
2022-11-28 18:52:35 +00:00
Teo Voinea
95abeeacef
Improve logging around notification failures (#2653)
Co-authored-by: Teo Voinea <Teodor.Voinea@microsoft.com>
2022-11-25 09:04:42 -05:00
Noah McGregor Harper
e0634a3365
Deploy update (#2650)
* Remove Python App Function Deployment Code and  Code.

* Updating yml and zip names.

* Fixing ci.yml.

* Typo.

* Format

* Trying to remove python service.

* Updating directories.

* Removing flag.

* Format.

* Fixng api-service-net ref.

* Re-add requirement.

* Fixing refs in bicep.

* Specifying version.

* Removing dotnet refs in integration tests.

* Updating role assignment naming convention.

* Adding ignore.

* Update src/deny.toml

Co-authored-by: George Pollard <porges@porg.es>

* Update version.

* Removing onefuzztypes dependency.

* Switch app service plan to windows.

* Update test version.

* Changing version.

* Returning version.

* Trying to add onefuzz types back.

* Force pipenv version.

* Fix.

* Syntax.

* Renaming.

* Trying different version.

* Removing build step.

* Fixing bicep parameter.

* Retrying run with older version.

* Trying pipenv with another version.

* Forcing pipenv version in tox.

* Adding pipenv fix and updating version.

Co-authored-by: George Pollard <porges@porg.es>
2022-11-23 17:48:30 -08:00
Cheick Keita
6c981f613d
Making machine identity a parameter of the agent config (#2649)
* making machine identity a parameter

* build fix

* build fix

* format
2022-11-23 16:32:19 -08:00
Cheick Keita
ae827b2dda
Fix invocation of functionapp in the deployment script (#2645) 2022-11-21 21:13:53 -08:00
Justin Moore
abb7b4781e
Add gitignore for C# artifacts (#2640) 2022-11-21 21:35:43 +00:00
George Pollard
06849b29f0
Bump azure_* crates (#2637) 2022-11-21 20:38:44 +00:00
George Pollard
04d39a3f28
Setup cargo-deny (#2638)
Using [`cargo-deny`](https://embarkstudios.github.io/cargo-deny/) to ensure that disallowed dependencies removed in #2423 do not accidentally make their way back in.

`cargo-deny` subsumes the `cargo-audit` functionality, so switch to the `cargo-deny` version.

Setting this up required explicitly stating the license which was not in some of our `Cargo.toml` files.
2022-11-21 08:23:20 -05:00
George Pollard
894dcc62be
Update to .NET 7 (#2615)
Update service code to .NET 7.0.

Also update the version installed onto agent nodes via setup scripts to 7.0.100.
2022-11-18 10:39:45 +13:00
Cheick Keita
1f46388e6d
Bug fixes related to the unmanaged nodes (#2632)
* bug fixes related to the unmanaged nodes
- fix the token request in the client
- adding a is_unmnaged field to the agentConfig
- fix typo in the appId claim type
- fix typo in the UnmanagedNode claim value
- fix query PoolOperation.GetByClientId

* remove unused import

* build fix

* change unmanaged  field to managed

* Apply suggestions from code review

Co-authored-by: Teo Voinea <58236992+tevoinea@users.noreply.github.com>

Co-authored-by: Teo Voinea <58236992+tevoinea@users.noreply.github.com>
2022-11-17 20:51:52 +00:00
George Pollard
5bc3dac1ae
Fix clippy failure (#2634) 2022-11-17 20:38:14 +00:00
Adam
21205198c9
Update OMI to 1.6.10.2 on Ubuntu VMs (#2629)
* updating OMI version
2022-11-16 15:37:31 -08:00
Kanan B
40c85b8c16
Make the --container_type parameter required (#2631)
* Make --container_type a required parameter (#2574)

* Fix targetUrl so it no longer contains full exe path

* Make --container_type a required parameter (#2574)
2022-11-16 11:38:42 -08:00
George Pollard
07befc58b2
Fix SignalR client code not reading responses correctly (#2626)
The `Response` object must have the JSON loaded via `json()` (`backend.request` was updated but this callsite was missed).

Also, import the `Onefuzz` type so that this is type-checked to avoid the same problem in future.
2022-11-15 02:16:06 +00:00
Kanan B
b41e61a784
Fix targetUrl so it no longer contains full exe path (#2625) 2022-11-14 14:13:05 -08:00
George Pollard
c5840eb69b
Use broker or browser login instead of device flow (#2612)
Update CLI to attempt broker or browser-based authentication first; if you `Ctrl-C` to cancel it, you can fall back to device code login.

Also updated the MSAL dependency to latest version and pass `allow_broker=True` which will allow the use of Web Account Manager (WAM), if it is available.

Using browser auth requires the `http://localhost` redirect URI, and using the broker requires a special custom URI including the app ID (see code).
2022-11-14 20:13:36 +00:00
George Pollard
0caac2fc00
Only add "re-opened" comment if bug was reopened (#2623) 2022-11-15 08:51:59 +13:00
George Pollard
b4750ae52c
Make Proxy TelemetryKey optional (#2619) 2022-11-14 03:18:36 +00:00
George Pollard
0b19af3381
Switch to quick-xml (#2611) 2022-11-14 12:45:05 +13:00
Cheick Keita
f5851d0764
Update the Authorization layer to allow unmanaged nodes (#2584)
* create the config_path folder if it does not exist

* Update the Authorization layer to allow unmanaged nodes

* check the role assignment

* fix merge

* build fix

* fix typo

* formatting

* formatting
2022-11-10 17:58:20 +00:00
Cheick Keita
ff85b808e9
Adding an optional retry_limit to onefuzz debug connect (#2609)
* adding an optional retry_limit to onefuzz debug connect

* make none the default retry limit

* fix logic

* fix default
2022-11-10 01:51:35 +00:00
George Pollard
c50a9daf20
Bump cpufeatures to 0.2.5 (#2608) 2022-11-09 20:33:20 +00:00
Noah McGregor Harper
09e564f6fd
Catching Model Exceptions During VMSS Update (#2607)
* Catching Update VMSS Exceptions.

* Adding additional exception handling.

* Updating wording.

* Adding handler for model error when catching scale protection exceptions.

* Fixing error messaging.
2022-11-08 15:20:49 -08:00
Noah McGregor Harper
f20c482052
Adding Additional Information to Decommission Functionality. (#2605)
* Adding Additional Information to Decommission Function.

* Separating instance and machine ids
2022-11-08 10:04:18 -08:00
George Pollard
2a7a072333
Fixing status top bug (#2604) 2022-11-08 01:03:55 +00:00
Cheick Keita
7e9ff9cd11
Add endpoint to download the agent binaries (#2600)
* Add endpoint to download the agent binaries

* build fix

* fix types

* adding test

* format

* remove comments

* address pr comment

* build fix
2022-11-07 15:28:22 -08:00
Cheick Keita
fb8a4e36f1
Add a registration command to allow an application registration to connect to the Onefuzz (#2571) 2022-11-03 16:41:38 +00:00
George Pollard
6cc7caf67b
Handle duplicate containers across accounts (#2596) 2022-11-03 01:21:35 +00:00
George Pollard
f1f6400dad
Improve AppInsights setup (#2597)
Switch to using the package provided by Azure Functions to set up Application Insights.

Using the provided setup method `AddApplicationInsightsTelemetry` will configure the dependency collector by default as well as set up some other things that we haven’t done manually, including application version, which would be very useful.

This also means that `operation_Id` is populated more consistently which permits joining `traces` or `dependencies` on the `requests` table.

This also means that the end-to-end data in AI is now correct, so the chart works:
![MicrosoftTeams-image](https://user-images.githubusercontent.com/12575/199629195-f442c9b4-2bb3-4f78-a532-3936cf6bf74f.png)
2022-11-03 01:09:51 +00:00
Cheick Keita
22cb201805
Fix the notification get request (#2591)
* Fix the notification get request

* remove unused import
2022-11-02 17:56:50 -07:00
Justin Moore
d655fcd5c4
Optimize the output side of srcview (#2515) 2022-11-01 17:59:37 -07:00
Teo Voinea
33e2d5c954
Delete nodes on decommission instead of just releasing scale in protection (#2586)
Co-authored-by: Teo Voinea <Teodor.Voinea@microsoft.com>
2022-11-01 08:48:42 -04:00
Stas
8be6e3c946
do not break the for loop, and cache all instance ids (#2548)
Co-authored-by: stas <statis@microsoft.com>
2022-10-29 11:43:57 -07:00
Cheick Keita
d7d55c77fc
create the config_path folder if it does not exist (#2583) 2022-10-28 17:18:49 -07:00
George Pollard
36265895ee
Reduce fetches to VMSS (#2577)
* Remove one GET from ListInstanceIds

* Only invoke ListInstanceIds if needed

* Format code

* Downgrade Exception log to Verbose

* Insert InstanceID during CleanupNodes

Co-authored-by: Cheick Keita <chkeita@microsoft.com>
2022-10-28 19:45:27 +00:00
Noah McGregor Harper
c277cd1ec3
Update References to Ubuntu 18.04 to 20.04 (#2535)
* Update References to 18.04 to 20.04

* Fixing Ubuntu Image.

* Fix image.
2022-10-28 10:17:57 -07:00
George Pollard
fc2f29dc6f
Reduce number of ARM calls in ListVmss (#2539)
1. `ListVmss` was invoking `GetVirtualMachineScaleSetAsync` and then `GetVirtualMachineScaleSetVms`; instead we can get the VMSS resource directly and invoke `GetVirtualMachineScaleSetVms` on it.

2. I also removed some unneeded invocations of `AsAsyncEnumerable` which might be turning async enumerables into sync enumerables.

3. Do the same `HasData` check in the instance ID lookup that we were already doing in `ListVmss`.

After this change (#3) we no longer do any `GET`s against individual VMSS VMs during the `check-pr` run: 
![image](https://user-images.githubusercontent.com/12575/198410236-250ed8d7-f7a1-4c2d-8bf4-2ebd1c1a0f7a.png)
2022-10-27 22:40:34 +00:00
George Pollard
228fe99bc1
Update to released version of appinsights-rs (#2573) 2022-10-27 14:01:31 -07:00
Cheick Keita
7232981d95
Use records to unpack the request parameters in AgentRegistration (#2570) 2022-10-27 10:26:34 -07:00
Teo Voinea
85437df253
Bump PyGithub (#2569)
Co-authored-by: Teo Voinea <Teodor.Voinea@microsoft.com>
2022-10-27 16:04:52 +00:00
Teo Voinea
76cde40234
Better logging with webhooks (#2568)
Co-authored-by: Teo Voinea <Teodor.Voinea@microsoft.com>
2022-10-27 11:53:22 -04:00