Commit Graph

2050 Commits

Author SHA1 Message Date
edf537137b Add multi-var target spec, improve errors (#2136) 2022-07-07 11:20:45 -07:00
aa2e76e7e9 Fix execution of TimerProxy (#2133)
* Fix execution of timerProxy
- fixed behavior of Container.SaveBlob (file was not being overwritten)
- added support for default value in the entity converter
- fix scaleset size data type
- added initialization of Subnet
- renamed TestHook Info to _Info to prevent clashing of name function mame
- added target framework to ApiServiceProject to help local debugging

* fix unit tests

* fix unit tests

* remove unused property

* fix typo

* removing partial class TimerProxy
2022-07-07 09:25:51 -07:00
0f0fd8ee9c Enable TimerRepro (#2108) 2022-07-07 15:25:20 +00:00
478788f607 Remove stubbed support for macOS builds (#2135)
Closes #2134.
2022-07-06 23:23:47 +00:00
c0895a5b87 Disable macos build for agent (#2134) 2022-07-07 10:44:38 +12:00
f37224e8bb Add dotnet coverage task (#2062)
* checkpoint

* some more progress

* more progress

* More progress

* Now it's time to test it

* It works locally 🎉

* Attempting clean build

* fmt

* temporarily stub out macos

* missed a few

* please be the last one

* .

* .

* .

* noop change to unstuck actions

* .

* .

* Fix setup script

* Some fixes

* It works except for a race condition -- use a directory watcher to fix it

* It works end to end!

* Execute the commands using tokio's structs and timeout mechanism

* It works.... for real this time

* Undo timer changes

* Cleanup

* 🧹

* Fix import

* .

* PR comments

* Fix clippy

* Clippy whyyy

* Only check dotnet path once

* fmt

* Fix a couple more comments
2022-07-06 16:13:45 -04:00
bc48069a94 Import LibFuzzerDotnetLoader, build .NET fuzzing tools (#2126) 2022-07-06 11:51:47 -07:00
728d60588c Use 'os-provided' Python (#2118)
Using `3.8` for the Python devcontainer feature version means it builds from source every time the devcontainer is rebuilt, which is very slow. Using `os-provided` will give us a recent enough version (3.10).

Also install `python-is-python3` so that the bare `python` command points to Python 3 (our scripts seem to expect this).
2022-07-05 21:31:38 +00:00
4d7a01e157 Fixed (#2122) 2022-07-05 20:32:04 +00:00
52b760b89a Porting divergent code introduced in #2116 (#2121)
Make the log sas url last as long as the job duration
2022-07-05 11:35:42 -07:00
ab0b974eb1 Cache radamsa build (#2114) 2022-07-03 21:09:31 +00:00
61fc091f88 Make the log sas url last as long as the job duration (#2116) 2022-07-02 01:40:02 +00:00
262ad689f3 Enable C# functions (#2093)
Enables 2 more C# refactor functions in code. They're ready to be tested in canary.

Improves the precommit script in devcontainer.
2022-07-01 03:57:36 +00:00
a9fe5e6c17 Refactor agent_registration to C# (#2107)
Port `agent_registration` from Python to C#.
2022-07-01 02:38:52 +00:00
2453127298 Bump tokio from 1.18.2 to 1.19.2 in /src/proxy-manager (#2022)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.18.2 to 1.19.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-01 14:00:05 +12:00
308ffa6caa Update devcontainer to 22.04 (#2110) 2022-07-01 00:54:33 +00:00
8d8a8b01d2 remove unused Poolname validation (#2094) 2022-06-30 23:23:53 +00:00
870a51a1e4 Enabling TimerProxy (#2089)
* Testing TimerProzy

* Update src/deployment/deploy.py
2022-06-30 22:47:37 +00:00
a017b10d17 Add correct routes and auth to "agents" Functions (#2109)
The routes and auth were missing from the "agents" functions.
2022-06-30 22:05:07 +00:00
fb0e055ca9 Add azcopy upload support and switch to the default max_concurrency (#1556)
* Add azcopy copy support and switch to the default max_concurrency

* Modify upload_file_data to use upload_file for consistency

* Fix lint issues

* Apply black formatting rules

* Apply isort formatting rules

Co-authored-by: Stas <stishkin@live.com>
2022-06-30 11:30:52 -07:00
ed69cb48b5 Fix Racing condition in monitor.rs (#2105)
* Fix Racing condition in monitor.rs

* address pr comment

* address pr comment

* emmiting a warning instead of stopping the monitoring
2022-06-30 09:29:51 -07:00
ea60410df9 Enable C# functions in code & fill out missing functionality (#2084)
C# HTTP functions won’t take effect by default so it is safe to enable them in code. Also implement required authentication code.
2022-06-30 01:33:12 +00:00
1c679e48fe More documentation (#2095)
Create an index (README) for the `docs` folder, and add a basic architecture diagram.
2022-06-30 00:37:48 +00:00
63bff8bedc Add caching to Storage.cs (#2102)
* Add caching to Storage.cs

* make Storage disposable

* make Storage disposable
2022-06-29 23:40:31 +00:00
7e0565dac1 Treat warnings as errors (#2103) 2022-06-29 21:25:18 +00:00
b8b0a3ed9b typo (#2101) 2022-06-29 19:04:58 +00:00
30daae2215 Potential solution for TLS errors in OneFuzz (#2087)
* proposed fix from here:

https://github.com/Azure/azure-functions-durable-python/issues/194#issuecomment-710670377

* Update src/api-service/__app__/__init__.py

Co-authored-by: George Pollard <porges@porg.es>

Co-authored-by: stas <statis@microsoft.com>
Co-authored-by: George Pollard <porges@porg.es>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2022-06-29 08:24:40 -07:00
358c57a5b3 Release 5.7.0 (#2088) 5.7.0 2022-06-28 15:48:48 -07:00
2a6a0d9996 Raise AnalysisLevel to 6.0-Recommended, fix warnings (#2086)
Raise the .NET Code Analysis Level to `6.0-Minimum` and then `6.0-Recommended`.
2022-06-28 21:27:33 +00:00
3c3aeaa569 Add test for StatefulOrm static constructor (#2092)
`StatefulOrm` performs checks in the static constructor. Make sure that all these checks pass.
2022-06-29 08:19:44 +12:00
f6fb980cb9 Revert "Update appinsights to released version (#2033)" (#2091)
#2033 appears to be breaking agent since we are getting nested tokio runtimes. Revert it!
2022-06-27 22:53:32 +00:00
e947f35996 Use GetRequiredService instead of GetService + check (#2085)
A small tidy-up.
2022-06-27 22:19:52 +00:00
2eae1793ad Integration tests project (#2083)
Move integration tests into their own project. This makes it easier to run unit tests if you don't want to (or don't have) `azurite` running, since you can `dotnet test` just that directory.

Also add a check into the `AzuriteStorage` class that will abort the entire test run if `azurite` is not running, which is simpler than reading lots of socket exceptions.
2022-06-27 21:14:51 +00:00
0259759407 Update appinsights to released version (#2033)
I noticed that there was a note to update `appinsights` to a released version once one was available, and now one is.

The `close` function was changed to be `async` in the latest version, so `try_flush_and_close` must also become async. Luckily, the three places we use this already have an async context available.
2022-06-27 04:23:12 +00:00
d61fe48a55 Migrate timer_task part 3 (#2066)
* Adding unit tests for the scheduler

* formatting

* fix unit test

* bug fixes

* proting a couple more missong functions

* more bug fixes

* fixing queries and serilization

* fix sas url generation

* fix condition

* [testing] enabling timer_tasks for testing

* Update src/ApiService/Tests/SchedulerTests.cs

Co-authored-by: George Pollard <porges@porg.es>

* address PR comments

* build fix

* address PR comment

* removing renamed function

* resolve merge

* Update src/ApiService/Tests/Integration/AzuriteStorage.cs

Co-authored-by: George Pollard <porges@porg.es>

* - Added verification of the state transition functions
- disabled validation on PoolName

* Update src/deployment/deploy.py

Co-authored-by: George Pollard <porges@porg.es>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2022-06-24 16:22:08 +00:00
fb9af4b811 Implement the download Function for C# (#2076)
Port the `download` function to the C# service.
2022-06-23 15:48:24 +12:00
4eec0bfc45 Implement the node C# function (#2072)
1. Ports the `node` function from Python to C#.
2. Adds a missing authentication check.
3. Add validated string type `PoolName` for consistency with Python version.
2022-06-23 13:44:14 +12:00
1eeefce85c Implement info Function in C# (#2061)
Adds implementation of the `info` function. 

Added support for blobs in the function integration test stuff.

Simplified usage of integration test classes by removing the account name parameter.
2022-06-23 13:06:40 +12:00
4cb7bba2c9 Add underscore naming convention to .editorconfig (#2070) 2022-06-22 15:47:10 +12:00
bbc060bbed Add null analysis attributes to result types (#2069) 2022-06-21 23:41:32 +00:00
3b8c5bbf29 Add bash-completion to devcontainer image (#2058) 2022-06-21 20:58:40 +00:00
8f30801c48 Add nsg config to check-pr (#2067)
Co-authored-by: Noah McGregor Harper <74685766+nharper285@users.noreply.github.com>
2022-06-21 12:32:02 -07:00
76d06efc8c Removing Bad Directory. (#2063) 2022-06-20 10:02:56 -07:00
4ecd43fc0f Convert agent_events to C# & add tests (#2032)
### Function implementation

Added implementation of `agent_events` function and some basic tests.

Fixed several issues encountered in `TestOperations`.

### Additional JsonConverter

The existing Python code for `agent_events` figures out which class to use only based upon the shape of the input, without a discriminator. I have added an additional `JsonConverter` named `SubclassConverter<T>` which will pick a subclass of `T` to deserialize into based upon what properties exist in the JSON.

### Enum helpers

Converted some enum helpers to extension methods to make them a bit more readable.
2022-06-16 22:28:09 +00:00
cdec3c9e8d Test infrastructure for C# Azure Function testing (#2055)
Add support for function tests and the ability to run them against either real Azure Storage or the Azurite emulator.

See follow-up PR #2032 for actual usage.
2022-06-16 21:50:02 +00:00
e9147ba9a7 Add context to every future in the supervisor task (#1976)
* Add context to every futur in the supervisor task

* adding more context info for the sync operations

* adding more logging in the monitoring of the process pipe
2022-06-16 12:01:22 -07:00
2225cb3fb8 Update onefuzz-agent to clap 3 (#2049)
Mostly a maintenance update (dependabot was trying to do this and failing), but it does have a small improvement in that the `debug node_event state_update` help:

```console
$ target/debug/onefuzz-agent debug node_event state_update --help
onefuzz-agent-debug-node_event-state_update 0.1.0

USAGE:
    onefuzz-agent debug node_event state_update <state>

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

ARGS:
    <state>    
```

Now displays the applicable enum values:

```console
$ target/debug/onefuzz-agent debug node_event state_update --help
onefuzz-agent-debug-node_event-state_update 

USAGE:
    onefuzz-agent debug node_event state_update <STATE>

ARGS:
    <STATE>    [possible values: init, free, setting-up, rebooting, ready, busy, done]

OPTIONS:
    -h, --help    Print help information
```

`structopt` dependency removed as it was integrated into `clap`.

Updating the other projects that use `clap` is more involved as they use the lower-level interface.
2022-06-16 14:15:40 +12:00
e73aa705fd fix typo (#2054) 2022-06-15 12:56:23 -07:00
286ebcc07b Adding Node State to Node Heartbeat (#2053) 2022-06-15 18:30:10 +00:00
8b826187a5 flushing the logs when the task is done (#2017)
* flushing the logs when the task is done

* removed teh flush logevent
flushig is now done by sending a message to the task_logger directly
moved the initialization of the logger to cmd.rs

* build fix

* include a duration in the flush message

* exit with a done state instead of panicing
2022-06-15 17:53:59 +00:00