Commit Graph

1471 Commits

Author SHA1 Message Date
8dbc97129a Bump tokio from 1.19.2 to 1.20.0 in /src/proxy-manager (#2164)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.19.2 to 1.20.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.19.2...tokio-1.20.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-21 10:43:48 +12:00
4fa6e74241 Enable .NET functions in check-pr for Agent-specific functions (#2119)
Enable the .NET functions for the agent by sending the agent the URI for the `-net` service.

Also fix some things causing failures when using the .NET functions (`CouldShrinkScaleset` was not implemented).

Improve error handling around table serialization/deserialization, fix an issue with int64/long mismatch between Python & C# code.

----

For `check-pr` testing:

1. There's a new parameter `enable_dotnet` which maps directly to the `--enable_dotnet` switch on `deploy.py`.
2. If you put `agent` there, all the `agent_*` functions will be enabled for .NET and disabled for Python.
3. If `agent_can_schedule` is disabled on the Python side, it will automatically tell the agent to use the .NET functions.

So to test the .NET agent functions, do a `check-pr` run with `enable_dotnet` set to `agent` and it should all work.
2022-07-20 20:40:30 +00:00
b1a3e7530a Drop support for pre 3.0.0 authentication in the cli (#2173) 2022-07-20 13:28:31 -07:00
7f90c13724 Laying groundwork for TimerRepro implementation (#2168)
* Add some pre-reqs for TimerRepro

* Format and add some API to creds

* PR comment
2022-07-15 18:33:25 +00:00
3347e7e67b Use .get instead of lookup (#2165) 2022-07-14 23:55:30 +00:00
fe8b75045f Drop the global event sender on close (#2125)
* anyhow

* rename flush to done

* format

* log when messages are skipped
2022-07-14 01:18:50 +00:00
ae4c1b9bec Allow autoscale with a minimum of 0 (#2162) 2022-07-14 12:37:46 +12:00
d71c558447 allow and set onefuzz cli autoscale min value default to zero (#2112) 2022-07-13 08:59:03 -07:00
0b8093b89e Reuse Agent artifacts if nothing in src/agent changes (#2115)
The agent build takes most of the CI runtime, so improve it by only rebuilding if the pre-reqs or something inside `src/agent` changes.

We will always skip the cache for builds on tags and from the `main` branch, so that version is stamped correctly there.
2022-07-12 22:09:32 +00:00
ec515a57fd Implement the containers function for C# (#2078)
Fairly straightforward but required implementation of a few more functions on the `IContainers`/`Containers` class.
2022-07-12 22:00:22 +00:00
9d5d86a03b Pin Rust to 1.62.0 (#2153) 2022-07-12 20:51:20 +00:00
69b11f60c2 Add GoodBad C# example project, integration test (#2148) 2022-07-12 12:07:30 -07:00
c1f9d738d9 Depend on updated SharpFuzz package (#2149) 2022-07-12 10:43:21 -07:00
3878793147 Trim whitespaces from options passed to executables (#2151)
* Trim whitespaces from options passed to executables

* check for None

* refactor to a function
5.8.0
2022-07-11 23:37:48 +00:00
7d704a9688 Fix clippy (#2152) 2022-07-11 19:00:51 -04:00
4878356e48 Release 5.8.0 (#2146)
* Release 5.8.0
2022-07-11 12:09:18 -07:00
28e42ab8e8 Update Function Scaling Policy and Diagnostic Settings. (#2140)
* Update Function Scaling Policy and Diagnostic Settings.

* Adding changes to scaling policy

* UPdating cooldown.

* Changing time windows.

* Updating duration.
2022-07-08 12:03:14 -07:00
34a8d721bb Add LibFuzzerDotnetLoader docs (#2142) 2022-07-08 08:24:56 -07:00
e37f56dd05 Remove some allow(clippy::uninit_vec) (#2111)
Closes #1595. Leaving one instance of `allow(clippy::uninit_vec)` in `pipe_handle.rs` as it is more difficult to remove. (Maybe there's  another crate we could use for pipes?)

To satisfy clippy, instead of creating the vec and then resizing it straight away (exposing uninitialized data), write into the `spare_capacity_mut()` (which is `&mut [MaybeUninit<_>]`) and then resize it afterwards.
2022-07-08 01:24:11 +00:00
6bf77a6230 Add logging (#2141)
- Add logging
- Catch terminal exceptions and log as errors before re-throwing
- Nest some exceptions
- Warn when falling back to non-zero cost delegate for legacy compat
2022-07-07 17:41:53 -07:00
ff1d088fa1 include state filter when searching for expired jobs and tasks (#2138) 2022-07-07 14:53:51 -07:00
edf537137b Add multi-var target spec, improve errors (#2136) 2022-07-07 11:20:45 -07:00
aa2e76e7e9 Fix execution of TimerProxy (#2133)
* Fix execution of timerProxy
- fixed behavior of Container.SaveBlob (file was not being overwritten)
- added support for default value in the entity converter
- fix scaleset size data type
- added initialization of Subnet
- renamed TestHook Info to _Info to prevent clashing of name function mame
- added target framework to ApiServiceProject to help local debugging

* fix unit tests

* fix unit tests

* remove unused property

* fix typo

* removing partial class TimerProxy
2022-07-07 09:25:51 -07:00
0f0fd8ee9c Enable TimerRepro (#2108) 2022-07-07 15:25:20 +00:00
478788f607 Remove stubbed support for macOS builds (#2135)
Closes #2134.
2022-07-06 23:23:47 +00:00
c0895a5b87 Disable macos build for agent (#2134) 2022-07-07 10:44:38 +12:00
f37224e8bb Add dotnet coverage task (#2062)
* checkpoint

* some more progress

* more progress

* More progress

* Now it's time to test it

* It works locally 🎉

* Attempting clean build

* fmt

* temporarily stub out macos

* missed a few

* please be the last one

* .

* .

* .

* noop change to unstuck actions

* .

* .

* Fix setup script

* Some fixes

* It works except for a race condition -- use a directory watcher to fix it

* It works end to end!

* Execute the commands using tokio's structs and timeout mechanism

* It works.... for real this time

* Undo timer changes

* Cleanup

* 🧹

* Fix import

* .

* PR comments

* Fix clippy

* Clippy whyyy

* Only check dotnet path once

* fmt

* Fix a couple more comments
2022-07-06 16:13:45 -04:00
bc48069a94 Import LibFuzzerDotnetLoader, build .NET fuzzing tools (#2126) 2022-07-06 11:51:47 -07:00
728d60588c Use 'os-provided' Python (#2118)
Using `3.8` for the Python devcontainer feature version means it builds from source every time the devcontainer is rebuilt, which is very slow. Using `os-provided` will give us a recent enough version (3.10).

Also install `python-is-python3` so that the bare `python` command points to Python 3 (our scripts seem to expect this).
2022-07-05 21:31:38 +00:00
4d7a01e157 Fixed (#2122) 2022-07-05 20:32:04 +00:00
52b760b89a Porting divergent code introduced in #2116 (#2121)
Make the log sas url last as long as the job duration
2022-07-05 11:35:42 -07:00
ab0b974eb1 Cache radamsa build (#2114) 2022-07-03 21:09:31 +00:00
61fc091f88 Make the log sas url last as long as the job duration (#2116) 2022-07-02 01:40:02 +00:00
262ad689f3 Enable C# functions (#2093)
Enables 2 more C# refactor functions in code. They're ready to be tested in canary.

Improves the precommit script in devcontainer.
2022-07-01 03:57:36 +00:00
a9fe5e6c17 Refactor agent_registration to C# (#2107)
Port `agent_registration` from Python to C#.
2022-07-01 02:38:52 +00:00
2453127298 Bump tokio from 1.18.2 to 1.19.2 in /src/proxy-manager (#2022)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.18.2 to 1.19.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-01 14:00:05 +12:00
308ffa6caa Update devcontainer to 22.04 (#2110) 2022-07-01 00:54:33 +00:00
8d8a8b01d2 remove unused Poolname validation (#2094) 2022-06-30 23:23:53 +00:00
870a51a1e4 Enabling TimerProxy (#2089)
* Testing TimerProzy

* Update src/deployment/deploy.py
2022-06-30 22:47:37 +00:00
a017b10d17 Add correct routes and auth to "agents" Functions (#2109)
The routes and auth were missing from the "agents" functions.
2022-06-30 22:05:07 +00:00
fb0e055ca9 Add azcopy upload support and switch to the default max_concurrency (#1556)
* Add azcopy copy support and switch to the default max_concurrency

* Modify upload_file_data to use upload_file for consistency

* Fix lint issues

* Apply black formatting rules

* Apply isort formatting rules

Co-authored-by: Stas <stishkin@live.com>
2022-06-30 11:30:52 -07:00
ed69cb48b5 Fix Racing condition in monitor.rs (#2105)
* Fix Racing condition in monitor.rs

* address pr comment

* address pr comment

* emmiting a warning instead of stopping the monitoring
2022-06-30 09:29:51 -07:00
ea60410df9 Enable C# functions in code & fill out missing functionality (#2084)
C# HTTP functions won’t take effect by default so it is safe to enable them in code. Also implement required authentication code.
2022-06-30 01:33:12 +00:00
1c679e48fe More documentation (#2095)
Create an index (README) for the `docs` folder, and add a basic architecture diagram.
2022-06-30 00:37:48 +00:00
63bff8bedc Add caching to Storage.cs (#2102)
* Add caching to Storage.cs

* make Storage disposable

* make Storage disposable
2022-06-29 23:40:31 +00:00
7e0565dac1 Treat warnings as errors (#2103) 2022-06-29 21:25:18 +00:00
b8b0a3ed9b typo (#2101) 2022-06-29 19:04:58 +00:00
30daae2215 Potential solution for TLS errors in OneFuzz (#2087)
* proposed fix from here:

https://github.com/Azure/azure-functions-durable-python/issues/194#issuecomment-710670377

* Update src/api-service/__app__/__init__.py

Co-authored-by: George Pollard <porges@porg.es>

Co-authored-by: stas <statis@microsoft.com>
Co-authored-by: George Pollard <porges@porg.es>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2022-06-29 08:24:40 -07:00
358c57a5b3 Release 5.7.0 (#2088) 5.7.0 2022-06-28 15:48:48 -07:00
2a6a0d9996 Raise AnalysisLevel to 6.0-Recommended, fix warnings (#2086)
Raise the .NET Code Analysis Level to `6.0-Minimum` and then `6.0-Recommended`.
2022-06-28 21:27:33 +00:00