Commit Graph

1377 Commits

Author SHA1 Message Date
c667e3aa2c Bump regex from 1.5.5 to 1.6.0 in /src/agent (#2127)
Bumps [regex](https://github.com/rust-lang/regex) from 1.5.5 to 1.6.0.
- [Release notes](https://github.com/rust-lang/regex/releases)
- [Changelog](https://github.com/rust-lang/regex/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/regex/compare/1.5.5...1.6.0)

---
updated-dependencies:
- dependency-name: regex
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-22 16:39:32 +12:00
b2f5b9a2d1 Bump bytes from 1.1.0 to 1.2.0 in /src/agent (#2175)
Bumps [bytes](https://github.com/tokio-rs/bytes) from 1.1.0 to 1.2.0.
- [Release notes](https://github.com/tokio-rs/bytes/releases)
- [Changelog](https://github.com/tokio-rs/bytes/blob/master/CHANGELOG.md)
- [Commits](https://github.com/tokio-rs/bytes/compare/v1.1.0...v1.2.0)

---
updated-dependencies:
- dependency-name: bytes
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-22 15:21:33 +12:00
8c636c37df Bump Azure.Storage.Blobs & Azure.Storage.Queues (#2181)
* Bump Azure.Storage.Queues in /src/ApiService/ApiService

Bumps [Azure.Storage.Queues](https://github.com/Azure/azure-sdk-for-net) from 12.9.0 to 12.11.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-net/releases)
- [Commits](https://github.com/Azure/azure-sdk-for-net/compare/Azure.Storage.Queues_12.9.0...Azure.Storage.Queues_12.11.0)

---
updated-dependencies:
- dependency-name: Azure.Storage.Queues
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Bump Azure.Storage.Blobs in /src/ApiService/ApiService

Bumps [Azure.Storage.Blobs](https://github.com/Azure/azure-sdk-for-net) from 12.11.0 to 12.13.0.
- [Release notes](https://github.com/Azure/azure-sdk-for-net/releases)
- [Commits](https://github.com/Azure/azure-sdk-for-net/compare/Azure.Storage.Blobs_12.11.0...Azure.Storage.Blobs_12.13.0)

---
updated-dependencies:
- dependency-name: Azure.Storage.Blobs
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>

* Update lockfiles

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: George Pollard <gpollard@microsoft.com>
2022-07-22 00:31:29 +00:00
5083d9e461 Bump symbolic from 8.6.0 to 8.8.0 in /src/agent (#2056)
Bumps [symbolic](https://github.com/getsentry/symbolic) from 8.6.0 to 8.8.0.
- [Release notes](https://github.com/getsentry/symbolic/releases)
- [Changelog](https://github.com/getsentry/symbolic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/getsentry/symbolic/compare/8.6.0...8.8.0)

---
updated-dependencies:
- dependency-name: symbolic
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-21 23:16:36 +00:00
94db49995f Move functions into Functions namespace (#2176) 2022-07-21 20:46:31 +00:00
08fb29fc6b ignore the state when a job_id is specified in tasks list (#2171) 2022-07-21 18:25:39 +00:00
8dbc97129a Bump tokio from 1.19.2 to 1.20.0 in /src/proxy-manager (#2164)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.19.2 to 1.20.0.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/compare/tokio-1.19.2...tokio-1.20.0)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-21 10:43:48 +12:00
4fa6e74241 Enable .NET functions in check-pr for Agent-specific functions (#2119)
Enable the .NET functions for the agent by sending the agent the URI for the `-net` service.

Also fix some things causing failures when using the .NET functions (`CouldShrinkScaleset` was not implemented).

Improve error handling around table serialization/deserialization, fix an issue with int64/long mismatch between Python & C# code.

----

For `check-pr` testing:

1. There's a new parameter `enable_dotnet` which maps directly to the `--enable_dotnet` switch on `deploy.py`.
2. If you put `agent` there, all the `agent_*` functions will be enabled for .NET and disabled for Python.
3. If `agent_can_schedule` is disabled on the Python side, it will automatically tell the agent to use the .NET functions.

So to test the .NET agent functions, do a `check-pr` run with `enable_dotnet` set to `agent` and it should all work.
2022-07-20 20:40:30 +00:00
b1a3e7530a Drop support for pre 3.0.0 authentication in the cli (#2173) 2022-07-20 13:28:31 -07:00
7f90c13724 Laying groundwork for TimerRepro implementation (#2168)
* Add some pre-reqs for TimerRepro

* Format and add some API to creds

* PR comment
2022-07-15 18:33:25 +00:00
3347e7e67b Use .get instead of lookup (#2165) 2022-07-14 23:55:30 +00:00
fe8b75045f Drop the global event sender on close (#2125)
* anyhow

* rename flush to done

* format

* log when messages are skipped
2022-07-14 01:18:50 +00:00
ae4c1b9bec Allow autoscale with a minimum of 0 (#2162) 2022-07-14 12:37:46 +12:00
d71c558447 allow and set onefuzz cli autoscale min value default to zero (#2112) 2022-07-13 08:59:03 -07:00
0b8093b89e Reuse Agent artifacts if nothing in src/agent changes (#2115)
The agent build takes most of the CI runtime, so improve it by only rebuilding if the pre-reqs or something inside `src/agent` changes.

We will always skip the cache for builds on tags and from the `main` branch, so that version is stamped correctly there.
2022-07-12 22:09:32 +00:00
ec515a57fd Implement the containers function for C# (#2078)
Fairly straightforward but required implementation of a few more functions on the `IContainers`/`Containers` class.
2022-07-12 22:00:22 +00:00
9d5d86a03b Pin Rust to 1.62.0 (#2153) 2022-07-12 20:51:20 +00:00
69b11f60c2 Add GoodBad C# example project, integration test (#2148) 2022-07-12 12:07:30 -07:00
c1f9d738d9 Depend on updated SharpFuzz package (#2149) 2022-07-12 10:43:21 -07:00
3878793147 Trim whitespaces from options passed to executables (#2151)
* Trim whitespaces from options passed to executables

* check for None

* refactor to a function
5.8.0
2022-07-11 23:37:48 +00:00
7d704a9688 Fix clippy (#2152) 2022-07-11 19:00:51 -04:00
4878356e48 Release 5.8.0 (#2146)
* Release 5.8.0
2022-07-11 12:09:18 -07:00
28e42ab8e8 Update Function Scaling Policy and Diagnostic Settings. (#2140)
* Update Function Scaling Policy and Diagnostic Settings.

* Adding changes to scaling policy

* UPdating cooldown.

* Changing time windows.

* Updating duration.
2022-07-08 12:03:14 -07:00
34a8d721bb Add LibFuzzerDotnetLoader docs (#2142) 2022-07-08 08:24:56 -07:00
e37f56dd05 Remove some allow(clippy::uninit_vec) (#2111)
Closes #1595. Leaving one instance of `allow(clippy::uninit_vec)` in `pipe_handle.rs` as it is more difficult to remove. (Maybe there's  another crate we could use for pipes?)

To satisfy clippy, instead of creating the vec and then resizing it straight away (exposing uninitialized data), write into the `spare_capacity_mut()` (which is `&mut [MaybeUninit<_>]`) and then resize it afterwards.
2022-07-08 01:24:11 +00:00
6bf77a6230 Add logging (#2141)
- Add logging
- Catch terminal exceptions and log as errors before re-throwing
- Nest some exceptions
- Warn when falling back to non-zero cost delegate for legacy compat
2022-07-07 17:41:53 -07:00
ff1d088fa1 include state filter when searching for expired jobs and tasks (#2138) 2022-07-07 14:53:51 -07:00
edf537137b Add multi-var target spec, improve errors (#2136) 2022-07-07 11:20:45 -07:00
aa2e76e7e9 Fix execution of TimerProxy (#2133)
* Fix execution of timerProxy
- fixed behavior of Container.SaveBlob (file was not being overwritten)
- added support for default value in the entity converter
- fix scaleset size data type
- added initialization of Subnet
- renamed TestHook Info to _Info to prevent clashing of name function mame
- added target framework to ApiServiceProject to help local debugging

* fix unit tests

* fix unit tests

* remove unused property

* fix typo

* removing partial class TimerProxy
2022-07-07 09:25:51 -07:00
0f0fd8ee9c Enable TimerRepro (#2108) 2022-07-07 15:25:20 +00:00
478788f607 Remove stubbed support for macOS builds (#2135)
Closes #2134.
2022-07-06 23:23:47 +00:00
c0895a5b87 Disable macos build for agent (#2134) 2022-07-07 10:44:38 +12:00
f37224e8bb Add dotnet coverage task (#2062)
* checkpoint

* some more progress

* more progress

* More progress

* Now it's time to test it

* It works locally 🎉

* Attempting clean build

* fmt

* temporarily stub out macos

* missed a few

* please be the last one

* .

* .

* .

* noop change to unstuck actions

* .

* .

* Fix setup script

* Some fixes

* It works except for a race condition -- use a directory watcher to fix it

* It works end to end!

* Execute the commands using tokio's structs and timeout mechanism

* It works.... for real this time

* Undo timer changes

* Cleanup

* 🧹

* Fix import

* .

* PR comments

* Fix clippy

* Clippy whyyy

* Only check dotnet path once

* fmt

* Fix a couple more comments
2022-07-06 16:13:45 -04:00
bc48069a94 Import LibFuzzerDotnetLoader, build .NET fuzzing tools (#2126) 2022-07-06 11:51:47 -07:00
728d60588c Use 'os-provided' Python (#2118)
Using `3.8` for the Python devcontainer feature version means it builds from source every time the devcontainer is rebuilt, which is very slow. Using `os-provided` will give us a recent enough version (3.10).

Also install `python-is-python3` so that the bare `python` command points to Python 3 (our scripts seem to expect this).
2022-07-05 21:31:38 +00:00
4d7a01e157 Fixed (#2122) 2022-07-05 20:32:04 +00:00
52b760b89a Porting divergent code introduced in #2116 (#2121)
Make the log sas url last as long as the job duration
2022-07-05 11:35:42 -07:00
ab0b974eb1 Cache radamsa build (#2114) 2022-07-03 21:09:31 +00:00
61fc091f88 Make the log sas url last as long as the job duration (#2116) 2022-07-02 01:40:02 +00:00
262ad689f3 Enable C# functions (#2093)
Enables 2 more C# refactor functions in code. They're ready to be tested in canary.

Improves the precommit script in devcontainer.
2022-07-01 03:57:36 +00:00
a9fe5e6c17 Refactor agent_registration to C# (#2107)
Port `agent_registration` from Python to C#.
2022-07-01 02:38:52 +00:00
2453127298 Bump tokio from 1.18.2 to 1.19.2 in /src/proxy-manager (#2022)
Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.18.2 to 1.19.2.
- [Release notes](https://github.com/tokio-rs/tokio/releases)
- [Commits](https://github.com/tokio-rs/tokio/commits)

---
updated-dependencies:
- dependency-name: tokio
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-01 14:00:05 +12:00
308ffa6caa Update devcontainer to 22.04 (#2110) 2022-07-01 00:54:33 +00:00
8d8a8b01d2 remove unused Poolname validation (#2094) 2022-06-30 23:23:53 +00:00
870a51a1e4 Enabling TimerProxy (#2089)
* Testing TimerProzy

* Update src/deployment/deploy.py
2022-06-30 22:47:37 +00:00
a017b10d17 Add correct routes and auth to "agents" Functions (#2109)
The routes and auth were missing from the "agents" functions.
2022-06-30 22:05:07 +00:00
fb0e055ca9 Add azcopy upload support and switch to the default max_concurrency (#1556)
* Add azcopy copy support and switch to the default max_concurrency

* Modify upload_file_data to use upload_file for consistency

* Fix lint issues

* Apply black formatting rules

* Apply isort formatting rules

Co-authored-by: Stas <stishkin@live.com>
2022-06-30 11:30:52 -07:00
ed69cb48b5 Fix Racing condition in monitor.rs (#2105)
* Fix Racing condition in monitor.rs

* address pr comment

* address pr comment

* emmiting a warning instead of stopping the monitoring
2022-06-30 09:29:51 -07:00
ea60410df9 Enable C# functions in code & fill out missing functionality (#2084)
C# HTTP functions won’t take effect by default so it is safe to enable them in code. Also implement required authentication code.
2022-06-30 01:33:12 +00:00
1c679e48fe More documentation (#2095)
Create an index (README) for the `docs` folder, and add a basic architecture diagram.
2022-06-30 00:37:48 +00:00