Commit Graph

1082 Commits

Author SHA1 Message Date
404c12354c Use u32 to represent module offsets (#811)
In #765, we introduced an assumption that all module-relative instruction offsets are representable by a `u32`. Move to an explicit `u32` repr for offsets, and add validation to check for overflow at conversion sites.
2021-04-15 18:15:41 -07:00
4d86765be3 Simplify handling of OS-sensitive test cases (#810)
- Add `module!` test helper to let us always define tests paths using POSIX-style paths
- Use `serde_json::json!` macro for maintainable definitions of expected test results
2021-04-15 14:48:08 -07:00
d9eda00f12 disable expect_crash_on_failure by default in more places (#807)
dotnet, as well as the _create_tasks helper still used the old default for requiring crash files on libFuzzer exiting non-zero
2021-04-15 20:19:10 +00:00
c5e0163068 catch VM SKU & VM Image generation mismatch failures (#803) 2021-04-14 14:34:12 -04:00
0e9b6e379f Add example job pipelines (#778) 2021-04-14 14:11:53 -04:00
b599aea5cd allow extra fields in Events (#805) 2021-04-14 13:09:21 -04:00
c8572cd55a remove timestamp from WebhookMessageLog model (#804) 2021-04-14 12:49:46 -04:00
e413aec03d Upgrade to tokio 1.5.0 (#800)
Upgrade to tokio 1.5.0
replaced tokio::sync::mpsc:: with [flume ](https://crates.io/crates/flume) because try_recv was removed
2021-04-14 10:07:28 +00:00
3f52f34b6a Allow the local run to work without copying the synced dirs (#794)
Added an option to local run to not create a job directory
2021-04-13 23:53:37 +00:00
627463d94b only record the first failure if a task has multiple failures (#797) 2021-04-13 17:36:56 -04:00
50ee2abf28 add basic context to every retried request (#798) 2021-04-13 17:36:46 -04:00
5135aa1a27 remove unused import from coverage example (#799) 2021-04-13 17:36:36 -04:00
c570f55088 show task error details in logs (#802) 2021-04-13 17:34:15 -04:00
470e95c833 add Timestamp to multiple models (#796)
Expose the Azure storage table's "Timestamp" for the models where Timestamp should be user-accessible and remove the Timestamp field from models that did not sign up for it.

The behavior where Timestamp is only set by Azure Storage is kept.
2021-04-13 19:03:25 +00:00
39464dc606 simplify removing UserInfo from events prior to logging (#795) 2021-04-13 15:47:08 +00:00
46b8bdccbc add TaskConfig to crash_reported and regression_reported events (#793)
resolves #757 and #758
2021-04-13 10:24:12 +00:00
6a817e6c9e Use dbghelp APIs to find PDB for PE when disassembling (#779)
- Use default dbghelp user search path
- Add module dir to search path
- Allow reusing existing symbol handler context
2021-04-12 11:37:44 -07:00
338a803a0d invert expect_crash_on_failure config option defaults (#792) 2021-04-12 14:08:36 -04:00
a0291350c5 release 2.13.0 (#791) 2.13.0 2021-04-12 12:54:08 -04:00
542129bd1f invert the TUI event filter (#787) 2021-04-09 19:37:17 -04:00
ef8e200438 embed coverage debugger scripts (#783) 2021-04-09 19:16:41 -04:00
a91b5aae89 don't remove inputs once processed (#788) 2021-04-09 17:41:44 -04:00
ba009cd30a combine execs/sec and iteration counts from multiple workers (#786) 2021-04-09 17:21:08 -04:00
030ab4b87f limit the amount of data recorded from libFuzzer's stderr (#785) 2021-04-09 15:44:57 -04:00
4f6432bdc3 don't refresh system stats when only reporting process stats (#784) 2021-04-09 14:12:13 -04:00
4086e7695e Wiring up the UI to the events (#776)
## Summary of the Pull Request
- The UI now receives the telemetry events
- A new section for the coverage has been added 
- All synced dir are now monitored by the UI
- Gracefully exit from the UI 


depends on #663
2021-04-09 17:05:48 +00:00
bf5f3d0ebb always use unix file endings when building libfuzzer-qemu wrappers (#782) 2021-04-09 09:56:56 -04:00
80b3533f83 Report the setup failure in the task when available (#781) 2021-04-09 08:57:56 -04:00
75f0315eb9 add retry with backoff to azcopy calls (#701)
This adds a retry wrapper around azcopy.

something to note:  tokio's Command doesn't allow for clone, which makes this unfortunately difficult to generalize to any command.
2021-04-09 00:35:35 +00:00
97a73a43b6 set default value for ephemeral disks at the server request parser (#780) 2021-04-08 11:18:52 -04:00
cb0701b2f2 Added missing parameter for multi tenant deployment (#773)
## Info on Pull Request

This is a quick fix for multi tenant authentication. After doing some more testing, I realized I was missing a parameter in `azuredeploy.json`. So added missing code from previous [PR ](https://github.com/microsoft/onefuzz/pull/563/files). 

## Validation Steps Performed

I have tested these changes and was successfully able to deploy in both single tenant and multi tenant environments.
2021-04-07 21:45:56 +00:00
3e7b3df34f local run UI (#663)
## Summary of the Pull Request

This PR add a UI to the local run. 
- The UI currently monitors the logs and  some of the directory created (the rest will be wired in a coming PR)
- pressing 'q' will quit the PR
- By default, the job directory is deleted when the ui quits unless the parameter 'keep_job_dir' is specified
2021-04-06 21:44:37 +00:00
794400adf1 ignore benign error messages when checking integration test logs (#775) 2021-04-06 15:35:54 -04:00
88af5f4dd3 allow deployment to non-default subscriptions (#774) 2021-04-06 13:06:35 -04:00
a33aba26f2 format deployment registration code (#768) 2021-04-06 09:26:13 +00:00
d279fe3486 simplify context for storage queue enqueue (#767) 2021-04-05 21:58:01 +00:00
5e289d7a24 release 2.12.0 (#770) 2.12.0 2021-04-05 16:01:40 -04:00
e21eafd135 clarify telemetry key names at the service level (#769) 2021-04-05 15:23:03 -04:00
c452a8aff3 allow upper case acronymns (#771) 2021-04-05 13:41:33 -04:00
2aea006c50 fix checking status codes for failure (#766)
This fixes retrying on bad status codes.

Note, this adds support to specify specific error codes as "successful".  This is important for the If-None-Match conditional upload case (blob uploading).  The response we get back is 409 (Conflict).

Previously, we would "fail fast" the 409.  However, what we want is "409 is basically success here" and every other call 409 should be a failure.
2021-04-03 20:12:50 +00:00
eb7571e531 verify libfuzzer targets at the start of the task (#752) 2021-04-02 23:11:46 -04:00
ca12904684 add log checking to refactored integration check (#700)
In practice, Application Insights can take up to 3 minutes before something sent to it is available via KQL.

This PR logs a start and stop marker such that the integration tests only search for logs during the integration tests. This reduces the complexity when using the integration tests during the development process.

Note: this migrated the new functionality from #356 into the latest integration test tools.
2021-04-02 21:49:19 +00:00
9c1540aca8 Use custom serialization for block coverage (#765)
Update command block coverage serialization to support a more compact JSON format.
2021-04-02 11:57:27 -07:00
7e5cf780a6 Added support for multi tenant authentication (#746)
## Summary of the Pull Request

_What is this about?_

## PR Checklist
* [x] Applies to work item: #562 
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/onefuzz) and sign the CLI.
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

## Info on Pull Request

The end-to-end changes needed to have onefuzz deployed with multi-tenant authentication.

## Validation Steps Performed

_How does someone test & validate?_
2021-04-02 14:39:20 +00:00
624a7f77e8 set ephemeral disks off by default (#761) 2021-04-01 16:26:17 -04:00
f6adad3ed9 updates related to cargo clippy 1.51.0 (#730) 2021-04-01 13:08:21 -04:00
03fc386106 update ADO deploy dependencies (#764) 2021-04-01 12:38:34 -04:00
7ef8223f38 Borrow Recorder in debug event handler (#762)
Enables accessing recorded coverage without using the top-level `record()` function.
2021-03-31 15:01:01 -07:00
6aca32ed9c use usize for worker_id (#755) 2021-03-30 22:27:10 -04:00
522ae4c5d9 remove supervisor-is-done upon test completion (#754) 2021-03-30 19:58:43 -04:00