Commit Graph

719 Commits

Author SHA1 Message Date
470e95c833 add Timestamp to multiple models (#796)
Expose the Azure storage table's "Timestamp" for the models where Timestamp should be user-accessible and remove the Timestamp field from models that did not sign up for it.

The behavior where Timestamp is only set by Azure Storage is kept.
2021-04-13 19:03:25 +00:00
39464dc606 simplify removing UserInfo from events prior to logging (#795) 2021-04-13 15:47:08 +00:00
46b8bdccbc add TaskConfig to crash_reported and regression_reported events (#793)
resolves #757 and #758
2021-04-13 10:24:12 +00:00
6a817e6c9e Use dbghelp APIs to find PDB for PE when disassembling (#779)
- Use default dbghelp user search path
- Add module dir to search path
- Allow reusing existing symbol handler context
2021-04-12 11:37:44 -07:00
338a803a0d invert expect_crash_on_failure config option defaults (#792) 2021-04-12 14:08:36 -04:00
a0291350c5 release 2.13.0 (#791) 2.13.0 2021-04-12 12:54:08 -04:00
542129bd1f invert the TUI event filter (#787) 2021-04-09 19:37:17 -04:00
ef8e200438 embed coverage debugger scripts (#783) 2021-04-09 19:16:41 -04:00
a91b5aae89 don't remove inputs once processed (#788) 2021-04-09 17:41:44 -04:00
ba009cd30a combine execs/sec and iteration counts from multiple workers (#786) 2021-04-09 17:21:08 -04:00
030ab4b87f limit the amount of data recorded from libFuzzer's stderr (#785) 2021-04-09 15:44:57 -04:00
4f6432bdc3 don't refresh system stats when only reporting process stats (#784) 2021-04-09 14:12:13 -04:00
4086e7695e Wiring up the UI to the events (#776)
## Summary of the Pull Request
- The UI now receives the telemetry events
- A new section for the coverage has been added 
- All synced dir are now monitored by the UI
- Gracefully exit from the UI 


depends on #663
2021-04-09 17:05:48 +00:00
bf5f3d0ebb always use unix file endings when building libfuzzer-qemu wrappers (#782) 2021-04-09 09:56:56 -04:00
80b3533f83 Report the setup failure in the task when available (#781) 2021-04-09 08:57:56 -04:00
75f0315eb9 add retry with backoff to azcopy calls (#701)
This adds a retry wrapper around azcopy.

something to note:  tokio's Command doesn't allow for clone, which makes this unfortunately difficult to generalize to any command.
2021-04-09 00:35:35 +00:00
97a73a43b6 set default value for ephemeral disks at the server request parser (#780) 2021-04-08 11:18:52 -04:00
cb0701b2f2 Added missing parameter for multi tenant deployment (#773)
## Info on Pull Request

This is a quick fix for multi tenant authentication. After doing some more testing, I realized I was missing a parameter in `azuredeploy.json`. So added missing code from previous [PR ](https://github.com/microsoft/onefuzz/pull/563/files). 

## Validation Steps Performed

I have tested these changes and was successfully able to deploy in both single tenant and multi tenant environments.
2021-04-07 21:45:56 +00:00
3e7b3df34f local run UI (#663)
## Summary of the Pull Request

This PR add a UI to the local run. 
- The UI currently monitors the logs and  some of the directory created (the rest will be wired in a coming PR)
- pressing 'q' will quit the PR
- By default, the job directory is deleted when the ui quits unless the parameter 'keep_job_dir' is specified
2021-04-06 21:44:37 +00:00
794400adf1 ignore benign error messages when checking integration test logs (#775) 2021-04-06 15:35:54 -04:00
88af5f4dd3 allow deployment to non-default subscriptions (#774) 2021-04-06 13:06:35 -04:00
a33aba26f2 format deployment registration code (#768) 2021-04-06 09:26:13 +00:00
d279fe3486 simplify context for storage queue enqueue (#767) 2021-04-05 21:58:01 +00:00
5e289d7a24 release 2.12.0 (#770) 2.12.0 2021-04-05 16:01:40 -04:00
e21eafd135 clarify telemetry key names at the service level (#769) 2021-04-05 15:23:03 -04:00
c452a8aff3 allow upper case acronymns (#771) 2021-04-05 13:41:33 -04:00
2aea006c50 fix checking status codes for failure (#766)
This fixes retrying on bad status codes.

Note, this adds support to specify specific error codes as "successful".  This is important for the If-None-Match conditional upload case (blob uploading).  The response we get back is 409 (Conflict).

Previously, we would "fail fast" the 409.  However, what we want is "409 is basically success here" and every other call 409 should be a failure.
2021-04-03 20:12:50 +00:00
eb7571e531 verify libfuzzer targets at the start of the task (#752) 2021-04-02 23:11:46 -04:00
ca12904684 add log checking to refactored integration check (#700)
In practice, Application Insights can take up to 3 minutes before something sent to it is available via KQL.

This PR logs a start and stop marker such that the integration tests only search for logs during the integration tests. This reduces the complexity when using the integration tests during the development process.

Note: this migrated the new functionality from #356 into the latest integration test tools.
2021-04-02 21:49:19 +00:00
9c1540aca8 Use custom serialization for block coverage (#765)
Update command block coverage serialization to support a more compact JSON format.
2021-04-02 11:57:27 -07:00
7e5cf780a6 Added support for multi tenant authentication (#746)
## Summary of the Pull Request

_What is this about?_

## PR Checklist
* [x] Applies to work item: #562 
* [x] CLA signed. If not, go over [here](https://cla.opensource.microsoft.com/microsoft/onefuzz) and sign the CLI.
* [x] Tests added/passed
* [ ] Requires documentation to be updated
* [x] I've discussed this with core contributors already. If not checked, I'm ready to accept this work might be rejected in favor of a different grand plan. Issue number where discussion took place: #xxx

## Info on Pull Request

The end-to-end changes needed to have onefuzz deployed with multi-tenant authentication.

## Validation Steps Performed

_How does someone test & validate?_
2021-04-02 14:39:20 +00:00
624a7f77e8 set ephemeral disks off by default (#761) 2021-04-01 16:26:17 -04:00
f6adad3ed9 updates related to cargo clippy 1.51.0 (#730) 2021-04-01 13:08:21 -04:00
03fc386106 update ADO deploy dependencies (#764) 2021-04-01 12:38:34 -04:00
7ef8223f38 Borrow Recorder in debug event handler (#762)
Enables accessing recorded coverage without using the top-level `record()` function.
2021-03-31 15:01:01 -07:00
6aca32ed9c use usize for worker_id (#755) 2021-03-30 22:27:10 -04:00
522ae4c5d9 remove supervisor-is-done upon test completion (#754) 2021-03-30 19:58:43 -04:00
560fa63ac3 Add command block coverage merging (#756)
- Add coverage merging for module and command block coverage
- Initialize module and offset tracking on demand
- Add tests
2021-03-30 16:31:54 -07:00
3096f99e86 enable using ephemeral disks by default (#461) 2021-03-30 18:48:44 -04:00
3eb7c8643b set expect_crash_on_failure default to False on libFuzzer tasks (#748) 2021-03-30 21:51:15 +00:00
b75916d62f Yield on each enqueue and pop on the local queue (#750) 2021-03-30 21:19:20 +00:00
5055bf6c38 add libfuzzer regression tasks to local fuzzing (#744) 2021-03-30 20:58:07 +00:00
cf1051bad1 Update Linux Recorder API for task (#753)
Clean up the interface of the Linux `Recorder` struct, and make it more consistent with the Windows version. Hold a mutable ref to a `ModuleCache` to enable in-memory cache re-use.
2021-03-30 11:42:52 -07:00
e2c9b39ee8 release 2.11.1 (#751) 2.11.1 2021-03-30 09:45:52 -04:00
9bd2e90e53 fix message queue send (#749) 2021-03-29 23:01:47 -04:00
541695927f release 2.11.0 (#747) 2.11.0 2021-03-29 14:38:43 -04:00
e8ce384bd9 cleanup local job_id directory upon clean exit (#738) 2021-03-27 16:39:09 -04:00
19d19a1e0e add names for poller logging (#742) 2021-03-26 23:04:41 -04:00
b43a45187b optionally ignore dotfiles in syncdir monitors (#741) 2021-03-26 22:40:36 -04:00
fffaab2d25 change the scale of timeouts to seconds instead of minutes (#739) 2021-03-26 22:20:10 -04:00