Commit Graph

1082 Commits

Author SHA1 Message Date
560fa63ac3 Add command block coverage merging (#756)
- Add coverage merging for module and command block coverage
- Initialize module and offset tracking on demand
- Add tests
2021-03-30 16:31:54 -07:00
3096f99e86 enable using ephemeral disks by default (#461) 2021-03-30 18:48:44 -04:00
3eb7c8643b set expect_crash_on_failure default to False on libFuzzer tasks (#748) 2021-03-30 21:51:15 +00:00
b75916d62f Yield on each enqueue and pop on the local queue (#750) 2021-03-30 21:19:20 +00:00
5055bf6c38 add libfuzzer regression tasks to local fuzzing (#744) 2021-03-30 20:58:07 +00:00
cf1051bad1 Update Linux Recorder API for task (#753)
Clean up the interface of the Linux `Recorder` struct, and make it more consistent with the Windows version. Hold a mutable ref to a `ModuleCache` to enable in-memory cache re-use.
2021-03-30 11:42:52 -07:00
e2c9b39ee8 release 2.11.1 (#751) 2.11.1 2021-03-30 09:45:52 -04:00
9bd2e90e53 fix message queue send (#749) 2021-03-29 23:01:47 -04:00
541695927f release 2.11.0 (#747) 2.11.0 2021-03-29 14:38:43 -04:00
e8ce384bd9 cleanup local job_id directory upon clean exit (#738) 2021-03-27 16:39:09 -04:00
19d19a1e0e add names for poller logging (#742) 2021-03-26 23:04:41 -04:00
b43a45187b optionally ignore dotfiles in syncdir monitors (#741) 2021-03-26 22:40:36 -04:00
fffaab2d25 change the scale of timeouts to seconds instead of minutes (#739) 2021-03-26 22:20:10 -04:00
324f812bb8 cleanup syncdir events (#740) 2021-03-26 21:53:55 -04:00
dc9b6fa1b0 address telemetry log message locality (#736) 2021-03-26 18:45:36 -04:00
5c149413a5 Enable coverage module filtering (only) for both platforms (#733)
- Add coverage filtering to Windows generic coverage recording
- Temporarily disable deserialization of symbol filter rules
2021-03-26 15:16:28 -07:00
cf0d047a23 bubble out invalid messages on input queues (#731) 2021-03-26 15:50:48 +00:00
2a0365b9a2 Adding timeout to local run (#735) 2021-03-25 19:07:38 -04:00
5f10169783 simplify returns (#732) 2021-03-25 12:27:31 -04:00
c02ede76b3 fix local generic analysis (#729) 2021-03-24 22:03:34 -04:00
fd6f9eb0c3 add symbol and module names to StackFrame (#723)
This exposes the module_info and symbol name from debugger in the StackFrame.  This enables the stack minimization function work on function names.
2021-03-24 19:07:28 +00:00
5fcb777799 update AFL++ to 3.12c (#728) 2021-03-24 12:30:07 -04:00
c06d439c92 update azure-cli, azure-cli-core, and msal (#721) 2021-03-23 20:22:54 -04:00
50ce56f754 update to include regexes from google/clusterfuzz #2282 (#722) 2021-03-23 19:43:21 -04:00
d6b2ff9bae hardcode wget to use TLSv1.2 (#717) 2021-03-23 19:21:13 -04:00
92b5139a0a Removing UserInfo from notifications logging (#724) 2021-03-23 18:47:05 -04:00
1706a91291 Removing UserInfo from 'created task' logging (#725) 2021-03-23 18:45:18 -04:00
de5c843eb5 work around AAD service principal race condition (#716)
This works around an issue in AAD service principal creation.  The race condition in AAD shows up as:

```
INFO:deploy:checking if RBAC already exists
INFO:deploy:creating Application registration
INFO:deploy:creating service principal
Traceback (most recent call last):
  File "deploy.py", line 926, in <module>
    main()
  File "deploy.py", line 920, in main
    state[1](client)
  File "deploy.py", line 303, in setup_rbac
    client.service_principals.create(service_principal_params)
  File "/tmp/tmpp2x7ybfg/deploy-venv/lib/python3.8/site-packages/azure/graphrbac/operations/service_principals_operations.py", line 87, in create
    raise models.GraphErrorException(self._deserialize, response)
azure.graphrbac.models.graph_error_py3.GraphErrorException: When using this permission, the backing application of the service principal being created must in the local tenant
```

The azure-cli has the same issue, as seen in https://github.com/Azure/azure-cli/issues/14767
2021-03-23 21:02:36 +00:00
3ef7db64c3 Update Windows generic coverage recording (#699)
- Reimplement Windows generic coverage using new coverage format
- Remove old format
- Update and unify examples
2021-03-23 13:07:57 -07:00
7522bfd3ab release 2.10.0 (#720) 2.10.0 2021-03-23 12:56:39 -04:00
e80b658a1a expose new minimized stack fields in the crash report model (#719) 2021-03-23 12:29:08 -04:00
2b61256001 run until end of LLVMFuzzerTestOneInput for collecting coverage (#713) 2021-03-23 11:08:00 -04:00
516b1e000e expose minimized_stack_depth functionality in the CLI/API (#715) 2021-03-23 10:09:34 -04:00
92d8299412 make asan_log optional in CrashReport (#714) 2021-03-23 02:36:20 -04:00
24a20b6c28 automatically retry supervisor requests (#704)
## Summary of the Pull Request

closes #677
2021-03-23 06:08:09 +00:00
3113f887c4 remove generic- from the command prefixes (#710) 2021-03-22 20:39:32 -04:00
b4e0d89329 enable stack minimization for all crash dumps (#705)
This builds upon #591 to expand the stack minimization to crash reporting mechanisms.

Example (see #703 for an example without the new functionality):

```
$ onefuzz-agent local test-input /tmp/fuzz.exe /etc/passwd
{
  "crash_report": {
    "input_sha256": "a35b3ce1038750e9175a6dcd3f64c8d4e85720affb12cc11f5d0b6889274d06e",
    "executable": "/tmp/fuzz.exe",
    "crash_type": "SIGABRT",
    "crash_site": "0x7ffff7e0d18b in gsignal+0xcb (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x4618b)",
    "call_stack": [
      "#0 0x7ffff7e0d18b in gsignal+0xcb (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x4618b)",
      "#1 0x7ffff7dec859 in abort+0x12b (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x25859)",
      "#2 0x7ffff7e573ee in <unknown> (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x903ee)",
      "#3 0x7ffff7ef9b4a in __fortify_fail+0x2a (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x132b4a)",
      "#4 0x7ffff7ef83e6 in __chk_fail+0x16 (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x1313e6)",
      "#5 0x7ffff7ef7e09 in __strncpy_chk+0x19 (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x130e09)",
      "#6 0x400a54 in from_file+0xa4 (/tmp/fuzz.exe+0xa54)",
      "#7 0x7ffff7dee0b3 in __libc_start_main+0xf3 (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x270b3)",
      "#8 0x40077a in _start+0x2a (/tmp/fuzz.exe+0x77a)"
    ],
    "call_stack_sha256": "99625a7c103136e02910b65c7b60f1bbd1a7612242d6838da52d968369039409",
    "minimized_stack": [
      "__fortify_fail",
      "__chk_fail",
      "from_file"
    ],
    "minimized_stack_sha256": "237f13bfa384c6c2bc06369099373efbb36995a9ad00fd5469d354b5fc672ba1",
    "minimized_stack_function_names": [
      "__fortify_fail",
      "__chk_fail",
      "from_file"
    ],
    "minimized_stack_function_names_sha256": "237f13bfa384c6c2bc06369099373efbb36995a9ad00fd5469d354b5fc672ba1",
    "asan_log": "",
    "task_id": "00000000-0000-0000-0000-000000000000",
    "job_id": "00000000-0000-0000-0000-000000000000"
  }
}
$
```
2021-03-23 00:13:28 +00:00
8250007294 Fix parsing of list of synced dir (#711) 2021-03-22 17:04:00 -04:00
16d3f3cf10 show the full error on proxy loop failure (#702) 2021-03-22 19:55:28 +00:00
02c92bfb1f add second resolution to log timestamp and print to stdout logs (#709) 2021-03-22 14:32:49 -04:00
7be4f3bbc1 add a single-shot crash report utility (#703)
Adds `test-input` and `test-input-libfuzzer`, which print the CrashTestResult in json form.

While many of the existing tasks make sense running in a managed loop, crash report generation is something that having a single one-off is useful.

Example:
```
$ onefuzz-agent local test-input /tmp/fuzz.exe /tmp/crash.txt
{
  "crash_report": {
    "input_sha256": "a35b3ce1038750e9175a6dcd3f64c8d4e85720affb12cc11f5d0b6889274d06e",
    "executable": "/tmp/fuzz.exe",
    "crash_type": "SIGABRT",
    "crash_site": "0x7f0d9d4ad18b in gsignal+0xcb (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x4618b)",
    "call_stack": [
      "#0 0x7f0d9d4ad18b in gsignal+0xcb (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x4618b)",
      "#1 0x7f0d9d48c859 in abort+0x12b (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x25859)",
      "#2 0x7f0d9d4f73ee in <unknown> (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x903ee)",
      "#3 0x7f0d9d599b4a in __fortify_fail+0x2a (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x132b4a)",
      "#4 0x7f0d9d5983e6 in __chk_fail+0x16 (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x1313e6)",
      "#5 0x7f0d9d597e09 in __strncpy_chk+0x19 (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x130e09)",
      "#6 0x400a54 in from_file+0xa4 (/tmp/fuzz.exe+0xa54)",
      "#7 0x7f0d9d48e0b3 in __libc_start_main+0xf3 (/usr/lib/x86_64-linux-gnu/libc-2.31.so+0x270b3)",
      "#8 0x40077a in _start+0x2a (/tmp/fuzz.exe+0x77a)"
    ],
    "call_stack_sha256": "6906234fb235690cc2843a1a55f49ff68b424e54bec55f9b8258415d97b3e638",
    "task_id": "00000000-0000-0000-0000-000000000000",
    "job_id": "00000000-0000-0000-0000-000000000000"
  }
}
$
```
2021-03-22 17:46:33 +00:00
cf6c4e5632 add regexes from STACK_FRAME_IGNORE_REGEXES_IF_SYMBOLIZED (#706) 2021-03-22 16:13:41 +00:00
c0d7381427 add clusterfuzz's stack-trace dataset (#707) 2021-03-22 11:52:33 -04:00
71f487506f update pydantic to 1.8.1 (#698) 2021-03-19 18:02:03 -04:00
14b295f337 add privacy statement to CLI (#695) 2021-03-19 12:52:37 -04:00
d57abb6956 include the full failure when we give up from retrying (#696) 2021-03-19 11:37:43 -04:00
e38281face rename salvo telemetry to "upcoming feature" (#693)
Until salvo is released, this should be "upcoming feature" to avoid questions.
2021-03-19 14:59:08 +00:00
769b614367 add git-bisect howto (#691) 2021-03-19 14:26:20 +00:00
2417c37a60 add LoadLibrary integration test (#689) 2021-03-19 09:46:26 -04:00
3d226391fd update azure-mgmt-network to 18.0.0 (#687) 2021-03-19 09:49:42 +00:00