Capture crash dumps from libfuzzer, when provided (#2793)

Enables capturing crashdumps generated by ASAN at point of failure.

This helps in several ways:
- provides a crash dump in the case that we cannot repro a failure later
- some people have stated that crash dumps would be more useful to their team than the repro VM
- we should be able to use these for automated submission to Watson or similar

---

Crash dumps are automatically collected (for libfuzzer) jobs, if we find any. They should be activated by enabling crash dumps in ASAN, via:

- On Linux: `ASAN_OPTIONS=disable_coredump=0:abort_on_error=1:unmap_shadow_on_exit=1`
  - OneFuzz will override the (Ubuntu) crash dump options to generate core dumps instead and then upload them.
- On Windows: `ASAN_SAVE_DUMPS=my_dump.dmp`
  - OneFuzz will look for any `*.dmp` files in the working directory and then upload them.

In both cases, the crash dump will be renamed to match the crashing input, if possible, and uploaded to a new `crashdumps` container.

---

Also updated: the “simple” LibFuzzer test has been updated to be compiled with `cl.exe` instead of `clang` on Windows, so that we are exercising the MSVC implementation of ASAN/LibFuzzer, and the CI image has been updated to `windows-2022`. The restriction to an old version of the Windows SDK has been removed.
This commit is contained in:
George Pollard
2023-08-10 09:55:27 +12:00
committed by GitHub
parent 74ae105074
commit a364051923
21 changed files with 437 additions and 168 deletions

View File

@ -1087,11 +1087,10 @@ class Tasks(Endpoint):
if tags is None:
tags = {}
containers_submit = []
for container_type, container in containers:
containers_submit.append(
models.TaskContainers(name=container, type=container_type)
)
containers_submit = [
models.TaskContainers(name=container, type=container_type)
for container_type, container in containers
]
config = models.TaskConfig(
containers=containers_submit,
@ -1210,6 +1209,7 @@ class JobContainers(Endpoint):
) -> None:
SAFE_TO_REMOVE = [
enums.ContainerType.crashes,
enums.ContainerType.crashdumps,
enums.ContainerType.setup,
enums.ContainerType.inputs,
enums.ContainerType.reports,

View File

@ -45,6 +45,7 @@ class AFL(Command):
supervisor_env: Optional[Dict[str, str]] = None,
supervisor_input_marker: str = "@@",
tags: Optional[Dict[str, str]] = None,
target_env: Optional[Dict[str, str]] = None,
wait_for_running: bool = False,
wait_for_files: Optional[List[ContainerType]] = None,
afl_container: Optional[Container] = None,
@ -162,6 +163,7 @@ class AFL(Command):
stats_format=StatsFormat.AFL,
task_wait_for_files=ContainerType.inputs,
tags=helper.tags,
target_env=target_env,
debug=debug,
ensemble_sync_delay=ensemble_sync_delay,
)

View File

@ -132,6 +132,7 @@ class Libfuzzer(Command):
fuzzer_containers = [
(ContainerType.setup, containers[ContainerType.setup]),
(ContainerType.crashes, containers[ContainerType.crashes]),
(ContainerType.crashdumps, containers[ContainerType.crashdumps]),
(ContainerType.inputs, containers[ContainerType.inputs]),
]
@ -416,6 +417,7 @@ class Libfuzzer(Command):
ContainerType.setup,
ContainerType.inputs,
ContainerType.crashes,
ContainerType.crashdumps,
ContainerType.reports,
ContainerType.unique_reports,
ContainerType.unique_inputs,
@ -726,6 +728,7 @@ class Libfuzzer(Command):
ContainerType.setup,
ContainerType.inputs,
ContainerType.crashes,
ContainerType.crashdumps,
ContainerType.coverage,
ContainerType.reports,
ContainerType.unique_reports,
@ -753,6 +756,7 @@ class Libfuzzer(Command):
fuzzer_containers = [
(ContainerType.setup, containers[ContainerType.setup]),
(ContainerType.crashes, containers[ContainerType.crashes]),
(ContainerType.crashdumps, containers[ContainerType.crashdumps]),
(ContainerType.inputs, containers[ContainerType.inputs]),
(ContainerType.tools, fuzzer_tools_container),
]
@ -960,6 +964,7 @@ class Libfuzzer(Command):
ContainerType.setup,
ContainerType.inputs,
ContainerType.crashes,
ContainerType.crashdumps,
ContainerType.reports,
ContainerType.unique_reports,
ContainerType.no_repro,
@ -978,6 +983,7 @@ class Libfuzzer(Command):
fuzzer_containers = [
(ContainerType.setup, helper.containers[ContainerType.setup]),
(ContainerType.crashes, helper.containers[ContainerType.crashes]),
(ContainerType.crashdumps, helper.containers[ContainerType.crashdumps]),
(ContainerType.inputs, helper.containers[ContainerType.inputs]),
]