docs to md

This commit is contained in:
Andrea Fioraldi
2020-02-03 13:02:16 +01:00
parent fd03c64346
commit e2eedefc65
14 changed files with 32 additions and 27 deletions

1
QuickStartGuide.md Symbolic link
View File

@ -0,0 +1 @@
docs/QuickStartGuide.md

View File

@ -1 +0,0 @@
docs/QuickStartGuide.txt

View File

@ -433,11 +433,11 @@ see [http://lcamtuf.coredump.cx/afl/plot/](http://lcamtuf.coredump.cx/afl/plot/)
Every instance of afl-fuzz takes up roughly one core. This means that on
multi-core systems, parallelization is necessary to fully utilize the hardware.
For tips on how to fuzz a common target on multiple cores or multiple networked
machines, please refer to [docs/parallel_fuzzing.txt](docs/parallel_fuzzing.txt).
machines, please refer to [docs/parallel_fuzzing.md](docs/parallel_fuzzing.md).
The parallel fuzzing mode also offers a simple way for interfacing AFL to other
fuzzers, to symbolic or concolic execution engines, and so forth; again, see the
last section of [docs/parallel_fuzzing.txt](docs/parallel_fuzzing.txt) for tips.
last section of [docs/parallel_fuzzing.md](docs/parallel_fuzzing.md) for tips.
## 10) Fuzzer dictionaries

View File

@ -45,7 +45,7 @@ if [ "$DIR" = "" ]; then
echo "Usage: $0 [ -s ] afl_sync_dir" 1>&2
echo 1>&2
echo "The -s option causes the tool to skip all the per-fuzzer trivia and show" 1>&2
echo "just the summary results. See docs/parallel_fuzzing.txt for additional tips." 1>&2
echo "just the summary results. See docs/parallel_fuzzing.md for additional tips." 1>&2
echo 1>&2
exit 1

View File

@ -51,4 +51,4 @@ following files:
- README.md - A general introduction to AFL,
- docs/perf_tips.txt - Simple tips on how to fuzz more quickly,
- docs/status_screen.txt - An explanation of the tidbits shown in the UI,
- docs/parallel_fuzzing.txt - Advice on running AFL on multiple cores.
- docs/parallel_fuzzing.md - Advice on running AFL on multiple cores.

View File

@ -14,7 +14,7 @@ See dictionaries/README.dictionaries to learn how.
%
You can get the most out of your hardware by parallelizing AFL jobs.
See docs/parallel_fuzzing.txt for step-by-step tips.
See docs/parallel_fuzzing.md for step-by-step tips.
%

View File

@ -1,12 +1,9 @@
=========================
Tips for parallel fuzzing
=========================
# Tips for parallel fuzzing
This document talks about synchronizing afl-fuzz jobs on a single machine
or across a fleet of systems. See README for the general instruction manual.
1) Introduction
---------------
## 1) Introduction
Every copy of afl-fuzz will take up one CPU core. This means that on an
n-core system, you can almost always run around n concurrent fuzzing jobs with
@ -33,8 +30,7 @@ several instances in parallel. See docs/power_schedules.txt
Alternatively running other AFL spinoffs in parallel can be of value,
e.g. Angora (https://github.com/AngoraFuzzer/Angora/)
2) Single-system parallelization
--------------------------------
## 2) Single-system parallelization
If you wish to parallelize a single job across multiple cores on a local
system, simply create a new, empty output directory ("sync dir") that will be
@ -43,12 +39,16 @@ for every instance - say, "fuzzer01", "fuzzer02", etc.
Run the first one ("master", -M) like this:
```
$ ./afl-fuzz -i testcase_dir -o sync_dir -M fuzzer01 [...other stuff...]
```
...and then, start up secondary (-S) instances like this:
```
$ ./afl-fuzz -i testcase_dir -o sync_dir -S fuzzer02 [...other stuff...]
$ ./afl-fuzz -i testcase_dir -o sync_dir -S fuzzer03 [...other stuff...]
```
Each fuzzer will keep its state in a separate subdirectory, like so:
@ -68,9 +68,11 @@ Note that running multiple -M instances is wasteful, although there is an
experimental support for parallelizing the deterministic checks. To leverage
that, you need to create -M instances like so:
```
$ ./afl-fuzz -i testcase_dir -o sync_dir -M masterA:1/3 [...]
$ ./afl-fuzz -i testcase_dir -o sync_dir -M masterB:2/3 [...]
$ ./afl-fuzz -i testcase_dir -o sync_dir -M masterC:3/3 [...]
```
...where the first value after ':' is the sequential ID of a particular master
instance (starting at 1), and the second value is the total number of fuzzers to
@ -86,15 +88,16 @@ WARNING: Exercise caution when explicitly specifying the -f option. Each fuzzer
must use a separate temporary file; otherwise, things will go south. One safe
example may be:
```
$ ./afl-fuzz [...] -S fuzzer10 -f file10.txt ./fuzzed/binary @@
$ ./afl-fuzz [...] -S fuzzer11 -f file11.txt ./fuzzed/binary @@
$ ./afl-fuzz [...] -S fuzzer12 -f file12.txt ./fuzzed/binary @@
```
This is not a concern if you use @@ without -f and let afl-fuzz come up with the
file name.
3) Multi-system parallelization
-------------------------------
## 3) Multi-system parallelization
The basic operating principle for multi-system parallelization is similar to
the mechanism explained in section 2. The key difference is that you need to
@ -106,18 +109,22 @@ write a simple script that performs two actions:
that includes host name in the fuzzer ID, so that you can do something
like:
```sh
for s in {1..10}; do
ssh user@host${s} "tar -czf - sync/host${s}_fuzzid*/[qf]*" >host${s}.tgz
done
```
- Distributes and unpacks these files on all the remaining machines, e.g.:
```sh
for s in {1..10}; do
for d in {1..10}; do
test "$s" = "$d" && continue
ssh user@host${d} 'tar -kxzf -' <host${s}.tgz
done
done
```
There is an example of such a script in experimental/distributed_fuzzing/;
you can also find a more featured, experimental tool developed by
@ -167,8 +174,7 @@ It is *not* advisable to skip the synchronization script and run the fuzzers
directly on a network filesystem; unexpected latency and unkillable processes
in I/O wait state can mess things up.
4) Remote monitoring and data collection
----------------------------------------
## 4) Remote monitoring and data collection
You can use screen, nohup, tmux, or something equivalent to run remote
instances of afl-fuzz. If you redirect the program's output to a file, it will
@ -192,8 +198,7 @@ Keep in mind that crashing inputs are *not* automatically propagated to the
master instance, so you may still want to monitor for crashes fleet-wide
from within your synchronization or health checking scripts (see afl-whatsup).
5) Asymmetric setups
--------------------
## 5) Asymmetric setups
It is perhaps worth noting that all of the following is permitted:

View File

@ -140,7 +140,7 @@ options to use non-instrumented system-wide copies instead.
The fuzzer is designed to need ~1 core per job. This means that on a, say,
4-core system, you can easily run four parallel fuzzing jobs with relatively
little performance hit. For tips on how to do that, see parallel_fuzzing.txt.
little performance hit. For tips on how to do that, see parallel_fuzzing.md.
The afl-gotcpu utility can help you understand if you still have idle CPU
capacity on your system. (It won't tell you about memory bandwidth, cache

View File

@ -218,7 +218,7 @@ now. It tells you about the current stage, which can be any of:
splices together two random inputs from the queue at some arbitrarily
selected midpoint.
- sync - a stage used only when -M or -S is set (see parallel_fuzzing.txt).
- sync - a stage used only when -M or -S is set (see parallel_fuzzing.md).
No real fuzzing is involved, but the tool scans the output from other
fuzzers and imports test cases as necessary. The first time this is done,
it may take several minutes or so.
@ -370,7 +370,7 @@ comparing it to the number of logical cores on the system.
If the value is shown in green, you are using fewer CPU cores than available on
your system and can probably parallelize to improve performance; for tips on
how to do that, see parallel_fuzzing.txt.
how to do that, see parallel_fuzzing.md.
If the value is shown in red, your CPU is *possibly* oversubscribed, and
running additional fuzzers may not give you any benefits.

View File

@ -485,7 +485,7 @@ This allows for extreme flexibility in fuzzer setup, including running synced
instances against different parsers of a common data format, often with
synergistic effects.
For more information about this design, see parallel_fuzzing.txt.
For more information about this design, see parallel_fuzzing.md.
12) Binary-only instrumentation
-------------------------------

View File

@ -20,7 +20,7 @@ Here's a quick overview of the stuff you can find in this directory:
with additional gdb metadata.
- distributed_fuzzing - a sample script for synchronizing fuzzer instances
across multiple machines (see parallel_fuzzing.txt).
across multiple machines (see parallel_fuzzing.md).
- libpng_no_checksum - a sample patch for removing CRC checks in libpng.

View File

@ -1719,7 +1719,7 @@ void get_core_count(void) {
} else if (cur_runnable + 1 <= cpu_core_count) {
OKF("Try parallel jobs - see %s/parallel_fuzzing.txt.", doc_path);
OKF("Try parallel jobs - see %s/parallel_fuzzing.md.", doc_path);
}

View File

@ -133,7 +133,7 @@ static void usage(u8* argv0) {
"Other stuff:\n"
" -T text - text banner to show on the screen\n"
" -M / -S id - distributed mode (see parallel_fuzzing.txt)\n"
" -M / -S id - distributed mode (see parallel_fuzzing.md)\n"
" -I command - execute this command/script when a new crash is "
"found\n"
" -B bitmap.txt - mutate a specific test case, use the out/fuzz_bitmap "

View File

@ -19,7 +19,7 @@
This tool provides a fairly accurate measurement of CPU preemption rate.
It is meant to complement the quick-and-dirty load average widget shown
in the afl-fuzz UI. See docs/parallel_fuzzing.txt for more info.
in the afl-fuzz UI. See docs/parallel_fuzzing.md for more info.
For some work loads, the tool may actually suggest running more instances
than you have CPU cores. This can happen if the tested program is spending