ExternalVendorCode/AFLplusplus
1
0
Fork 0
mirror of https://github.com/AFLplusplus/AFLplusplus.git synced 2025-06-09 00:31:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

Clean up docs folder

Browse Source
This commit is contained in:
llzmb 2021-11-21 18:00:01 +01:00
parent 36514a2e4f
commit 492dbe9fb2
5 changed files with 109 additions and 106 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
README.md
View File

@ -132,9 +132,6 @@ The following branches exist:
* [dev](https://github.com/AFLplusplus/AFLplusplus/tree/dev): development state of AFL++ - bleeding edge and you might catch a checkout which does not compile or has a bug. *We only accept PRs in dev!!* * [dev](https://github.com/AFLplusplus/AFLplusplus/tree/dev): development state of AFL++ - bleeding edge and you might catch a checkout which does not compile or has a bug. *We only accept PRs in dev!!*
* (any other): experimental branches to work on specific features or testing new functionality or changes. * (any other): experimental branches to work on specific features or testing new functionality or changes.
For releases, please see the [Releases tab](https://github.com/AFLplusplus/AFLplusplus/releases).
Also take a look at the list of [important changes in AFL++](docs/important_changes.md).
## Help wanted ## Help wanted
We have several [ideas](docs/ideas.md) we would like to see in AFL++ to make it We have several [ideas](docs/ideas.md) we would like to see in AFL++ to make it
@ -233,4 +230,4 @@ presented at WOOT'20:
} }
``` ```
</details> </details>

5
docs/FAQ.md
View File

@ -83,7 +83,8 @@ If you find an interesting or important question missing, submit it via
However, if there is only the binary program and no source code available, then the standard non-instrumented mode is not effective. However, if there is only the binary program and no source code available, then the standard non-instrumented mode is not effective.
To learn how these binaries can be fuzzed, read [binaryonly_fuzzing.md](binaryonly_fuzzing.md). To learn how these binaries can be fuzzed, read
[fuzzing_binary-only_targets.md](fuzzing_binary-only_targets.md).
</p></details> </p></details>
<details> <details>
@ -143,7 +144,7 @@ If you find an interesting or important question missing, submit it via
Target: x86_64-unknown-linux-gnu Target: x86_64-unknown-linux-gnu
Thread model: posix Thread model: posix
InstalledDir: /prg/tmp/llvm-project/build/bin InstalledDir: /prg/tmp/llvm-project/build/bin
clang-13: note: diagnostic msg: clang-13: note: diagnostic msg:
******************** ********************
``` ```

18
docs/best_practices.md
View File

@ -4,20 +4,26 @@
### Targets ### Targets
* [Fuzzing a binary-only target](#fuzzing-a-binary-only-target) * [Fuzzing a target with source code available](#fuzzing-a-target-with-source-code-available)
* [Fuzzing a GUI program](#fuzzing-a-gui-program) * [Fuzzing a binary-only target](#fuzzing-a-binary-only-target)
* [Fuzzing a network service](#fuzzing-a-network-service) * [Fuzzing a GUI program](#fuzzing-a-gui-program)
* [Fuzzing a network service](#fuzzing-a-network-service)
### Improvements ### Improvements
* [Improving speed](#improving-speed) * [Improving speed](#improving-speed)
* [Improving stability](#improving-stability) * [Improving stability](#improving-stability)
## Targets ## Targets
### Fuzzing a target with source code available
To learn how to fuzz a target if source code is available, see [fuzzing_in_depth.md](fuzzing_in_depth.md).
### Fuzzing a binary-only target ### Fuzzing a binary-only target
For a comprehensive guide, see [binaryonly_fuzzing.md](binaryonly_fuzzing.md). For a comprehensive guide, see
[fuzzing_binary-only_targets.md](fuzzing_binary-only_targets.md).
### Fuzzing a GUI program ### Fuzzing a GUI program

182
docs/parallel_fuzzing.md
View File

@ -1,28 +1,28 @@
# Tips for parallel fuzzing # Tips for parallel fuzzing
This document talks about synchronizing afl-fuzz jobs on a single machine This document talks about synchronizing afl-fuzz jobs on a single machine or
or across a fleet of systems. See README.md for the general instruction manual. across a fleet of systems. See README.md for the general instruction manual.
Note that this document is rather outdated. please refer to the main document Note that this document is rather outdated. please refer to the main document
section on multiple core usage [fuzzing_expert.md#Using multiple cores](fuzzing_expert.md#b-using-multiple-cores) section on multiple core usage
[fuzzing_in_depth.md:b) Using multiple cores](fuzzing_in_depth.md#b-using-multiple-cores)
for up to date strategies! for up to date strategies!
## 1) Introduction ## 1) Introduction
Every copy of afl-fuzz will take up one CPU core. This means that on an Every copy of afl-fuzz will take up one CPU core. This means that on an n-core
n-core system, you can almost always run around n concurrent fuzzing jobs with system, you can almost always run around n concurrent fuzzing jobs with
virtually no performance hit (you can use the afl-gotcpu tool to make sure). virtually no performance hit (you can use the afl-gotcpu tool to make sure).
In fact, if you rely on just a single job on a multi-core system, you will In fact, if you rely on just a single job on a multi-core system, you will be
be underutilizing the hardware. So, parallelization is always the right way to underutilizing the hardware. So, parallelization is always the right way to go.
go.
When targeting multiple unrelated binaries or using the tool in When targeting multiple unrelated binaries or using the tool in
"non-instrumented" (-n) mode, it is perfectly fine to just start up several "non-instrumented" (-n) mode, it is perfectly fine to just start up several
fully separate instances of afl-fuzz. The picture gets more complicated when fully separate instances of afl-fuzz. The picture gets more complicated when you
you want to have multiple fuzzers hammering a common target: if a hard-to-hit want to have multiple fuzzers hammering a common target: if a hard-to-hit but
but interesting test case is synthesized by one fuzzer, the remaining instances interesting test case is synthesized by one fuzzer, the remaining instances will
will not be able to use that input to guide their work. not be able to use that input to guide their work.
To help with this problem, afl-fuzz offers a simple way to synchronize test To help with this problem, afl-fuzz offers a simple way to synchronize test
cases on the fly. cases on the fly.
@ -30,15 +30,15 @@ cases on the fly.
It is a good idea to use different power schedules if you run several instances It is a good idea to use different power schedules if you run several instances
in parallel (`-p` option). in parallel (`-p` option).
Alternatively running other AFL spinoffs in parallel can be of value, Alternatively running other AFL spinoffs in parallel can be of value, e.g.
e.g. Angora (https://github.com/AngoraFuzzer/Angora/) Angora (https://github.com/AngoraFuzzer/Angora/)
## 2) Single-system parallelization ## 2) Single-system parallelization
If you wish to parallelize a single job across multiple cores on a local If you wish to parallelize a single job across multiple cores on a local system,
system, simply create a new, empty output directory ("sync dir") that will be simply create a new, empty output directory ("sync dir") that will be shared by
shared by all the instances of afl-fuzz; and then come up with a naming scheme all the instances of afl-fuzz; and then come up with a naming scheme for every
for every instance - say, "fuzzer01", "fuzzer02", etc. instance - say, "fuzzer01", "fuzzer02", etc.
Run the first one ("main node", -M) like this: Run the first one ("main node", -M) like this:
@ -57,18 +57,18 @@ Each fuzzer will keep its state in a separate subdirectory, like so:
/path/to/sync_dir/fuzzer01/ /path/to/sync_dir/fuzzer01/
Each instance will also periodically rescan the top-level sync directory Each instance will also periodically rescan the top-level sync directory for any
for any test cases found by other fuzzers - and will incorporate them into test cases found by other fuzzers - and will incorporate them into its own
its own fuzzing when they are deemed interesting enough. fuzzing when they are deemed interesting enough. For performance reasons only -M
For performance reasons only -M main node syncs the queue with everyone, the main node syncs the queue with everyone, the -S secondary nodes will only sync
-S secondary nodes will only sync from the main node. from the main node.
The difference between the -M and -S modes is that the main instance will The difference between the -M and -S modes is that the main instance will still
still perform deterministic checks; while the secondary instances will perform deterministic checks; while the secondary instances will proceed
proceed straight to random tweaks. straight to random tweaks.
Note that you must always have one -M main instance! Note that you must always have one -M main instance! Running multiple -M
Running multiple -M instances is wasteful! instances is wasteful!
You can also monitor the progress of your jobs from the command line with the You can also monitor the progress of your jobs from the command line with the
provided afl-whatsup tool. When the instances are no longer finding new paths, provided afl-whatsup tool. When the instances are no longer finding new paths,
@ -90,18 +90,18 @@ file name.
## 3) Multiple -M mains ## 3) Multiple -M mains
There is support for parallelizing the deterministic checks. There is support for parallelizing the deterministic checks. This is only needed
This is only needed where where
1. many new paths are found fast over a long time and it looks unlikely that 1. many new paths are found fast over a long time and it looks unlikely that
main node will ever catch up, and main node will ever catch up, and
2. deterministic fuzzing is actively helping path discovery (you can see this 2. deterministic fuzzing is actively helping path discovery (you can see this
in the main node for the first for lines in the "fuzzing strategy yields" in the main node for the first for lines in the "fuzzing strategy yields"
section. If the ration `found/attemps` is high, then it is effective. It section. If the ration `found/attempts` is high, then it is effective. It
most commonly isn't.) most commonly isn't.)
Only if both are true it is beneficial to have more than one main. Only if both are true it is beneficial to have more than one main. You can
You can leverage this by creating -M instances like so: leverage this by creating -M instances like so:
``` ```
./afl-fuzz -i testcase_dir -o sync_dir -M mainA:1/3 [...] ./afl-fuzz -i testcase_dir -o sync_dir -M mainA:1/3 [...]
@ -115,27 +115,26 @@ distribute the deterministic fuzzing across. Note that if you boot up fewer
fuzzers than indicated by the second number passed to -M, you may end up with fuzzers than indicated by the second number passed to -M, you may end up with
poor coverage. poor coverage.
## 4) Syncing with non-AFL fuzzers or independant instances ## 4) Syncing with non-AFL fuzzers or independent instances
A -M main node can be told with the `-F other_fuzzer_queue_directory` option A -M main node can be told with the `-F other_fuzzer_queue_directory` option to
to sync results from other fuzzers, e.g. libfuzzer or honggfuzz. sync results from other fuzzers, e.g. libfuzzer or honggfuzz.
Only the specified directory will by synced into afl, not subdirectories. Only the specified directory will by synced into afl, not subdirectories. The
The specified directory does not need to exist yet at the start of afl. specified directory does not need to exist yet at the start of afl.
The `-F` option can be passed to the main node several times. The `-F` option can be passed to the main node several times.
## 5) Multi-system parallelization ## 5) Multi-system parallelization
The basic operating principle for multi-system parallelization is similar to The basic operating principle for multi-system parallelization is similar to the
the mechanism explained in section 2. The key difference is that you need to mechanism explained in section 2. The key difference is that you need to write a
write a simple script that performs two actions: simple script that performs two actions:
- Uses SSH with authorized_keys to connect to every machine and retrieve - Uses SSH with authorized_keys to connect to every machine and retrieve a tar
a tar archive of the /path/to/sync_dir/<main_node(s)> directory local to archive of the /path/to/sync_dir/<main_node(s)> directory local to the
the machine. machine. It is best to use a naming scheme that includes host name and it's
It is best to use a naming scheme that includes host name and it's being being a main node (e.g. main1, main2) in the fuzzer ID, so that you can do
a main node (e.g. main1, main2) in the fuzzer ID, so that you can do
something like: something like:
```sh ```sh
@ -163,70 +162,70 @@ There are other (older) more featured, experimental tools:
However these do not support syncing just main nodes (yet). However these do not support syncing just main nodes (yet).
When developing custom test case sync code, there are several optimizations When developing custom test case sync code, there are several optimizations to
to keep in mind: keep in mind:
- The synchronization does not have to happen very often; running the - The synchronization does not have to happen very often; running the task
task every 60 minutes or even less often at later fuzzing stages is every 60 minutes or even less often at later fuzzing stages is fine
fine
- There is no need to synchronize crashes/ or hangs/; you only need to - There is no need to synchronize crashes/ or hangs/; you only need to copy
copy over queue/* (and ideally, also fuzzer_stats). over queue/* (and ideally, also fuzzer_stats).
- It is not necessary (and not advisable!) to overwrite existing files; - It is not necessary (and not advisable!) to overwrite existing files; the -k
the -k option in tar is a good way to avoid that. option in tar is a good way to avoid that.
- There is no need to fetch directories for fuzzers that are not running - There is no need to fetch directories for fuzzers that are not running
locally on a particular machine, and were simply copied over onto that locally on a particular machine, and were simply copied over onto that
system during earlier runs. system during earlier runs.
- For large fleets, you will want to consolidate tarballs for each host, - For large fleets, you will want to consolidate tarballs for each host, as
as this will let you use n SSH connections for sync, rather than n*(n-1). this will let you use n SSH connections for sync, rather than n*(n-1).
You may also want to implement staged synchronization. For example, you You may also want to implement staged synchronization. For example, you
could have 10 groups of systems, with group 1 pushing test cases only could have 10 groups of systems, with group 1 pushing test cases only to
to group 2; group 2 pushing them only to group 3; and so on, with group group 2; group 2 pushing them only to group 3; and so on, with group
eventually 10 feeding back to group 1. eventually 10 feeding back to group 1.
This arrangement would allow test interesting cases to propagate across This arrangement would allow test interesting cases to propagate across the
the fleet without having to copy every fuzzer queue to every single host. fleet without having to copy every fuzzer queue to every single host.
- You do not want a "main" instance of afl-fuzz on every system; you should - You do not want a "main" instance of afl-fuzz on every system; you should
run them all with -S, and just designate a single process somewhere within run them all with -S, and just designate a single process somewhere within
the fleet to run with -M. the fleet to run with -M.
- Syncing is only necessary for the main nodes on a system. It is possible - Syncing is only necessary for the main nodes on a system. It is possible to
to run main-less with only secondaries. However then you need to find out run main-less with only secondaries. However then you need to find out which
which secondary took over the temporary role to be the main node. Look for secondary took over the temporary role to be the main node. Look for the
the `is_main_node` file in the fuzzer directories, eg. `sync-dir/hostname-*/is_main_node` `is_main_node` file in the fuzzer directories, eg.
`sync-dir/hostname-*/is_main_node`
It is *not* advisable to skip the synchronization script and run the fuzzers It is *not* advisable to skip the synchronization script and run the fuzzers
directly on a network filesystem; unexpected latency and unkillable processes directly on a network filesystem; unexpected latency and unkillable processes in
in I/O wait state can mess things up. I/O wait state can mess things up.
## 6) Remote monitoring and data collection ## 6) Remote monitoring and data collection
You can use screen, nohup, tmux, or something equivalent to run remote You can use screen, nohup, tmux, or something equivalent to run remote instances
instances of afl-fuzz. If you redirect the program's output to a file, it will of afl-fuzz. If you redirect the program's output to a file, it will
automatically switch from a fancy UI to more limited status reports. There is automatically switch from a fancy UI to more limited status reports. There is
also basic machine-readable information which is always written to the also basic machine-readable information which is always written to the
fuzzer_stats file in the output directory. Locally, that information can be fuzzer_stats file in the output directory. Locally, that information can be
interpreted with afl-whatsup. interpreted with afl-whatsup.
In principle, you can use the status screen of the main (-M) instance to In principle, you can use the status screen of the main (-M) instance to monitor
monitor the overall fuzzing progress and decide when to stop. In this the overall fuzzing progress and decide when to stop. In this mode, the most
mode, the most important signal is just that no new paths are being found important signal is just that no new paths are being found for a longer while.
for a longer while. If you do not have a main instance, just pick any If you do not have a main instance, just pick any single secondary instance to
single secondary instance to watch and go by that. watch and go by that.
You can also rely on that instance's output directory to collect the You can also rely on that instance's output directory to collect the synthesized
synthesized corpus that covers all the noteworthy paths discovered anywhere corpus that covers all the noteworthy paths discovered anywhere within the
within the fleet. Secondary (-S) instances do not require any special fleet. Secondary (-S) instances do not require any special monitoring, other
monitoring, other than just making sure that they are up. than just making sure that they are up.
Keep in mind that crashing inputs are *not* automatically propagated to the Keep in mind that crashing inputs are *not* automatically propagated to the main
main instance, so you may still want to monitor for crashes fleet-wide instance, so you may still want to monitor for crashes fleet-wide from within
from within your synchronization or health checking scripts (see afl-whatsup). your synchronization or health checking scripts (see afl-whatsup).
## 7) Asymmetric setups ## 7) Asymmetric setups
@ -238,21 +237,20 @@ It is perhaps worth noting that all of the following is permitted:
out_dir/<fuzzer_id>/queue/* and writing their own finds to sequentially out_dir/<fuzzer_id>/queue/* and writing their own finds to sequentially
numbered id:nnnnnn files in out_dir/<ext_tool_id>/queue/*. numbered id:nnnnnn files in out_dir/<ext_tool_id>/queue/*.
- Running some of the synchronized fuzzers with different (but related) - Running some of the synchronized fuzzers with different (but related) target
target binaries. For example, simultaneously stress-testing several binaries. For example, simultaneously stress-testing several different JPEG
different JPEG parsers (say, IJG jpeg and libjpeg-turbo) while sharing parsers (say, IJG jpeg and libjpeg-turbo) while sharing the discovered test
the discovered test cases can have synergistic effects and improve the cases can have synergistic effects and improve the overall coverage.
overall coverage.
(In this case, running one -M instance per target is necessary.) (In this case, running one -M instance per target is necessary.)
- Having some of the fuzzers invoke the binary in different ways. - Having some of the fuzzers invoke the binary in different ways. For example,
For example, 'djpeg' supports several DCT modes, configurable with 'djpeg' supports several DCT modes, configurable with a command-line flag,
a command-line flag, while 'dwebp' supports incremental and one-shot while 'dwebp' supports incremental and one-shot decoding. In some scenarios,
decoding. In some scenarios, going after multiple distinct modes and then going after multiple distinct modes and then pooling test cases will improve
pooling test cases will improve coverage. coverage.
- Much less convincingly, running the synchronized fuzzers with different - Much less convincingly, running the synchronized fuzzers with different
starting test cases (e.g., progressive and standard JPEG) or dictionaries. starting test cases (e.g., progressive and standard JPEG) or dictionaries.
The synchronization mechanism ensures that the test sets will get fairly The synchronization mechanism ensures that the test sets will get fairly
homogeneous over time, but it introduces some initial variability. homogeneous over time, but it introduces some initial variability.

5
qemu_mode/README.md
View File

@ -217,5 +217,6 @@ them at run time, can be a faster alternative. That said, static rewriting is
fraught with peril, because it depends on being able to properly and fully model fraught with peril, because it depends on being able to properly and fully model
program control flow without actually executing each and every code path. program control flow without actually executing each and every code path.
Checkout the "Fuzzing binary-only targets" section in our main README.md and Check out
the docs/binaryonly_fuzzing.md document for more information and hints. [docs/fuzzing_binary-only_targets.md](../docs/fuzzing_binary-only_targets.md)
for more information and hints.