push to stable (#1734)

* afl++ -> AFL++

* update readme

* more debug

* slightly different weighting algo (#1719)

* better seed selection

* slightly different weighting calculation

* remove unnecessary memset

* Add "Hangs saved" to afl-whatsup (#1717)

The hangs could show long or infinite loops. This is important.

Co-authored-by: van Hauser <vh@thc.org>

* nits

* afl-showmap: Start a only a single fork server (#1718)

A forkserver is started by afl_fsrv_get_mapsize() when dynamically
finding the map size.  When an input directory option is specified a
second fork server was also started.  This commit re-arranges the inits
for several forkserver struct members so that we can re-use the server
started by the get_mapsize() call when not in coresight/qemu/unicorn
modes and just start the server otherwise.

* Source Code Coverage support for Nyx (Part 1) (#1720)

* Additional source code reformatting in afl-compiler-rt

* Add source code coverage support to afl-compiler-rt (for use with Nyx)

* doc, code format

* llvm 17 changes

* more llvm 17

* add frida mode tutorial

* fix effector map

* docs

* Should memset EFF_ALEN(len) of eff_map (#1722)

* fix reallocs

* fix afl-system-config for macos

* afl-fuzz.c: Document -i - in --help (#1725)

afl-fuzz.c: Document `-i -` in `--help`, to write that `-i` can be passed '-' to resume the prior fuzzing job. Also reference AFL_AUTORESUME so users know they can set that parameter to sidestep the issue entirely.

* tritondse custom mutator attempt

* tritondse fixes

* update libnyx (#1727)

* GNUmakefile: Update LLVM instructions (#1728)

Update LLVM instructions, because versions higher than 14 are supported and to be explicit that LLD is also required

* disable macos in the ci, works fine for me

* fix makefile

* better tritondse support

* next steps for tritondse

* qemuafl: Persistent mode for PPC32 targets

* update qemu_mode

* afl-clang-lto incomptable with -flto=thin

* add @responsefile support for afl-cc

---------

Co-authored-by: fxlb <devel.fx.lebail@orange.fr>
Co-authored-by: Nick Potenski <nick.potenski@garmin.com>
Co-authored-by: Christian Holler (:decoder) <choller@mozilla.com>
Co-authored-by: lazymio <mio@lazym.io>
Co-authored-by: Moshe Kaplan <me@moshekaplan.com>
Co-authored-by: Sergej Schumilo <sergej@schumilo.de>
Co-authored-by: Dominik Maier <domenukk@gmail.com>

docs/Changelog.md

@@ -11,13 +11,17 @@
     - new env `AFL_IGNORE_PROBLEMS_COVERAGE` to ignore coverage from
       loaded libs after forkserver initialization (required by Mozilla)
   - afl-cc:
+    - added @responsefile support
     - new env `AFL_LLVM_LTO_SKIPINIT` to support the AFL++ based WASM
       (https://github.com/fgsect/WAFL) project
+    - error and print help if afl-clan-lto is used with lto=thin
   - afl-showmap:
     - added custom mutator post_process and send support
     - add `-I filelist` option, an alternative to `-i in_dir`
   - afl-cmin + afl-cmin.bash:
     - `-T threads` parallel task support, can be a huge speedup!
+  - qemu_mode:
+    - Persistent mode +QASAN support for ppc32 tragets by @worksbutnottested
   - a new grammar custom mutator atnwalk was submitted by @voidptr127 !
 
 
@@ -229,7 +233,7 @@
     afl-showmap and other tools.
   - afl-cc:
     - detect overflow reads on initial input buffer for asan
-    - new cmplog mode (incompatible with older afl++ versions)
+    - new cmplog mode (incompatible with older AFL++ versions)
     - support llvm IR select instrumentation for default PCGUARD and LTO
     - fix for shared linking on MacOS
     - better selective instrumentation AFL_LLVM_{ALLOW|DENY}LIST

docs/FAQ.md

@@ -171,6 +171,14 @@ If you find an interesting or important question missing, submit it via
   The more "unstable" edges there are, the harder it is for AFL++ to identify
   valid new paths.
 
+  If you fuzz in persistent mode (`AFL_LOOP` or `LLVMFuzzerTestOneInput()`
+  harnesses, a large number of unstable edges can mean that the target keeps
+  internal state and therefore it is possible that crashes cannot be replayed.
+  In such a case do either **not** fuzz in persistent mode (remove `AFL_LOOP()`
+  from your harness or call `LLVMFuzzerTestOneInput()` harnesses with `@@`),
+  or set a low  `AFL_LOOP` value, e.g. 100, and enable `AFL_PERSISTENT_RECORD`
+  in `config.h` with the same value.
+
   A value above 90% is usually fine and a value above 80% is also still ok, and
   even a value above 20% can still result in successful finds of bugs. However,
   it is recommended that for values below 90% or 80% you should take

docs/INSTALL.md

@@ -51,7 +51,7 @@ make source-only
 
 These build targets exist:
 
-* all: the main afl++ binaries and llvm/gcc instrumentation
+* all: the main AFL++ binaries and llvm/gcc instrumentation
 * binary-only: everything for binary-only fuzzing: frida_mode, nyx_mode,
   qemu_mode, frida_mode, unicorn_mode, coresight_mode, libdislocator,
   libtokencap
@@ -79,22 +79,20 @@ make STATIC=1
 These build options exist:
 
 * STATIC - compile AFL++ static
+* CODE_COVERAGE - compile the target for code coverage (see docs/instrumentation/README.llvm.md)
 * ASAN_BUILD - compiles AFL++ with memory sanitizer for debug purposes
-* UBSAN_BUILD - compiles AFL++ tools with undefined behaviour sanitizer for
-  debug purposes
+* UBSAN_BUILD - compiles AFL++ tools with undefined behaviour sanitizer for debug purposes
 * DEBUG - no optimization, -ggdb3, all warnings and -Werror
 * LLVM_DEBUG - shows llvm deprecation warnings
 * PROFILING - compile afl-fuzz with profiling information
 * INTROSPECTION - compile afl-fuzz with mutation introspection
 * NO_PYTHON - disable python support
-* NO_SPLICING - disables splicing mutation in afl-fuzz, not recommended for
-  normal fuzzing
+* NO_SPLICING - disables splicing mutation in afl-fuzz, not recommended for normal fuzzing
 * NO_NYX - disable building nyx mode dependencies
 * NO_CORESIGHT - disable building coresight (arm64 only)
 * NO_UNICORN_ARM64 - disable building unicorn on arm64
 * AFL_NO_X86 - if compiling on non-intel/amd platforms
-* LLVM_CONFIG - if your distro doesn't use the standard name for llvm-config
-  (e.g., Debian)
+* LLVM_CONFIG - if your distro doesn't use the standard name for llvm-config (e.g., Debian)
 
 e.g.: `make LLVM_CONFIG=llvm-config-14`
 

docs/best_practices.md

@@ -131,6 +131,11 @@ jitter, or is a hash map function etc., then it should not be instrumented.
 To be able to exclude these functions (based on AFL++'s measured stability), the
 following process will allow to identify functions with variable edges.
 
+Note that this is only useful for non-persistent targets!
+If a persistent target is unstable whereas when run non-persistent is fine,
+then this means that the target is keeping internal state, which is bad for
+fuzzing. Fuzz such targets **without** persistent mode.
+
 Four steps are required to do this and it also requires quite some knowledge of
 coding and/or disassembly and is effectively possible only with `afl-clang-fast`
 `PCGUARD` and `afl-clang-lto` `LTO` instrumentation.

docs/custom_mutators.md

@@ -304,6 +304,34 @@ Note: for some distributions, you might also need the package `python[3]-apt`.
 In case your setup is different, set the necessary variables like this:
 `PYTHON_INCLUDE=/path/to/python/include LDFLAGS=-L/path/to/python/lib make`.
 
+### Helpers
+
+For C/C++ custom mutators you get a pointer to `afl_state_t *afl` in the
+`afl_custom_init()` which contains all information that you need.
+Note that if you access it, you need to recompile your custom mutator if
+you update AFL++ because the structure might have changed!
+
+For mutators written in Python, Rust, GO, etc. there are a few environment
+variables set to help you to get started:
+
+`AFL_CUSTOM_INFO_PROGRAM` - the program name of the target that is executed.
+If your custom mutator is used with modes like Qemu (`-Q`), this will still
+contain the target program, not afl-qemu-trace.
+
+`AFL_CUSTOM_INFO_PROGRAM_INPUT` - if the `-f` parameter is used with afl-fuzz
+then this value is found in this environment variable.
+
+`AFL_CUSTOM_INFO_PROGRAM_ARGV` - this contains the parameters given to the
+target program and still has the `@@` identifier in there.
+
+Note: If `AFL_CUSTOM_INFO_PROGRAM_INPUT` is empty and `AFL_CUSTOM_INFO_PROGRAM_ARGV`
+is either empty or does not contain `@@` then the target gets the input via
+`stdin`.
+
+`AFL_CUSTOM_INFO_OUT` - This is the output directory for this fuzzer instance,
+so if `afl-fuzz` was called with `-o out -S foobar`, then this will be set to
+`out/foobar`.
+
 ### Custom Mutator Preparation
 
 For C/C++ mutators, the source code must be compiled as a shared object:

docs/tutorials.md

@@ -20,6 +20,10 @@ training, then we can highly recommend the following:
 
 * [https://github.com/antonio-morales/Fuzzing101](https://github.com/antonio-morales/Fuzzing101)
 
+Here is good workflow description for frida_mode:
+
+* [https://blog.quarkslab.com/android-greybox-fuzzing-with-afl-frida-mode.html](https://blog.quarkslab.com/android-greybox-fuzzing-with-afl-frida-mode.html)
+
 If you are interested in fuzzing structured data (where you define what the
 structure is), these links have you covered (some are outdated though):