Dev (#949)

* use atomic read-modify-write increment for LLVM CLASSIC

* Change other LLVM modes to atomic increments

* sync (#886)

* Create FUNDING.yml

* Update FUNDING.yml

* moved custom_mutator examples

* unicorn speedtest makefile cleanup

* fixed example location

* fix qdbi

* update util readme

* work in progress: not working correctly yet

* Frida persistent (#880)

* Added x64 support for persistent mode (function call only), in-memory teest cases and complog

* Review changes, fix NeverZero and code to parse the .text section of the main executable. Excluded ranges TBC

* Various minor fixes and finished support for AFL_INST_LIBS

* Review changes

Co-authored-by: Your Name <you@example.com>

* nits

* fix frida mode

* Integer overflow/underflow fixes in libdislocator (#889)

* libdislocator: fixing integer overflow in 'max_mem' variable and setting 'max_mem' type to 'size_t'

* libdislocator: fixing potential integer underflow in 'total_mem' variable due to its different values in different threads

* Bumped warnings up to the max and fixed remaining issues (#890)

Co-authored-by: Your Name <you@example.com>

* nits

* frida mode - support non-pie

* nits

* nit

* update grammar mutator

* Fixes for aarch64, OSX and other minor issues (#891)

Co-authored-by: Your Name <you@example.com>

* nits

* nits

* fix PCGUARD, build aflpp_driver with fPIC

* Added representative fuzzbench test and test for libxml (#893)

* Added representative fuzzbench test and test for libxml

* Added support for building FRIDA from source with FRIDA_SOURCE=1

Co-authored-by: Your Name <you@example.com>

* nits

* update changelog

* typos

* still not working

* fixed potential double free in custom trim (#881)

* error handling, freeing mem

* frida: complog -> cmplog

* fix statsd writing

* let aflpp_qemu_driver_hook.so build fail gracefully

* fix stdin trimming

* Support for AFL_ENTRYPOINT (#898)

Co-authored-by: Your Name <you@example.com>

* remove the input file .cur_input at the end of the fuzzing, if AFL_TMPDIR is used

* reverse push (#901)

* Create FUNDING.yml

* Update FUNDING.yml

* disable QEMU static pie

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>

* clarify that no modifications are required.

* add new test for frida_mode (please review)

* typos

* fix persistent mode (64-bit)

* set ARCH for linux intel 32-bit for frida-gum-devkit

* prepare for 32-bit support (later)

* not on qemu 3 anymore

* unicorn mips fixes

* instrumentation further move to C++11 (#900)

* unicorn fixes

* first working NeverZero implementation

* more unicorn fixes

* Fix memory errors when trim causes testcase growth (#881) (#903)

* Revert "fixed potential double free in custom trim (#881)"

This reverts commit e9d2f72382.

* Revert "fix custom trim for increasing data"

This reverts commit 86a8ef168d.

* Fix memory errors when trim causes testcase growth

Modify trim_case_custom to avoid writing into in_buf because
some custom mutators can cause the testcase to grow rather than
shrink.

Instead of modifying in_buf directly, we write the update out
to the disk when trimming is complete, and then the caller is
responsible for refreshing the in-memory buffer from the file.

This is still a bit sketchy because it does need to modify q->len in
order to notify the upper layers that something changed, and it could
end up telling upper layer code that the q->len is *bigger* than
the buffer (q->testcase_buf) that contains it, which is asking
for trouble down the line somewhere...

* Fix an unlikely situation

Put back some `unlikely()` calls that were in
the e9d2f72382 commit that was
reverted.

* add some comments

* typo

* Exit on time (#904)

* Variable AFL_EXIT_ON_TIME description has been added.
Variables AFL_EXIT_ON_TIME and afl_exit_on_time has been added.
afl->exit_on_time variable initialization has been added.
The asignment of a value to the afl->afl_env.afl_exit_on_time variable from
environment variables has been added.
Code to exit on timeout if new path not found has been added.

* Type of afl_exit_on_time variable has been changed.
Variable exit_on_time has been added to the afl_state_t structure.

* Command `export AFL_EXIT_WHEN_DONE=1` has been added.

* Millisecond to second conversion has been added.
Call get_cur_time() has been added.

* Revert to using the saved current time value.

* Useless check has been removed.

* fix new path to custom-mutators

* ensure crashes/README.txt exists

* fix

* Changes to bump FRIDA version and to clone FRIDA repo in to build directory rather than use a submodule as the FRIDA build scripts don't like it (#906)

Co-authored-by: Your Name <you@example.com>

* Fix numeric overflow in cmplog implementation (#907)

Co-authored-by: Your Name <you@example.com>

* testcase fixes for unicorn

* remove merge conflict artifacts

* fix afl-plot

* Changes to remove binaries from frida_mode (#913)

Co-authored-by: Your Name <you@example.com>

* Frida cmplog fail fast (#914)

* Changes to remove binaries from frida_mode

* Changes to make cmplog fail fast

Co-authored-by: Your Name <you@example.com>

* afl-plot: relative time

* arch linux and mac os support for afl-system-config

* typo

* code-format

* update documentation

* github workflow for qemu

* OSX-specific improvements (#912)

* Fix afl-cc to work correctly by default on OSX using xcode

- CLANG_ENV_VAR must be set for afl-as to work
- Use clang mode by default if no specific compiler selected

* Add OSX-specific documentation for configuring shared memory

* Fixes to memory operands for complog (#916)

Co-authored-by: Your Name <you@example.com>

* fix a few cur_time uses

* added bounds check to pivot_inputs (fixes #921)

* additional safety checks for restarts

* restrict afl-showmap in_file size

* fix seed crash disable

* add warning for afl-showmap partial read

* no core dumps

* AFL_PRINT_FILENAMES added

* more documentation for AFL_EXIT_ON_TIME

* Flushing for AFL_PRINT_FILENAMES

* FASAN Support (#918)

* FASAN Support

* Fix handling of Address Sanitizer DSO

* Changes to identification of Address Sanitizer DSO

Co-authored-by: Your Name <you@example.com>

* Support for x86 (#920)

Co-authored-by: Your Name <you@example.com>

* Update frida_mode readme (#925)

* libqasan: use syscalls for read and write

* update readme

* Minor integration tweaks (#926)

Co-authored-by: Your Name <you@example.com>

* merge

* fix afl-fuzz.c frida preload

* cleaned up AFL_PRINT_FILENAMES env

* Changes to have persistent mode exit at the end of the loop (#928)

Co-authored-by: Your Name <you@example.com>

* fix llvm-dict2file

* push to stable (#931) (#932)

* sync (#886)

* Create FUNDING.yml

* Update FUNDING.yml

* moved custom_mutator examples

* unicorn speedtest makefile cleanup

* fixed example location

* fix qdbi

* update util readme

* Frida persistent (#880)

* Added x64 support for persistent mode (function call only), in-memory teest cases and complog

* Review changes, fix NeverZero and code to parse the .text section of the main executable. Excluded ranges TBC

* Various minor fixes and finished support for AFL_INST_LIBS

* Review changes

Co-authored-by: Your Name <you@example.com>

* nits

* fix frida mode

* Integer overflow/underflow fixes in libdislocator (#889)

* libdislocator: fixing integer overflow in 'max_mem' variable and setting 'max_mem' type to 'size_t'

* libdislocator: fixing potential integer underflow in 'total_mem' variable due to its different values in different threads

* Bumped warnings up to the max and fixed remaining issues (#890)

Co-authored-by: Your Name <you@example.com>

* nits

* frida mode - support non-pie

* nits

* nit

* update grammar mutator

* Fixes for aarch64, OSX and other minor issues (#891)

Co-authored-by: Your Name <you@example.com>

* nits

* nits

* fix PCGUARD, build aflpp_driver with fPIC

* Added representative fuzzbench test and test for libxml (#893)

* Added representative fuzzbench test and test for libxml

* Added support for building FRIDA from source with FRIDA_SOURCE=1

Co-authored-by: Your Name <you@example.com>

* nits

* update changelog

* typos

* fixed potential double free in custom trim (#881)

* error handling, freeing mem

* frida: complog -> cmplog

* fix statsd writing

* let aflpp_qemu_driver_hook.so build fail gracefully

* fix stdin trimming

* Support for AFL_ENTRYPOINT (#898)

Co-authored-by: Your Name <you@example.com>

* remove the input file .cur_input at the end of the fuzzing, if AFL_TMPDIR is used

* reverse push (#901)

* Create FUNDING.yml

* Update FUNDING.yml

* disable QEMU static pie

Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>

* clarify that no modifications are required.

* add new test for frida_mode (please review)

* typos

* fix persistent mode (64-bit)

* set ARCH for linux intel 32-bit for frida-gum-devkit

* prepare for 32-bit support (later)

* not on qemu 3 anymore

* unicorn mips fixes

* instrumentation further move to C++11 (#900)

* unicorn fixes

* more unicorn fixes

* Fix memory errors when trim causes testcase growth (#881) (#903)

* Revert "fixed potential double free in custom trim (#881)"

This reverts commit e9d2f72382.

* Revert "fix custom trim for increasing data"

This reverts commit 86a8ef168d.

* Fix memory errors when trim causes testcase growth

Modify trim_case_custom to avoid writing into in_buf because
some custom mutators can cause the testcase to grow rather than
shrink.

Instead of modifying in_buf directly, we write the update out
to the disk when trimming is complete, and then the caller is
responsible for refreshing the in-memory buffer from the file.

This is still a bit sketchy because it does need to modify q->len in
order to notify the upper layers that something changed, and it could
end up telling upper layer code that the q->len is *bigger* than
the buffer (q->testcase_buf) that contains it, which is asking
for trouble down the line somewhere...

* Fix an unlikely situation

Put back some `unlikely()` calls that were in
the e9d2f72382 commit that was
reverted.

* typo

* Exit on time (#904)

* Variable AFL_EXIT_ON_TIME description has been added.
Variables AFL_EXIT_ON_TIME and afl_exit_on_time has been added.
afl->exit_on_time variable initialization has been added.
The asignment of a value to the afl->afl_env.afl_exit_on_time variable from
environment variables has been added.
Code to exit on timeout if new path not found has been added.

* Type of afl_exit_on_time variable has been changed.
Variable exit_on_time has been added to the afl_state_t structure.

* Command `export AFL_EXIT_WHEN_DONE=1` has been added.

* Millisecond to second conversion has been added.
Call get_cur_time() has been added.

* Revert to using the saved current time value.

* Useless check has been removed.

* fix new path to custom-mutators

* ensure crashes/README.txt exists

* fix

* Changes to bump FRIDA version and to clone FRIDA repo in to build directory rather than use a submodule as the FRIDA build scripts don't like it (#906)

Co-authored-by: Your Name <you@example.com>

* Fix numeric overflow in cmplog implementation (#907)

Co-authored-by: Your Name <you@example.com>

* testcase fixes for unicorn

* remove merge conflict artifacts

* fix afl-plot

* Changes to remove binaries from frida_mode (#913)

Co-authored-by: Your Name <you@example.com>

* Frida cmplog fail fast (#914)

* Changes to remove binaries from frida_mode

* Changes to make cmplog fail fast

Co-authored-by: Your Name <you@example.com>

* afl-plot: relative time

* arch linux and mac os support for afl-system-config

* typo

* code-format

* update documentation

* github workflow for qemu

* OSX-specific improvements (#912)

* Fix afl-cc to work correctly by default on OSX using xcode

- CLANG_ENV_VAR must be set for afl-as to work
- Use clang mode by default if no specific compiler selected

* Add OSX-specific documentation for configuring shared memory

* Fixes to memory operands for complog (#916)

Co-authored-by: Your Name <you@example.com>

* fix a few cur_time uses

* added bounds check to pivot_inputs (fixes #921)

* additional safety checks for restarts

* restrict afl-showmap in_file size

* fix seed crash disable

* add warning for afl-showmap partial read

* no core dumps

* AFL_PRINT_FILENAMES added

* more documentation for AFL_EXIT_ON_TIME

* Flushing for AFL_PRINT_FILENAMES

* FASAN Support (#918)

* FASAN Support

* Fix handling of Address Sanitizer DSO

* Changes to identification of Address Sanitizer DSO

Co-authored-by: Your Name <you@example.com>

* Support for x86 (#920)

Co-authored-by: Your Name <you@example.com>

* Update frida_mode readme (#925)

* libqasan: use syscalls for read and write

* update readme

* Minor integration tweaks (#926)

Co-authored-by: Your Name <you@example.com>

* merge

* fix afl-fuzz.c frida preload

* cleaned up AFL_PRINT_FILENAMES env

* Changes to have persistent mode exit at the end of the loop (#928)

Co-authored-by: Your Name <you@example.com>

* fix llvm-dict2file

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>

Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: hexcoder <hexcoder-@users.noreply.github.com>
Co-authored-by: hexcoder- <heiko@hexco.de>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>

* improve error msg

* Added documentation for wine LoadLibrary workaround (#933)

* Fix cmake target compilation command example (#934)

- Fix typo DCMAKE_C_COMPILERC -> DCMAKE_C_COMPILER.
- Add `cd build` after `mkdir build`.

* showmap passes queue items in alphabetical order

* added tmp files to gitignore

* lenient dict parsing, no map size enum for binary fuzzing

* added info about showmap queue directions

* update binary-only doc

* turn off map size detection if skip_bin_check is set

* Typo

* update docs

* update afl-system-config

* Set kill signal before using it in afl-showmap (#935)

* fix afl-cc help output

* add libafl to binary-only doc

* update docs

* less executions on variable paths

* AFL_SKIP_CRASHES is obsolete since 3.0

* add AFL_TRY_AFFINITY

* Typo

* Typo

* Typo/wording

* tweaks

* typos

* fix afl-whatsup help output

* fix afl-plot output

* fix for MacOS

* fix cmpcov doc for qemu

* fix tmpfile removal

* update dockerfile

* Frida (#940)

* Added re2 test

* Added libpcap test

* Fix validation of setting of ADDR_NO_RANDOMIZE

* Added support for printing original and instrumented code

Co-authored-by: Your Name <you@example.com>

* Support for AFL_FRIDA_PERSISTENT_RET (#941)

Co-authored-by: Your Name <you@example.com>

* Changes to add missing exclusion of ranges (#943)

Co-authored-by: Your Name <you@example.com>

* add --afl-noopt to afl-cc

* docs: fix link to README in QuickStartGuide (#946)

* Support writing Stalker stats (#945)

* Support writing Stalker stats

* Fixed string handling in print functions

Co-authored-by: Your Name <you@example.com>

* afl-cmin help fix, aflpp_driver - + @@ support

* fix for afl-showmap

* support new env var AFL_LLVM_THREADSAFE_INST to enable atomic counters.
add new test case for that.

* add documentation for AFL_LLVM_THREADSAFE_INST

* add support for AFL_LLVM_THREADSAFE_INST to other LLVM passes

* add missing include for _exit()

* threadsafe doc fixes, code format

* Wording: "never zero" -> NeverZero

* fix afl_custom_post_process with multiple custom mutators

* fix docs

* debug ck_write

* fixed potential diff by 0

* fixes

* fix classic threadsafe counters

Co-authored-by: van Hauser <vh@thc.org>
Co-authored-by: Dominik Maier <domenukk@gmail.com>
Co-authored-by: WorksButNotTested <62701594+WorksButNotTested@users.noreply.github.com>
Co-authored-by: Your Name <you@example.com>
Co-authored-by: Dmitry Zheregelya <zheregelya.d@gmail.com>
Co-authored-by: Andrea Fioraldi <andreafioraldi@gmail.com>
Co-authored-by: David CARLIER <devnexen@gmail.com>
Co-authored-by: realmadsci <71108352+realmadsci@users.noreply.github.com>
Co-authored-by: Roman M. Iudichev <SecNotice@ya.ru>
Co-authored-by: Dustin Spicuzza <dustin@virtualroadside.com>
Co-authored-by: 0x4d5a-ctf <51098072+0x4d5a-ctf@users.noreply.github.com>
Co-authored-by: Tommy Chiang <oToToT@users.noreply.github.com>
Co-authored-by: buherator <buherator@silentsignal.hu>
Co-authored-by: Dag Heyman Kajevic <dag.heyman@gmail.com>

README.md

@@ -90,6 +90,7 @@ behaviours and defaults:
 
   | Feature/Instrumentation  | afl-gcc | llvm      | gcc_plugin | frida_mode | qemu_mode        |unicorn_mode |
   | -------------------------|:-------:|:---------:|:----------:|:----------:|:----------------:|:------------:|
+  | Threadsafe counters      |         |     x(3)  |            |            |                  |              |
   | NeverZero                | x86[_64]|     x(1)  |     x      |     x      |         x        |       x      |
   | Persistent Mode          |         |     x     |     x      |  x86[_64]  | x86[_64]/arm[64] |       x      |
   | LAF-Intel / CompCov      |         |     x     |            |            | x86[_64]/arm[64] | x86[_64]/arm |
@@ -104,7 +105,7 @@ behaviours and defaults:
 
   1. default for LLVM >= 9.0, env var for older version due an efficiency bug in previous llvm versions
   2. GCC creates non-performant code, hence it is disabled in gcc_plugin
-  3. (currently unassigned)
+  3. with `AFL_LLVM_THREADSAFE_INST`, disables NeverZero
   4. with pcguard mode and LTO mode for LLVM 11 and newer
   5. upcoming, development in the branch
   6. not compatible with LTO instrumentation and needs at least LLVM v4.1

custom_mutators/examples/post_library_gif.so.c

@@ -45,6 +45,7 @@
    1) If you don't want to modify the test case, simply set `*out_buf = in_buf`
       and return the original `len`.
 
+   NOTE: the following is currently NOT true, we abort in this case!
    2) If you want to skip this test case altogether and have AFL generate a
       new one, return 0 or set `*out_buf = NULL`.
       Use this sparingly - it's faster than running the target program
@@ -60,7 +61,7 @@
 
       *** Feel free to reuse the original 'in_buf' BUFFER and return it. ***
 
-    Aight. The example below shows a simple postprocessor that tries to make
+    Alright. The example below shows a simple postprocessor that tries to make
     sure that all input files start with "GIF89a".
 
     PS. If you don't like C, you can try out the unix-based wrapper from

docs/Changelog.md

@@ -22,13 +22,14 @@ sending a mail to <afl-users+subscribe@googlegroups.com>.
       to allow replay of non-reproducable crashes, see
       AFL_PERSISTENT_RECORD in config.h and docs/envs.h
     - fixed a bug when trimming for stdin targets
-    - default cmplog level (-l) is now 2, better efficiency.
+    - cmplog -l: default cmplog level is now 2, better efficiency.
-    - cmplog level 3 (-l 3) now performs redqueen on everything.
+      level 3 now performs redqueen on everything. use with care.
-      use with care.
+    - better fuzzing strategy yield display for enabled options
-    - better fuzzing strategy yields for enabled options
     - ensure one fuzzer sync per cycle
     - fix afl_custom_queue_new_entry original file name when syncing
       from fuzzers
+    - fixed a crash when more than one custom mutator was used together
+      with afl_custom_post_process
     - on a crashing seed potentially the wrong input was disabled
     - added AFL_EXIT_ON_SEED_ISSUES env that will exit if a seed in
       -i dir crashes the target or results in a timeout. By default
@@ -41,6 +42,8 @@ sending a mail to <afl-users+subscribe@googlegroups.com>.
       it fails
   - afl-cc:
     - We do not support llvm versions prior 6.0 anymore
+    - added thread safe counters to all modes (`AFL_LLVM_THREADSAFE_INST`),
+      note that this disables NeverZero counters.
     - Fix for -pie compiled binaries with default afl-clang-fast PCGUARD
     - Leak Sanitizer (AFL_USE_LSAN) added by Joshua Rogers, thanks!
     - Removed InsTrim instrumentation as it is not as good as PCGUARD

docs/custom_mutators.md

@@ -123,6 +123,7 @@ def deinit():  # optional for Python
     Note that this function is optional - but it makes sense to use it.
     You would only skip this if `post_process` is used to fix checksums etc.
     so if you are using it e.g. as a post processing library.
+    Note that a length > 0 *must* be returned!
 
 - `describe` (optional):
 

docs/env_variables.md

@@ -231,6 +231,12 @@ Then there are a few specific features that are only available in instrumentatio
 
   See [instrumentation/README.instrument_list.md](../instrumentation/README.instrument_list.md) for more information.
 
+### Thread safe instrumentation counters (in all modes)
+
+   - Setting `AFL_LLVM_THREADSAFE_INST` will inject code that implements thread
+     safe counters. The overhead is a little bit higher compared to the older
+     non-thread safe case. Note that this disables neverzero (see below).
+
 ### NOT_ZERO
 
    - Setting `AFL_LLVM_NOT_ZERO=1` during compilation will use counters

frida_mode/src/persistent/persistent_x64.c

@@ -1,3 +1,4 @@
+#include <unistd.h>
 #include "frida-gum.h"
 
 #include "config.h"

frida_mode/src/stats/stats.c

@@ -99,7 +99,7 @@ void stats_vprint(int fd, char *format, va_list ap) {
   int  ret;
   int  len;
 
-  if(vsnprintf(buffer, sizeof(buffer) - 1, format, ap) < 0) { return; }
+  if (vsnprintf(buffer, sizeof(buffer) - 1, format, ap) < 0) { return; }
 
   len = strnlen(buffer, sizeof(buffer));
   IGNORED_RETURN(write(fd, buffer, len));

include/debug.h

@@ -362,7 +362,12 @@ static inline const char *colorfilter(const char *x) {
                                                                           \
     s32 _len = (s32)(len);                                                \
     s32 _res = write(_fd, (buf), _len);                                   \
-    if (_res != _len) RPFATAL(_res, "Short write to %s, fd %d", fn, _fd); \
+    if (_res != _len) {                                                   \
+                                                                          \
+      RPFATAL(_res, "Short write to %s, fd %d (%d of %d bytes)", fn, _fd, \
+              _res, _len);                                                \
+                                                                          \
+    }                                                                     \
                                                                           \
   } while (0)
 

include/envs.h

@@ -126,6 +126,7 @@ static char *afl_environment_variables[] = {
     "AFL_NGRAM_SIZE",
     "AFL_LLVM_NOT_ZERO",
     "AFL_LLVM_INSTRUMENT_FILE",
+    "AFL_LLVM_THREADSAFE_INST",
     "AFL_LLVM_SKIP_NEVERZERO",
     "AFL_NO_AFFINITY",
     "AFL_TRY_AFFINITY",

instrumentation/README.llvm.md

@@ -144,6 +144,11 @@ is not optimal and was only fixed in llvm 9.
 You can set this with AFL_LLVM_NOT_ZERO=1
 See [README.neverzero.md](README.neverzero.md)
 
+Support for thread safe counters has been added for all modes.
+Activate it with `AFL_LLVM_THREADSAFE_INST=1`. The tradeoff is better precision
+in multi threaded apps for a slightly higher instrumentation overhead.
+This also disables the nozero counter default for performance reasons.
+
 ## 4) Snapshot feature
 
 To speed up fuzzing you can use a linux loadable kernel module which enables

instrumentation/README.neverzero.md

@@ -16,11 +16,12 @@ at a very little cost (one instruction per edge).
 (The alternative of saturated counters has been tested also and proved to be
 inferior in terms of path discovery.)
 
-This is implemented in afl-gcc and afl-gcc-fast, however for llvm_mode this is optional if
+This is implemented in afl-gcc and afl-gcc-fast, however for llvm_mode this is
-the llvm version is below 9 - as there is a perfomance bug that is only fixed
+optional if multithread safe counters are selected or the llvm version is below
-in version 9 and onwards.
+9 - as there are severe performance costs in these cases.
 
-If you want to enable this for llvm versions below 9 then set
+If you want to enable this for llvm versions below 9 or thread safe counters
+then set
 
 ```
 export AFL_LLVM_NOT_ZERO=1
@@ -33,3 +34,8 @@ AFL_LLVM_SKIP_NEVERZERO=1
 ```
 If the target does not have extensive loops or functions that are called
 a lot then this can give a small performance boost.
+
+Please note that the default counter implementations are not thread safe!
+
+Support for thread safe counters in mode LLVM CLASSIC can be activated with setting
+`AFL_LLVM_THREADSAFE_INST=1`.

instrumentation/SanitizerCoverageLTO.so.cc

@@ -236,7 +236,8 @@ class ModuleSanitizerCoverage {
   uint32_t                         inst = 0;
   uint32_t                         afl_global_id = 0;
   uint64_t                         map_addr = 0;
-  char *                           skip_nozero = NULL;
+  const char *                     skip_nozero = NULL;
+  const char *                     use_threadsafe_counters = nullptr;
   std::vector<BasicBlock *>        BlockList;
   DenseMap<Value *, std::string *> valueMap;
   std::vector<std::string>         dictionary;
@@ -437,6 +438,7 @@ bool ModuleSanitizerCoverage::instrumentModule(
     be_quiet = 1;
 
   skip_nozero = getenv("AFL_LLVM_SKIP_NEVERZERO");
+  use_threadsafe_counters = getenv("AFL_LLVM_THREADSAFE_INST");
 
   if ((ptr = getenv("AFL_LLVM_LTO_STARTID")) != NULL)
     if ((afl_global_id = atoi(ptr)) < 0)
@@ -1208,7 +1210,7 @@ void ModuleSanitizerCoverage::instrumentFunction(
     return;  // Should not instrument sanitizer init functions.
   if (F.getName().startswith("__sanitizer_"))
     return;  // Don't instrument __sanitizer_* callbacks.
-  // Don't touch available_externally functions, their actual body is elewhere.
+  // Don't touch available_externally functions, their actual body is elsewhere.
   if (F.getLinkage() == GlobalValue::AvailableExternallyLinkage) return;
   // Don't instrument MSVC CRT configuration helpers. They may run before normal
   // initialization.
@@ -1495,9 +1497,16 @@ void ModuleSanitizerCoverage::InjectCoverageAtBlock(Function &F, BasicBlock &BB,
     }
 
     /* Update bitmap */
+    if (use_threadsafe_counters) {                                /* Atomic */
+
+      IRB.CreateAtomicRMW(llvm::AtomicRMWInst::BinOp::Add, MapPtrIdx, One,
+                          llvm::AtomicOrdering::Monotonic);
+
+    } else {
 
       LoadInst *Counter = IRB.CreateLoad(MapPtrIdx);
-    Counter->setMetadata(Mo->getMDKindID("nosanitize"), MDNode::get(*Ct, None));
+      Counter->setMetadata(Mo->getMDKindID("nosanitize"),
+                           MDNode::get(*Ct, None));
 
       Value *Incr = IRB.CreateAdd(Counter, One);
 
@@ -1512,6 +1521,8 @@ void ModuleSanitizerCoverage::InjectCoverageAtBlock(Function &F, BasicBlock &BB,
       IRB.CreateStore(Incr, MapPtrIdx)
           ->setMetadata(Mo->getMDKindID("nosanitize"), MDNode::get(*Ct, None));
 
+    }
+
     // done :)
 
     inst++;

instrumentation/SanitizerCoveragePCGUARD.so.cc

@@ -86,7 +86,8 @@ const char SanCovPCsSectionName[] = "sancov_pcs";
 
 const char SanCovLowestStackName[] = "__sancov_lowest_stack";
 
-static char *skip_nozero;
+static const char *skip_nozero;
+static const char *use_threadsafe_counters;
 
 namespace {
 
@@ -386,6 +387,7 @@ bool ModuleSanitizerCoverage::instrumentModule(
     be_quiet = 1;
 
   skip_nozero = getenv("AFL_LLVM_SKIP_NEVERZERO");
+  use_threadsafe_counters = getenv("AFL_LLVM_THREADSAFE_INST");
 
   initInstrumentList();
   scanForDangerousFunctions(&M);
@@ -1067,9 +1069,16 @@ void ModuleSanitizerCoverage::InjectCoverageAtBlock(Function &F, BasicBlock &BB,
 
     /* Load counter for CurLoc */
 
-    Value *   MapPtrIdx = IRB.CreateGEP(MapPtr, CurLoc);
+    Value *MapPtrIdx = IRB.CreateGEP(MapPtr, CurLoc);
-    LoadInst *Counter = IRB.CreateLoad(MapPtrIdx);
 
+    if (use_threadsafe_counters) {
+
+      IRB.CreateAtomicRMW(llvm::AtomicRMWInst::BinOp::Add, MapPtrIdx, One,
+                          llvm::AtomicOrdering::Monotonic);
+
+    } else {
+
+      LoadInst *Counter = IRB.CreateLoad(MapPtrIdx);
       /* Update bitmap */
 
       Value *Incr = IRB.CreateAdd(Counter, One);
@@ -1084,6 +1093,8 @@ void ModuleSanitizerCoverage::InjectCoverageAtBlock(Function &F, BasicBlock &BB,
 
       IRB.CreateStore(Incr, MapPtrIdx);
 
+    }
+
     // done :)
 
     //    IRB.CreateCall(SanCovTracePCGuard, Offset)->setCannotMerge();

instrumentation/afl-llvm-lto-instrumentation.so.cc

@@ -93,7 +93,8 @@ class AFLLTOPass : public ModulePass {
   uint32_t               function_minimum_size = 1;
   uint32_t               inst_blocks = 0, inst_funcs = 0, total_instr = 0;
   unsigned long long int map_addr = 0x10000;
-  char *                 skip_nozero = NULL;
+  const char *           skip_nozero = NULL;
+  const char *           use_threadsafe_counters = nullptr;
 
 };
 
@@ -131,6 +132,8 @@ bool AFLLTOPass::runOnModule(Module &M) {
 
     be_quiet = 1;
 
+  use_threadsafe_counters = getenv("AFL_LLVM_THREADSAFE_INST");
+
   if ((ptr = getenv("AFL_LLVM_DOCUMENT_IDS")) != NULL) {
 
     if ((documentFile = fopen(ptr, "a")) == NULL)
@@ -839,6 +842,13 @@ bool AFLLTOPass::runOnModule(Module &M) {
 
           /* Update bitmap */
 
+          if (use_threadsafe_counters) {
+
+            IRB.CreateAtomicRMW(llvm::AtomicRMWInst::BinOp::Add, MapPtrIdx, One,
+                                llvm::AtomicOrdering::Monotonic);
+
+          } else {
+
             LoadInst *Counter = IRB.CreateLoad(MapPtrIdx);
             Counter->setMetadata(M.getMDKindID("nosanitize"),
                                  MDNode::get(C, None));
@@ -854,7 +864,10 @@ bool AFLLTOPass::runOnModule(Module &M) {
             }
 
             IRB.CreateStore(Incr, MapPtrIdx)
-              ->setMetadata(M.getMDKindID("nosanitize"), MDNode::get(C, None));
+                ->setMetadata(M.getMDKindID("nosanitize"),
+                              MDNode::get(C, None));
+
+          }
 
           // done :)
 

instrumentation/afl-llvm-pass.so.cc

@@ -85,7 +85,8 @@ class AFLCoverage : public ModulePass {
   uint32_t    ctx_k = 0;
   uint32_t    map_size = MAP_SIZE;
   uint32_t    function_minimum_size = 1;
-  char *   ctx_str = NULL, *caller_str = NULL, *skip_nozero = NULL;
+  const char *ctx_str = NULL, *caller_str = NULL, *skip_nozero = NULL;
+  const char *use_threadsafe_counters = nullptr;
 
 };
 
@@ -182,6 +183,38 @@ bool AFLCoverage::runOnModule(Module &M) {
   char *neverZero_counters_str = getenv("AFL_LLVM_NOT_ZERO");
 #endif
   skip_nozero = getenv("AFL_LLVM_SKIP_NEVERZERO");
+  use_threadsafe_counters = getenv("AFL_LLVM_THREADSAFE_INST");
+
+  if ((isatty(2) && !getenv("AFL_QUIET")) || !!getenv("AFL_DEBUG")) {
+
+    if (use_threadsafe_counters) {
+
+      // disabled unless there is support for other modules as well
+      // (increases documentation complexity)
+      /*      if (!getenv("AFL_LLVM_NOT_ZERO")) { */
+
+      skip_nozero = "1";
+      SAYF(cCYA "afl-llvm-pass" VERSION cRST " using thread safe counters\n");
+
+      /*
+
+            } else {
+
+              SAYF(cCYA "afl-llvm-pass" VERSION cRST
+                        " using thread safe not-zero-counters\n");
+
+            }
+
+      */
+
+    } else {
+
+      SAYF(cCYA "afl-llvm-pass" VERSION cRST
+                " using non-thread safe instrumentation\n");
+
+    }
+
+  }
 
   unsigned PrevLocSize = 0;
   unsigned PrevCallerSize = 0;
@@ -388,7 +421,6 @@ bool AFLCoverage::runOnModule(Module &M) {
 #endif
 
   // other constants we need
-  ConstantInt *Zero = ConstantInt::get(Int8Ty, 0);
   ConstantInt *One = ConstantInt::get(Int8Ty, 1);
 
   Value *   PrevCtx = NULL;     // CTX sensitive coverage
@@ -410,6 +442,7 @@ bool AFLCoverage::runOnModule(Module &M) {
 
     if (F.size() < function_minimum_size) continue;
 
+    std::list<Value *> todo;
     for (auto &BB : F) {
 
       BasicBlock::iterator IP = BB.getFirstInsertionPt();
@@ -628,6 +661,33 @@ bool AFLCoverage::runOnModule(Module &M) {
 
       /* Update bitmap */
 
+      if (use_threadsafe_counters) {                              /* Atomic */
+                                     /*
+                                     #if LLVM_VERSION_MAJOR < 9
+                                             if (neverZero_counters_str !=
+                                                 NULL) {  // with llvm 9 we make this the default as the bug
+                                     in llvm
+                                                          // is then fixed
+                                     #else
+                                             if (!skip_nozero) {
+                             
+                                     #endif
+                                               // register MapPtrIdx in a todo list
+                                               todo.push_back(MapPtrIdx);
+                             
+                                             } else {
+
+                                     */
+        IRB.CreateAtomicRMW(llvm::AtomicRMWInst::BinOp::Add, MapPtrIdx, One,
+                            llvm::AtomicOrdering::Monotonic);
+        /*
+
+                }
+
+        */
+
+      } else {
+
         LoadInst *Counter = IRB.CreateLoad(MapPtrIdx);
         Counter->setMetadata(M.getMDKindID("nosanitize"), MDNode::get(C, None));
 
@@ -635,14 +695,15 @@ bool AFLCoverage::runOnModule(Module &M) {
 
 #if LLVM_VERSION_MAJOR < 9
         if (neverZero_counters_str !=
-          NULL) {  // with llvm 9 we make this the default as the bug in llvm is
+            NULL) {  // with llvm 9 we make this the default as the bug in llvm
-                   // then fixed
+                     // is then fixed
 #else
         if (!skip_nozero) {
 
 #endif
           /* hexcoder: Realize a counter that skips zero during overflow.
-         * Once this counter reaches its maximum value, it next increments to 1
+           * Once this counter reaches its maximum value, it next increments to
+           * 1
            *
            * Instead of
            * Counter + 1 -> Counter
@@ -651,6 +712,7 @@ bool AFLCoverage::runOnModule(Module &M) {
            * Counter + OverflowFlag -> Counter
            */
 
+          ConstantInt *Zero = ConstantInt::get(Int8Ty, 0);
           auto         cf = IRB.CreateICmpEQ(Incr, Zero);
           auto         carry = IRB.CreateZExt(cf, Int8Ty);
           Incr = IRB.CreateAdd(Incr, carry);
@@ -660,6 +722,8 @@ bool AFLCoverage::runOnModule(Module &M) {
         IRB.CreateStore(Incr, MapPtrIdx)
             ->setMetadata(M.getMDKindID("nosanitize"), MDNode::get(C, None));
 
+      }                                                  /* non atomic case */
+
       /* Update prev_loc history vector (by placing cur_loc at the head of the
          vector and shuffle the other elements back by one) */
 
@@ -715,6 +779,120 @@ bool AFLCoverage::runOnModule(Module &M) {
 
     }
 
+#if 0
+    if (use_threadsafe_counters) {                       /*Atomic NeverZero */
+      // handle the list of registered blocks to instrument
+      for (auto val : todo) {
+
+        /* hexcoder: Realize a thread-safe counter that skips zero during
+         * overflow. Once this counter reaches its maximum value, it next
+         * increments to 1
+         *
+         * Instead of
+         * Counter + 1 -> Counter
+         * we inject now this
+         * Counter + 1 -> {Counter, OverflowFlag}
+         * Counter + OverflowFlag -> Counter
+         */
+
+        /* equivalent c code looks like this
+         * Thanks to
+         https://preshing.com/20150402/you-can-do-any-kind-of-atomic-read-modify-write-operation/
+
+            int old = atomic_load_explicit(&Counter, memory_order_relaxed);
+            int new;
+            do {
+
+                 if (old == 255) {
+
+                   new = 1;
+
+                 } else {
+
+                   new = old + 1;
+
+                 }
+
+            } while (!atomic_compare_exchange_weak_explicit(&Counter, &old, new,
+
+         memory_order_relaxed, memory_order_relaxed));
+
+         */
+
+        Value *              MapPtrIdx = val;
+        Instruction *        MapPtrIdxInst = cast<Instruction>(val);
+        BasicBlock::iterator it0(&(*MapPtrIdxInst));
+        ++it0;
+        IRBuilder<> IRB(&(*it0));
+
+        // load the old counter value atomically
+        LoadInst *Counter = IRB.CreateLoad(MapPtrIdx);
+        Counter->setAlignment(llvm::Align());
+        Counter->setAtomic(llvm::AtomicOrdering::Monotonic);
+        Counter->setMetadata(M.getMDKindID("nosanitize"), MDNode::get(C, None));
+
+        BasicBlock *BB = IRB.GetInsertBlock();
+        // insert a basic block with the corpus of a do while loop
+        // the calculation may need to repeat, if atomic compare_exchange is not
+        // successful
+
+        BasicBlock::iterator it(*Counter);
+        it++;  // split after load counter
+        BasicBlock *end_bb = BB->splitBasicBlock(it);
+        end_bb->setName("injected");
+
+        // insert the block before the second half of the split
+        BasicBlock *do_while_bb =
+            BasicBlock::Create(C, "injected", end_bb->getParent(), end_bb);
+
+        // set terminator of BB from target end_bb to target do_while_bb
+        auto term = BB->getTerminator();
+        BranchInst::Create(do_while_bb, BB);
+        term->eraseFromParent();
+
+        // continue to fill instructions into the do_while loop
+        IRB.SetInsertPoint(do_while_bb, do_while_bb->getFirstInsertionPt());
+
+        PHINode *PN = IRB.CreatePHI(Int8Ty, 2);
+
+        // compare with maximum value 0xff
+        auto *Cmp = IRB.CreateICmpEQ(Counter, ConstantInt::get(Int8Ty, -1));
+
+        // increment the counter
+        Value *Incr = IRB.CreateAdd(Counter, One);
+
+        // select the counter value or 1
+        auto *Select = IRB.CreateSelect(Cmp, One, Incr);
+
+        // try to save back the new counter value
+        auto *CmpXchg = IRB.CreateAtomicCmpXchg(
+            MapPtrIdx, PN, Select, llvm::AtomicOrdering::Monotonic,
+            llvm::AtomicOrdering::Monotonic);
+        CmpXchg->setAlignment(llvm::Align());
+        CmpXchg->setWeak(true);
+        CmpXchg->setMetadata(M.getMDKindID("nosanitize"), MDNode::get(C, None));
+
+        // get the result of trying to update the Counter
+        Value *Success =
+            IRB.CreateExtractValue(CmpXchg, ArrayRef<unsigned>({1}));
+        // get the (possibly updated) value of Counter
+        Value *OldVal =
+            IRB.CreateExtractValue(CmpXchg, ArrayRef<unsigned>({0}));
+
+        // initially we use Counter
+        PN->addIncoming(Counter, BB);
+        // on retry, we use the updated value
+        PN->addIncoming(OldVal, do_while_bb);
+
+        // if the cmpXchg was not successful, retry
+        IRB.CreateCondBr(Success, end_bb, do_while_bb);
+
+      }
+
+    }
+
+#endif
+
   }
 
   /*

qemu_mode/libqasan/libqasan.c

@@ -70,8 +70,7 @@ __attribute__((constructor)) void __libqasan_init() {
 
   __libqasan_init_hooks();
 
-  if (getenv("AFL_INST_LIBS") || getenv("QASAN_HOTPACH"))
+  if (getenv("AFL_INST_LIBS") || getenv("QASAN_HOTPACH")) __libqasan_hotpatch();
-    __libqasan_hotpatch();
 
   if (getenv("AFL_INST_LIBS") || getenv("QASAN_HOTPACH")) __libqasan_hotpatch();
 

src/afl-cc.c

@@ -1777,6 +1777,8 @@ int main(int argc, char **argv, char **envp) {
         SAYF(
             "\nLLVM/LTO/afl-clang-fast/afl-clang-lto specific environment "
             "variables:\n"
+            "  AFL_LLVM_THREADSAFE_INST: instrument with thread safe counters, "
+            "disables neverzero\n"
 
             COUNTER_BEHAVIOUR
 

src/afl-common.c

@@ -479,10 +479,18 @@ void print_suggested_envs(char *mispelled_env) {
 
       size_t end = start + strcspn(afl_env + start, "_") + 1;
       memcpy(reduced, afl_env, start);
-      if (end < afl_env_len)
+      if (end < afl_env_len) {
+
         memcpy(reduced + start, afl_env + end, afl_env_len - end);
+
+      }
+
+      if (afl_env_len + start >= end) {
+
         reduced[afl_env_len - end + start] = 0;
 
+      }
+
       int distance = string_distance_levenshtein(reduced, env_name);
       if (distance < ENV_SIMILARITY_TRESHOLD && seen[j] == 0) {
 

src/afl-fuzz-one.c

@@ -561,6 +561,7 @@ u8 fuzz_one_original(afl_state_t *afl) {
 
       if (afl->cmplog_lvl == 3 ||
           (afl->cmplog_lvl == 2 && afl->queue_cur->tc_ref) ||
+          afl->queue_cur->favored ||
           !(afl->fsrv.total_execs % afl->queued_paths) ||
           get_cur_time() - afl->last_path_time > 300000) {  // 300 seconds
 

src/afl-fuzz-redqueen.c

@@ -438,7 +438,7 @@ static u8 colorization(afl_state_t *afl, u8 *buf, u32 len,
   if (taint) {
 
     if (afl->colorize_success && afl->cmplog_lvl < 3 &&
-        (len / positions == 1 && positions > CMPLOG_POSITIONS_MAX &&
+        (positions > CMPLOG_POSITIONS_MAX && len / positions == 1 &&
          afl->active_paths / afl->colorize_success > CMPLOG_CORPUS_PERCENT)) {
 
 #ifdef _DEBUG

src/afl-fuzz-run.c

@@ -107,27 +107,21 @@ write_to_testcase(afl_state_t *afl, void *mem, u32 len) {
         new_size =
             el->afl_custom_post_process(el->data, new_mem, new_size, &new_buf);
 
+        if (unlikely(!new_buf && new_size <= 0)) {
+
+          FATAL("Custom_post_process failed (ret: %lu)",
+                (long unsigned)new_size);
+
         }
 
         new_mem = new_buf;
 
+      }
+
     });
 
-    if (unlikely(!new_buf && (new_size <= 0))) {
+    /* everything as planned. use the potentially new data. */
-
+    afl_fsrv_write_to_testcase(&afl->fsrv, new_mem, new_size);
-      FATAL("Custom_post_process failed (ret: %lu)", (long unsigned)new_size);
-
-    } else if (likely(new_buf)) {
-
-      /* everything as planned. use the new data. */
-      afl_fsrv_write_to_testcase(&afl->fsrv, new_buf, new_size);
-
-    } else {
-
-      /* custom mutators do not has a custom_post_process function */
-      afl_fsrv_write_to_testcase(&afl->fsrv, mem, len);
-
-    }
 
   } else {
 
@@ -188,17 +182,17 @@ static void write_with_gap(afl_state_t *afl, u8 *mem, u32 len, u32 skip_at,
         new_size =
             el->afl_custom_post_process(el->data, new_mem, new_size, &new_buf);
 
-        if (unlikely(!new_buf || (new_size <= 0))) {
+        if (unlikely(!new_buf || new_size <= 0)) {
 
           FATAL("Custom_post_process failed (ret: %lu)",
                 (long unsigned)new_size);
 
         }
 
-      }
-
         new_mem = new_buf;
 
+      }
+
     });
 
   }

src/afl-fuzz.c

@@ -2066,13 +2066,10 @@ int main(int argc, char **argv_orig, char **envp) {
               break;
             case 4:
               afl->expand_havoc = 5;
-              if (afl->cmplog_lvl && afl->cmplog_lvl < 3) afl->cmplog_lvl = 3;
+              // if (afl->cmplog_lvl && afl->cmplog_lvl < 3) afl->cmplog_lvl =
+              // 3;
               break;
             case 5:
-              // if not in sync mode, enable deterministic mode?
-              // if (!afl->sync_id) afl->skip_deterministic = 0;
-              afl->expand_havoc = 6;
-            case 6:
               // nothing else currently
               break;
 

test/test-llvm.sh

@@ -43,6 +43,36 @@ test -e ../afl-clang-fast -a -e ../split-switches-pass.so && {
     $ECHO "$RED[!] llvm_mode failed"
     CODE=1
   }
+  AFL_LLVM_INSTRUMENT=CLASSIC AFL_LLVM_THREADSAFE_INST=1 ../afl-clang-fast -o test-instr.ts ../test-instr.c > /dev/null 2>&1
+  test -e test-instr.ts && {
+    $ECHO "$GREEN[+] llvm_mode threadsafe compilation succeeded"
+    echo 0 | AFL_QUIET=1 ../afl-showmap -m ${MEM_LIMIT} -o test-instr.ts.0 -r -- ./test-instr.ts > /dev/null 2>&1
+    AFL_QUIET=1 ../afl-showmap -m ${MEM_LIMIT} -o test-instr.ts.1 -r -- ./test-instr.ts < /dev/null > /dev/null 2>&1
+    test -e test-instr.ts.0 -a -e test-instr.ts.1 && {
+      diff test-instr.ts.0 test-instr.ts.1 > /dev/null 2>&1 && {
+        $ECHO "$RED[!] llvm_mode threadsafe instrumentation should be different on different input but is not"
+        CODE=1
+      } || {
+        $ECHO "$GREEN[+] llvm_mode threadsafe instrumentation present and working correctly"
+        TUPLES=`echo 0|AFL_QUIET=1 ../afl-showmap -m ${MEM_LIMIT} -o /dev/null -- ./test-instr.ts 2>&1 | grep Captur | awk '{print$3}'`
+        test "$TUPLES" -gt 2 -a "$TUPLES" -lt 8 && {
+          $ECHO "$GREEN[+] llvm_mode run reported $TUPLES threadsafe instrumented locations which is fine"
+        } || {
+          $ECHO "$RED[!] llvm_mode threadsafe instrumentation produces weird numbers: $TUPLES"
+          CODE=1
+        }
+        test "$TUPLES" -lt 3 && SKIP=1
+        true
+      }
+    } || {
+      $ECHO "$RED[!] llvm_mode threadsafe instrumentation failed"
+      CODE=1
+    }
+    rm -f test-instr.ts.0 test-instr.ts.1
+  } || {
+    $ECHO "$RED[!] llvm_mode (threadsafe) failed"
+    CODE=1
+  }
   ../afl-clang-fast -DTEST_SHARED_OBJECT=1 -z defs -fPIC -shared -o test-instr.so ../test-instr.c > /dev/null 2>&1
   test -e test-instr.so && {
     $ECHO "$GREEN[+] llvm_mode shared object with -z defs compilation succeeded"