push to stable (#1967)

* Output afl-clang-fast stuffs only if necessary (#1912)

* afl-cc header

* afl-cc common declarations

 - Add afl-cc-state.c
 - Strip includes, find_object, debug/be_quiet/have_*/callname setting from afl-cc.c
 - Use debugf_args in main
 - Modify execvp stuffs to fit new aflcc struct

* afl-cc show usage

* afl-cc mode selecting

1. compiler_mode by callname in argv[0]
2. compiler_mode by env "AFL_CC_COMPILER"
3. compiler_mode/instrument_mode by command line options "--afl-..."
4. instrument_mode/compiler_mode by various env vars including "AFL_LLVM_INSTRUMENT"
5. final checking steps
6. print "... - mode: %s-%s\n"
7. determine real argv[0] according to compiler_mode

* afl-cc macro defs

* afl-cc linking behaviors

* afl-cc fsanitize behaviors

* afl-cc misc

* afl-cc body update

* afl-cc all-in-one

formated with custom-format.py

* nits

---------

Co-authored-by: vanhauser-thc <vh@thc.org>

* changelog

* update grammar mutator

* lto llvm 12+

* docs(custom_mutators): fix missing ':' (#1953)

* Fix broken LTO mode and response file support (#1948)

* Strip `-Wl,-no-undefined` during compilation (#1952)

Make the compiler wrapper stripping `-Wl,-no-undefined` in addition to `-Wl,--no-undefined`.
Both versions of the flag are accepted by clang and, therefore, used by building systems in the wild (e.g., samba will not build without this fix).

* Remove dead code in write_to_testcase (#1955)

The custom_mutators_count check in if case is duplicate with if condition.
The else case is custom_mutators_count == 0, neither custom_mutator_list iteration nor sent check needed.

Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>

* update qemuafl

* WIP: Add ability to generate drcov trace using QEMU backend (#1956)

* Document new drcov QEMU plugin

* Add link to lightkeeper for QEMU drcov file loading

---------

Co-authored-by: Jean-Romain Garnier <jean-romain.garnier@airbus.com>

* code format

* changelog

* sleep on uid != 0 afl-system-config

* fix segv about skip_next, warn on unsupported cases of linking options (#1958)

* todos

* ensure afl-cc only allows available compiler modes

* update grammar mutator

* disable aslr on apple

* fix for arm64

* help selective instrumentation

* typos

* macos

* add compiler test script

* apple fixes

* bump nyx submodules (#1963)

* fix docs

* update changelog

* update grammar mutator

* improve compiler test script

* gcc asan workaround (#1966)

* fix github merge fuckup

* fix

* Fix afl-cc (#1968)

- Check if too many cmdline params here, each time before insert a new param.
 - Check if it is "-fsanitize=..." before we do sth.
 - Remove improper param_st transfer.

* Avoid adding llvmnative instrumentation when linking rust sanitizer runtime (#1969)

* Dynamic instrumentation filtering for LLVM native (#1971)

* Add two dynamic instrumentation filter methods to runtime

* Always use pc-table with native pcguard

* Add make_symbol_list.py and README

* changelog

* todos

* new forkserver check

* fix

* nyx test for CI

* improve nyx docs

* Fixes to afl-cc and documentation (#1974)

* Always compile with -ldl when building for CODE_COVERAGE

When building with CODE_COVERAGE, the afl runtime contains code that
calls `dladdr` which requires -ldl. Under most circumstances, clang
already adds this (e.g. when building with pc-table), but there are some
circumstances where it isn't added automatically.

* Add visibility declaration to __afl_connected

When building with hidden visibility, the use of __AFL_LOOP inside such
code can cause linker errors due to __afl_connected being declared
"hidden".

* Update docs to clarify that CODE_COVERAGE=1 is required for dynamic_covfilter

* nits

* nyx build script updates

* test error output

* debug ci

* debug ci

* Improve afl-cc (#1975)

* update response file support

 - full support of rsp file
 - fix some segv issues

* Improve afl-cc

 - remove dead code about allow/denylist options of sancov
 - missing `if (!aflcc->have_msan)`
 - add docs for each function
 - typo

* enable nyx

* debug ci

* debug ci

* debug ci

* debug ci

* debug ci

* debug ci

* debug ci

* debug ci

* fix ci

* clean test script

* NO_NYX

* NO_NYX

* fix ci

* debug ci

* fix ci

* finalize ci fix

---------

Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>
Co-authored-by: Sonic <50692172+SonicStark@users.noreply.github.com>
Co-authored-by: Xeonacid <h.dwwwwww@gmail.com>
Co-authored-by: Nils Bars <nils.bars@rub.de>
Co-authored-by: Jean-Romain Garnier <7504819+JRomainG@users.noreply.github.com>
Co-authored-by: Jean-Romain Garnier <jean-romain.garnier@airbus.com>
Co-authored-by: Sergej Schumilo <sergej@schumilo.de>
Co-authored-by: Christian Holler (:decoder) <choller@mozilla.com>

.github/workflows/ci.yml

@@ -20,20 +20,18 @@ jobs:
       AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES: 1
     steps:
       - uses: actions/checkout@v3
+      - name: update
+        run: sudo apt-get update && sudo apt-get upgrade -y
       - name: debug
         run: apt-cache search plugin-dev | grep gcc-; echo; apt-cache search clang-format- | grep clang-format-
-      - name: update
-        run: sudo apt-get update
-        # && sudo apt-get upgrade -y
       - name: install packages
-        #run: sudo apt-get install -y -m -f --install-suggests build-essential git libtool libtool-bin automake bison libglib2.0-0 clang llvm-dev libc++-dev findutils libcmocka-dev python3-dev python3-setuptools ninja-build python3-pip
+        run: sudo apt-get install -y -m -f build-essential gcc-10 g++-10 git libtool libtool-bin automake flex bison libglib2.0-0 clang-12 llvm-12-dev libc++-dev findutils libcmocka-dev python3-dev python3-setuptools ninja-build python3-pip gcc-10-plugin-dev
-        run: sudo apt-get install -y -m -f build-essential git libtool libtool-bin automake flex bison libglib2.0-0 clang llvm-dev libc++-dev findutils libcmocka-dev python3-dev python3-setuptools ninja-build python3-pip
       - name: compiler installed
         run: gcc -v; echo; clang -v
       - name: install gcc plugin
         run: sudo apt-get install -y -m -f --install-suggests $(readlink /usr/bin/gcc)-plugin-dev
       - name: build afl++
-        run: make distrib ASAN_BUILD=1 NO_NYX=1
+        run: export NO_NYX=1; export ASAN_BUILD=1; export LLVM_CONFIG=llvm-config-12; make ASAN_BUILD=1 NO_NYX=1 LLVM_CONFIG=llvm-config-12 distrib
       - name: run tests
         run: sudo -E ./afl-system-config; make tests
  # macos:

Dockerfile

@@ -16,8 +16,8 @@ ENV NO_CORESIGHT=1
 ENV NO_NYX=1
 
 ### Only change these if you know what you are doing:
-# LLVM 15 does not look good so we stay at 14 to still have LTO
+# Current recommended LLVM version is 16
-ENV LLVM_VERSION=14
+ENV LLVM_VERSION=16
 # GCC 12 is producing compile errors for some targets so we stay at GCC 11
 ENV GCC_VERSION=11
 
@@ -88,7 +88,7 @@ ARG TEST_BUILD
 
 RUN sed -i.bak 's/^	-/	/g' GNUmakefile && \
     make clean && make distrib && \
-    ([ "${TEST_BUILD}" ] || (make install && make clean)) && \
+    ([ "${TEST_BUILD}" ] || (make install)) && \
     mv GNUmakefile.bak GNUmakefile
 
 RUN echo "set encoding=utf-8" > /root/.vimrc && \

GNUmakefile

@@ -66,6 +66,10 @@ ifdef MSAN_BUILD
   override LDFLAGS += -fsanitize=memory
 endif
 
+ifdef CODE_COVERAGE
+  override CFLAGS += -D__AFL_CODE_COVERAGE=1
+endif
+
 ifeq "$(findstring android, $(shell $(CC) --version 2>/dev/null))" ""
 ifeq "$(shell echo 'int main() {return 0; }' | $(CC) $(CFLAGS) -Werror -x c - -flto=full -o .test 2>/dev/null && echo 1 || echo 0 ; rm -f .test )" "1"
 	CFLAGS_FLTO ?= -flto=full
@@ -395,7 +399,7 @@ help:
 	@echo INTROSPECTION - compile afl-fuzz with mutation introspection
 	@echo NO_PYTHON - disable python support
 	@echo NO_SPLICING - disables splicing mutation in afl-fuzz, not recommended for normal fuzzing
-	@echo NO_UTF - do not use UTF-8 for line rendering in status screen (fallback to G1 box drawing, of vanilla AFL)
+	@echo "NO_UTF - do not use UTF-8 for line rendering in status screen (fallback to G1 box drawing, of vanilla AFL)"
 	@echo NO_NYX - disable building nyx mode dependencies
 	@echo "NO_CORESIGHT - disable building coresight (arm64 only)"
 	@echo NO_UNICORN_ARM64 - disable building unicorn on arm64
@@ -649,16 +653,16 @@ endif
 	# -$(MAKE) -C utils/plot_ui
 	-$(MAKE) -C frida_mode
 ifneq "$(SYS)" "Darwin"
-  ifeq "$(ARCH)" "aarch64"
+ifeq "$(ARCH)" "aarch64"
   ifndef NO_CORESIGHT
 	-$(MAKE) -C coresight_mode
   endif
-  endif
+endif
-  ifeq "$(SYS)" "Linux"
+ifeq "$(SYS)" "Linux"
-    ifndef NO_NYX
+ifndef NO_NYX
 	-cd nyx_mode && ./build_nyx_support.sh
-    endif
+endif
-  endif
+endif
 	-cd qemu_mode && sh ./build_qemu_support.sh
   ifeq "$(ARCH)" "aarch64"
     ifndef NO_UNICORN_ARM64

TODO.md

@@ -2,26 +2,21 @@
 
 ## Must
 
+ - UI revamp
+ - hardened_usercopy=0 page_alloc.shuffle=0
+ - add value_profile but only enable after 15 minutes without finds
+ - cmplog max len, cmplog max items envs?
  - adapt MOpt to new mutation engine
    - Update afl->pending_not_fuzzed for MOpt
- - cmplog rtn sanity check on fixed length? + no length 1
+ - cmplog rtn sanity check on fixed length? currently we ignore the length
  - afl-showmap -f support
  - afl-fuzz multicore wrapper script
  - when trimming then perform crash detection
- - either -L0 and/or -p mmopt results in zero new coverage
+ - problem: either -L0 and/or -p mmopt results in zero new coverage
 
 
 ## Should
 
-<<<<<<< Updated upstream
- - add value_profile but only enable after 15 minutes without finds?
-=======
- - afl-showmap -f support
- - afl-fuzz multicore wrapper script
- - UI revamp
- - hardened_usercopy=0 page_alloc.shuffle=0
- - add value_profile but only enable after 15 minutes without finds
->>>>>>> Stashed changes
  - afl-crash-analysis
  - support persistent and deferred fork server in afl-showmap?
  - better autodetection of shifting runtime timeout values

afl-cmin

@@ -1,11 +1,15 @@
 #!/usr/bin/env sh
+SYS=$(uname -s)
+test "$SYS" = "Darwin" && {
+  echo Error: afl-cmin does not work on Apple currently. please use afl-cmin.bash instead.
+  exit 1
+}
 export AFL_QUIET=1
 export ASAN_OPTIONS=detect_leaks=0
 THISPATH=`dirname ${0}`
 export PATH="${THISPATH}:$PATH"
 awk -f - -- ${@+"$@"} <<'EOF'
 #!/usr/bin/awk -f
-
 # awk script to minimize a test corpus of input files
 #
 # based on afl-cmin bash script written by Michal Zalewski

custom_mutators/grammar_mutator/GRAMMAR_VERSION

@@ -1 +1 @@
-ff4e5a2
+5ed4f8d

docs/Changelog.md

@@ -12,20 +12,25 @@
   - afl-cc:
     - large rewrite by @SonicStark which fixes a few corner cases, thanks!
     - LTO mode now requires llvm 12+
+    - workaround for ASAN with gcc_plugin mode
   - instrumentation:
     - LLVM 18 support, thanks to @devnexen!
-    - Injection (SQL, LDAP, XSS) feature now available, see
+    - Injection (SQL, LDAP, XSS) fuzzing feature now available, see
       `instrumentation/README.injections.md` how to activate/use/expand.
     - compcov/LAF-intel:
       - floating point splitting bug fix by @hexcoder
       - due a bug in LLVM 17 integer splitting is disabled there!
       - when splitting floats was selected, integers were always split as well,
         fixed to require AFL_LLVM_LAF_SPLIT_COMPARES or _ALL as it should
+    - dynamic instrumentation filtering for LLVM NATIVE, thanks @Mozilla!
+      see utils/dynamic_covfilter/README.md
   - qemu_mode:
     - plugins are now activated by default and a new module is included that
       produces drcov compatible traces for lighthouse/lightkeeper/...
       thanks to @JRomainG to submitting!
+  - updated Nyx checkout (fixes a bug) and some QOL
   - updated the custom grammar mutator
+  - document afl-cmin does not work on macOS (but afl-cmin.bash does)
 
 
 ### Version ++4.09c (release)

instrumentation/README.lto.md

@@ -2,7 +2,7 @@
 
 ## TL;DR:
 
-This version requires a LLVM 11 or newer.
+This version requires a LLVM 12 or newer.
 
 1. Use afl-clang-lto/afl-clang-lto++ because the resulting binaries run
    slightly faster and give better coverage.
@@ -10,7 +10,7 @@ This version requires a LLVM 11 or newer.
 2. You can use it together with COMPCOV, COMPLOG and the instrument file
    listing features.
 
-3. It only works with LLVM 11 or newer.
+3. It only works with LLVM 12 or newer.
 
 4. AUTODICTIONARY feature (see below)
 
@@ -60,7 +60,7 @@ AUTODICTIONARY: 11 strings found
 [+] Instrumented 12071 locations with no collisions (on average 1046 collisions would be in afl-gcc/afl-clang-fast) (non-hardened mode).
 ```
 
-## Getting LLVM 11+
+## Getting LLVM 12+
 
 ### Installing llvm
 
@@ -73,7 +73,7 @@ chmod +x llvm.sh
 sudo ./llvm.sh 15 all
 ```
 
-LLVM 11 to 16 should be available in all current Linux repositories.
+LLVM 12 to 18 should be available in all current Linux repositories.
 
 ## How to build afl-clang-lto
 
@@ -277,7 +277,7 @@ AS=llvm-as  ...
 afl-clang-lto is still work in progress.
 
 Known issues:
-* Anything that LLVM 11+ cannot compile, afl-clang-lto cannot compile either -
+* Anything that LLVM 12+ cannot compile, afl-clang-lto cannot compile either -
   obviously.
 * Anything that does not compile with LTO, afl-clang-lto cannot compile either -
   obviously.
@@ -319,7 +319,7 @@ Still more problems came up though as this only works without bugs from LLVM 9
 onwards, and with high optimization the link optimization ruins the instrumented
 control flow graph.
 
-This is all now fixed with LLVM 11+. The llvm's own linker is now able to load
+This is all now fixed with LLVM 12+. The llvm's own linker is now able to load
 passes and this bypasses all problems we had.
 
 Happy end :)

instrumentation/SanitizerCoveragePCGUARD.so.cc

@@ -627,6 +627,13 @@ void ModuleSanitizerCoverageAFL::instrumentFunction(
 
   }
 
+  if (debug) {
+
+    fprintf(stderr, "SanitizerCoveragePCGUARD: instrumenting %s in %s\n",
+            F.getName().str().c_str(), F.getParent()->getName().str().c_str());
+
+  }
+
   InjectCoverage(F, BlocksToInstrument, IsLeafFunc);
   // InjectTraceForCmp(F, CmpTraceTargets);
   // InjectTraceForSwitch(F, SwitchTraceTargets);

instrumentation/afl-compiler-rt.o.c

@@ -22,6 +22,10 @@
     #define __USE_GNU
   #endif
   #include <dlfcn.h>
+
+__attribute__((weak)) void __sanitizer_symbolize_pc(void *, const char *fmt,
+                                                    char  *out_buf,
+                                                    size_t out_buf_size);
 #endif
 
 #ifdef __ANDROID__
@@ -124,8 +128,8 @@ struct afl_module_info_t {
   uintptr_t base_address;
 
   // PC Guard start/stop
-  u32 start;
+  u32 *start;
-  u32 stop;
+  u32 *stop;
 
   // PC Table begin/end
   const uintptr_t *pcs_beg;
@@ -147,6 +151,18 @@ afl_module_info_t *__afl_module_info = NULL;
 
 u32        __afl_pcmap_size = 0;
 uintptr_t *__afl_pcmap_ptr = NULL;
+
+typedef struct {
+
+  uintptr_t start;
+  u32       len;
+
+} FilterPCEntry;
+
+u32            __afl_filter_pcs_size = 0;
+FilterPCEntry *__afl_filter_pcs = NULL;
+u8            *__afl_filter_pcs_module = NULL;
+
 #endif  // __AFL_CODE_COVERAGE
 
 /* 1 if we are running in afl, and the forkserver was started, else 0 */
@@ -1587,15 +1603,116 @@ void __sanitizer_cov_trace_pc_guard(uint32_t *guard) {
 }
 
 #ifdef __AFL_CODE_COVERAGE
-void __sanitizer_cov_pcs_init(const uintptr_t *pcs_beg,
+void afl_read_pc_filter_file(const char *filter_file) {
-                              const uintptr_t *pcs_end) {
 
-  if (__afl_debug) {
+  FILE *file;
+  char  ch;
 
-    fprintf(stderr, "DEBUG: __sanitizer_cov_pcs_init called\n");
+  file = fopen(filter_file, "r");
+  if (file == NULL) {
+
+    perror("Error opening file");
+    return;
 
   }
 
+  // Check how many PCs we expect to read
+  while ((ch = fgetc(file)) != EOF) {
+
+    if (ch == '\n') { __afl_filter_pcs_size++; }
+
+  }
+
+  // Rewind to actually read the PCs
+  fseek(file, 0, SEEK_SET);
+
+  __afl_filter_pcs = malloc(__afl_filter_pcs_size * sizeof(FilterPCEntry));
+  if (!__afl_filter_pcs) {
+
+    perror("Error allocating PC array");
+    return;
+
+  }
+
+  for (size_t i = 0; i < __afl_filter_pcs_size; i++) {
+
+    fscanf(file, "%lx", &(__afl_filter_pcs[i].start));
+    ch = fgetc(file);  // Read tab
+    fscanf(file, "%u", &(__afl_filter_pcs[i].len));
+    ch = fgetc(file);  // Read tab
+
+    if (!__afl_filter_pcs_module) {
+
+      // Read the module name and store it.
+      // TODO: We only support one module here right now although
+      // there is technically no reason to support multiple modules
+      // in one go.
+      size_t max_module_len = 255;
+      size_t i = 0;
+      __afl_filter_pcs_module = malloc(max_module_len);
+      while (i < max_module_len - 1 &&
+             (__afl_filter_pcs_module[i] = fgetc(file)) != '\t') {
+
+        ++i;
+
+      }
+
+      __afl_filter_pcs_module[i] = '\0';
+      fprintf(stderr, "DEBUGXXX: Read module name %s\n",
+              __afl_filter_pcs_module);
+
+    }
+
+    while ((ch = fgetc(file)) != '\n' && ch != EOF)
+      ;
+
+  }
+
+  fclose(file);
+
+}
+
+u32 locate_in_pcs(uintptr_t needle, u32 *index) {
+
+  size_t lower_bound = 0;
+  size_t upper_bound = __afl_filter_pcs_size - 1;
+
+  while (lower_bound < __afl_filter_pcs_size && lower_bound <= upper_bound) {
+
+    size_t current_index = lower_bound + (upper_bound - lower_bound) / 2;
+
+    if (__afl_filter_pcs[current_index].start <= needle) {
+
+      if (__afl_filter_pcs[current_index].start +
+              __afl_filter_pcs[current_index].len >
+          needle) {
+
+        // Hit
+        *index = current_index;
+        return 1;
+
+      } else {
+
+        lower_bound = current_index + 1;
+
+      }
+
+    } else {
+
+      if (!current_index) { break; }
+      upper_bound = current_index - 1;
+
+    }
+
+  }
+
+  return 0;
+
+}
+
+void __sanitizer_cov_pcs_init(const uintptr_t *pcs_beg,
+                              const uintptr_t *pcs_end) {
+
   // If for whatever reason, we cannot get dlinfo here, then pc_guard_init also
   // couldn't get it and we'd end up attributing to the wrong module.
   Dl_info dlinfo;
@@ -1608,6 +1725,16 @@ void __sanitizer_cov_pcs_init(const uintptr_t *pcs_beg,
 
   }
 
+  if (__afl_debug) {
+
+    fprintf(
+        stderr,
+        "DEBUG: (%u) __sanitizer_cov_pcs_init called for module %s with %ld "
+        "PCs\n",
+        getpid(), dlinfo.dli_fname, pcs_end - pcs_beg);
+
+  }
+
   afl_module_info_t *last_module_info = __afl_module_info;
   while (last_module_info && last_module_info->next) {
 
@@ -1623,34 +1750,78 @@ void __sanitizer_cov_pcs_init(const uintptr_t *pcs_beg,
 
   }
 
+  if (strcmp(dlinfo.dli_fname, last_module_info->name)) {
+
+    // This can happen with modules being loaded after the forkserver
+    // where we decide to not track the module. In that case we must
+    // not track it here either.
+    fprintf(
+        stderr,
+        "WARNING: __sanitizer_cov_pcs_init module info mismatch: %s vs %s\n",
+        dlinfo.dli_fname, last_module_info->name);
+    return;
+
+  }
+
   last_module_info->pcs_beg = pcs_beg;
   last_module_info->pcs_end = pcs_end;
 
+  // This is a direct filter based on symbolizing inside the runtime.
+  // It should only be used with smaller binaries to avoid long startup
+  // times. Currently, this only supports a single token to scan for.
+  const char *pc_filter = getenv("AFL_PC_FILTER");
+
+  // This is a much faster PC filter based on pre-symbolized input data
+  // that is sorted for fast lookup through binary search. This method
+  // of filtering is suitable even for very large binaries.
+  const char *pc_filter_file = getenv("AFL_PC_FILTER_FILE");
+  if (pc_filter_file && !__afl_filter_pcs) {
+
+    afl_read_pc_filter_file(pc_filter_file);
+
+  }
+
   // Now update the pcmap. If this is the last module coming in, after all
   // pre-loaded code, then this will also map all of our delayed previous
   // modules.
-
+  //
-  if (!__afl_pcmap_ptr) { return; }
-
   for (afl_module_info_t *mod_info = __afl_module_info; mod_info;
        mod_info = mod_info->next) {
 
     if (mod_info->mapped) { continue; }
 
+    if (!mod_info->start) {
+
+      fprintf(stderr,
+              "ERROR: __sanitizer_cov_pcs_init called with mod_info->start == "
+              "NULL (%s)\n",
+              mod_info->name);
+      abort();
+
+    }
+
     PCTableEntry *start = (PCTableEntry *)(mod_info->pcs_beg);
     PCTableEntry *end = (PCTableEntry *)(mod_info->pcs_end);
 
+    if (!*mod_info->stop) { continue; }
+
     u32 in_module_index = 0;
 
     while (start < end) {
 
-      if (mod_info->start + in_module_index >= __afl_map_size) {
+      if (*mod_info->start + in_module_index >= __afl_map_size) {
 
-        fprintf(stderr, "ERROR: __sanitizer_cov_pcs_init out of bounds?!\n");
+        fprintf(stderr,
+                "ERROR: __sanitizer_cov_pcs_init out of bounds?! Start: %u "
+                "Stop: %u Map Size: %u (%s)\n",
+                *mod_info->start, *mod_info->stop, __afl_map_size,
+                mod_info->name);
         abort();
 
       }
 
+      u32 orig_start_index = *mod_info->start;
+
       uintptr_t PC = start->PC;
 
       // This is what `GetPreviousInstructionPc` in sanitizer runtime does
@@ -1660,7 +1831,58 @@ void __sanitizer_cov_pcs_init(const uintptr_t *pcs_beg,
       // Calculate relative offset in module
       PC = PC - mod_info->base_address;
 
-      __afl_pcmap_ptr[mod_info->start + in_module_index] = PC;
+      if (__afl_pcmap_ptr) {
+
+        __afl_pcmap_ptr[orig_start_index + in_module_index] = PC;
+
+      }
+
+      if (pc_filter) {
+
+        char PcDescr[1024];
+        // This function is a part of the sanitizer run-time.
+        // To use it, link with AddressSanitizer or other sanitizer.
+        __sanitizer_symbolize_pc((void *)start->PC, "%p %F %L", PcDescr,
+                                 sizeof(PcDescr));
+
+        if (strstr(PcDescr, pc_filter)) {
+
+          if (__afl_debug)
+            fprintf(
+                stderr,
+                "DEBUG: Selective instrumentation match: %s (PC %p Index %u)\n",
+                PcDescr, (void *)start->PC,
+                *(mod_info->start + in_module_index));
+          // No change to guard needed
+
+        } else {
+
+          // Null out the guard to disable this edge
+          *(mod_info->start + in_module_index) = 0;
+
+        }
+
+      }
+
+      if (__afl_filter_pcs && strstr(mod_info->name, __afl_filter_pcs_module)) {
+
+        u32 result_index;
+        if (locate_in_pcs(PC, &result_index)) {
+
+          if (__afl_debug)
+            fprintf(stderr,
+                    "DEBUG: Selective instrumentation match: (PC %lx File "
+                    "Index %u PC Index %u)\n",
+                    PC, result_index, in_module_index);
+
+        } else {
+
+          // Null out the guard to disable this edge
+          *(mod_info->start + in_module_index) = 0;
+
+        }
+
+      }
 
       start++;
       in_module_index++;
@@ -1671,8 +1893,10 @@ void __sanitizer_cov_pcs_init(const uintptr_t *pcs_beg,
 
     if (__afl_debug) {
 
-      fprintf(stderr, "DEBUG: __sanitizer_cov_pcs_init initialized %u PCs\n",
+      fprintf(stderr,
-              in_module_index);
+              "DEBUG: __sanitizer_cov_pcs_init successfully mapped %s with %u "
+              "PCs\n",
+              mod_info->name, in_module_index);
 
     }
 
@@ -1706,9 +1930,9 @@ void __sanitizer_cov_trace_pc_guard_init(uint32_t *start, uint32_t *stop) {
     fprintf(
         stderr,
         "DEBUG: Running __sanitizer_cov_trace_pc_guard_init: %p-%p (%lu edges) "
-        "after_fs=%u\n",
+        "after_fs=%u *start=%u\n",
         start, stop, (unsigned long)(stop - start),
-        __afl_already_initialized_forkserver);
+        __afl_already_initialized_forkserver, *start);
 
   }
 
@@ -1740,8 +1964,8 @@ void __sanitizer_cov_trace_pc_guard_init(uint32_t *start, uint32_t *stop) {
       mod_info->id = last_module_info ? last_module_info->id + 1 : 0;
       mod_info->name = strdup(dlinfo.dli_fname);
       mod_info->base_address = (uintptr_t)dlinfo.dli_fbase;
-      mod_info->start = 0;
+      mod_info->start = NULL;
-      mod_info->stop = 0;
+      mod_info->stop = NULL;
       mod_info->pcs_beg = NULL;
       mod_info->pcs_end = NULL;
       mod_info->mapped = 0;
@@ -1757,8 +1981,12 @@ void __sanitizer_cov_trace_pc_guard_init(uint32_t *start, uint32_t *stop) {
 
       }
 
-      fprintf(stderr, "[pcmap] Module: %s Base Address: %p\n", dlinfo.dli_fname,
+      if (__afl_debug) {
-              dlinfo.dli_fbase);
+
+        fprintf(stderr, "[pcmap] Module: %s Base Address: %p\n",
+                dlinfo.dli_fname, dlinfo.dli_fbase);
+
+      }
 
     }
 
@@ -1861,12 +2089,17 @@ void __sanitizer_cov_trace_pc_guard_init(uint32_t *start, uint32_t *stop) {
 #ifdef __AFL_CODE_COVERAGE
   if (mod_info) {
 
-    mod_info->start = *orig_start;
+    if (!mod_info->start) {
-    mod_info->stop = *(stop - 1);
+
+      mod_info->start = orig_start;
+      mod_info->stop = stop - 1;
+
+    }
+
     if (__afl_debug) {
 
       fprintf(stderr, "DEBUG: [pcmap] Start Index: %u Stop Index: %u\n",
-              mod_info->start, mod_info->stop);
+              *(mod_info->start), *(mod_info->stop));
 
     }
 

instrumentation/afl-llvm-common.cc

@@ -201,7 +201,7 @@ void initInstrumentList() {
 
     if (debug)
       DEBUGF("loaded allowlist with %zu file and %zu function entries\n",
-             allowListFiles.size(), allowListFunctions.size());
+             allowListFiles.size() / 4, allowListFunctions.size() / 4);
 
   }
 
@@ -276,7 +276,7 @@ void initInstrumentList() {
 
     if (debug)
       DEBUGF("loaded denylist with %zu file and %zu function entries\n",
-             denyListFiles.size(), denyListFunctions.size());
+             denyListFiles.size() / 4, denyListFunctions.size() / 4);
 
   }
 

nyx_mode/LIBNYX_VERSION

@@ -1 +1 @@
-512058a
+6833d23

nyx_mode/QEMU_NYX_VERSION

@@ -1 +1 @@
-02a6f2aed3
+1def26f83e

nyx_mode/README.md

@@ -84,9 +84,17 @@ Then the final step: we generate the Nyx package configuration:
 python3 nyx_mode/packer/packer/nyx_config_gen.py PACKAGE-DIRECTORY Kernel
 ```
 
-
 ## Fuzzing with Nyx mode
 
+Note that you need to load the kvm kernel modules for Nyx:
+```
+sudo modprobe -r kvm-intel
+sudo modprobe -r kvm
+sudo modprobe  kvm enable_vmware_backdoor=y
+sudo modprobe  kvm-intel
+cat /sys/module/kvm/parameters/enable_vmware_backdoor | grep -q Y && echi OK || echo KVM module problem
+```
+
 All the hard parts are done, fuzzing with Nyx mode is easy - just supply the
 `PACKAGE-DIRECTORY` as fuzzing target and specify the `-X` option to afl-fuzz:
 
@@ -94,16 +102,8 @@ All the hard parts are done, fuzzing with Nyx mode is easy - just supply the
 afl-fuzz -i in -o out -X -- ./PACKAGE-DIRECTORY
 ```
 
-Most likely your first run will fail because the Linux modules have to be
+If you get a forkserver error upon starting then you did not load the Linux
-specially set up, but afl-fuzz will tell you this on startup and how to rectify
+kvm kernel modules, see above.
-the situation:
-
-```
-sudo modprobe -r kvm-intel # or kvm-amd for AMD processors
-sudo modprobe -r kvm
-sudo modprobe kvm enable_vmware_backdoor=y
-sudo modprobe kvm-intel # or kvm-amd for AMD processors
-```
 
 If you want to fuzz in parallel (and you should!), then this has to be done in a
 special way:

nyx_mode/build_nyx_support.sh

@@ -9,6 +9,21 @@ echo
 
 echo "[*] Performing basic sanity checks..."
 
+if [ "$CI" = "true" ]; then
+
+  echo "[-] Error: nyx_mode cannot be tested in the Github CI, skipping ..."
+  exit 0
+
+fi
+
+
+if [ -n "$NO_NYX" ]; then
+
+  echo "[-] Error: the NO_NYX environment variable is set, please unset."
+  exit 0
+
+fi
+
 if [ ! "$(uname -s)" = "Linux" ]; then
 
   echo "[-] Error: Nyx mode is only available on Linux."
@@ -23,11 +38,17 @@ if [ ! "$(uname -m)" = "x86_64" ]; then
 
 fi
 
+cargo help > /dev/null 2>&1 || {
+   echo "[-] Error: Rust is not installed."
+   exit 0
+}
+
 echo "[*] Making sure all Nyx is checked out"
 
 
 if git status 1>/dev/null 2>&1; then
 
+  set +e
   git submodule init
   echo "[*] initializing QEMU-Nyx submodule"
   git submodule update ./QEMU-Nyx 2>/dev/null # ignore errors
@@ -35,6 +56,7 @@ if git status 1>/dev/null 2>&1; then
   git submodule update ./packer 2>/dev/null # ignore errors
   echo "[*] initializing libnyx submodule"
   git submodule update ./libnyx 2>/dev/null # ignore errors
+  set -e
 
 else
 
@@ -48,20 +70,57 @@ test -e packer/.git || { echo "[-] packer not checked out, please install git or
 test -e libnyx/.git || { echo "[-] libnyx not checked out, please install git or check your internet connection." ; exit 1 ; }
 test -e QEMU-Nyx/.git || { echo "[-] QEMU-Nyx not checked out, please install git or check your internet connection." ; exit 1 ; }
 
-echo "[*] checking packer init.cpio.gz ..."
+
-if [ ! -f "packer/linux_initramfs/init.cpio.gz" ]; then
+QEMU_NYX_VERSION="$(cat ./QEMU_NYX_VERSION)"
-    (cd packer/linux_initramfs/ && sh pack.sh)
+cd "./QEMU-Nyx" || exit 1
+if [ -n "$NO_CHECKOUT" ]; then
+  echo "[*] Skipping checkout to $QEMU_NYX_VERSION"
+else
+  echo "[*] Checking out $QEMU_NYX_VERSION"
+  set +e
+  sh -c 'git stash' 1>/dev/null 2>/dev/null
+  git pull 1>/dev/null 2>/dev/null
+  git checkout "$QEMU_NYX_VERSION" || echo Warning: could not check out to commit $QEMU_NYX_VERSION
+  set -e
 fi
+cd - > /dev/null
+
+PACKER_VERSION="$(cat ./PACKER_VERSION)"
+cd "./packer" || exit 1
+if [ -n "$NO_CHECKOUT" ]; then
+  echo "[*] Skipping checkout to $PACKER_VERSION"
+else
+  echo "[*] Checking out $PACKER_VERSION"
+  set +e
+  sh -c 'git stash' 1>/dev/null 2>/dev/null
+  git pull 1>/dev/null 2>/dev/null
+  git checkout "$PACKER_VERSION" || echo Warning: could not check out to commit $PACKER_VERSION
+  set -e
+fi
+cd - > /dev/null
+
+LIBNYX_VERSION="$(cat ./LIBNYX_VERSION)"
+cd "./libnyx/" || exit 1
+if [ -n "$NO_CHECKOUT" ]; then
+  echo "[*] Skipping checkout to $LIBNYX_VERSION"
+else
+  echo "[*] Checking out $LIBNYX_VERSION"
+  set +e
+  sh -c 'git stash' 1>/dev/null 2>/dev/null
+  git pull 1>/dev/null 2>/dev/null
+  git checkout "$LIBNYX_VERSION" || echo Warning: could not check out to commit $LIBNYX_VERSION
+  set -e
+fi
+cd - > /dev/null
+
+echo "[*] checking packer init.cpio.gz ..."
+(cd packer/linux_initramfs/ && sh pack.sh)
 
 echo "[*] Checking libnyx ..."
-if [ ! -f "libnyx/libnyx/target/release/liblibnyx.a" ]; then
+(cd libnyx/libnyx && cargo build --release)
-    (cd libnyx/libnyx && cargo build --release)
-fi
 
 echo "[*] Checking QEMU-Nyx ..."
-if [ ! -f "QEMU-Nyx/x86_64-softmmu/qemu-system-x86_64" ]; then
+(cd QEMU-Nyx && ./compile_qemu_nyx.sh static )
-    (cd QEMU-Nyx && ./compile_qemu_nyx.sh static)
-fi
 
 echo "[*] Checking libnyx.so ..."
 cp libnyx/libnyx/target/release/liblibnyx.so ../libnyx.so

nyx_mode/update_ref.sh

@@ -41,7 +41,7 @@ cd ..
 rm "$UC_VERSION_FILE"
 echo "$NEW_VERSION" > "$UC_VERSION_FILE"
 
-echo "Done. New XXX version is $NEW_VERSION."
+echo "Done. New libnyx version is $NEW_VERSION."
 
 
 UC_VERSION_FILE='./PACKER_VERSION'
@@ -68,7 +68,7 @@ cd ..
 rm "$UC_VERSION_FILE"
 echo "$NEW_VERSION" > "$UC_VERSION_FILE"
 
-echo "Done. New XXX version is $NEW_VERSION."
+echo "Done. New packer version is $NEW_VERSION."
 
 
 UC_VERSION_FILE='./QEMU_NYX_VERSION'
@@ -95,5 +95,5 @@ cd ..
 rm "$UC_VERSION_FILE"
 echo "$NEW_VERSION" > "$UC_VERSION_FILE"
 
-echo "Done. New XXX version is $NEW_VERSION."
+echo "Done. New QEMU-Nyx version is $NEW_VERSION."
 

src/afl-forkserver.c

@@ -1017,6 +1017,12 @@ void afl_fsrv_start(afl_forkserver_t *fsrv, char **argv,
 
   if (rlen == 4) {
 
+    if (status >= 0x41464c00 && status <= 0x41464cff) {
+
+      FATAL("Target uses the new forkserver model, you need to switch to a newer afl-fuzz too!");
+
+    }
+
     if (!be_quiet) { OKF("All right - fork server is up."); }
 
     if (getenv("AFL_DEBUG")) {

src/afl-fuzz-init.c

@@ -124,6 +124,9 @@ void bind_to_free_cpu(afl_state_t *afl) {
     }
 
     WARNF("Not binding to a CPU core (AFL_NO_AFFINITY set).");
+  #ifdef __linux__
+    if (afl->fsrv.nyx_mode) { afl->fsrv.nyx_bind_cpu_id = 0; }
+  #endif
     return;
 
   }
@@ -151,6 +154,9 @@ void bind_to_free_cpu(afl_state_t *afl) {
     } else {
 
       OKF("CPU binding request using -b %d successful.", afl->cpu_to_bind);
+  #ifdef __linux__
+      if (afl->fsrv.nyx_mode) { afl->fsrv.nyx_bind_cpu_id = afl->cpu_to_bind; }
+  #endif
 
     }
 

src/afl-fuzz.c

@@ -165,7 +165,7 @@ static void usage(u8 *argv0, int more_help) {
       "\n"
 
       "Mutator settings:\n"
-      "  -a            - target input format, \"text\" or \"binary\" (default: "
+      "  -a type       - target input format, \"text\" or \"binary\" (default: "
       "generic)\n"
       "  -g minlength  - set min length of generated fuzz input (default: 1)\n"
       "  -G maxlength  - set max length of generated fuzz input (default: "
@@ -1915,6 +1915,15 @@ int main(int argc, char **argv_orig, char **envp) {
   bind_to_free_cpu(afl);
   #endif                                                   /* HAVE_AFFINITY */
 
+  #ifdef __linux__
+  if (afl->fsrv.nyx_mode && afl->fsrv.nyx_bind_cpu_id == 0xFFFFFFFF) {
+
+    afl->fsrv.nyx_bind_cpu_id = 0;
+
+  }
+
+  #endif
+
   #ifdef __HAIKU__
   /* Prioritizes performance over power saving */
   set_scheduler_mode(SCHEDULER_MODE_LOW_LATENCY);

test/test-all.sh

@@ -16,6 +16,8 @@
 
 . ./test-frida-mode.sh
 
+. ./test-nyx-mode.sh
+
 . ./test-unicorn-mode.sh
 
 . ./test-custom-mutators.sh

test/test-basic.sh

@@ -2,6 +2,7 @@
 
 . ./test-pre.sh
 
+OS=$(uname -s)
 
 AFL_GCC=afl-gcc
 $ECHO "$BLUE[*] Testing: ${AFL_GCC}, afl-showmap, afl-fuzz, afl-cmin and afl-tmin"
@@ -61,7 +62,7 @@ test "$SYS" = "i686" -o "$SYS" = "x86_64" -o "$SYS" = "amd64" -o "$SYS" = "i86pc
    }
    # now we want to be sure that afl-fuzz is working
    # make sure crash reporter is disabled on Mac OS X
-   (test "$(uname -s)" = "Darwin" && test $(launchctl list 2>/dev/null | grep -q '\.ReportCrash$') && {
+   (test "$OS" = "Darwin" && test $(launchctl list 2>/dev/null | grep -q '\.ReportCrash$') && {
     $ECHO "$RED[!] we cannot run afl-fuzz with enabled crash reporter. Run 'sudo sh afl-system-config'.$RESET"
     true
    }) || {
@@ -84,6 +85,9 @@ test "$SYS" = "i686" -o "$SYS" = "x86_64" -o "$SYS" = "amd64" -o "$SYS" = "i86pc
     }
     echo 000000000000000000000000 > in/in2
     echo 111 > in/in3
+    test "$OS" = "Darwin" && {
+      $ECHO "$GREY[*] afl-cmin not available on macOS, cannot test afl-cmin"
+    } || {
       mkdir -p in2
       ../afl-cmin -m ${MEM_LIMIT} -i in -o in2 -- ./test-instr.plain >/dev/null 2>&1 # why is afl-forkserver writing to stderr?
       CNT=`ls in2/* 2>/dev/null | wc -l`
@@ -94,6 +98,7 @@ test "$SYS" = "i686" -o "$SYS" = "x86_64" -o "$SYS" = "amd64" -o "$SYS" = "i86pc
             ;;
       esac
       rm -f in2/in*
+    }
     export AFL_QUIET=1
     if command -v bash >/dev/null ; then {
       ../afl-cmin.bash -m ${MEM_LIMIT} -i in -o in2 -- ./test-instr.plain >/dev/null
@@ -182,7 +187,7 @@ test "$SYS" = "i686" -o "$SYS" = "x86_64" -o "$SYS" = "amd64" -o "$SYS" = "i86pc
    }
    # now we want to be sure that afl-fuzz is working
    # make sure crash reporter is disabled on Mac OS X
-   (test "$(uname -s)" = "Darwin" && test $(launchctl list 2>/dev/null | grep -q '\.ReportCrash$') && {
+   (test "$OS" = "Darwin" && test $(launchctl list 2>/dev/null | grep -q '\.ReportCrash$') && {
     $ECHO "$RED[!] we cannot run afl-fuzz with enabled crash reporter. Run 'sudo sh afl-system-config'.$RESET"
     true
    }) || {
@@ -204,7 +209,10 @@ test "$SYS" = "i686" -o "$SYS" = "x86_64" -o "$SYS" = "amd64" -o "$SYS" = "i86pc
       }
     }
     echo 000000000000000000000000 > in/in2
-    echo AAA > in/in3
+    echo AAA > in/in2
+    test "$OS" = "Darwin" && {
+      $ECHO "$GREY[*] afl-cmin not available on macOS, cannot test afl-cmin"
+    } || {
       mkdir -p in2
       ../afl-cmin -m ${MEM_LIMIT} -i in -o in2 -- ./test-instr.plain >/dev/null 2>&1 # why is afl-forkserver writing to stderr?
       CNT=`ls in2/* 2>/dev/null | wc -l`
@@ -223,6 +231,7 @@ test "$SYS" = "i686" -o "$SYS" = "x86_64" -o "$SYS" = "amd64" -o "$SYS" = "i86pc
             ;;
       esac
       rm -f in2/in*
+    }
     export AFL_QUIET=1
     if command -v bash >/dev/null ; then {
       ../afl-cmin.bash -m ${MEM_LIMIT} -i in -o in2 -- ./test-instr.plain >/dev/null

test/test-compilers.sh

@@ -0,0 +1,7 @@
+#!/bin/sh
+echo Testing compilers ...
+for cc in afl-cc afl-gcc afl-clang afl-clang-fast afl-clang-lto afl-gcc-fast; do
+  test -e ../$cc && { { ../$cc -o t ../test-instr.c >/dev/null 2<&1 && echo Success: $cc ; } || echo Failing: $cc ; } || echo Missing: $cc
+done
+rm -f t
+echo Done!

test/test-llvm.sh

@@ -2,6 +2,8 @@
 
 . ./test-pre.sh
 
+OS=$(uname -s)
+
 $ECHO "$BLUE[*] Testing: llvm_mode, afl-showmap, afl-fuzz, afl-cmin and afl-tmin"
 test -e ../afl-clang-fast -a -e ../split-switches-pass.so && {
   ../afl-clang-fast -o test-instr.plain ../test-instr.c > /dev/null 2>&1
@@ -123,7 +125,7 @@ test -e ../afl-clang-fast -a -e ../split-switches-pass.so && {
   }
   # now we want to be sure that afl-fuzz is working
   # make sure crash reporter is disabled on Mac OS X
-  (test "$(uname -s)" = "Darwin" && test $(launchctl list 2>/dev/null | grep -q '\.ReportCrash$') && {
+  (test "$OS" = "Darwin" && test $(launchctl list 2>/dev/null | grep -q '\.ReportCrash$') && {
     $ECHO "$RED[!] we cannot run afl-fuzz with enabled crash reporter. Run 'sudo sh afl-system-config'.$RESET"
     CODE=1
     true
@@ -146,9 +148,12 @@ test -e ../afl-clang-fast -a -e ../split-switches-pass.so && {
       }
     }
     test "$SYS" = "i686" -o "$SYS" = "x86_64" -o "$SYS" = "amd64" -o "$SYS" = "i86pc" || {
+      mkdir -p in2
       echo 000000000000000000000000 > in/in2
       echo 111 > in/in3
-      mkdir -p in2
+      test "$OS" = "Darwin" && {
+        $ECHO "$GREY[*] afl-cmin not available on macOS, cannot test afl-cmin"
+      } || {
         ../afl-cmin -m ${MEM_LIMIT} -i in -o in2 -- ./test-instr.plain >/dev/null 2>&1 # why is afl-forkserver writing to stderr?
         CNT=`ls in2/* 2>/dev/null | wc -l`
         case "$CNT" in
@@ -158,6 +163,7 @@ test -e ../afl-clang-fast -a -e ../split-switches-pass.so && {
               ;;
         esac
         rm -f in2/in*
+      }
       export AFL_QUIET=1
       if type bash >/dev/null ; then {
         ../afl-cmin.bash -m ${MEM_LIMIT} -i in -o in2 -- ./test-instr.plain >/dev/null

test/test-nyx-mode.sh

@@ -0,0 +1,79 @@
+#!/bin/sh
+
+. ./test-pre.sh
+
+$ECHO "$BLUE[*] Testing: nyx_mode"
+
+test "$CI" = "true" && {
+  $ECHO "$YELLOW[-] nyx_mode cannot be tested in the Github CI, skipping ..."
+  exit 0
+}
+
+unset AFL_CC
+
+test -e ../libnyx.so && {
+  ../afl-cc -o test-instr ../test-instr.c > errors 2>&1
+  test -e test-instr && {
+    {
+      rm -rf nyx-test in out
+      $ECHO "$GREY[*] running nyx_packer"
+      python3 ../nyx_mode/packer/packer/nyx_packer.py \
+        ./test-instr \
+        nyx-test \
+        afl \
+        instrumentation \
+        --fast_reload_mode \
+        --purge > /dev/null 2>&1
+
+      test -e nyx-test/test-instr && {
+
+        $ECHO "$GREY[*] running nyx_config_gen"
+        python3 ../nyx_mode/packer/packer/nyx_config_gen.py nyx-test Kernel > /dev/null 2>&1
+        
+        test -e nyx-test/config.ron && {
+          sudo modprobe -r kvm-intel
+          sudo modprobe -r kvm
+          sudo modprobe  kvm enable_vmware_backdoor=y
+          sudo modprobe  kvm-intel
+          #cat /sys/module/kvm/parameters/enable_vmware_backdoor 
+
+          mkdir -p in
+          echo 00000 > in/in
+          $ECHO "$GREY[*] running afl-fuzz for nyx_mode, this will take approx 10 seconds"
+          {
+            AFL_DEBUG=1 ../afl-fuzz -i in -o out -V05 -X -- ./nyx-test >>errors 2>&1
+          } >>errors 2>&1
+          test -n "$( ls out/default/queue/id:000002* 2>/dev/null )" && {
+            $ECHO "$GREEN[+] afl-fuzz is working correctly with nyx_mode"
+            RUNTIME=`grep execs_done out/default/fuzzer_stats | awk '{print$3}'`
+            rm -rf errors nyx-test test-instr in out
+          } || {
+            echo CUT------------------------------------------------------------------CUT
+            cat errors
+            echo CUT------------------------------------------------------------------CUT
+            $ECHO "$RED[!] afl-fuzz is not working correctly with nyx_mode"
+            CODE=1
+          }
+        } || {
+          $ECHO "$RED[!] nyx_packer failed, likely install requirements not met."
+          CODE=1
+        }
+      } || {
+       $ECHO "$RED[!] nyx_packer failed, likely install requirements not met."
+       CODE=1
+      }
+      #rm -rf test-instr in out errors nyx-test
+    }
+  } || {
+    echo CUT------------------------------------------------------------------CUT
+    cat errors
+    echo CUT------------------------------------------------------------------CUT
+    $ECHO "$RED[!] afl-cc compilation of test targets failed - what is going on??"
+    CODE=1
+  }
+} || {
+  $ECHO "$YELLOW[-] nyx_mode is not compiled, cannot test"
+  INCOMPLETE=1
+}
+
+. ./test-post.sh

test/test-pre.sh

@@ -20,7 +20,7 @@ echo foobar | grep -qE 'asd|oob' 2>/dev/null || { echo Error: grep command does
 test -e ./test-all.sh || cd $(dirname $0) || exit 1
 test -e ./test-all.sh || { echo Error: you must be in the test/ directory ; exit 1 ; }
 export AFL_PATH=`pwd`/..
-export AFL_NO_AFFINITY=1 # workaround for travis that fails for no avail cores 
+export AFL_TRY_AFFINITY=1 # workaround for travis that fails for no avail cores
 
 echo 1 > test.1
 echo 1 > test.2

utils/dynamic_covfilter/README.md

@@ -0,0 +1,60 @@
+# Dynamic Instrumentation Filter
+
+Sometimes it can be beneficial to limit the instrumentation feedback to
+specific code locations. It is possible to do so at compile-time by simply
+not instrumenting any undesired locations. However, there are situations
+where doing this dynamically without requiring a new build can be beneficial.
+Especially when dealing with larger builds, it is much more convenient to
+select the target code locations at runtime instead of doing so at build time.
+
+There are two ways of doing this in AFL++. Both approaches require a build of
+AFL++ with `CODE_COVERAGE=1`, so make sure to build AFL++ first by invoking
+
+`CODE_COVERAGE=1 make`
+
+Once you have built AFL++, you can choose out of two approaches:
+
+## Simple Selection with `AFL_PC_FILTER`
+
+This approach requires a build with `AFL_INSTRUMENTATION=llvmnative` or
+`llvmcodecov` as well as an AddressSanitizer build with debug information.
+
+By setting the environment variable `AFL_PC_FILTER` to a string, the runtime
+symbolizer is enabled in the AFL++ runtime. At startup, the runtime will call
+the `__sanitizer_symbolize_pc` API to resolve every PC in every loaded module.
+The runtime then matches the result using `strstr` and disables the PC guard
+if the symbolized PC does not contain the specified string.
+
+This approach has the benefit of being very easy to use. The downside is that
+it causes significant startup delays with large binaries and that it requires
+an AddressSanitizer build.
+
+This method has no additional runtime overhead after startup.
+
+## Selection using pre-symbolized data file with `AFL_PC_FILTER_FILE`
+
+To avoid large startup time delays, a specific module can be pre-symbolized
+using the `make_symbol_list.py` script. This script outputs a sorted list of
+functions with their respective relative offsets and lengths in the target
+binary:
+
+`python3 make_symbol_list.py libxul.so > libxul.symbols.txt`
+
+The resulting list can be filtered, e.g. using grep:
+
+`grep -i "webgl" libxul.symbols.txt > libxul.webgl.symbols.txt`
+
+Finally, you can run with `AFL_PC_FILTER_FILE=libxul.webgl.symbols.txt` to
+restrict instrumentation feedback to the given locations. This approach only
+has a minimal startup time delay due to the implementation only using binary
+search on the given file per PC rather than reading debug information for every
+PC. It also works well with Nyx, where symbolizing is usually disabled for the
+target process to avoid delays with frequent crashes.
+
+Similar to the previous method, This approach requires a build with 
+`AFL_INSTRUMENTATION=llvmnative` or `llvmcodecov` as well debug information.
+However, it does not require the ASan runtime as it doesn't do the symbolizing
+in process. Due to the way it maps PCs to symbols, it is less accurate when it
+comes to includes and inlines (it assumes all PCs within a function belong to
+that function and originate from the same file). For most purposes, this should
+be a reasonable simplification to quickly process even the largest binaries.

utils/dynamic_covfilter/make_symbol_list.py

@@ -0,0 +1,73 @@
+# This Source Code Form is subject to the terms of the Mozilla Public
+# License, v. 2.0. If a copy of the MPL was not distributed with this
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
+#
+# Written by Christian Holler <decoder at mozilla dot com>
+
+import json
+import os
+import sys
+import subprocess
+
+if len(sys.argv) != 2:
+    print("Usage: %s binfile" % os.path.basename(sys.argv[0]))
+    sys.exit(1)
+
+binfile = sys.argv[1]
+
+addr2len = {}
+addrs = []
+
+output = subprocess.check_output(["objdump", "-t", binfile]).decode("utf-8")
+for line in output.splitlines():
+    line = line.replace("\t", " ")
+    components = [x for x in line.split(" ") if x]
+    if not components:
+        continue
+    try:
+        start_addr = int(components[0], 16)
+    except ValueError:
+        continue
+
+    # Length has variable position in objdump output
+    length = None
+    for comp in components[1:]:
+        if len(comp) == 16:
+            try:
+                length = int(comp, 16)
+                break
+            except:
+                continue
+
+    if length is None:
+        print("ERROR: Couldn't determine function section length: %s" % line)
+
+    func = components[-1]
+    
+    addrs.append(start_addr)
+    addr2len[str(hex(start_addr))] = str(length)
+
+# The search implementation in the AFL runtime expects everything sorted.
+addrs.sort()
+addrs = [str(hex(addr)) for addr in addrs]
+
+# We symbolize in one go to speed things up with large binaries.
+output = subprocess.check_output([
+    "llvm-addr2line",
+    "--output-style=JSON",
+    "-f", "-C", "-a", "-e",
+    binfile],
+    input="\n".join(addrs).encode("utf-8")).decode("utf-8")
+
+output = output.strip().splitlines()
+for line in output:
+    output = json.loads(line)
+    if "Symbol" in output and output["Address"] in addr2len:
+        final_output = [
+            output["Address"],
+            addr2len[output["Address"]],
+            os.path.basename(output["ModuleName"]),
+            output["Symbol"][0]["FileName"],
+            output["Symbol"][0]["FunctionName"]
+        ]
+        print("\t".join(final_output))