deprecated AFL_POST_LIBRARY

README.md

@@ -672,8 +672,9 @@ Here are some of the most important caveats for AFL:
 
     To work around this, you can comment out the relevant checks (see
     examples/libpng_no_checksum/ for inspiration); if this is not possible,
-    you can also write a postprocessor, as explained in
+    you can also write a postprocessor, one of the hooks of custom mutators.
-    examples/post_library/ (with AFL_POST_LIBRARY)
+    See [docs/custom_mutators.md](docs/custom_mutators.md) on how to use
+    `AFL_CUSTOM_MUTATOR_LIBRARY`
 
   - There are some unfortunate trade-offs with ASAN and 64-bit binaries. This
     isn't due to any specific fault of afl-fuzz; see [docs/notes_for_asan.md](docs/notes_for_asan.md)

docs/Changelog.md

@@ -15,6 +15,8 @@ sending a mail to <afl-users+subscribe@googlegroups.com>.
      - better python detection
      - an old, old bug in afl that would show negative stability in rare
        circumstances is now hopefully fixed
+     - AFL_POST_LIBRARY was deprecated, use AFL_CUSTOM_MUTATOR_LIBRARY
+       instead (see docs/custom_mutators.md)
   - llvm_mode:
      - afl-clang-fast/lto now do not skip single block functions. This
        behaviour can be reactivated with AFL_LLVM_SKIPSINGLEBLOCK

docs/env_variables.md

@@ -310,9 +310,8 @@ checks or alter some of the more exotic semantics of the tool:
     else. This makes the "own finds" counter in the UI more accurate.
     Beyond counter aesthetics, not much else should change.
 
-  - Setting AFL_POST_LIBRARY allows you to configure a postprocessor for
+  - Note that AFL_POST_LIBRARY is deprecated, use AFL_CUSTOM_MUTATOR_LIBRARY
-    mutated files - say, to fix up checksums. See examples/post_library/
+    instead (see below).
-    for more.
 
   - Setting AFL_CUSTOM_MUTATOR_LIBRARY to a shared library with
     afl_custom_fuzz() creates additional mutations through this library.

docs/sister_projects.md

@@ -56,13 +56,6 @@ functionality is now available as the "persistent" feature described in
 
 http://llvm.org/docs/LibFuzzer.html
 
-## AFL fixup shim (Ben Nagy)
-
-Allows AFL_POST_LIBRARY postprocessors to be written in arbitrary languages
-that don't have C / .so bindings. Includes examples in Go.
-
-https://github.com/bnagy/aflfix
-
 ## TriforceAFL (Tim Newsham and Jesse Hertz)
 
 Leverages QEMU full system emulation mode to allow AFL to target operating

examples/post_library/post_library.so.c

@@ -1,159 +0,0 @@
-/*
-   american fuzzy lop++ - postprocessor library example
-   --------------------------------------------------
-
-   Originally written by Michal Zalewski
-   Edited by Dominik Maier, 2020
-
-   Copyright 2015 Google Inc. All rights reserved.
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at:
-
-     http://www.apache.org/licenses/LICENSE-2.0
-
-   Postprocessor libraries can be passed to afl-fuzz to perform final cleanup
-   of any mutated test cases - for example, to fix up checksums in PNG files.
-
-   Please heed the following warnings:
-
-   1) In almost all cases, it is more productive to comment out checksum logic
-      in the targeted binary (as shown in ../libpng_no_checksum/). One possible
-      exception is the process of fuzzing binary-only software in QEMU mode.
-
-   2) The use of postprocessors for anything other than checksums is
-   questionable and may cause more harm than good. AFL is normally pretty good
-   about dealing with length fields, magic values, etc.
-
-   3) Postprocessors that do anything non-trivial must be extremely robust to
-      gracefully handle malformed data and other error conditions - otherwise,
-      they will crash and take afl-fuzz down with them. Be wary of reading past
-      *len and of integer overflows when calculating file offsets.
-
-   In other words, THIS IS PROBABLY NOT WHAT YOU WANT - unless you really,
-   honestly know what you're doing =)
-
-   With that out of the way: the postprocessor library is passed to afl-fuzz
-   via AFL_POST_LIBRARY. The library must be compiled with:
-
-     gcc -shared -Wall -O3 post_library.so.c -o post_library.so
-
-   AFL will call the afl_postprocess() function for every mutated output buffer.
-   From there, you have three choices:
-
-   1) If you don't want to modify the test case, simply set `*out_buf = in_buf`
-      and return the original `len`.
-
-   2) If you want to skip this test case altogether and have AFL generate a
-      new one, return 0 or set `*out_buf = NULL`.
-      Use this sparingly - it's faster than running the target program
-      with patently useless inputs, but still wastes CPU time.
-
-   3) If you want to modify the test case, allocate an appropriately-sized
-      buffer, move the data into that buffer, make the necessary changes, and
-      then return the new pointer as out_buf. Return an appropriate len
-   afterwards.
-
-      Note that the buffer will *not* be freed for you. To avoid memory leaks,
-      you need to free it or reuse it on subsequent calls (as shown below).
-
-      *** Feel free to reuse the original 'in_buf' BUFFER and return it. ***
-
-    Aight. The example below shows a simple postprocessor that tries to make
-    sure that all input files start with "GIF89a".
-
-    PS. If you don't like C, you can try out the unix-based wrapper from
-    Ben Nagy instead: https://github.com/bnagy/aflfix
-
- */
-
-#include <stdio.h>
-#include <stdlib.h>
-#include <string.h>
-
-/* Header that must be present at the beginning of every test case: */
-
-#define HEADER "GIF89a"
-
-typedef struct post_state {
-
-  unsigned char *buf;
-  size_t         size;
-
-} post_state_t;
-
-void *afl_postprocess_init(void *afl, unsigned int seed) {
-
-  post_state_t *state = malloc(sizeof(post_state_t));
-  if (!state) {
-
-    perror("malloc");
-    return NULL;
-
-  }
-
-  state->buf = calloc(sizeof(unsigned char), 4096);
-  if (!state->buf) { return NULL; }
-
-  return state;
-
-}
-
-/* The actual postprocessor routine called by afl-fuzz: */
-
-size_t afl_postprocess(post_state_t *data, unsigned char *in_buf,
-                       unsigned int len, unsigned char **out_buf) {
-
-  /* Skip execution altogether for buffers shorter than 6 bytes (just to
-     show how it's done). We can trust len to be sane. */
-
-  if (len < strlen(HEADER)) return 0;
-
-  /* Do nothing for buffers that already start with the expected header. */
-
-  if (!memcmp(in_buf, HEADER, strlen(HEADER))) {
-
-    *out_buf = in_buf;
-    return len;
-
-  }
-
-  /* Allocate memory for new buffer, reusing previous allocation if
-     possible. */
-
-  *out_buf = realloc(data->buf, len);
-
-  /* If we're out of memory, the most graceful thing to do is to return the
-     original buffer and give up on modifying it. Let AFL handle OOM on its
-     own later on. */
-
-  if (!*out_buf) {
-
-    *out_buf = in_buf;
-    return len;
-
-  }
-
-  /* Copy the original data to the new location. */
-
-  memcpy(*out_buf, in_buf, len);
-
-  /* Insert the new header. */
-
-  memcpy(*out_buf, HEADER, strlen(HEADER));
-
-  /* Return the new len. It hasn't changed, so it's just len. */
-
-  return len;
-
-}
-
-/* Gets called afterwards */
-void afl_postprocess_deinit(post_state_t *data) {
-
-  free(data->buf);
-  free(data);
-
-}
-

examples/post_library/post_library_png.so.c

@@ -1,156 +0,0 @@
-/*
-   american fuzzy lop++ - postprocessor for PNG
-   ------------------------------------------
-
-   Originally written by Michal Zalewski
-
-   Copyright 2015 Google Inc. All rights reserved.
-   Adapted to the new API, 2020 by Dominik Maier
-
-   Licensed under the Apache License, Version 2.0 (the "License");
-   you may not use this file except in compliance with the License.
-   You may obtain a copy of the License at:
-
-     http://www.apache.org/licenses/LICENSE-2.0
-
-   See post_library.so.c for a general discussion of how to implement
-   postprocessors. This specific postprocessor attempts to fix up PNG
-   checksums, providing a slightly more complicated example than found
-   in post_library.so.c.
-
-   Compile with:
-
-     gcc -shared -Wall -O3 post_library_png.so.c -o post_library_png.so -lz
-
- */
-
-#include <stdio.h>
-#include <stdlib.h>
-#include <stdint.h>
-#include <string.h>
-#include <zlib.h>
-
-#include <arpa/inet.h>
-
-/* A macro to round an integer up to 4 kB. */
-
-#define UP4K(_i) ((((_i) >> 12) + 1) << 12)
-
-typedef struct post_state {
-
-  unsigned char *buf;
-  size_t         size;
-
-} post_state_t;
-
-void *afl_postprocess_init(void *afl, unsigned int seed) {
-
-  post_state_t *state = malloc(sizeof(post_state_t));
-  if (!state) {
-
-    perror("malloc");
-    return NULL;
-
-  }
-
-  state->buf = calloc(sizeof(unsigned char), 4096);
-  if (!state->buf) { return NULL; }
-
-  return state;
-
-}
-
-size_t afl_postprocess(post_state_t *data, const unsigned char *in_buf,
-                       unsigned int len, const unsigned char **out_buf) {
-
-  unsigned char *new_buf = (unsigned char *)in_buf;
-  unsigned int   pos = 8;
-
-  /* Don't do anything if there's not enough room for the PNG header
-     (8 bytes). */
-
-  if (len < 8) {
-
-    *out_buf = in_buf;
-    return len;
-
-  }
-
-  /* Minimum size of a zero-length PNG chunk is 12 bytes; if we
-     don't have that, we can bail out. */
-
-  while (pos + 12 <= len) {
-
-    unsigned int chunk_len, real_cksum, file_cksum;
-
-    /* Chunk length is the first big-endian dword in the chunk. */
-
-    chunk_len = ntohl(*(uint32_t *)(in_buf + pos));
-
-    /* Bail out if chunk size is too big or goes past EOF. */
-
-    if (chunk_len > 1024 * 1024 || pos + 12 + chunk_len > len) break;
-
-    /* Chunk checksum is calculated for chunk ID (dword) and the actual
-       payload. */
-
-    real_cksum = htonl(crc32(0, in_buf + pos + 4, chunk_len + 4));
-
-    /* The in-file checksum is the last dword past the chunk data. */
-
-    file_cksum = *(uint32_t *)(in_buf + pos + 8 + chunk_len);
-
-    /* If the checksums do not match, we need to fix the file. */
-
-    if (real_cksum != file_cksum) {
-
-      /* First modification? Make a copy of the input buffer. Round size
-         up to 4 kB to minimize the number of reallocs needed. */
-
-      if (new_buf == in_buf) {
-
-        if (len <= data->size) {
-
-          new_buf = data->buf;
-
-        } else {
-
-          new_buf = realloc(data->buf, UP4K(len));
-          if (!new_buf) {
-
-            *out_buf = in_buf;
-            return len;
-
-          }
-
-          data->buf = new_buf;
-          data->size = UP4K(len);
-          memcpy(new_buf, in_buf, len);
-
-        }
-
-      }
-
-      *(uint32_t *)(new_buf + pos + 8 + chunk_len) = real_cksum;
-
-    }
-
-    /* Skip the entire chunk and move to the next one. */
-
-    pos += 12 + chunk_len;
-
-  }
-
-  *out_buf = new_buf;
-  return len;
-
-}
-
-/* Gets called afterwards */
-void afl_postprocess_deinit(post_state_t *data) {
-
-  free(data->buf);
-  free(data);
-
-}
-

include/afl-fuzz.h

@@ -323,9 +323,8 @@ typedef struct afl_env_vars {
       afl_bench_until_crash, afl_debug_child_output, afl_autoresume,
       afl_cal_fast;
 
-  u8 *afl_tmpdir, *afl_post_library, *afl_custom_mutator_library,
+  u8 *afl_tmpdir, *afl_custom_mutator_library, *afl_python_module, *afl_path,
-      *afl_python_module, *afl_path, *afl_hang_tmout, *afl_skip_crashes,
+      *afl_hang_tmout, *afl_skip_crashes, *afl_preload;
-      *afl_preload;
 
 } afl_env_vars_t;
 

src/afl-common.c

@@ -76,7 +76,7 @@ char *afl_environment_variables[] = {
     "AFL_NO_X86",  // not really an env but we dont want to warn on it
     "AFL_MAP_SIZE", "AFL_MAPSIZE", "AFL_PATH", "AFL_PERFORMANCE_FILE",
     //"AFL_PERSISTENT", // not implemented anymore, so warn additionally
-    "AFL_POST_LIBRARY", "AFL_PRELOAD", "AFL_PYTHON_MODULE", "AFL_QEMU_COMPCOV",
+    "AFL_PRELOAD", "AFL_PYTHON_MODULE", "AFL_QEMU_COMPCOV",
     "AFL_QEMU_COMPCOV_DEBUG", "AFL_QEMU_DEBUG_MAPS", "AFL_QEMU_DISABLE_CACHE",
     "AFL_QEMU_PERSISTENT_ADDR", "AFL_QEMU_PERSISTENT_CNT",
     "AFL_QEMU_PERSISTENT_GPR", "AFL_QEMU_PERSISTENT_HOOK",

src/afl-fuzz-init.c

@@ -280,54 +280,6 @@ cpuset_destroy(c);
 
 #endif                                                     /* HAVE_AFFINITY */
 
-/* Load postprocessor, if available. */
-
-void setup_post(afl_state_t *afl) {
-
-  void *dh;
-  u8 *  fn = afl->afl_env.afl_post_library;
-  u8    tbuf[6];
-  u32   tlen = 6;
-  strncpy(tbuf, "hello", tlen);
-
-  if (!fn) { return; }
-
-  ACTF("Loading postprocessor from '%s'...", fn);
-
-  dh = dlopen(fn, RTLD_NOW);
-  if (!dh) { FATAL("%s", dlerror()); }
-
-  struct custom_mutator *mutator;
-  mutator = ck_alloc(sizeof(struct custom_mutator));
-  memset(mutator, 0, sizeof(struct custom_mutator));
-
-  mutator->afl_custom_post_process = dlsym(dh, "afl_postprocess");
-  if (!mutator->afl_custom_post_process) {
-
-    FATAL("Symbol 'afl_postprocess' not found.");
-
-  }
-
-  mutator->afl_custom_init = dlsym(dh, "afl_postprocess_init");
-  if (!mutator->afl_custom_init) {
-
-    WARNF("optional symbol 'afl_postprocess_init' not found.");
-
-  }
-
-  mutator->afl_custom_deinit = dlsym(dh, "afl_postprocess_deinit");
-  if (!mutator->afl_custom_post_process) {
-
-    WARNF("optional symbol 'afl_postprocess_deinit' not found.");
-
-  }
-
-  afl->post_library_mutator = mutator;
-
-  OKF("Postprocessor installed successfully.");
-
-}
-
 /* Shuffle an array of pointers. Might be slightly biased. */
 
 static void shuffle_ptrs(afl_state_t *afl, void **ptrs, u32 cnt) {

src/afl-fuzz-state.c

@@ -313,8 +313,10 @@ void read_afl_environment(afl_state_t *afl, char **envp) {
 
                               afl_environment_variable_len)) {
 
-            afl->afl_env.afl_post_library =
+            FATAL(
-                (u8 *)get_afl_env(afl_environment_variables[i]);
+                "AFL_POST_LIBRARY is deprecated, use "
+                "AFL_CUSTOM_MUTATOR_LIBRARY instead, see "
+                "docs/custom_mutators.md");
 
           } else if (!strncmp(env, "AFL_CUSTOM_MUTATOR_LIBRARY",
 

src/afl-fuzz.c

@@ -187,7 +187,6 @@ static void usage(afl_state_t *afl, u8 *argv0, int more_help) {
       "AFL_NO_SNAPSHOT: do not use the snapshot feature (if the snapshot lkm is loaded)\n"
       "AFL_NO_UI: switch status screen off\n"
       "AFL_PATH: path to AFL support binaries\n"
-      "AFL_POST_LIBRARY: postprocess generated test cases before use as target input\n"
       "AFL_PYTHON_MODULE: mutate and trim inputs with the specified Python module\n"
       "AFL_QUIET: suppress forkserver status messages\n"
       "AFL_PRELOAD: LD_PRELOAD / DYLD_INSERT_LIBRARIES settings for target\n"
@@ -1060,8 +1059,6 @@ int main(int argc, char **argv_orig, char **envp) {
   afl->fsrv.trace_bits =
       afl_shm_init(&afl->shm, afl->fsrv.map_size, afl->dumb_mode);
 
-  setup_post(afl);
-
   if (!afl->in_bitmap) { memset(afl->virgin_bits, 255, afl->fsrv.map_size); }
   memset(afl->virgin_tmout, 255, afl->fsrv.map_size);
   memset(afl->virgin_crash, 255, afl->fsrv.map_size);

test/test.sh

@@ -60,7 +60,6 @@ unset AFL_QEMU_PERSISTENT_GPR
 unset AFL_QEMU_PERSISTENT_RET
 unset AFL_QEMU_PERSISTENT_HOOK
 unset AFL_QEMU_PERSISTENT_CNT
-unset AFL_POST_LIBRARY
 unset AFL_CUSTOM_MUTATOR_LIBRARY
 unset AFL_PYTHON_MODULE
 unset AFL_PRELOAD