removed overlooked post_lib references, added post_lib examples to examples/custom_mutators

docs/Changelog.md

@@ -14,6 +14,7 @@ sending a mail to <afl-users+subscribe@googlegroups.com>.
      - -S slaves now only sync from the master to increase performance,
        the -M master stilly syncs from everyone. Added checks that exactly
        one master is present
+  - added former post_library examples to examples/custom_mutators/
 
 
 ### Version ++2.65c (release):

docs/QuickStartGuide.md

@@ -10,8 +10,9 @@ how to hit the ground running:
    If testing a network service, modify it to run in the foreground and read
    from stdin. When fuzzing a format that uses checksums, comment out the
    checksum verification code, too.
-   If this is not possible (e.g. in -Q(emu) mode) then use AFL_POST_LIBRARY
+
-   to calculate the values with your own library.
+   If this is not possible (e.g. in -Q(emu) mode) then use
+   AFL_CUSTOM_MUTATOR_LIBRARY to calculate the values with your own library.
 
    The program must crash properly when a fault is encountered. Watch out for
    custom SIGSEGV or SIGABRT handlers and background processes. For tips on

docs/life_pro_tips.md

@@ -82,8 +82,8 @@ You can find a simple solution in examples/argv_fuzzing.
 
 ## Attacking a format that uses checksums? 
 
-Remove the checksum-checking code or
+Remove the checksum-checking code or use a postprocessor!
-use a postprocessor! See examples/post_library/ for more.
+See examples/custom_mutators/ for more.
 
 ## Dealing with a very slow target or hoping for instant results? 
 

examples/README.md

@@ -2,7 +2,15 @@
 
 Here's a quick overview of the stuff you can find in this directory:
 
-  - custom_mutators      - example custom mutators in python an c
+  - afl_network_proxy    - fuzz a target over the network: afl-fuzz on
+                           a host, target on an embedded system.
+
+  - afl_proxy            - skeleton file example to show how to fuzz
+                           something where you gather coverage data via
+                           different means, e.g. hw debugger
+
+  - afl_untracer         - fuzz binary-only libraries much faster but with
+                           less coverage than qemu_mode
 
   - argv_fuzzing         - a simple wrapper to allow cmdline to be fuzzed
                            (e.g., to test setuid programs).
@@ -23,6 +31,9 @@ Here's a quick overview of the stuff you can find in this directory:
   - crash_triage         - a very rudimentary example of how to annotate crashes
                            with additional gdb metadata.
 
+  - custom_mutators      - examples for the afl++ custom mutator interface in
+                           C and Python
+
   - distributed_fuzzing  - a sample script for synchronizing fuzzer instances
                            across multiple machines (see parallel_fuzzing.md).
 
@@ -31,8 +42,6 @@ Here's a quick overview of the stuff you can find in this directory:
   - persistent_demo      - an example of how to use the LLVM persistent process
                            mode to speed up certain fuzzing jobs.
 
-  - post_library         - an example of how to build postprocessors for AFL.
-
   - socket_fuzzing       - a LD_PRELOAD library 'redirects' a socket to stdin
                            for fuzzing access with afl++
 

examples/custom_mutators/README.md

@@ -15,6 +15,10 @@ example.c - this is a simple example written in C and should be compiled to a
 example.py - this is the template you can use, the functions are there but they
            are empty
 
+post_library_gif.so.c - fix a fuzz input to ensure it is valid for GIF
+
+post_library_png.so.c - fix a fuzz input to ensure it is valid for PNG
+
 simple-chunk-replace.py - this is a simple example where chunks are replaced
 
 common.py - this can be used for common functions and helpers.

examples/custom_mutators/post_library_gif.so.c

@@ -0,0 +1,159 @@
+/*
+   american fuzzy lop++ - postprocessor library example
+   --------------------------------------------------
+
+   Originally written by Michal Zalewski
+   Edited by Dominik Maier, 2020
+
+   Copyright 2015 Google Inc. All rights reserved.
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at:
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+   Postprocessor libraries can be passed to afl-fuzz to perform final cleanup
+   of any mutated test cases - for example, to fix up checksums in PNG files.
+
+   Please heed the following warnings:
+
+   1) In almost all cases, it is more productive to comment out checksum logic
+      in the targeted binary (as shown in ../libpng_no_checksum/). One possible
+      exception is the process of fuzzing binary-only software in QEMU mode.
+
+   2) The use of postprocessors for anything other than checksums is
+   questionable and may cause more harm than good. AFL is normally pretty good
+   about dealing with length fields, magic values, etc.
+
+   3) Postprocessors that do anything non-trivial must be extremely robust to
+      gracefully handle malformed data and other error conditions - otherwise,
+      they will crash and take afl-fuzz down with them. Be wary of reading past
+      *len and of integer overflows when calculating file offsets.
+
+   In other words, THIS IS PROBABLY NOT WHAT YOU WANT - unless you really,
+   honestly know what you're doing =)
+
+   With that out of the way: the postprocessor library is passed to afl-fuzz
+   via AFL_POST_LIBRARY. The library must be compiled with:
+
+     gcc -shared -Wall -O3 post_library.so.c -o post_library.so
+
+   AFL will call the afl_custom_post_process() function for every mutated output
+   buffer. From there, you have three choices:
+
+   1) If you don't want to modify the test case, simply set `*out_buf = in_buf`
+      and return the original `len`.
+
+   2) If you want to skip this test case altogether and have AFL generate a
+      new one, return 0 or set `*out_buf = NULL`.
+      Use this sparingly - it's faster than running the target program
+      with patently useless inputs, but still wastes CPU time.
+
+   3) If you want to modify the test case, allocate an appropriately-sized
+      buffer, move the data into that buffer, make the necessary changes, and
+      then return the new pointer as out_buf. Return an appropriate len
+   afterwards.
+
+      Note that the buffer will *not* be freed for you. To avoid memory leaks,
+      you need to free it or reuse it on subsequent calls (as shown below).
+
+      *** Feel free to reuse the original 'in_buf' BUFFER and return it. ***
+
+    Aight. The example below shows a simple postprocessor that tries to make
+    sure that all input files start with "GIF89a".
+
+    PS. If you don't like C, you can try out the unix-based wrapper from
+    Ben Nagy instead: https://github.com/bnagy/aflfix
+
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+
+/* Header that must be present at the beginning of every test case: */
+
+#define HEADER "GIF89a"
+
+typedef struct post_state {
+
+  unsigned char *buf;
+  size_t         size;
+
+} post_state_t;
+
+void *afl afl_custom_init(void *afl) {
+
+  post_state_t *state = malloc(sizeof(post_state_t));
+  if (!state) {
+
+    perror("malloc");
+    return NULL;
+
+  }
+
+  state->buf = calloc(sizeof(unsigned char), 4096);
+  if (!state->buf) { return NULL; }
+
+  return state;
+
+}
+
+/* The actual postprocessor routine called by afl-fuzz: */
+
+size_t afl_custom_post_process(post_state_t *data, unsigned char *in_buf,
+                               unsigned int len, unsigned char **out_buf) {
+
+  /* Skip execution altogether for buffers shorter than 6 bytes (just to
+     show how it's done). We can trust len to be sane. */
+
+  if (len < strlen(HEADER)) return 0;
+
+  /* Do nothing for buffers that already start with the expected header. */
+
+  if (!memcmp(in_buf, HEADER, strlen(HEADER))) {
+
+    *out_buf = in_buf;
+    return len;
+
+  }
+
+  /* Allocate memory for new buffer, reusing previous allocation if
+     possible. */
+
+  *out_buf = realloc(data->buf, len);
+
+  /* If we're out of memory, the most graceful thing to do is to return the
+     original buffer and give up on modifying it. Let AFL handle OOM on its
+     own later on. */
+
+  if (!*out_buf) {
+
+    *out_buf = in_buf;
+    return len;
+
+  }
+
+  /* Copy the original data to the new location. */
+
+  memcpy(*out_buf, in_buf, len);
+
+  /* Insert the new header. */
+
+  memcpy(*out_buf, HEADER, strlen(HEADER));
+
+  /* Return the new len. It hasn't changed, so it's just len. */
+
+  return len;
+
+}
+
+/* Gets called afterwards */
+void afl_custom_deinit(post_state_t *data) {
+
+  free(data->buf);
+  free(data);
+
+}
+

examples/custom_mutators/post_library_png.so.c

@@ -0,0 +1,157 @@
+/*
+   american fuzzy lop++ - postprocessor for PNG
+   ------------------------------------------
+
+   Originally written by Michal Zalewski
+
+   Copyright 2015 Google Inc. All rights reserved.
+   Adapted to the new API, 2020 by Dominik Maier
+
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at:
+
+     http://www.apache.org/licenses/LICENSE-2.0
+
+   See post_library.so.c for a general discussion of how to implement
+   postprocessors. This specific postprocessor attempts to fix up PNG
+   checksums, providing a slightly more complicated example than found
+   in post_library.so.c.
+
+   Compile with:
+
+     gcc -shared -Wall -O3 post_library_png.so.c -o post_library_png.so -lz
+
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+#include <zlib.h>
+
+#include <arpa/inet.h>
+
+/* A macro to round an integer up to 4 kB. */
+
+#define UP4K(_i) ((((_i) >> 12) + 1) << 12)
+
+typedef struct post_state {
+
+  unsigned char *buf;
+  size_t         size;
+
+} post_state_t;
+
+void *afl_custom_init(void *afl) {
+
+  post_state_t *state = malloc(sizeof(post_state_t));
+  if (!state) {
+
+    perror("malloc");
+    return NULL;
+
+  }
+
+  state->buf = calloc(sizeof(unsigned char), 4096);
+  if (!state->buf) { return NULL; }
+
+  return state;
+
+}
+
+size_t afl_custom_post_process(post_state_t *data, const unsigned char *in_buf,
+                               unsigned int          len,
+                               const unsigned char **out_buf) {
+
+  unsigned char *new_buf = (unsigned char *)in_buf;
+  unsigned int   pos = 8;
+
+  /* Don't do anything if there's not enough room for the PNG header
+     (8 bytes). */
+
+  if (len < 8) {
+
+    *out_buf = in_buf;
+    return len;
+
+  }
+
+  /* Minimum size of a zero-length PNG chunk is 12 bytes; if we
+     don't have that, we can bail out. */
+
+  while (pos + 12 <= len) {
+
+    unsigned int chunk_len, real_cksum, file_cksum;
+
+    /* Chunk length is the first big-endian dword in the chunk. */
+
+    chunk_len = ntohl(*(uint32_t *)(in_buf + pos));
+
+    /* Bail out if chunk size is too big or goes past EOF. */
+
+    if (chunk_len > 1024 * 1024 || pos + 12 + chunk_len > len) break;
+
+    /* Chunk checksum is calculated for chunk ID (dword) and the actual
+       payload. */
+
+    real_cksum = htonl(crc32(0, in_buf + pos + 4, chunk_len + 4));
+
+    /* The in-file checksum is the last dword past the chunk data. */
+
+    file_cksum = *(uint32_t *)(in_buf + pos + 8 + chunk_len);
+
+    /* If the checksums do not match, we need to fix the file. */
+
+    if (real_cksum != file_cksum) {
+
+      /* First modification? Make a copy of the input buffer. Round size
+         up to 4 kB to minimize the number of reallocs needed. */
+
+      if (new_buf == in_buf) {
+
+        if (len <= data->size) {
+
+          new_buf = data->buf;
+
+        } else {
+
+          new_buf = realloc(data->buf, UP4K(len));
+          if (!new_buf) {
+
+            *out_buf = in_buf;
+            return len;
+
+          }
+
+          data->buf = new_buf;
+          data->size = UP4K(len);
+          memcpy(new_buf, in_buf, len);
+
+        }
+
+      }
+
+      *(uint32_t *)(new_buf + pos + 8 + chunk_len) = real_cksum;
+
+    }
+
+    /* Skip the entire chunk and move to the next one. */
+
+    pos += 12 + chunk_len;
+
+  }
+
+  *out_buf = new_buf;
+  return len;
+
+}
+
+/* Gets called afterwards */
+void afl_custom_deinit(post_state_t *data) {
+
+  free(data->buf);
+  free(data);
+
+}
+

include/afl-fuzz.h

@@ -547,8 +547,6 @@ typedef struct afl_state {
 
   /* afl_postprocess API - Now supported via custom mutators */
 
-  struct custom_mutator *post_library_mutator;
-
   /* CmpLog */
 
   char *           cmplog_binary;

qemu_mode/README.md

@@ -157,7 +157,7 @@ match.
 ## 10) Gotchas, feedback, bugs
 
 If you need to fix up checksums or do other cleanup on mutated test cases, see
-examples/post_library/ for a viable solution.
+examples/custom_mutators/ for a viable solution.
 
 Do not mix QEMU mode with ASAN, MSAN, or the likes; QEMU doesn't appreciate
 the "shadow VM" trick employed by the sanitizers and will probably just

src/afl-fuzz-mutators.c

@@ -108,9 +108,6 @@ void setup_custom_mutators(afl_state_t *afl) {
 
 #endif
 
-  if (afl->post_library_mutator)
-    list_append(&afl->custom_mutator_list, afl->post_library_mutator);
-
 }
 
 void destroy_custom_mutators(afl_state_t *afl) {