documentation update

TODO

@@ -1,6 +1,6 @@
 Roadmap 2.53d:
 ==============
- - indent all the code: clang-format -style=Google
+ - indent all the code: .clang-format
 
  - update docs/sister_projects.txt
 
@@ -18,16 +18,41 @@ gcc_plugin:
 
 qemu_mode:
  - deferred mode with AFL_DEFERRED_QEMU=0xaddress
+   @andrea - dont we have that already with AFL_ENTRYPOINT?
 
 unit testing / or large testcase campaign
 
 
 Roadmap 2.54d:
 ==============
- - expand MAP size to 256k (current L2 cache size on processors)
+Problem: Average targets (tiff, jpeg, unrar) go through 1500 edges.
-   -> 18 bit map
+         At afl's default map that means ~16 collisions and ~3 wrappings.
- - llvm_mode: dynamic map size and collission free basic block IDs
+  Solution #1: increase map size.
-
+    every +1 decreases fuzzing speed by ~10% and halfs the collisions
+    birthday paradox predicts at collisions at this # of edges:
+	2^16 = 302
+	2^17 = 427
+	2^18 = 603
+	2^19 = 853
+	2^20 = 1207
+	2^21 = 1706
+	2^22 = 2412
+	2^23 = 3411
+	2^24 = 4823
+    Its an easy solution but also not a good one.
+  Solution #2: use dynamic map size and collision free basic block IDs
+    This only works in llvm_mode and llvm >= 9 though
+    A potential good future solution
+  Solution #3: write instruction pointers to a big shared map
+    512kb/1MB shared map and the instrumented code writes the instruction
+    pointer into the map. Map must be big enough but could be command line
+    controlled.
+    Good: complete coverage information, nothing is lost. choice of analysis
+          impacts speed, but this can be decided by user options
+    Neutral: a little bit slower but no loss of coverage
+    Bad: completely changes how afl uses the map and the scheduling.
+    Overall another very good solution
+    
 qemu_mode:
  - persistent mode patching the return address (WinAFL style)
  - instrument only comparison with immediate values by default when using compcov

docs/ChangeLog

@@ -17,6 +17,8 @@ sending a mail to <afl-users+subscribe@googlegroups.com>.
 Version ++2.53d (dev):
 ----------------------
 
+  - llvm 9 is now supported (still needs testing)
+  - fix building qemu on some Ubuntus (thanks to floyd!)
   - custom mutator by a loaded library is now supported (thanks to kyakdan!)
   - fix for a few features to support different map sized than 2^16
   - afl-showmap: new option -r now shows the real values in the buckets (stock
@@ -26,7 +28,6 @@ Version ++2.53d (dev):
   - ... your patch? :)
 
 
-
 --------------------------
 Version ++2.53c (release):
 --------------------------

docs/env_variables.txt

@@ -257,10 +257,14 @@ The QEMU wrapper used to instrument binary-only code supports several settings:
     Use this if you are unsure if the entrypoint might be wrong - but
     use it directly, e.g. afl-qemu-trace ./program 
 
-  - If you want to specify a specific entrypoint into the binary (this can
+  - AFL_ENTRYPOINT allows you to specify a specific entrypoint into the
-    be very good for the performance!), use AFL_ENTRYPOINT for this.
+    binary (this can be very good for the performance!).
     The entrypoint is specified as hex address, e.g. 0x4004110
 
+  - AFL_QEMU_COMPCOV is for a sub-project in qemu_mode called ./libcompcov
+    which implements laf-intel for qemu. It also needs AFL_PRELOAD and
+    you can find more information in qemu_mode/libcompcov/README.compcov
+
 5) Settings for afl-cmin
 ------------------------
 

llvm_mode/README.llvm

@@ -8,8 +8,7 @@ Fast LLVM-based instrumentation for afl-fuzz
 1) Introduction
 ---------------
 
-! llvm_mode works with llvm version 3.8.1 up to 8.x !
+! llvm_mode works with llvm versions 3.8.1 up to 9 !
-! llvm version 9 does not work yet !
 
 The code in this directory allows you to instrument programs for AFL using
 true compiler-level instrumentation, instead of the more crude

qemu_mode/README.qemu

@@ -46,7 +46,19 @@ Note: if you want the QEMU helper to be installed on your system for all
 users, you need to build it before issuing 'make install' in the parent
 directory.
 
-3) Notes on linking
+3) Options
+----------
+
+There is ./libcompcov/ which implements laf-intel (splitting memcmp,
+strncmp, etc. to make these conditions easier solvable by afl-fuzz).
+Highly recommended.
+
+Another option is the environment variable AFL_ENTRYPOINT which allows
+move the forkserver to a different part, e.g. just before the file is
+opened (e.g. way after command line parsing and config file loading, etc)
+which can be a huge speed improvement.
+
+4) Notes on linking
 -------------------
 
 The feature is supported only on Linux. Supporting BSD may amount to porting
@@ -68,7 +80,7 @@ practice, this means two things:
 Setting AFL_INST_LIBS=1 can be used to circumvent the .text detection logic
 and instrument every basic block encountered.
 
-4) Benchmarking
+5) Benchmarking
 ---------------
 
 If you want to compare the performance of the QEMU instrumentation with that of
@@ -84,7 +96,7 @@ Comparative measurements of execution speed or instrumentation coverage will be
 fairly meaningless if the optimization levels or instrumentation scopes don't
 match.
 
-5) Gotchas, feedback, bugs
+6) Gotchas, feedback, bugs
 --------------------------
 
 If you need to fix up checksums or do other cleanup on mutated test cases, see
@@ -106,7 +118,7 @@ with -march=core2, can help.
 Beyond that, this is an early-stage mechanism, so fields reports are welcome.
 You can send them to <afl-users@googlegroups.com>.
 
-6) Alternatives: static rewriting
+7) Alternatives: static rewriting
 ---------------------------------
 
 Statically rewriting binaries just once, instead of attempting to translate
@@ -114,12 +126,11 @@ them at run time, can be a faster alternative. That said, static rewriting is
 fraught with peril, because it depends on being able to properly and fully model
 program control flow without actually executing each and every code path.
 
-If you want to experiment with this mode of operation, there is a module
+The best implementation is this one:
-contributed by Aleksandar Nikolich:
 
   https://github.com/vanhauser-thc/afl-dyninst
-  https://groups.google.com/forum/#!topic/afl-users/HlSQdbOTlpg
 
-At this point, the author reports the possibility of hiccups with stripped
+The issue however is Dyninst which is not rewriting the binaries so that
-binaries. That said, if we can get it to be comparably reliable to QEMU, we may
+they run stable. a lot of crashes happen, especially in C++ programs that
-decide to switch to this mode, but I had no time to play with it yet.
+use throw/catch. Try it first, and if it works for you be happy as it is
+2-3x as fast as qemu_mode.