rename whitelist -> instrumentlist

This commit is contained in:
van Hauser 2020-06-30 17:28:21 +02:00
parent 878b27af76
commit 06264df168
32 changed files with 227 additions and 204 deletions

View File

@ -2,7 +2,7 @@
<img align="right" src="https://raw.githubusercontent.com/andreafioraldi/AFLplusplus-website/master/static/logo_256x256.png" alt="AFL++ Logo">
![Travis State](https://api.travis-ci.com/AFLplusplus/AFLplusplus.svg?branch=master)
![Travis State](https://api.travis-ci.com/AFLplusplus/AFLplusplus.svg?branch=stable)
Release Version: [2.65c](https://github.com/AFLplusplus/AFLplusplus/releases)
@ -40,7 +40,7 @@
* InsTrim, a very effective CFG llvm_mode instrumentation implementation for large targets: [https://github.com/csienslab/instrim](https://github.com/csienslab/instrim)
* C. Holler's afl-fuzz Python mutator module and llvm_mode whitelist support: [https://github.com/choller/afl](https://github.com/choller/afl)
* C. Holler's afl-fuzz Python mutator module and llvm_mode instrument file support: [https://github.com/choller/afl](https://github.com/choller/afl)
* Custom mutator by a library (instead of Python) by kyakdan
@ -70,7 +70,7 @@
| Persistent mode | | x | x | x86[_64]/arm[64] | x |
| LAF-Intel / CompCov | | x | | x86[_64]/arm[64] | x86[_64]/arm |
| CmpLog | | x | | x86[_64]/arm[64] | |
| Whitelist | | x | x | (x)(3) | |
| Instrument file list | | x | x | (x)(3) | |
| Non-colliding coverage | | x(4) | | (x)(5) | |
| InsTrim | | x | | | |
| Ngram prev_loc coverage | | x(6) | | | |
@ -297,7 +297,7 @@ Using the LAF Intel performance enhancements are also recommended, see
[llvm_mode/README.laf-intel.md](llvm_mode/README.laf-intel.md)
Using partial instrumentation is also recommended, see
[llvm_mode/README.whitelist.md](llvm_mode/README.whitelist.md)
[llvm_mode/README.instrument_file.md](llvm_mode/README.instrument_file.md)
When testing libraries, you need to find or write a simple program that reads
data from stdin or from a file and passes it to the tested library. In such a

View File

@ -10,6 +10,10 @@ sending a mail to <afl-users+subscribe@googlegroups.com>.
### Version ++2.65d (dev)
- renamed the main branch on Github to "stable"
- renamed master/slave to main/secondary
- renamed blacklist/whitelist to ignorelist/instrumentlist ->
AFL_LLVM_INSTRUMENT_FILE and AFL_GCC_INSTRUMENT_FILE
- afl-fuzz:
- -S secondary nodes now only sync from the main node to increase
performance, the -M main node still syncs from everyone. Added checks
@ -40,8 +44,8 @@ sending a mail to <afl-users+subscribe@googlegroups.com>.
- WHITELIST feature now supports wildcards (thanks to sirmc)
- small change to cmplog to make it work with current llvm 11-dev
- added AFL_LLVM_LAF_ALL, sets all laf-intel settings
- LTO whitelist functionality rewritten, now main, _init etc functions
need not to be whitelisted anymore
- LTO instrument_files functionality rewritten, now main, _init etc functions
need not to be instrument_filesed anymore
- fixed crash in compare-transform-pass when strcasecmp/strncasecmp was
tried to be instrumented with LTO
- fixed crash in cmplog with LTO
@ -249,7 +253,7 @@ sending a mail to <afl-users+subscribe@googlegroups.com>.
the original script is still present as afl-cmin.bash
- afl-showmap: -i dir option now allows processing multiple inputs using the
forkserver. This is for enhanced speed in afl-cmin.
- added blacklist and whitelisting function check in all modules of llvm_mode
- added blacklist and instrument_filesing function check in all modules of llvm_mode
- added fix from Debian project to compile libdislocator and libtokencap
- libdislocator: AFL_ALIGNED_ALLOC to force size alignment to max_align_t
@ -304,7 +308,7 @@ sending a mail to <afl-users+subscribe@googlegroups.com>.
performance loss of ~10%
- added test/test-performance.sh script
- (re)added gcc_plugin, fast inline instrumentation is not yet finished,
however it includes the whitelisting and persistance feature! by hexcoder-
however it includes the instrument_filesing and persistance feature! by hexcoder-
- gcc_plugin tests added to testing framework
@ -392,7 +396,7 @@ sending a mail to <afl-users+subscribe@googlegroups.com>.
- more cpu power for afl-system-config
- added forkserver patch to afl-tmin, makes it much faster (originally from
github.com/nccgroup/TriforceAFL)
- added whitelist support for llvm_mode via AFL_LLVM_WHITELIST to allow
- added instrument_files support for llvm_mode via AFL_LLVM_WHITELIST to allow
only to instrument what is actually interesting. Gives more speed and less
map pollution (originally by choller@mozilla)
- added Python Module mutator support, python2.7-dev is autodetected.

View File

@ -28,7 +28,7 @@ afl-qemu-optimize-map.diff by mh(at)mh-sec(dot)de
+ AFLfast additions (github.com/mboehme/aflfast) were incorporated.
+ Qemu 3.1 upgrade with enhancement patches (github.com/andreafioraldi/afl)
+ Python mutator modules support (github.com/choller/afl)
+ Whitelisting in LLVM mode (github.com/choller/afl)
+ Instrument file list in LLVM mode (github.com/choller/afl)
+ forkserver patch for afl-tmin (github.com/nccgroup/TriforceAFL)

View File

@ -204,14 +204,14 @@ Then there are a few specific features that are only available in llvm_mode:
See llvm_mode/README.laf-intel.md for more information.
### WHITELIST
### INSTRUMENT_FILE
This feature allows selectively instrumentation of the source
- Setting AFL_LLVM_WHITELIST with a filename will only instrument those
- Setting AFL_LLVM_INSTRUMENT_FILE with a filename will only instrument those
files that match the names listed in this file.
See llvm_mode/README.whitelist.md for more information.
See llvm_mode/README.instrument_file.md for more information.
### NOT_ZERO
@ -236,14 +236,14 @@ Then there are a few specific features that are only available in llvm_mode:
Then there are a few specific features that are only available in the gcc_plugin:
### WHITELIST
### INSTRUMENT_FILE
This feature allows selective instrumentation of the source
- Setting AFL_GCC_WHITELIST with a filename will only instrument those
- Setting AFL_GCC_INSTRUMENT_FILE with a filename will only instrument those
files that match the names listed in this file (one filename per line).
See gcc_plugin/README.whitelist.md for more information.
See gcc_plugin/README.instrument_file.md for more information.
## 3) Settings for afl-fuzz

View File

@ -66,8 +66,8 @@ then using laf-intel (see llvm_mode/README.laf-intel.md) will help `afl-fuzz` a
to get to the important parts in the code.
If you are only interested in specific parts of the code being fuzzed, you can
whitelist the files that are actually relevant. This improves the speed and
accuracy of afl. See llvm_mode/README.whitelist.md
instrument_files the files that are actually relevant. This improves the speed and
accuracy of afl. See llvm_mode/README.instrument_file.md
Also use the InsTrim mode on larger binaries, this improves performance and
coverage a lot.

View File

@ -156,7 +156,7 @@ install: all
install -m 755 ../afl-gcc-fast $${DESTDIR}$(BIN_PATH)
install -m 755 ../afl-gcc-pass.so ../afl-gcc-rt.o $${DESTDIR}$(HELPER_PATH)
install -m 644 -T README.md $${DESTDIR}$(DOC_PATH)/README.gcc_plugin.md
install -m 644 -T README.whitelist.md $${DESTDIR}$(DOC_PATH)/README.gcc_plugin.whitelist.md
install -m 644 -T README.instrument_file.md $${DESTDIR}$(DOC_PATH)/README.gcc_plugin.instrument_file.md
clean:
rm -f *.o *.so *~ a.out core core.[1-9][0-9]* test-instr .test-instr0 .test-instr1 .test2

View File

@ -152,7 +152,7 @@ install: all
install -m 755 ../afl-gcc-fast $${DESTDIR}$(BIN_PATH)
install -m 755 ../afl-gcc-pass.so ../afl-gcc-rt.o $${DESTDIR}$(HELPER_PATH)
install -m 644 -T README.md $${DESTDIR}$(DOC_PATH)/README.gcc_plugin.md
install -m 644 -T README.whitelist.md $${DESTDIR}$(DOC_PATH)/README.gcc_plugin.whitelist.md
install -m 644 -T README.instrument_file.md $${DESTDIR}$(DOC_PATH)/README.gcc_plugin.instrument_file.md
clean:
rm -f *.o *.so *~ a.out core core.[1-9][0-9]* test-instr .test-instr0 .test-instr1 .test2

View File

@ -32,12 +32,12 @@ subdirectory. There is nothing specifically to do :)
In order to build with partial instrumentation, you need to build with
afl-gcc-fast and afl-g++-fast respectively. The only required change is
that you need to set the environment variable AFL_GCC_WHITELIST when calling
that you need to set the environment variable AFL_GCC_INSTRUMENT_FILE when calling
the compiler.
The environment variable must point to a file containing all the filenames
that should be instrumented. For matching, the filename that is being compiled
must end in the filename entry contained in this whitelist (to avoid breaking
must end in the filename entry contained in this instrument list (to avoid breaking
the matching when absolute paths are used during compilation).
For example if your source tree looks like this:
@ -50,14 +50,14 @@ project/feature_b/b1.cpp
project/feature_b/b2.cpp
```
and you only want to test feature_a, then create a whitelist file containing:
and you only want to test feature_a, then create a instrument list file containing:
```
feature_a/a1.cpp
feature_a/a2.cpp
```
However if the whitelist file contains only this, it works as well:
However if the instrument list file contains only this, it works as well:
```
a1.cpp
@ -67,7 +67,7 @@ a2.cpp
but it might lead to files being unwantedly instrumented if the same filename
exists somewhere else in the project directories.
The created whitelist file is then set to AFL_GCC_WHITELIST when you compile
your program. For each file that didn't match the whitelist, the compiler will
The created instrument list file is then set to AFL_GCC_INSTRUMENT_FILE when you compile
your program. For each file that didn't match the instrument list, the compiler will
issue a warning at the end stating that no blocks were instrumented. If you
didn't intend to instrument that file, then you can safely ignore that warning.

View File

@ -306,8 +306,7 @@ int main(int argc, char **argv, char **envp) {
if (argc < 2 || strcmp(argv[1], "-h") == 0) {
printf(
cCYA
printf(cCYA
"afl-gcc-fast" VERSION cRST
" initially by <aseipp@pobox.com>, maintainer: hexcoder-\n"
"\n"
@ -342,7 +341,8 @@ int main(int argc, char **argv, char **envp) {
"AFL_USE_ASAN: activate address sanitizer\n"
"AFL_USE_MSAN: activate memory sanitizer\n"
"AFL_USE_UBSAN: activate undefined behaviour sanitizer\n"
"AFL_GCC_WHITELIST: enable whitelisting (selective instrumentation)\n"
"AFL_GCC_INSTRUMENT_FILE: enable selective instrumentation by "
"filename\n"
"\nafl-gcc-fast was built for gcc %s with the gcc binary path of "
"\"%s\".\n\n",
@ -357,10 +357,13 @@ int main(int argc, char **argv, char **envp) {
SAYF(cCYA "afl-gcc-fast" VERSION cRST
" initially by <aseipp@pobox.com>, maintainer: hexcoder-\n");
if (getenv("AFL_GCC_WHITELIST") == NULL) {
if (getenv("AFL_GCC_INSTRUMENT_FILE") == NULL &&
getenv("AFL_GCC_WHITELIST") == NULL) {
SAYF(cYEL "Warning:" cRST
" using afl-gcc-fast without using AFL_GCC_WHITELIST currently "
SAYF(
cYEL
"Warning:" cRST
" using afl-gcc-fast without using AFL_GCC_INSTRUMENT_FILE currently "
"produces worse results than afl-gcc. Even better, use "
"llvm_mode for now.\n");

View File

@ -2,7 +2,7 @@
// There are some TODOs in this file:
// - fix instrumentation via external call
// - fix inline instrumentation
// - implement whitelist feature
// - implement instrument list feature
// - dont instrument blocks that are uninteresting
// - implement neverZero
//
@ -95,7 +95,7 @@
static int be_quiet = 0;
static unsigned int inst_ratio = 100;
static bool inst_ext = true;
static std::list<std::string> myWhitelist;
static std::list<std::string> myInstrumentList;
static unsigned int ext_call_instrument(function *fun) {
@ -414,7 +414,7 @@ class afl_pass : public gimple_opt_pass {
unsigned int execute(function *fun) override {
if (!myWhitelist.empty()) {
if (!myInstrumentList.empty()) {
bool instrumentBlock = false;
std::string instFilename;
@ -436,8 +436,8 @@ class afl_pass : public gimple_opt_pass {
/* Continue only if we know where we actually are */
if (!instFilename.empty()) {
for (std::list<std::string>::iterator it = myWhitelist.begin();
it != myWhitelist.end(); ++it) {
for (std::list<std::string>::iterator it = myInstrumentList.begin();
it != myInstrumentList.end(); ++it) {
/* We don't check for filename equality here because
* filenames might actually be full paths. Instead we
@ -462,13 +462,14 @@ class afl_pass : public gimple_opt_pass {
}
/* Either we couldn't figure out our location or the location is
* not whitelisted, so we skip instrumentation. */
* not in the instrument list, so we skip instrumentation. */
if (!instrumentBlock) {
if (!be_quiet) {
if (!instFilename.empty())
SAYF(cYEL "[!] " cBRI "Not in whitelist, skipping %s line %u...\n",
SAYF(cYEL "[!] " cBRI
"Not in instrument list, skipping %s line %u...\n",
instFilename.c_str(), instLine);
else
SAYF(cYEL "[!] " cBRI "No filename information found, skipping it");
@ -562,26 +563,32 @@ int plugin_init(struct plugin_name_args * plugin_info,
}
char *instWhiteListFilename = getenv("AFL_GCC_WHITELIST");
if (instWhiteListFilename) {
char *instInstrumentListFilename = getenv("AFL_GCC_INSTRUMENT_FILE");
if (!instInstrumentListFilename)
instInstrumentListFilename = getenv("AFL_GCC_WHITELIST");
if (instInstrumentListFilename) {
std::string line;
std::ifstream fileStream;
fileStream.open(instWhiteListFilename);
if (!fileStream) PFATAL("Unable to open AFL_GCC_WHITELIST");
fileStream.open(instInstrumentListFilename);
if (!fileStream) PFATAL("Unable to open AFL_GCC_INSTRUMENT_FILE");
getline(fileStream, line);
while (fileStream) {
myWhitelist.push_back(line);
myInstrumentList.push_back(line);
getline(fileStream, line);
}
} else if (!be_quiet && getenv("AFL_LLVM_WHITELIST"))
} else if (!be_quiet && (getenv("AFL_LLVM_WHITELIST") ||
getenv("AFL_LLVM_INSTRUMENT_FILE"))) {
SAYF(cYEL "[-] " cRST
"AFL_LLVM_WHITELIST environment variable detected - did you mean "
"AFL_GCC_WHITELIST?\n");
"AFL_LLVM_INSTRUMENT_FILE environment variable detected - did "
"you mean AFL_GCC_INSTRUMENT_FILE?\n");
}
/* Go go gadget */
register_callback(plugin_info->base_name, PLUGIN_INFO, NULL,

View File

@ -253,7 +253,7 @@ ifeq "$(TEST_MMAP)" "1"
LDFLAGS += -Wno-deprecated-declarations
endif
PROGS = ../afl-clang-fast ../afl-llvm-pass.so ../afl-ld-lto ../afl-llvm-lto-whitelist.so ../afl-llvm-lto-instrumentation.so ../afl-llvm-lto-instrim.so ../libLLVMInsTrim.so ../afl-llvm-rt.o ../afl-llvm-rt-32.o ../afl-llvm-rt-64.o ../compare-transform-pass.so ../split-compares-pass.so ../split-switches-pass.so ../cmplog-routines-pass.so ../cmplog-instructions-pass.so
PROGS = ../afl-clang-fast ../afl-llvm-pass.so ../afl-ld-lto ../afl-llvm-lto-instrumentlist.so ../afl-llvm-lto-instrumentation.so ../afl-llvm-lto-instrim.so ../libLLVMInsTrim.so ../afl-llvm-rt.o ../afl-llvm-rt-32.o ../afl-llvm-rt-64.o ../compare-transform-pass.so ../split-compares-pass.so ../split-switches-pass.so ../cmplog-routines-pass.so ../cmplog-instructions-pass.so
# If prerequisites are not given, warn, do not build anything, and exit with code 0
ifeq "$(LLVMVER)" ""
@ -332,7 +332,7 @@ ifeq "$(LLVM_MIN_4_0_1)" "0"
endif
$(CXX) $(CLANG_CPPFL) -DLLVMInsTrim_EXPORTS -fno-rtti -fPIC -std=$(LLVM_STDCXX) -shared $< -o $@ $(CLANG_LFL) afl-llvm-common.o
../afl-llvm-lto-whitelist.so: afl-llvm-lto-whitelist.so.cc afl-llvm-common.o
../afl-llvm-lto-instrumentlist.so: afl-llvm-lto-instrumentlist.so.cc afl-llvm-common.o
ifeq "$(LLVM_LTO)" "1"
$(CXX) $(CLANG_CPPFL) -fno-rtti -fPIC -std=$(LLVM_STDCXX) -shared $< -o $@ $(CLANG_LFL) afl-llvm-common.o
endif
@ -403,7 +403,7 @@ all_done: test_build
install: all
install -d -m 755 $${DESTDIR}$(BIN_PATH) $${DESTDIR}$(HELPER_PATH) $${DESTDIR}$(DOC_PATH) $${DESTDIR}$(MISC_PATH)
if [ -f ../afl-clang-fast -a -f ../libLLVMInsTrim.so -a -f ../afl-llvm-rt.o ]; then set -e; install -m 755 ../afl-clang-fast $${DESTDIR}$(BIN_PATH); ln -sf afl-clang-fast $${DESTDIR}$(BIN_PATH)/afl-clang-fast++; install -m 755 ../libLLVMInsTrim.so ../afl-llvm-pass.so ../afl-llvm-rt.o $${DESTDIR}$(HELPER_PATH); fi
if [ -f ../afl-clang-lto ]; then set -e; ln -sf afl-clang-fast $${DESTDIR}$(BIN_PATH)/afl-clang-lto; ln -sf afl-clang-fast $${DESTDIR}$(BIN_PATH)/afl-clang-lto++; install -m 755 ../afl-llvm-lto-instrumentation.so ../afl-llvm-lto-instrim.so ../afl-llvm-rt-lto*.o ../afl-llvm-lto-whitelist.so $${DESTDIR}$(HELPER_PATH); fi
if [ -f ../afl-clang-lto ]; then set -e; ln -sf afl-clang-fast $${DESTDIR}$(BIN_PATH)/afl-clang-lto; ln -sf afl-clang-fast $${DESTDIR}$(BIN_PATH)/afl-clang-lto++; install -m 755 ../afl-llvm-lto-instrumentation.so ../afl-llvm-lto-instrim.so ../afl-llvm-rt-lto*.o ../afl-llvm-lto-instrumentlist.so $${DESTDIR}$(HELPER_PATH); fi
if [ -f ../afl-ld-lto ]; then set -e; install -m 755 ../afl-ld-lto $${DESTDIR}$(BIN_PATH); fi
if [ -f ../afl-llvm-rt-32.o ]; then set -e; install -m 755 ../afl-llvm-rt-32.o $${DESTDIR}$(HELPER_PATH); fi
if [ -f ../afl-llvm-rt-64.o ]; then set -e; install -m 755 ../afl-llvm-rt-64.o $${DESTDIR}$(HELPER_PATH); fi

View File

@ -74,7 +74,7 @@ struct InsTrim : public ModulePass {
InsTrim() : ModulePass(ID), generator(0) {
initWhitelist();
initInstrumentList();
}
@ -271,7 +271,7 @@ struct InsTrim : public ModulePass {
}
if (!isInWhitelist(&F)) continue;
if (!isInInstrumentList(&F)) continue;
// if the function below our minimum size skip it (1 or 2)
if (F.size() < function_minimum_size) { continue; }

View File

@ -29,12 +29,12 @@ subdirectory. There is nothing specifically to do :)
In order to build with partial instrumentation, you need to build with
afl-clang-fast and afl-clang-fast++ respectively. The only required change is
that you need to set the environment variable AFL_LLVM_WHITELIST when calling
that you need to set the environment variable AFL_LLVM_INSTRUMENT_FILE when calling
the compiler.
The environment variable must point to a file containing all the filenames
that should be instrumented. For matching, the filename that is being compiled
must end in the filename entry contained in this whitelist (to avoid breaking
must end in the filename entry contained in this the instrument file list (to avoid breaking
the matching when absolute paths are used during compilation).
For example if your source tree looks like this:
@ -47,14 +47,14 @@ project/feature_b/b1.cpp
project/feature_b/b2.cpp
```
and you only want to test feature_a, then create a whitelist file containing:
and you only want to test feature_a, then create a the instrument file list file containing:
```
feature_a/a1.cpp
feature_a/a2.cpp
```
However if the whitelist file contains only this, it works as well:
However if the the instrument file list file contains only this, it works as well:
```
a1.cpp
@ -64,8 +64,8 @@ a2.cpp
but it might lead to files being unwantedly instrumented if the same filename
exists somewhere else in the project directories.
The created whitelist file is then set to AFL_LLVM_WHITELIST when you compile
your program. For each file that didn't match the whitelist, the compiler will
The created the instrument file list file is then set to AFL_LLVM_INSTRUMENT_FILE when you compile
your program. For each file that didn't match the the instrument file list, the compiler will
issue a warning at the end stating that no blocks were instrumented. If you
didn't intend to instrument that file, then you can safely ignore that warning.
@ -75,5 +75,5 @@ required anymore (and might hurt performance and crash detection, so better not
use -g).
## 4) UNIX-style filename pattern matching
You can add UNIX-style pattern matching in the whitelist entries. See `man
You can add UNIX-style pattern matching in the the instrument file list entries. See `man
fnmatch` for the syntax. We do not set any of the `fnmatch` flags.

View File

@ -7,7 +7,7 @@ This version requires a current llvm 11 compiled from the github master.
1. Use afl-clang-lto/afl-clang-lto++ because it is faster and gives better
coverage than anything else that is out there in the AFL world
2. You can use it together with llvm_mode: laf-intel and whitelisting
2. You can use it together with llvm_mode: laf-intel and the instrument file listing
features and can be combined with cmplog/Redqueen
3. It only works with llvm 11 (current github master state)
@ -108,7 +108,7 @@ make install
Just use afl-clang-lto like you did with afl-clang-fast or afl-gcc.
Also whitelisting (AFL_LLVM_WHITELIST -> [README.whitelist.md](README.whitelist.md)) and
Also the instrument file listing (AFL_LLVM_INSTRUMENT_FILE -> [README.instrument_file.md](README.instrument_file.md)) and
laf-intel/compcov (AFL_LLVM_LAF_* -> [README.laf-intel.md](README.laf-intel.md)) work.
InsTrim (control flow graph instrumentation) is supported and recommended!
(set `AFL_LLVM_INSTRUMENT=CFG`)

View File

@ -108,8 +108,8 @@ directory.
Several options are present to make llvm_mode faster or help it rearrange
the code to make afl-fuzz path discovery easier.
If you need just to instrument specific parts of the code, you can whitelist
which C/C++ files to actually instrument. See [README.whitelist](README.whitelist.md)
If you need just to instrument specific parts of the code, you can the instrument file list
which C/C++ files to actually instrument. See [README.instrument_file](README.instrument_file.md)
For splitting memcmp, strncmp, etc. please see [README.laf-intel](README.laf-intel.md)

View File

@ -1,10 +0,0 @@
TODO for afl-ld:
* handle libfoo.a object archives
TODO for afl-llvm-lto-instrumentation:
* better algo for putting stuff in the map?
* try to predict how long the instrumentation process will take
TODO for afl-llvm-lto-whitelist
* different solution then renaming?

View File

@ -227,13 +227,14 @@ static void edit_params(u32 argc, char **argv, char **envp) {
if (lto_mode) {
if (getenv("AFL_LLVM_WHITELIST") != NULL) {
if (getenv("AFL_LLVM_INSTRUMENT_FILE") != NULL ||
getenv("AFL_LLVM_WHITELIST")) {
cc_params[cc_par_cnt++] = "-Xclang";
cc_params[cc_par_cnt++] = "-load";
cc_params[cc_par_cnt++] = "-Xclang";
cc_params[cc_par_cnt++] =
alloc_printf("%s/afl-llvm-lto-whitelist.so", obj_path);
alloc_printf("%s/afl-llvm-lto-instrumentlist.so", obj_path);
}
@ -762,7 +763,7 @@ int main(int argc, char **argv, char **envp) {
#if LLVM_VERSION_MAJOR <= 6
instrument_mode = INSTRUMENT_AFL;
#else
if (getenv("AFL_LLVM_WHITELIST"))
if (getenv("AFL_LLVM_INSTRUMENT_FILE") || getenv("AFL_LLVM_WHITELIST"))
instrument_mode = INSTRUMENT_AFL;
else
instrument_mode = INSTRUMENT_PCGUARD;
@ -810,8 +811,11 @@ int main(int argc, char **argv, char **envp) {
"AFL_LLVM_NOT_ZERO and AFL_LLVM_SKIP_NEVERZERO can not be set "
"together");
if (instrument_mode == INSTRUMENT_PCGUARD && getenv("AFL_LLVM_WHITELIST"))
WARNF("Instrumentation type PCGUARD does not support AFL_LLVM_WHITELIST!");
if (instrument_mode == INSTRUMENT_PCGUARD &&
(getenv("AFL_LLVM_INSTRUMENT_FILE") || getenv("AFL_LLVM_WHITELIST")))
WARNF(
"Instrumentation type PCGUARD does not support "
"AFL_LLVM_INSTRUMENT_FILE!");
if (argc < 2 || strcmp(argv[1], "-h") == 0) {
@ -861,7 +865,8 @@ int main(int argc, char **argv, char **envp) {
"AFL_LLVM_LAF_TRANSFORM_COMPARES: transform library comparison "
"function calls\n"
"AFL_LLVM_LAF_ALL: enables all LAF splits/transforms\n"
"AFL_LLVM_WHITELIST: enable whitelisting (selective "
"AFL_LLVM_INSTRUMENT_FILE: enable the instrument file listing "
"(selective "
"instrumentation)\n"
"AFL_NO_BUILTIN: compile for use with libtokencap.so\n"
"AFL_PATH: path to instrumenting pass and runtime "

View File

@ -18,7 +18,7 @@
using namespace llvm;
static std::list<std::string> myWhitelist;
static std::list<std::string> myInstrumentList;
char *getBBName(const llvm::BasicBlock *BB) {
@ -44,7 +44,7 @@ char *getBBName(const llvm::BasicBlock *BB) {
}
/* Function that we never instrument or analyze */
/* Note: this ignore check is also called in isInWhitelist() */
/* Note: this ignore check is also called in isInInstrumentList() */
bool isIgnoreFunction(const llvm::Function *F) {
// Starting from "LLVMFuzzer" these are functions used in libfuzzer based
@ -83,19 +83,22 @@ bool isIgnoreFunction(const llvm::Function *F) {
}
void initWhitelist() {
void initInstrumentList() {
char *instWhiteListFilename = getenv("AFL_LLVM_WHITELIST");
if (instWhiteListFilename) {
char *instrumentListFilename = getenv("AFL_LLVM_INSTRUMENT_FILE");
if (!instrumentListFilename)
instrumentListFilename = getenv("AFL_LLVM_WHITELIST");
if (instrumentListFilename) {
std::string line;
std::ifstream fileStream;
fileStream.open(instWhiteListFilename);
if (!fileStream) report_fatal_error("Unable to open AFL_LLVM_WHITELIST");
fileStream.open(instrumentListFilename);
if (!fileStream)
report_fatal_error("Unable to open AFL_LLVM_INSTRUMENT_FILE");
getline(fileStream, line);
while (fileStream) {
myWhitelist.push_back(line);
myInstrumentList.push_back(line);
getline(fileStream, line);
}
@ -104,14 +107,14 @@ void initWhitelist() {
}
bool isInWhitelist(llvm::Function *F) {
bool isInInstrumentList(llvm::Function *F) {
// is this a function with code? If it is external we dont instrument it
// anyway and cant be in the whitelist. Or if it is ignored.
// anyway and cant be in the the instrument file list. Or if it is ignored.
if (!F->size() || isIgnoreFunction(F)) return false;
// if we do not have a whitelist return true
if (myWhitelist.empty()) return true;
// if we do not have a the instrument file list return true
if (myInstrumentList.empty()) return true;
// let's try to get the filename for the function
auto bb = &F->getEntryBlock();
@ -147,8 +150,8 @@ bool isInWhitelist(llvm::Function *F) {
/* Continue only if we know where we actually are */
if (!instFilename.str().empty()) {
for (std::list<std::string>::iterator it = myWhitelist.begin();
it != myWhitelist.end(); ++it) {
for (std::list<std::string>::iterator it = myInstrumentList.begin();
it != myInstrumentList.end(); ++it) {
/* We don't check for filename equality here because
* filenames might actually be full paths. Instead we
@ -185,8 +188,8 @@ bool isInWhitelist(llvm::Function *F) {
/* Continue only if we know where we actually are */
if (!instFilename.str().empty()) {
for (std::list<std::string>::iterator it = myWhitelist.begin();
it != myWhitelist.end(); ++it) {
for (std::list<std::string>::iterator it = myInstrumentList.begin();
it != myInstrumentList.end(); ++it) {
/* We don't check for filename equality here because
* filenames might actually be full paths. Instead we
@ -215,7 +218,7 @@ bool isInWhitelist(llvm::Function *F) {
else {
// we could not find out the location. in this case we say it is not
// in the whitelist
// in the the instrument file list
return false;

View File

@ -34,8 +34,8 @@ typedef long double max_align_t;
char * getBBName(const llvm::BasicBlock *BB);
bool isIgnoreFunction(const llvm::Function *F);
void initWhitelist();
bool isInWhitelist(llvm::Function *F);
void initInstrumentList();
bool isInInstrumentList(llvm::Function *F);
unsigned long long int calculateCollisions(uint32_t edges);
#endif

View File

@ -566,12 +566,13 @@ struct InsTrimLTO : public ModulePass {
functions++;
// whitelist check
// the instrument file list check
AttributeList Attrs = F.getAttributes();
if (Attrs.hasAttribute(-1, StringRef("skipinstrument"))) {
if (debug)
fprintf(stderr, "DEBUG: Function %s is not whitelisted\n",
fprintf(stderr,
"DEBUG: Function %s is not the instrument file listed\n",
F.getName().str().c_str());
continue;

View File

@ -198,12 +198,13 @@ bool AFLLTOPass::runOnModule(Module &M) {
if (F.size() < function_minimum_size) continue;
if (isIgnoreFunction(&F)) continue;
// whitelist check
// the instrument file list check
AttributeList Attrs = F.getAttributes();
if (Attrs.hasAttribute(-1, StringRef("skipinstrument"))) {
if (debug)
fprintf(stderr, "DEBUG: Function %s is not whitelisted\n",
fprintf(stderr,
"DEBUG: Function %s is not the instrument file listed\n",
F.getName().str().c_str());
continue;

View File

@ -53,27 +53,30 @@ using namespace llvm;
namespace {
class AFLwhitelist : public ModulePass {
class AFLcheckIfInstrument : public ModulePass {
public:
static char ID;
AFLwhitelist() : ModulePass(ID) {
AFLcheckIfInstrument() : ModulePass(ID) {
int entries = 0;
if (getenv("AFL_DEBUG")) debug = 1;
char *instWhiteListFilename = getenv("AFL_LLVM_WHITELIST");
if (instWhiteListFilename) {
char *instrumentListFilename = getenv("AFL_LLVM_INSTRUMENT_FILE");
if (!instrumentListFilename)
instrumentListFilename = getenv("AFL_LLVM_WHITELIST");
if (instrumentListFilename) {
std::string line;
std::ifstream fileStream;
fileStream.open(instWhiteListFilename);
if (!fileStream) report_fatal_error("Unable to open AFL_LLVM_WHITELIST");
fileStream.open(instrumentListFilename);
if (!fileStream)
report_fatal_error("Unable to open AFL_LLVM_INSTRUMENT_FILE");
getline(fileStream, line);
while (fileStream) {
myWhitelist.push_back(line);
myInstrumentList.push_back(line);
getline(fileStream, line);
entries++;
@ -81,11 +84,14 @@ class AFLwhitelist : public ModulePass {
} else
PFATAL("afl-llvm-lto-whitelist.so loaded without AFL_LLVM_WHITELIST?!");
PFATAL(
"afl-llvm-lto-instrumentlist.so loaded without "
"AFL_LLVM_INSTRUMENT_FILE?!");
if (debug)
SAYF(cMGN "[D] " cRST "loaded whitelist %s with %d entries\n",
instWhiteListFilename, entries);
SAYF(cMGN "[D] " cRST
"loaded the instrument file list %s with %d entries\n",
instrumentListFilename, entries);
}
@ -97,16 +103,16 @@ class AFLwhitelist : public ModulePass {
// }
protected:
std::list<std::string> myWhitelist;
std::list<std::string> myInstrumentList;
int debug = 0;
};
} // namespace
char AFLwhitelist::ID = 0;
char AFLcheckIfInstrument::ID = 0;
bool AFLwhitelist::runOnModule(Module &M) {
bool AFLcheckIfInstrument::runOnModule(Module &M) {
/* Show a banner */
@ -115,7 +121,7 @@ bool AFLwhitelist::runOnModule(Module &M) {
if ((isatty(2) && !getenv("AFL_QUIET")) || getenv("AFL_DEBUG") != NULL) {
SAYF(cCYA "afl-llvm-lto-whitelist" VERSION cRST
SAYF(cCYA "afl-llvm-lto-instrumentlist" VERSION cRST
" by Marc \"vanHauser\" Heuse <mh@mh-sec.de>\n");
} else if (getenv("AFL_QUIET"))
@ -131,7 +137,7 @@ bool AFLwhitelist::runOnModule(Module &M) {
BasicBlock::iterator IP = F.getEntryBlock().getFirstInsertionPt();
IRBuilder<> IRB(&(*IP));
if (!myWhitelist.empty()) {
if (!myInstrumentList.empty()) {
bool instrumentFunction = false;
@ -168,8 +174,8 @@ bool AFLwhitelist::runOnModule(Module &M) {
/* Continue only if we know where we actually are */
if (!instFilename.str().empty()) {
for (std::list<std::string>::iterator it = myWhitelist.begin();
it != myWhitelist.end(); ++it) {
for (std::list<std::string>::iterator it = myInstrumentList.begin();
it != myInstrumentList.end(); ++it) {
/* We don't check for filename equality here because
* filenames might actually be full paths. Instead we
@ -194,18 +200,19 @@ bool AFLwhitelist::runOnModule(Module &M) {
}
/* Either we couldn't figure out our location or the location is
* not whitelisted, so we skip instrumentation.
* not the instrument file listed, so we skip instrumentation.
* We do this by renaming the function. */
if (instrumentFunction == true) {
if (debug)
SAYF(cMGN "[D] " cRST "function %s is in whitelist\n",
SAYF(cMGN "[D] " cRST "function %s is in the instrument file list\n",
F.getName().str().c_str());
} else {
if (debug)
SAYF(cMGN "[D] " cRST "function %s is NOT in whitelist\n",
SAYF(cMGN "[D] " cRST
"function %s is NOT in the instrument file list\n",
F.getName().str().c_str());
auto & Ctx = F.getContext();
@ -219,7 +226,7 @@ bool AFLwhitelist::runOnModule(Module &M) {
} else {
PFATAL("Whitelist is empty");
PFATAL("InstrumentList is empty");
}
@ -229,16 +236,18 @@ bool AFLwhitelist::runOnModule(Module &M) {
}
static void registerAFLwhitelistpass(const PassManagerBuilder &,
static void registerAFLcheckIfInstrumentpass(const PassManagerBuilder &,
legacy::PassManagerBase &PM) {
PM.add(new AFLwhitelist());
PM.add(new AFLcheckIfInstrument());
}
static RegisterStandardPasses RegisterAFLwhitelistpass(
PassManagerBuilder::EP_ModuleOptimizerEarly, registerAFLwhitelistpass);
static RegisterStandardPasses RegisterAFLcheckIfInstrumentpass(
PassManagerBuilder::EP_ModuleOptimizerEarly,
registerAFLcheckIfInstrumentpass);
static RegisterStandardPasses RegisterAFLwhitelistpass0(
PassManagerBuilder::EP_EnabledOnOptLevel0, registerAFLwhitelistpass);
static RegisterStandardPasses RegisterAFLcheckIfInstrumentpass0(
PassManagerBuilder::EP_EnabledOnOptLevel0,
registerAFLcheckIfInstrumentpass);

View File

@ -74,7 +74,7 @@ class AFLCoverage : public ModulePass {
static char ID;
AFLCoverage() : ModulePass(ID) {
initWhitelist();
initInstrumentList();
}
@ -307,7 +307,7 @@ bool AFLCoverage::runOnModule(Module &M) {
fprintf(stderr, "FUNCTION: %s (%zu)\n", F.getName().str().c_str(),
F.size());
if (!isInWhitelist(&F)) continue;
if (!isInInstrumentList(&F)) continue;
if (F.size() < function_minimum_size) continue;

View File

@ -59,7 +59,7 @@ class CmpLogInstructions : public ModulePass {
static char ID;
CmpLogInstructions() : ModulePass(ID) {
initWhitelist();
initInstrumentList();
}
@ -170,7 +170,7 @@ bool CmpLogInstructions::hookInstrs(Module &M) {
/* iterate over all functions, bbs and instruction and add suitable calls */
for (auto &F : M) {
if (!isInWhitelist(&F)) continue;
if (!isInInstrumentList(&F)) continue;
for (auto &BB : F) {

View File

@ -59,7 +59,7 @@ class CmpLogRoutines : public ModulePass {
static char ID;
CmpLogRoutines() : ModulePass(ID) {
initWhitelist();
initInstrumentList();
}
@ -118,7 +118,7 @@ bool CmpLogRoutines::hookRtns(Module &M) {
/* iterate over all functions, bbs and instruction and add suitable calls */
for (auto &F : M) {
if (!isInWhitelist(&F)) continue;
if (!isInInstrumentList(&F)) continue;
for (auto &BB : F) {

View File

@ -58,7 +58,7 @@ class CompareTransform : public ModulePass {
static char ID;
CompareTransform() : ModulePass(ID) {
initWhitelist();
initInstrumentList();
}
@ -124,7 +124,7 @@ bool CompareTransform::transformCmps(Module &M, const bool processStrcmp,
* strcmp/memcmp/strncmp/strcasecmp/strncasecmp */
for (auto &F : M) {
if (!isInWhitelist(&F)) continue;
if (!isInInstrumentList(&F)) continue;
for (auto &BB : F) {

View File

@ -55,7 +55,7 @@ class SplitComparesTransform : public ModulePass {
static char ID;
SplitComparesTransform() : ModulePass(ID) {
initWhitelist();
initInstrumentList();
}
@ -102,7 +102,7 @@ bool SplitComparesTransform::simplifyCompares(Module &M) {
* all integer comparisons with >= and <= predicates to the icomps vector */
for (auto &F : M) {
if (!isInWhitelist(&F)) continue;
if (!isInInstrumentList(&F)) continue;
for (auto &BB : F) {

View File

@ -60,7 +60,7 @@ class SplitSwitchesTransform : public ModulePass {
static char ID;
SplitSwitchesTransform() : ModulePass(ID) {
initWhitelist();
initInstrumentList();
}
@ -312,7 +312,7 @@ bool SplitSwitchesTransform::splitSwitches(Module &M) {
* all switches to switches vector for later processing */
for (auto &F : M) {
if (!isInWhitelist(&F)) continue;
if (!isInInstrumentList(&F)) continue;
for (auto &BB : F) {

View File

@ -58,7 +58,7 @@ char *afl_environment_variables[] = {
//"AFL_DEFER_FORKSRV", // not implemented anymore, so warn additionally
"AFL_DISABLE_TRIM", "AFL_DONT_OPTIMIZE", "AFL_DUMB_FORKSRV",
"AFL_ENTRYPOINT", "AFL_EXIT_WHEN_DONE", "AFL_FAST_CAL", "AFL_FORCE_UI",
"AFL_GCC_WHITELIST", "AFL_GCJ", "AFL_HANG_TMOUT", "AFL_HARDEN",
"AFL_GCC_INSTRUMENT_FILE", "AFL_GCJ", "AFL_HANG_TMOUT", "AFL_HARDEN",
"AFL_I_DONT_CARE_ABOUT_MISSING_CRASHES", "AFL_IMPORT_FIRST",
"AFL_INST_LIBS", "AFL_INST_RATIO", "AFL_KEEP_TRACES", "AFL_KEEP_ASSEMBLY",
"AFL_LD_HARD_FAIL", "AFL_LD_LIMIT_MB", "AFL_LD_NO_CALLOC_OVER",
@ -71,7 +71,7 @@ char *afl_environment_variables[] = {
"AFL_LLVM_LAF_SPLIT_FLOATS", "AFL_LLVM_LAF_SPLIT_SWITCHES",
"AFL_LLVM_LAF_ALL", "AFL_LLVM_LAF_TRANSFORM_COMPARES", "AFL_LLVM_MAP_ADDR",
"AFL_LLVM_MAP_DYNAMIC", "AFL_LLVM_NGRAM_SIZE", "AFL_NGRAM_SIZE",
"AFL_LLVM_NOT_ZERO", "AFL_LLVM_WHITELIST", "AFL_LLVM_SKIP_NEVERZERO",
"AFL_LLVM_NOT_ZERO", "AFL_LLVM_INSTRUMENT_FILE", "AFL_LLVM_SKIP_NEVERZERO",
"AFL_NO_AFFINITY", "AFL_LLVM_LTO_STARTID", "AFL_LLVM_LTO_DONTWRITEID",
"AFL_NO_ARITH", "AFL_NO_BUILTIN", "AFL_NO_CPU_RED", "AFL_NO_FORKSRV",
"AFL_NO_UI", "AFL_NO_PYTHON", "AFL_UNTRACER_FILE", "AFL_LLVM_USE_TRACE_PC",

View File

@ -790,8 +790,8 @@ int main(int argc, char **argv_orig, char **envp) {
OKF("afl++ is open source, get it at "
"https://github.com/AFLplusplus/AFLplusplus");
OKF("Power schedules from github.com/mboehme/aflfast");
OKF("Python Mutator and llvm_mode whitelisting from github.com/choller/afl");
OKF("afl-tmin fork server patch from github.com/nccgroup/TriforceAFL");
OKF("Python Mutator and llvm_mode instrument file list from "
"github.com/choller/afl");
OKF("MOpt Mutator from github.com/puppet-meteor/MOpt-AFL");
if (afl->sync_id && afl->is_main_node &&

View File

@ -21,8 +21,8 @@ unset AFL_USE_ASAN
unset AFL_USE_MSAN
unset AFL_CC
unset AFL_PRELOAD
unset AFL_GCC_WHITELIST
unset AFL_LLVM_WHITELIST
unset AFL_GCC_INSTRUMENT_FILE
unset AFL_LLVM_INSTRUMENT_FILE
unset AFL_LLVM_INSTRIM
unset AFL_LLVM_LAF_SPLIT_SWITCHES
unset AFL_LLVM_LAF_TRANSFORM_COMPARES

View File

@ -62,8 +62,8 @@ unset AFL_USE_UBSAN
unset AFL_TMPDIR
unset AFL_CC
unset AFL_PRELOAD
unset AFL_GCC_WHITELIST
unset AFL_LLVM_WHITELIST
unset AFL_GCC_INSTRUMENT_FILE
unset AFL_LLVM_INSTRUMENT_FILE
unset AFL_LLVM_INSTRIM
unset AFL_LLVM_LAF_SPLIT_SWITCHES
unset AFL_LLVM_LAF_TRANSFORM_COMPARES
@ -386,20 +386,20 @@ test -e ../afl-clang-fast -a -e ../split-switches-pass.so && {
CODE=1
}
rm -f test-compcov.compcov test.out
echo foobar.c > whitelist.txt
AFL_DEBUG=1 AFL_LLVM_WHITELIST=whitelist.txt ../afl-clang-fast -o test-compcov test-compcov.c > test.out 2>&1
echo foobar.c > instrumentlist.txt
AFL_DEBUG=1 AFL_LLVM_INSTRUMENT_FILE=instrumentlist.txt ../afl-clang-fast -o test-compcov test-compcov.c > test.out 2>&1
test -e test-compcov && test_compcov_binary_functionality ./test-compcov && {
grep -q "No instrumentation targets found" test.out && {
$ECHO "$GREEN[+] llvm_mode whitelist feature works correctly"
$ECHO "$GREEN[+] llvm_mode instrumentlist feature works correctly"
} || {
$ECHO "$RED[!] llvm_mode whitelist feature failed"
$ECHO "$RED[!] llvm_mode instrumentlist feature failed"
CODE=1
}
} || {
$ECHO "$RED[!] llvm_mode whitelist feature compilation failed"
$ECHO "$RED[!] llvm_mode instrumentlist feature compilation failed"
CODE=1
}
rm -f test-compcov test.out whitelist.txt
rm -f test-compcov test.out instrumentlist.txt
../afl-clang-fast -o test-persistent ../examples/persistent_demo/persistent_demo.c > /dev/null 2>&1
test -e test-persistent && {
echo foo | ../afl-showmap -m ${MEM_LIMIT} -o /dev/null -q -r ./test-persistent && {
@ -459,20 +459,20 @@ test -e ../afl-clang-lto -a -e ../afl-llvm-lto-instrumentation.so && {
}
rm -f test-instr.plain
echo foobar.c > whitelist.txt
AFL_DEBUG=1 AFL_LLVM_WHITELIST=whitelist.txt ../afl-clang-lto -o test-compcov test-compcov.c > test.out 2>&1
echo foobar.c > instrumentlist.txt
AFL_DEBUG=1 AFL_LLVM_INSTRUMENT_FILE=instrumentlist.txt ../afl-clang-lto -o test-compcov test-compcov.c > test.out 2>&1
test -e test-compcov && {
grep -q "No instrumentation targets found" test.out && {
$ECHO "$GREEN[+] llvm_mode LTO whitelist feature works correctly"
$ECHO "$GREEN[+] llvm_mode LTO instrumentlist feature works correctly"
} || {
$ECHO "$RED[!] llvm_mode LTO whitelist feature failed"
$ECHO "$RED[!] llvm_mode LTO instrumentlist feature failed"
CODE=1
}
} || {
$ECHO "$RED[!] llvm_mode LTO whitelist feature compilation failed"
$ECHO "$RED[!] llvm_mode LTO instrumentlist feature compilation failed"
CODE=1
}
rm -f test-compcov test.out whitelist.txt
rm -f test-compcov test.out instrumentlist.txt
../afl-clang-lto -o test-persistent ../examples/persistent_demo/persistent_demo.c > /dev/null 2>&1
test -e test-persistent && {
echo foo | ../afl-showmap -m none -o /dev/null -q -r ./test-persistent && {
@ -569,20 +569,20 @@ test -e ../afl-gcc-fast -a -e ../afl-gcc-rt.o && {
rm -f test-instr.plain.gccpi
# now for the special gcc_plugin things
echo foobar.c > whitelist.txt
AFL_GCC_WHITELIST=whitelist.txt ../afl-gcc-fast -o test-compcov test-compcov.c > /dev/null 2>&1
echo foobar.c > instrumentlist.txt
AFL_GCC_INSTRUMENT_FILE=instrumentlist.txt ../afl-gcc-fast -o test-compcov test-compcov.c > /dev/null 2>&1
test -e test-compcov && test_compcov_binary_functionality ./test-compcov && {
echo 1 | ../afl-showmap -m ${MEM_LIMIT} -o - -r -- ./test-compcov 2>&1 | grep -q "Captured 1 tuples" && {
$ECHO "$GREEN[+] gcc_plugin whitelist feature works correctly"
$ECHO "$GREEN[+] gcc_plugin instrumentlist feature works correctly"
} || {
$ECHO "$RED[!] gcc_plugin whitelist feature failed"
$ECHO "$RED[!] gcc_plugin instrumentlist feature failed"
CODE=1
}
} || {
$ECHO "$RED[!] gcc_plugin whitelist feature compilation failed"
$ECHO "$RED[!] gcc_plugin instrumentlist feature compilation failed"
CODE=1
}
rm -f test-compcov test.out whitelist.txt
rm -f test-compcov test.out instrumentlist.txt
../afl-gcc-fast -o test-persistent ../examples/persistent_demo/persistent_demo.c > /dev/null 2>&1
test -e test-persistent && {
echo foo | ../afl-showmap -m ${MEM_LIMIT} -o /dev/null -q -r ./test-persistent && {