ExternalVendorCode/AFLplusplus
1
0
Fork 0
mirror of https://github.com/AFLplusplus/AFLplusplus.git synced 2025-06-10 17:21:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #2117 from AFLplusplus/dev

Browse Source 
push to stable
This commit is contained in:
van Hauser 2024-06-09 19:09:17 +02:00 committed by GitHub
commit 9f6b012fbf
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
25 changed files with 426 additions and 236 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@ -2,9 +2,9 @@
<img align="right" src="https://raw.githubusercontent.com/AFLplusplus/Website/main/static/aflpp_bg.svg" alt="AFL++ logo" width="250" heigh="250">
Release version: [4.20c](https://github.com/AFLplusplus/AFLplusplus/releases)
Release version: [4.21c](https://github.com/AFLplusplus/AFLplusplus/releases)
GitHub version: 4.21a
GitHub version: 4.21c
Repository:
[https://github.com/AFLplusplus/AFLplusplus](https://github.com/AFLplusplus/AFLplusplus)

3
TODO.md
View File

@ -2,6 +2,8 @@
## Must
- fast restart of afl-fuzz if cmdline + target hash is the same
- check for null ptr for xml/curl/g_ string transform functions
- hardened_usercopy=0 page_alloc.shuffle=0
- add value_profile but only enable after 15 minutes without finds
- cmplog max items env?
@ -11,7 +13,6 @@
- afl-showmap -f support
- afl-fuzz multicore wrapper script
- when trimming then perform crash detection
- cyclomatic complexity: 2 + calls + edges - blocks
## Should

2
custom_mutators/grammar_mutator/GRAMMAR_VERSION
View File

@ -1 +1 @@
5ed4f8d
95a6857

2
custom_mutators/grammar_mutator/grammar_mutator

@ -1 +1 @@
Subproject commit 5ed4f8d6e6524df9670af6b411b13031833d67d2
Subproject commit 95a685773e571620cb6e2788dbbdba333e1b9bfd

12
docs/Changelog.md
View File

@ -3,12 +3,16 @@
This is the list of all noteworthy changes made in every public
release of the tool. See README.md for the general instruction manual.
### Version ++4.21a (dev)
### Version ++4.21c (release)
* afl-fuzz
- fixed a regression in afl-fuzz that resulted in a 5-10% performace loss
do a switch from gettimeofday() to clock_gettime() which should be rather
three times faster. The reason for this is unknown.
- new queue selection algorithm based on 2 core years of queue data
analysis. gives a noticable improvement on coverage although the results
seem counterintuitive :-)
- added AFL_DISABLE_REDUNDANT for huge queues
- added `AFL_NO_SYNC` environment variable that does what you think it does
- fix AFL_PERSISTENT_RECORD
- run custom_post_process after standard trimming
- prevent filenames in the queue that have spaces
@ -19,6 +23,9 @@
- -V timing is now accurately the fuzz time (without syncing), before
long calibration times and syncing could result in now fuzzing being
made when the time was already run out until then, thanks to @eqv!
- fix -n uninstrumented mode when ending fuzzing
- enhanced the ASAN configuration
- make afl-fuzz use less memory with cmplog and fix a memleak
* afl-cc:
- re-enable i386 support that was accidently disabled
- fixes for LTO and outdated afl-gcc mode for i386
@ -32,7 +39,8 @@
* afl-showmap
- fix memory leak on shmem testcase usage (thanks to @ndrewh)
- minor fix to collect coverage -C (thanks to @bet4it)
* enhanced the ASAN configuration
* Fixed a shmem mmap bug (that rarely came up on MacOS)
* libtokencap: script generate_libtoken_dict.sh added by @a-shvedov
### Version ++4.20c (release)

3
docs/env_variables.md
View File

@ -588,6 +588,9 @@ checks or alter some of the more exotic semantics of the tool:
between fuzzing instances synchronization. Default sync time is 30 minutes,
note that time is halved for -M main nodes.
- `AFL_NO_SYNC` disables any syncing whatsoever and takes priority on all
other syncing parameters.
- Setting `AFL_TARGET_ENV` causes AFL++ to set extra environment variables for
the target binary. Example: `AFL_TARGET_ENV="VAR1=1 VAR2='a b c'" afl-fuzz
... `. This exists mostly for things like `LD_LIBRARY_PATH` but it would

13
frida_mode/src/instrument/instrument_x64_cache.c
View File

@ -49,14 +49,23 @@ void instrument_cache_init(void) {
if (setrlimit(RLIMIT_AS, &data_limit) != 0) {
FFATAL("Failed to setrlimit: %d", errno);
FWARNF("Failed to setrlimit: %d, you may need root or CAP_SYS_RESOURCE",
errno);
}
map_base =
gum_memory_allocate(NULL, instrument_cache_size, instrument_cache_size,
GUM_PAGE_READ | GUM_PAGE_WRITE);
if (map_base == MAP_FAILED) { FFATAL("Failed to map segment: %d", errno); }
if (map_base == MAP_FAILED) {
FFATAL(
"Failed to map segment: %d. This can be caused by failure to setrlimit."
"Disabling or reducing the size of the allocation using "
"AFL_FRIDA_INST_NO_CACHE or AFL_FRIDA_INST_CACHE_SIZE may help",
errno);
}
FOKF(cBLU "Instrumentation" cRST " - " cGRN "cache addr:" cYEL " [0x%016lX]",
GUM_ADDRESS(map_base));

31
frida_mode/src/ranges.c
View File

@ -194,24 +194,23 @@ static gboolean print_ranges_callback(const GumRangeDetails *details,
if (details->file == NULL) {
FVERBOSE("\t0x%016" G_GINT64_MODIFIER "x-0x%016" G_GINT64_MODIFIER
"X %c%c%c",
details->range->base_address,
details->range->base_address + details->range->size,
details->protection & GUM_PAGE_READ ? 'R' : '-',
details->protection & GUM_PAGE_WRITE ? 'W' : '-',
details->protection & GUM_PAGE_EXECUTE ? 'X' : '-');
OKF("\t0x%016" G_GINT64_MODIFIER "x-0x%016" G_GINT64_MODIFIER "X %c%c%c",
details->range->base_address,
details->range->base_address + details->range->size,
details->protection & GUM_PAGE_READ ? 'R' : '-',
details->protection & GUM_PAGE_WRITE ? 'W' : '-',
details->protection & GUM_PAGE_EXECUTE ? 'X' : '-');
} else {
FVERBOSE("\t0x%016" G_GINT64_MODIFIER "x-0x%016" G_GINT64_MODIFIER
"X %c%c%c %s(0x%016" G_GINT64_MODIFIER "x)",
details->range->base_address,
details->range->base_address + details->range->size,
details->protection & GUM_PAGE_READ ? 'R' : '-',
details->protection & GUM_PAGE_WRITE ? 'W' : '-',
details->protection & GUM_PAGE_EXECUTE ? 'X' : '-',
details->file->path, details->file->offset);
OKF("\t0x%016" G_GINT64_MODIFIER "x-0x%016" G_GINT64_MODIFIER
"X %c%c%c %s(0x%016" G_GINT64_MODIFIER "x)",
details->range->base_address,
details->range->base_address + details->range->size,
details->protection & GUM_PAGE_READ ? 'R' : '-',
details->protection & GUM_PAGE_WRITE ? 'W' : '-',
details->protection & GUM_PAGE_EXECUTE ? 'X' : '-', details->file->path,
details->file->offset);
}
@ -581,7 +580,7 @@ static GArray *merge_ranges(GArray *a) {
void ranges_print_debug_maps(void) {
FVERBOSE("Maps");
OKF("Maps");
gum_process_enumerate_ranges(GUM_PAGE_NO_ACCESS, print_ranges_callback, NULL);
}

5
include/afl-fuzz.h
View File

@ -457,7 +457,7 @@ typedef struct afl_env_vars {
afl_no_startup_calibration, afl_no_warn_instability,
afl_post_process_keep_original, afl_crashing_seeds_as_new_crash,
afl_final_sync, afl_ignore_seed_problems, afl_disable_redundant,
afl_sha1_filenames;
afl_sha1_filenames, afl_no_sync;
u8 *afl_tmpdir, *afl_custom_mutator_library, *afl_python_module, *afl_path,
*afl_hang_tmout, *afl_forksrv_init_tmout, *afl_preload,
@ -656,6 +656,7 @@ typedef struct afl_state {
switch_fuzz_mode, /* auto or fixed fuzz mode */
calibration_time_us, /* Time spend on calibration */
sync_time_us, /* Time spend on sync */
cmplog_time_us, /* Time spend on cmplog */
trim_time_us; /* Time spend on trimming */
u32 slowest_exec_ms, /* Slowest testcase non hang in ms */
@ -1226,6 +1227,7 @@ void show_init_stats(afl_state_t *);
void update_calibration_time(afl_state_t *afl, u64 *time);
void update_trim_time(afl_state_t *afl, u64 *time);
void update_sync_time(afl_state_t *afl, u64 *time);
void update_cmplog_time(afl_state_t *afl, u64 *time);
/* StatsD */
@ -1276,6 +1278,7 @@ void get_core_count(afl_state_t *);
void fix_up_sync(afl_state_t *);
void check_asan_opts(afl_state_t *);
void check_binary(afl_state_t *, u8 *);
u64 get_binary_hash(u8 *fn);
void check_if_tty(afl_state_t *);
void save_cmdline(afl_state_t *, u32, char **);
void read_foreign_testcases(afl_state_t *, int);

6
include/config.h
View File

@ -26,7 +26,7 @@
/* Version string: */
// c = release, a = volatile github dev, e = experimental branch
#define VERSION "++4.21a"
#define VERSION "++4.21c"
/******************************************************
* *
@ -324,9 +324,9 @@
#define SYNC_INTERVAL 8
/* Sync time (minimum time between syncing in ms, time is halfed for -M main
nodes) - default is 30 minutes: */
nodes) - default is 20 minutes: */
#define SYNC_TIME (30 * 60 * 1000)
#define SYNC_TIME (20 * 60 * 1000)
/* Output directory reuse grace period (minutes): */

9
include/envs.h
View File

@ -81,14 +81,13 @@ static char *afl_environment_variables[] = {
"AFL_LLVM_MAP_DYNAMIC", "AFL_LLVM_NGRAM_SIZE", "AFL_NGRAM_SIZE",
"AFL_LLVM_NO_RPATH", "AFL_LLVM_NOT_ZERO", "AFL_LLVM_INSTRUMENT_FILE",
"AFL_LLVM_THREADSAFE_INST", "AFL_LLVM_SKIP_NEVERZERO", "AFL_NO_AFFINITY",
"AFL_TRY_AFFINITY", "AFL_LLVM_LTO_DONTWRITEID",
"AFL_LLVM_LTO_SKIPINIT"
"AFL_LLVM_LTO_STARTID",
"AFL_FUZZER_LOOPCOUNT", "AFL_NO_ARITH", "AFL_NO_AUTODICT", "AFL_NO_BUILTIN",
"AFL_TRY_AFFINITY", "AFL_LLVM_LTO_DONTWRITEID", "AFL_LLVM_LTO_SKIPINIT",
"AFL_LLVM_LTO_STARTID", "AFL_FUZZER_LOOPCOUNT", "AFL_NO_ARITH",
"AFL_NO_AUTODICT", "AFL_NO_BUILTIN",
#if defined USE_COLOR && !defined ALWAYS_COLORED
"AFL_NO_COLOR", "AFL_NO_COLOUR",
#endif
"AFL_NO_CPU_RED",
"AFL_NO_CPU_RED", "AFL_NO_SYNC",
"AFL_NO_CFG_FUZZING", // afl.rs rust crate option
"AFL_NO_CRASH_README", "AFL_NO_FORKSRV", "AFL_NO_UI", "AFL_NO_PYTHON",
"AFL_NO_STARTUP_CALIBRATION", "AFL_NO_WARN_INSTABILITY",

2
instrumentation/afl-compiler-rt.o.c
View File

@ -1849,7 +1849,7 @@ void __sanitizer_cov_trace_pc_guard_init(uint32_t *start, uint32_t *stop) {
to avoid duplicate calls (which can happen as an artifact of the underlying
implementation in LLVM). */
if (__afl_final_loc < 5) __afl_final_loc = 5; // we skip the first 5 entries
if (__afl_final_loc < 4) __afl_final_loc = 4; // we skip the first 5 entries
*(start++) = ++__afl_final_loc;

3
src/afl-forkserver.c
View File

@ -1655,7 +1655,8 @@ void afl_fsrv_kill(afl_forkserver_t *fsrv) {
if (fsrv->fsrv_pid > 0) {
kill(fsrv->fsrv_pid, fsrv->fsrv_kill_signal);
waitpid(fsrv->fsrv_pid, NULL, 0);
usleep(25);
waitpid(fsrv->fsrv_pid, NULL, WNOHANG);
}

292
src/afl-fuzz-queue.c
View File

@ -60,32 +60,6 @@ inline u32 select_next_queue_entry(afl_state_t *afl) {
}
double compute_weight(afl_state_t *afl, struct queue_entry *q,
double avg_exec_us, double avg_bitmap_size,
double avg_top_size) {
double weight = 1.0;
if (likely(afl->schedule >= FAST && afl->schedule <= RARE)) {
u32 hits = afl->n_fuzz[q->n_fuzz_entry];
if (likely(hits)) { weight /= (log10(hits) + 1); }
}
if (likely(afl->schedule < RARE)) { weight *= (avg_exec_us / q->exec_us); }
weight *= (log(q->bitmap_size) / avg_bitmap_size);
weight *= (1 + (q->tc_ref / avg_top_size));
if (unlikely(weight < 0.1)) { weight = 0.1; }
if (unlikely(q->favored)) { weight *= 5; }
if (unlikely(!q->was_fuzzed)) { weight *= 2; }
if (unlikely(q->fs_redundant)) { weight *= 0.8; }
return weight;
}
/* create the alias table that allows weighted random selection - expensive */
void create_alias_table(afl_state_t *afl) {
@ -117,7 +91,7 @@ void create_alias_table(afl_state_t *afl) {
double avg_exec_us = 0.0;
double avg_bitmap_size = 0.0;
double avg_top_size = 0.0;
double avg_len = 0.0;
u32 active = 0;
for (i = 0; i < n; i++) {
@ -129,7 +103,7 @@ void create_alias_table(afl_state_t *afl) {
avg_exec_us += q->exec_us;
avg_bitmap_size += log(q->bitmap_size);
avg_top_size += q->tc_ref;
avg_len += q->len;
++active;
}
@ -138,7 +112,7 @@ void create_alias_table(afl_state_t *afl) {
avg_exec_us /= active;
avg_bitmap_size /= active;
avg_top_size /= active;
avg_len /= active;
for (i = 0; i < n; i++) {
@ -146,8 +120,59 @@ void create_alias_table(afl_state_t *afl) {
if (likely(!q->disabled)) {
q->weight =
compute_weight(afl, q, avg_exec_us, avg_bitmap_size, avg_top_size);
double weight = 1.0;
{ // inline does result in a compile error with LTO, weird
if (likely(afl->schedule >= FAST && afl->schedule <= RARE)) {
u32 hits = afl->n_fuzz[q->n_fuzz_entry];
if (likely(hits)) { weight /= (log10(hits) + 1); }
}
if (likely(afl->schedule < RARE)) {
double t = q->exec_us / avg_exec_us;
if (likely(t < 0.1)) {
// nothing
} else if (likely(t <= 0.25))
weight *= 0.9;
else if (likely(t <= 0.5)) {
// nothing
} else if (likely(t < 1.0))
weight *= 1.15;
else if (unlikely(t > 2.5 && t < 5.0))
weight *= 1.1;
// else nothing
}
double l = q->len / avg_len;
if (likely(l < 0.1))
weight *= 0.75;
else if (likely(l < 0.25))
weight *= 1.1;
else if (unlikely(l >= 10))
weight *= 1.1;
double bms = q->bitmap_size / avg_bitmap_size;
if (likely(bms < 0.5))
weight *= (1.0 + ((bms - 0.5) / 2));
else if (unlikely(bms > 1.33))
weight *= 1.1;
if (unlikely(!q->was_fuzzed)) { weight *= 2.5; }
if (unlikely(q->fs_redundant)) { weight *= 0.75; }
}
q->weight = weight;
q->perf_score = calculate_score(afl, q);
sum += q->weight;
@ -596,6 +621,8 @@ void add_to_queue(afl_state_t *afl, u8 *fname, u32 len, u8 passed_det) {
q->trace_mini = NULL;
q->testcase_buf = NULL;
q->mother = afl->queue_cur;
q->weight = 1.0;
q->perf_score = 100;
#ifdef INTROSPECTION
q->bitsmap_size = afl->bitsmap_size;
@ -1201,9 +1228,11 @@ inline void queue_testcase_retake(afl_state_t *afl, struct queue_entry *q,
u32 len = q->len;
if (len != old_len) {
// only realloc if necessary or useful
// (a custom trim can make the testcase larger)
if (unlikely(len > old_len || len < old_len + 1024)) {
afl->q_testcase_cache_size = afl->q_testcase_cache_size + len - old_len;
afl->q_testcase_cache_size += len - old_len;
q->testcase_buf = (u8 *)realloc(q->testcase_buf, len);
if (unlikely(!q->testcase_buf)) {
@ -1232,41 +1261,48 @@ inline void queue_testcase_retake_mem(afl_state_t *afl, struct queue_entry *q,
if (likely(q->testcase_buf)) {
u32 is_same = in == q->testcase_buf;
if (likely(in != q->testcase_buf)) {
if (likely(len != old_len)) {
// only realloc if we save memory
if (unlikely(len < old_len + 1024)) {
u8 *ptr = (u8 *)realloc(q->testcase_buf, len);
u8 *ptr = (u8 *)realloc(q->testcase_buf, len);
if (likely(ptr)) {
if (likely(ptr)) {
q->testcase_buf = ptr;
afl->q_testcase_cache_size = afl->q_testcase_cache_size + len - old_len;
q->testcase_buf = ptr;
afl->q_testcase_cache_size += len - old_len;
}
}
}
memcpy(q->testcase_buf, in, len);
if (unlikely(!is_same)) { memcpy(q->testcase_buf, in, len); }
}
}
}
/* Returns the testcase buf from the file behind this queue entry.
Increases the refcount. */
Increases the refcount. */
inline u8 *queue_testcase_get(afl_state_t *afl, struct queue_entry *q) {
u32 len = q->len;
if (likely(q->testcase_buf)) { return q->testcase_buf; }
/* first handle if no testcase cache is configured */
u32 len = q->len;
double weight = q->weight;
if (unlikely(!afl->q_testcase_max_cache_size)) {
// first handle if no testcase cache is configured, or if the
// weighting of the testcase is below average.
if (unlikely(weight < 1.0 || !afl->q_testcase_max_cache_size)) {
u8 *buf;
if (unlikely(q == afl->queue_cur)) {
if (likely(q == afl->queue_cur)) {
buf = (u8 *)afl_realloc((void **)&afl->testcase_buf, len);
@ -1292,118 +1328,113 @@ inline u8 *queue_testcase_get(afl_state_t *afl, struct queue_entry *q) {
}
/* now handle the testcase cache */
/* now handle the testcase cache and we know it is an interesting one */
if (unlikely(!q->testcase_buf)) {
/* Buf not cached, let's load it */
u32 tid = afl->q_testcase_max_cache_count;
static u32 do_once = 0; // because even threaded we would want this. WIP
/* Buf not cached, let's load it */
u32 tid = afl->q_testcase_max_cache_count;
static u32 do_once = 0; // because even threaded we would want this. WIP
while (unlikely(
(afl->q_testcase_cache_size + len >= afl->q_testcase_max_cache_size &&
afl->q_testcase_cache_count > 1) ||
afl->q_testcase_cache_count >= afl->q_testcase_max_cache_entries - 1)) {
while (unlikely(
(afl->q_testcase_cache_size + len >= afl->q_testcase_max_cache_size &&
afl->q_testcase_cache_count > 1) ||
afl->q_testcase_cache_count >= afl->q_testcase_max_cache_entries - 1)) {
/* We want a max number of entries to the cache that we learn.
Very simple: once the cache is filled by size - that is the max. */
/* We want a max number of entries to the cache that we learn.
Very simple: once the cache is filled by size - that is the max. */
if (unlikely(
afl->q_testcase_cache_size + len >=
afl->q_testcase_max_cache_size &&
(afl->q_testcase_cache_count < afl->q_testcase_max_cache_entries &&
afl->q_testcase_max_cache_count <
afl->q_testcase_max_cache_entries) &&
!do_once)) {
if (unlikely(afl->q_testcase_cache_size + len >=
afl->q_testcase_max_cache_size &&
(afl->q_testcase_cache_count <
afl->q_testcase_max_cache_entries &&
afl->q_testcase_max_cache_count <
afl->q_testcase_max_cache_entries) &&
!do_once)) {
if (afl->q_testcase_max_cache_count > afl->q_testcase_cache_count) {
if (afl->q_testcase_max_cache_count > afl->q_testcase_cache_count) {
afl->q_testcase_max_cache_entries = afl->q_testcase_max_cache_count + 1;
afl->q_testcase_max_cache_entries =
afl->q_testcase_max_cache_count + 1;
} else {
} else {
afl->q_testcase_max_cache_entries = afl->q_testcase_cache_count + 1;
}
do_once = 1;
// release unneeded memory
afl->q_testcase_cache = (struct queue_entry **)ck_realloc(
afl->q_testcase_cache,
(afl->q_testcase_max_cache_entries + 1) * sizeof(size_t));
afl->q_testcase_max_cache_entries = afl->q_testcase_cache_count + 1;
}
/* Cache full. We neet to evict one or more to map one.
Get a random one which is not in use */
do {
// if the cache (MB) is not enough for the queue then this gets
// undesirable because q_testcase_max_cache_count grows sometimes
// although the number of items in the cache will not change hence
// more and more loops
tid = rand_below(afl, afl->q_testcase_max_cache_count);
} while (afl->q_testcase_cache[tid] == NULL ||
afl->q_testcase_cache[tid] == afl->queue_cur);
struct queue_entry *old_cached = afl->q_testcase_cache[tid];
free(old_cached->testcase_buf);
old_cached->testcase_buf = NULL;
afl->q_testcase_cache_size -= old_cached->len;
afl->q_testcase_cache[tid] = NULL;
--afl->q_testcase_cache_count;
++afl->q_testcase_evictions;
if (tid < afl->q_testcase_smallest_free)
afl->q_testcase_smallest_free = tid;
do_once = 1;
// release unneeded memory
afl->q_testcase_cache = (struct queue_entry **)ck_realloc(
afl->q_testcase_cache,
(afl->q_testcase_max_cache_entries + 1) * sizeof(size_t));
}
if (unlikely(tid >= afl->q_testcase_max_cache_entries)) {
/* Cache full. We neet to evict one or more to map one.
Get a random one which is not in use */
// uh we were full, so now we have to search from start
tid = afl->q_testcase_smallest_free;
do {
}
// if the cache (MB) is not enough for the queue then this gets
// undesirable because q_testcase_max_cache_count grows sometimes
// although the number of items in the cache will not change hence
// more and more loops
tid = rand_below(afl, afl->q_testcase_max_cache_count);
// we need this while loop in case there were ever previous evictions but
// not in this call.
while (unlikely(afl->q_testcase_cache[tid] != NULL))
++tid;
} while (afl->q_testcase_cache[tid] == NULL ||
/* Map the test case into memory. */
afl->q_testcase_cache[tid] == afl->queue_cur);
int fd = open((char *)q->fname, O_RDONLY);
struct queue_entry *old_cached = afl->q_testcase_cache[tid];
free(old_cached->testcase_buf);
old_cached->testcase_buf = NULL;
afl->q_testcase_cache_size -= old_cached->len;
afl->q_testcase_cache[tid] = NULL;
--afl->q_testcase_cache_count;
++afl->q_testcase_evictions;
if (tid < afl->q_testcase_smallest_free)
afl->q_testcase_smallest_free = tid;
if (unlikely(fd < 0)) { PFATAL("Unable to open '%s'", (char *)q->fname); }
}
q->testcase_buf = (u8 *)malloc(len);
if (unlikely(tid >= afl->q_testcase_max_cache_entries)) {
if (unlikely(!q->testcase_buf)) {
// uh we were full, so now we have to search from start
tid = afl->q_testcase_smallest_free;
PFATAL("Unable to malloc '%s' with len %u", (char *)q->fname, len);
}
}
// we need this while loop in case there were ever previous evictions but
// not in this call.
while (unlikely(afl->q_testcase_cache[tid] != NULL))
++tid;
ck_read(fd, q->testcase_buf, len, q->fname);
close(fd);
/* Map the test case into memory. */
/* Register testcase as cached */
afl->q_testcase_cache[tid] = q;
afl->q_testcase_cache_size += len;
++afl->q_testcase_cache_count;
if (likely(tid >= afl->q_testcase_max_cache_count)) {
int fd = open((char *)q->fname, O_RDONLY);
afl->q_testcase_max_cache_count = tid + 1;
if (unlikely(fd < 0)) { PFATAL("Unable to open '%s'", (char *)q->fname); }
} else if (unlikely(tid == afl->q_testcase_smallest_free)) {
q->testcase_buf = (u8 *)malloc(len);
afl->q_testcase_smallest_free = tid + 1;
if (unlikely(!q->testcase_buf)) {
}
PFATAL("Unable to malloc '%s' with len %u", (char *)q->fname, len);
}
ck_read(fd, q->testcase_buf, len, q->fname);
close(fd);
/* Register testcase as cached */
afl->q_testcase_cache[tid] = q;
afl->q_testcase_cache_size += len;
++afl->q_testcase_cache_count;
if (likely(tid >= afl->q_testcase_max_cache_count)) {
afl->q_testcase_max_cache_count = tid + 1;
} else if (unlikely(tid == afl->q_testcase_smallest_free)) {
afl->q_testcase_smallest_free = tid + 1;
}
@ -1418,12 +1449,13 @@ inline void queue_testcase_store_mem(afl_state_t *afl, struct queue_entry *q,
u32 len = q->len;
if (unlikely(afl->q_testcase_cache_size + len >=
if (unlikely(q->weight < 1.0 ||
afl->q_testcase_cache_size + len >=
afl->q_testcase_max_cache_size ||
afl->q_testcase_cache_count >=
afl->q_testcase_max_cache_entries - 1)) {
// no space? will be loaded regularly later.
// no space or uninteresting? will be loaded regularly later.
return;
}

53
src/afl-fuzz-redqueen.c
View File

@ -322,7 +322,7 @@ static u8 colorization(afl_state_t *afl, u8 *buf, u32 len,
memcpy(backup, buf, len);
memcpy(changed, buf, len);
if (afl->cmplog_random_colorization) {
if (likely(afl->cmplog_random_colorization)) {
random_replace(afl, changed, len);
@ -402,6 +402,7 @@ static u8 colorization(afl_state_t *afl, u8 *buf, u32 len,
u32 i = 1;
u32 positions = 0;
while (i) {
restart:
@ -2937,7 +2938,8 @@ static u8 rtn_fuzz(afl_state_t *afl, u32 key, u8 *orig_buf, u8 *buf, u8 *cbuf,
// afl->queue_cur->exec_cksum
u8 input_to_state_stage(afl_state_t *afl, u8 *orig_buf, u8 *buf, u32 len) {
u8 r = 1;
u64 cmplog_start_us = get_cur_time_us();
u8 r = 1;
if (unlikely(!afl->pass_stats)) {
afl->pass_stats = ck_alloc(sizeof(struct afl_pass_stat) * CMP_MAP_W);
@ -2965,7 +2967,12 @@ u8 input_to_state_stage(afl_state_t *afl, u8 *orig_buf, u8 *buf, u32 len) {
if (!afl->queue_cur->taint || !afl->queue_cur->cmplog_colorinput) {
if (unlikely(colorization(afl, buf, len, &taint))) { return 1; }
if (unlikely(colorization(afl, buf, len, &taint))) {
update_cmplog_time(afl, &cmplog_start_us);
return 1;
}
// no taint? still try, create a dummy to prevent again colorization
if (!taint) {
@ -2974,6 +2981,7 @@ u8 input_to_state_stage(afl_state_t *afl, u8 *orig_buf, u8 *buf, u32 len) {
fprintf(stderr, "TAINT FAILED\n");
#endif
afl->queue_cur->colorized = CMPLOG_LVL_MAX;
update_cmplog_time(afl, &cmplog_start_us);
return 0;
}
@ -2994,17 +3002,20 @@ u8 input_to_state_stage(afl_state_t *afl, u8 *orig_buf, u8 *buf, u32 len) {
}
update_cmplog_time(afl, &cmplog_start_us);
struct tainted *t = taint;
#ifdef _DEBUG
while (t) {
#ifdef _DEBUG
fprintf(stderr, "T: idx=%u len=%u\n", t->pos, t->len);
#endif
t = t->next;
}
#endif
#if defined(_DEBUG) || defined(CMPLOG_INTROSPECTION)
u64 start_time = get_cur_time();
u32 cmp_locations = 0;
@ -3025,6 +3036,7 @@ u8 input_to_state_stage(afl_state_t *afl, u8 *orig_buf, u8 *buf, u32 len) {
}
update_cmplog_time(afl, &cmplog_start_us);
return 1;
}
@ -3048,6 +3060,7 @@ u8 input_to_state_stage(afl_state_t *afl, u8 *orig_buf, u8 *buf, u32 len) {
}
update_cmplog_time(afl, &cmplog_start_us);
return 1;
}
@ -3066,6 +3079,7 @@ u8 input_to_state_stage(afl_state_t *afl, u8 *orig_buf, u8 *buf, u32 len) {
u64 orig_hit_cnt, new_hit_cnt;
u64 orig_execs = afl->fsrv.total_execs;
orig_hit_cnt = afl->queued_items + afl->saved_crashes;
update_cmplog_time(afl, &cmplog_start_us);
afl->stage_name = "input-to-state";
afl->stage_short = "its";
@ -3142,33 +3156,35 @@ u8 input_to_state_stage(afl_state_t *afl, u8 *orig_buf, u8 *buf, u32 len) {
}
update_cmplog_time(afl, &cmplog_start_us);
}
r = 0;
exit_its:
if (afl->cmplog_lvl == CMPLOG_LVL_MAX) {
// if (afl->cmplog_lvl == CMPLOG_LVL_MAX) {
afl->queue_cur->colorized = CMPLOG_LVL_MAX;
afl->queue_cur->colorized = CMPLOG_LVL_MAX;
if (afl->queue_cur->cmplog_colorinput) {
if (afl->queue_cur->cmplog_colorinput) {
ck_free(afl->queue_cur->cmplog_colorinput);
ck_free(afl->queue_cur->cmplog_colorinput);
}
}
while (taint) {
while (taint) {
t = taint->next;
ck_free(taint);
taint = t;
t = taint->next;
ck_free(taint);
taint = t;
}
}
afl->queue_cur->taint = NULL;
afl->queue_cur->taint = NULL;
} else {
/*} else {
afl->queue_cur->colorized = LVL2;
@ -3182,7 +3198,7 @@ exit_its:
}
}
}*/
#ifdef CMPLOG_COMBINE
if (afl->queued_items + afl->saved_crashes > orig_hit_cnt + 1) {
@ -3270,6 +3286,7 @@ exit_its:
#endif
update_cmplog_time(afl, &cmplog_start_us);
return r;
}

2
src/afl-fuzz-run.c
View File

@ -666,6 +666,8 @@ abort_calibration:
void sync_fuzzers(afl_state_t *afl) {
if (unlikely(afl->afl_env.afl_no_sync)) { return; }
DIR *sd;
struct dirent *sd_ent;
u32 sync_cnt = 0, synced = 0, entries = 0;

9
src/afl-fuzz-skipdet.c
View File

@ -33,15 +33,15 @@ u8 is_det_timeout(u64 cur_ms, u8 is_flip) {
u8 should_det_fuzz(afl_state_t *afl, struct queue_entry *q) {
if (!afl->skipdet_g->virgin_det_bits) {
if (unlikely(!afl->skipdet_g->virgin_det_bits)) {
afl->skipdet_g->virgin_det_bits =
(u8 *)ck_alloc(sizeof(u8) * afl->fsrv.map_size);
}
if (!q->favored || q->passed_det) return 0;
if (!q->trace_mini) return 0;
if (likely(!q->favored || q->passed_det)) return 0;
if (unlikely(!q->trace_mini)) return 0;
if (!afl->skipdet_g->last_cov_undet)
afl->skipdet_g->last_cov_undet = get_cur_time();
@ -122,7 +122,8 @@ u8 skip_deterministic_stage(afl_state_t *afl, u8 *orig_buf, u8 *out_buf,
afl->stage_cur = 0;
orig_hit_cnt = afl->queued_items + afl->saved_crashes;
u8 *inf_eff_map = (u8 *)ck_alloc(sizeof(u8) * len);
static u8 *inf_eff_map;
inf_eff_map = (u8 *)ck_realloc(inf_eff_map, sizeof(u8) * len);
memset(inf_eff_map, 1, sizeof(u8) * len);
if (common_fuzz_stuff(afl, orig_buf, len)) { return 0; }

10
src/afl-fuzz-state.c
View File

@ -279,6 +279,13 @@ void read_afl_environment(afl_state_t *afl, char **envp) {
afl->afl_env.afl_final_sync =
get_afl_env(afl_environment_variables[i]) ? 1 : 0;
} else if (!strncmp(env, "AFL_NO_SYNC",
afl_environment_variable_len)) {
afl->afl_env.afl_no_sync =
get_afl_env(afl_environment_variables[i]) ? 1 : 0;
} else if (!strncmp(env, "AFL_CUSTOM_MUTATOR_ONLY",
afl_environment_variable_len)) {
@ -762,8 +769,9 @@ void afl_states_stop(void) {
if (el->fsrv.fsrv_pid > 0) {
kill(el->fsrv.fsrv_pid, el->fsrv.fsrv_kill_signal);
usleep(100);
/* Make sure the forkserver does not end up as zombie. */
waitpid(el->fsrv.fsrv_pid, NULL, 0);
waitpid(el->fsrv.fsrv_pid, NULL, WNOHANG);
}

31
src/afl-fuzz-stats.c
View File

@ -207,6 +207,12 @@ void load_stats_file(afl_state_t *afl) {
}
if (starts_with("cmplog_time", keystring)) {
afl->cmplog_time_us = strtoull(lptr, &nptr, 10) * 1000000;
}
if (starts_with("trim_time", keystring)) {
afl->trim_time_us = strtoull(lptr, &nptr, 10) * 1000000;
@ -322,8 +328,9 @@ void write_stats_file(afl_state_t *afl, u32 t_bytes, double bitmap_cvg,
if (getrusage(RUSAGE_CHILDREN, &rus)) { rus.ru_maxrss = 0; }
#endif
u64 runtime_ms = afl->prev_run_time + cur_time - afl->start_time;
u64 overhead_ms =
(afl->calibration_time_us + afl->sync_time_us + afl->trim_time_us) / 1000;
u64 overhead_ms = (afl->calibration_time_us + afl->sync_time_us +
afl->trim_time_us + afl->cmplog_time_us) /
1000;
if (!runtime_ms) { runtime_ms = 1; }
fprintf(
@ -337,6 +344,7 @@ void write_stats_file(afl_state_t *afl, u32 t_bytes, double bitmap_cvg,
"time_wo_finds : %llu\n"
"fuzz_time : %llu\n"
"calibration_time : %llu\n"
"cmplog_time : %llu\n"
"sync_time : %llu\n"
"trim_time : %llu\n"
"execs_done : %llu\n"
@ -385,8 +393,9 @@ void write_stats_file(afl_state_t *afl, u32 t_bytes, double bitmap_cvg,
? 0
: (cur_time - afl->last_find_time) / 1000),
(runtime_ms - MIN(runtime_ms, overhead_ms)) / 1000,
afl->calibration_time_us / 1000000, afl->sync_time_us / 1000000,
afl->trim_time_us / 1000000, afl->fsrv.total_execs,
afl->calibration_time_us / 1000000, afl->cmplog_time_us / 1000000,
afl->sync_time_us / 1000000, afl->trim_time_us / 1000000,
afl->fsrv.total_execs,
afl->fsrv.total_execs / ((double)(runtime_ms) / 1000),
afl->last_avg_execs_saved, afl->queued_items, afl->queued_favored,
afl->queued_discovered, afl->queued_imported, afl->queued_variable,
@ -2487,7 +2496,7 @@ void show_init_stats(afl_state_t *afl) {
}
void update_calibration_time(afl_state_t *afl, u64 *time) {
inline void update_calibration_time(afl_state_t *afl, u64 *time) {
u64 cur = get_cur_time_us();
afl->calibration_time_us += cur - *time;
@ -2495,7 +2504,7 @@ void update_calibration_time(afl_state_t *afl, u64 *time) {
}
void update_trim_time(afl_state_t *afl, u64 *time) {
inline void update_trim_time(afl_state_t *afl, u64 *time) {
u64 cur = get_cur_time_us();
afl->trim_time_us += cur - *time;
@ -2503,7 +2512,7 @@ void update_trim_time(afl_state_t *afl, u64 *time) {
}
void update_sync_time(afl_state_t *afl, u64 *time) {
inline void update_sync_time(afl_state_t *afl, u64 *time) {
u64 cur = get_cur_time_us();
afl->sync_time_us += cur - *time;
@ -2511,3 +2520,11 @@ void update_sync_time(afl_state_t *afl, u64 *time) {
}
inline void update_cmplog_time(afl_state_t *afl, u64 *time) {
u64 cur = get_cur_time_us();
afl->cmplog_time_us += cur - *time;
*time = cur;
}

71
src/afl-fuzz.c
View File

@ -335,6 +335,7 @@ static void usage(u8 *argv0, int more_help) {
"AFL_STATSD_PORT: change default statsd port (default: 8125)\n"
"AFL_STATSD_TAGS_FLAVOR: set statsd tags format (default: disable tags)\n"
" suported formats: dogstatsd, librato, signalfx, influxdb\n"
"AFL_NO_SYNC: disables all syncing\n"
"AFL_SYNC_TIME: sync time between fuzzing instances (in minutes)\n"
"AFL_FINAL_SYNC: sync a final time when exiting (will delay the exit!)\n"
"AFL_NO_CRASH_README: do not create a README in the crashes directory\n"
@ -914,8 +915,15 @@ int main(int argc, char **argv_orig, char **envp) {
u8 suffix = 'M';
if (mem_limit_given) { FATAL("Multiple -m options not supported"); }
mem_limit_given = 1;
if (mem_limit_given) {
WARNF("Overriding previous -m option.");
} else {
mem_limit_given = 1;
}
if (!optarg) { FATAL("Wrong usage of -m"); }
@ -1461,15 +1469,16 @@ int main(int argc, char **argv_orig, char **envp) {
#endif
configure_afl_kill_signals(&afl->fsrv, afl->afl_env.afl_child_kill_signal,
afl->afl_env.afl_fsrv_kill_signal,
(afl->fsrv.qemu_mode || afl->unicorn_mode
configure_afl_kill_signals(
&afl->fsrv, afl->afl_env.afl_child_kill_signal,
afl->afl_env.afl_fsrv_kill_signal,
(afl->fsrv.qemu_mode || afl->unicorn_mode || afl->fsrv.use_fauxsrv
#ifdef __linux__
|| afl->fsrv.nyx_mode
|| afl->fsrv.nyx_mode
#endif
)
? SIGKILL
: SIGTERM);
)
? SIGKILL
: SIGTERM);
setup_signal_handlers();
check_asan_opts(afl);
@ -2586,7 +2595,7 @@ int main(int argc, char **argv_orig, char **envp) {
(!afl->queue_cycle && afl->afl_env.afl_import_first)) &&
afl->sync_id)) {
if (!afl->queue_cycle && afl->afl_env.afl_import_first) {
if (unlikely(!afl->queue_cycle && afl->afl_env.afl_import_first)) {
OKF("Syncing queues from other fuzzer instances first ...");
@ -2597,6 +2606,12 @@ int main(int argc, char **argv_orig, char **envp) {
}
++afl->queue_cycle;
if (afl->afl_env.afl_no_ui) {
ACTF("Entering queue cycle %llu\n", afl->queue_cycle);
}
runs_in_current_cycle = (u32)-1;
afl->cur_skipped_items = 0;
@ -2605,7 +2620,7 @@ int main(int argc, char **argv_orig, char **envp) {
// queue is fully cycled.
time_t cursec = time(NULL);
struct tm *curdate = localtime(&cursec);
if (likely(!afl->afl_env.afl_pizza_mode)) {
if (unlikely(!afl->afl_env.afl_pizza_mode)) {
if (unlikely(curdate->tm_mon == 3 && curdate->tm_mday == 1)) {
@ -2650,13 +2665,6 @@ int main(int argc, char **argv_orig, char **envp) {
}
if (unlikely(afl->not_on_tty)) {
ACTF("Entering queue cycle %llu.", afl->queue_cycle);
fflush(stdout);
}
/* If we had a full queue cycle with no new finds, try
recombination strategies next. */
@ -2942,26 +2950,13 @@ int main(int argc, char **argv_orig, char **envp) {
if (likely(!afl->stop_soon && afl->sync_id)) {
if (likely(afl->skip_deterministic)) {
if (unlikely(afl->is_main_node)) {
if (unlikely(afl->is_main_node)) {
if (unlikely(cur_time > (afl->sync_time >> 1) + afl->last_sync_time)) {
if (unlikely(cur_time >
(afl->sync_time >> 1) + afl->last_sync_time)) {
if (!(sync_interval_cnt++ % (SYNC_INTERVAL / 3))) {
if (!(sync_interval_cnt++ % (SYNC_INTERVAL / 3))) {
sync_fuzzers(afl);
}
}
} else {
if (unlikely(cur_time > afl->sync_time + afl->last_sync_time)) {
if (!(sync_interval_cnt++ % SYNC_INTERVAL)) { sync_fuzzers(afl); }
sync_fuzzers(afl);
}
@ -2969,7 +2964,11 @@ int main(int argc, char **argv_orig, char **envp) {
} else {
sync_fuzzers(afl);
if (unlikely(cur_time > afl->sync_time + afl->last_sync_time)) {
if (!(sync_interval_cnt++ % SYNC_INTERVAL)) { sync_fuzzers(afl); }
}
}

18
src/afl-performance.c
View File

@ -95,6 +95,24 @@ inline u64 hash64(u8 *key, u32 len, u64 seed) {
}
/* Hash a file */
u64 get_binary_hash(u8 *fn) {
int fd = open(fn, O_RDONLY);
if (fd < 0) { PFATAL("Unable to open '%s'", fn); }
struct stat st;
if (fstat(fd, &st) < 0) { PFATAL("Unable to fstat '%s'", fn); }
u32 f_len = st.st_size;
u8 *f_data = mmap(0, f_len, PROT_READ, MAP_PRIVATE, fd, 0);
if (f_data == MAP_FAILED) { PFATAL("Unable to mmap file '%s'", fn); }
close(fd);
u64 hash = hash64(f_data, f_len, 0);
if (munmap(f_data, f_len)) { PFATAL("unmap() failed"); }
return hash;
}
// Public domain SHA1 implementation copied from:
// https://github.com/x42/liboauth/blob/7001b8256cd654952ec2515b055d2c5b243be600/src/sha1.c

6
src/afl-sharedmem.c
View File

@ -239,15 +239,15 @@ u8 *afl_shm_init(sharedmem_t *shm, size_t map_size,
if (shm->cmplog_g_shm_fd == -1) { PFATAL("shm_open() failed"); }
/* configure the size of the shared memory segment */
if (ftruncate(shm->cmplog_g_shm_fd, map_size)) {
if (ftruncate(shm->cmplog_g_shm_fd, sizeof(struct cmp_map))) {
PFATAL("setup_shm(): cmplog ftruncate() failed");
}
/* map the shared memory segment to the address space of the process */
shm->cmp_map = mmap(0, map_size, PROT_READ | PROT_WRITE, MAP_SHARED,
shm->cmplog_g_shm_fd, 0);
shm->cmp_map = mmap(0, sizeof(struct cmp_map), PROT_READ | PROT_WRITE,
MAP_SHARED, shm->cmplog_g_shm_fd, 0);
if (shm->cmp_map == MAP_FAILED) {
close(shm->cmplog_g_shm_fd);

2
unicorn_mode/unicornafl

@ -1 +1 @@
Subproject commit 764b66b21cd4a8124a5b6c9cc98d1214b2037196
Subproject commit 4b4fdab161c15529affcc1e785d779e318b882ab

18
utils/libtokencap/README.md
View File

@ -69,3 +69,21 @@ need to be changed for other OSes.
Current supported OSes are: Linux, Darwin, FreeBSD (thanks to @devnexen)
Also, the following example (generate_libtoken_dict.sh) shows how to use a script to capture tokens from the
files in the target output directory,
and then generate a dictionary file from those tokens.
#### usage:
```bash
./generate_libtoken_dict.sh -p /path/to/libtokencap.so -b /path/to/target/program -o /path/to/target/output -t 5 -- [-program_args]
```
#### description opts:
- ```-o``` : Path to target output directory ;
- ```-b``` : Path to target program binary ;
- ```-p``` : Path to LD_PRELOAD library ;
- ```-t``` : Timeout in seconds ;
- ```-- [-program_args]```: Any additional arguments required by the target binary can be specified after ```--```.
#### output:
A sorted and unique token dictionary file with the extension ``*.dict``
is created in the same directory as the target output containing tokens captured during the execution of the target binary.

55
utils/libtokencap/generate_libtoken_dict.sh Normal file
View File

@ -0,0 +1,55 @@
#help
usage() {
echo "Usage: $0 -o <target_output> -b <target_bin> -p <LD_PRELOAD_PATH> [-t <timeout_sec>] -- [target_args]"
echo "Options:"
echo " -o Path to target output directory"
echo " -b Path to target program binary"
echo " -p Path to LD_PRELOAD library"
echo " -t Timeout in seconds"
exit 1
}
#parse cli options
while getopts ":o:b:p:t:" opt; do
case $opt in
o) target_output="$OPTARG" ;;
b) target_bin="$OPTARG" ;;
p) LD_PRELOAD_PATH="$OPTARG" ;;
t) timeout_sec="$OPTARG" ;;
\?) echo "Invalid option: -$OPTARG" >&2; usage ;;
:) echo "Option -$OPTARG requires an argument." >&2; usage ;;
esac
done
#shift away the parsed opts
shift $((OPTIND - 1))
#check options
if [ -z "$target_output" ] || [ -z "$target_bin" ] || [ -z "$LD_PRELOAD_PATH" ]; then
echo "Error: Missing mandatory opts" >&2
usage
fi
# initialize vars
AFL_TOKEN_FILE="${PWD}/temp_output.txt"
AFL_DICT_FILE="${PWD}/$(basename "$target_bin")_tokens.dict"
#generate token-file
{
touch "$AFL_TOKEN_FILE"
for i in $(find "$target_output" -type f -name "id*"); do
LD_PRELOAD="$LD_PRELOAD_PATH" \
timeout -s SIGKILL "$timeout_sec" \
"$target_bin" "$@" "$i"
done
} >"$AFL_TOKEN_FILE"
# sort & remove duplicates
sort -u "$AFL_TOKEN_FILE" >"$AFL_DICT_FILE"
# delete temp-file
rm "$AFL_TOKEN_FILE"
# print done-message
echo "Token dictionary created: $AFL_DICT_FILE"
echo "Script completed successfully"