Files
AFLplusplus/afl-whatsup
van Hauser 602eceed8b push to stable (#1983)
* Output afl-clang-fast stuffs only if necessary (#1912)

* afl-cc header

* afl-cc common declarations

 - Add afl-cc-state.c
 - Strip includes, find_object, debug/be_quiet/have_*/callname setting from afl-cc.c
 - Use debugf_args in main
 - Modify execvp stuffs to fit new aflcc struct

* afl-cc show usage

* afl-cc mode selecting

1. compiler_mode by callname in argv[0]
2. compiler_mode by env "AFL_CC_COMPILER"
3. compiler_mode/instrument_mode by command line options "--afl-..."
4. instrument_mode/compiler_mode by various env vars including "AFL_LLVM_INSTRUMENT"
5. final checking steps
6. print "... - mode: %s-%s\n"
7. determine real argv[0] according to compiler_mode

* afl-cc macro defs

* afl-cc linking behaviors

* afl-cc fsanitize behaviors

* afl-cc misc

* afl-cc body update

* afl-cc all-in-one

formated with custom-format.py

* nits

---------

Co-authored-by: vanhauser-thc <vh@thc.org>

* changelog

* update grammar mutator

* lto llvm 12+

* docs(custom_mutators): fix missing ':' (#1953)

* Fix broken LTO mode and response file support (#1948)

* Strip `-Wl,-no-undefined` during compilation (#1952)

Make the compiler wrapper stripping `-Wl,-no-undefined` in addition to `-Wl,--no-undefined`.
Both versions of the flag are accepted by clang and, therefore, used by building systems in the wild (e.g., samba will not build without this fix).

* Remove dead code in write_to_testcase (#1955)

The custom_mutators_count check in if case is duplicate with if condition.
The else case is custom_mutators_count == 0, neither custom_mutator_list iteration nor sent check needed.

Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>

* update qemuafl

* WIP: Add ability to generate drcov trace using QEMU backend (#1956)

* Document new drcov QEMU plugin

* Add link to lightkeeper for QEMU drcov file loading

---------

Co-authored-by: Jean-Romain Garnier <jean-romain.garnier@airbus.com>

* code format

* changelog

* sleep on uid != 0 afl-system-config

* fix segv about skip_next, warn on unsupported cases of linking options (#1958)

* todos

* ensure afl-cc only allows available compiler modes

* update grammar mutator

* disable aslr on apple

* fix for arm64

* help selective instrumentation

* typos

* macos

* add compiler test script

* apple fixes

* bump nyx submodules (#1963)

* fix docs

* update changelog

* update grammar mutator

* improve compiler test script

* gcc asan workaround (#1966)

* fix github merge fuckup

* fix

* Fix afl-cc (#1968)

- Check if too many cmdline params here, each time before insert a new param.
 - Check if it is "-fsanitize=..." before we do sth.
 - Remove improper param_st transfer.

* Avoid adding llvmnative instrumentation when linking rust sanitizer runtime (#1969)

* Dynamic instrumentation filtering for LLVM native (#1971)

* Add two dynamic instrumentation filter methods to runtime

* Always use pc-table with native pcguard

* Add make_symbol_list.py and README

* changelog

* todos

* new forkserver check

* fix

* nyx test for CI

* improve nyx docs

* Fixes to afl-cc and documentation (#1974)

* Always compile with -ldl when building for CODE_COVERAGE

When building with CODE_COVERAGE, the afl runtime contains code that
calls `dladdr` which requires -ldl. Under most circumstances, clang
already adds this (e.g. when building with pc-table), but there are some
circumstances where it isn't added automatically.

* Add visibility declaration to __afl_connected

When building with hidden visibility, the use of __AFL_LOOP inside such
code can cause linker errors due to __afl_connected being declared
"hidden".

* Update docs to clarify that CODE_COVERAGE=1 is required for dynamic_covfilter

* nits

* nyx build script updates

* test error output

* debug ci

* debug ci

* Improve afl-cc (#1975)

* update response file support

 - full support of rsp file
 - fix some segv issues

* Improve afl-cc

 - remove dead code about allow/denylist options of sancov
 - missing `if (!aflcc->have_msan)`
 - add docs for each function
 - typo

* enable nyx

* debug ci

* debug ci

* debug ci

* debug ci

* debug ci

* debug ci

* debug ci

* debug ci

* fix ci

* clean test script

* NO_NYX

* NO_NYX

* fix ci

* debug ci

* fix ci

* finalize ci fix

* Enhancement on Deterministic stage (#1972)

* fuzzer: init commit based on aflpp 60dc37a8cf

* fuzzers: adding the skip variables and initialize

* log: profile the det/havoc finding

* log: add profile log output

* fuzzers: sperate log/skipdet module

* fuzzers: add quick eff_map calc

* fuzzers: add skip_eff_map in fuzz_one

* fuzzers: mark whole input space in eff_map

* fuzzers: add undet bit threshold to skip some seeds

* fuzzers: fix one byte overflow

* fuzzers: fix overflow

* fix code format

* add havoc only again

* code format

* remove log to INTROSPECTION, rename skipdet module

* rename skipdet module

* remove log to stats

* clean redundant code

* code format

* remove redundant code format check

* remove redundant doc

* remove redundant objects

* clean files

* change -d to default skipdet

* disable deterministic when using CUSTOM_MUTATOR

* revert fix

* final touches for skipdet

* remove unused var

* remove redundant eff struct (#1977)

* update QEMU-Nyx submodule (#1978)

* update QEMU-Nyx submodule (#1980)

* Fix type in AFL_NOOPT env variable in afl-cc help message (#1982)

* nits

* 2024 v4.10c release

* fixes

---------

Signed-off-by: Xeonacid <h.dwwwwww@gmail.com>
Co-authored-by: Sonic <50692172+SonicStark@users.noreply.github.com>
Co-authored-by: Xeonacid <h.dwwwwww@gmail.com>
Co-authored-by: Nils Bars <nils.bars@rub.de>
Co-authored-by: Jean-Romain Garnier <7504819+JRomainG@users.noreply.github.com>
Co-authored-by: Jean-Romain Garnier <jean-romain.garnier@airbus.com>
Co-authored-by: Sergej Schumilo <sergej@schumilo.de>
Co-authored-by: Christian Holler (:decoder) <choller@mozilla.com>
Co-authored-by: Han Zheng <35988108+kdsjZh@users.noreply.github.com>
Co-authored-by: Khaled Yakdan <yakdan@code-intelligence.com>
2024-02-03 10:55:51 +00:00

439 lines
11 KiB
Bash
Executable File

#!/bin/sh
#
# american fuzzy lop++ - status check tool
# ----------------------------------------
#
# Originally written by Michal Zalewski
#
# Copyright 2015 Google Inc. All rights reserved.
# Copyright 2019-2024 AFLplusplus Project. All rights reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at:
#
# https://www.apache.org/licenses/LICENSE-2.0
#
# This tool summarizes the status of any locally-running synchronized
# instances of afl-fuzz.
#
test "$1" = "-h" -o "$1" = "-hh" && {
echo "$0 status check tool for afl-fuzz by Michal Zalewski"
echo
echo "Usage: $0 [-s] [-d] afl_output_directory"
echo
echo Options:
echo " -d - include dead fuzzer stats"
echo " -m - just show minimal stats"
echo " -n - no color output"
echo " -s - skip details and output summary results only"
echo
exit 1
}
unset MINIMAL_ONLY
unset NO_COLOR
unset PROCESS_DEAD
unset SUMMARY_ONLY
unset RED
unset GREEN
unset YELLOW
unset BLUE
unset NC
unset RESET
if [ -z "$TERM" ]; then export TERM=vt220; fi
while [ "$1" = "-d" -o "$1" = "-m" -o "$1" = "-n" -o "$1" = "-s" ]; do
if [ "$1" = "-d" ]; then
PROCESS_DEAD=1
fi
if [ "$1" = "-m" ]; then
MINIMAL_ONLY=1
fi
if [ "$1" = "-n" ]; then
NO_COLOR=1
fi
if [ "$1" = "-s" ]; then
SUMMARY_ONLY=1
fi
shift
done
DIR="$1"
if [ "$DIR" = "" -o "$DIR" = "-h" -o "$DIR" = "--help" ]; then
echo "$0 status check tool for afl-fuzz by Michal Zalewski" 1>&2
echo 1>&2
echo "Usage: $0 [-d] [-m] [-n] [-s] afl_output_directory" 1>&2
echo 1>&2
echo Options: 1>&2
echo " -d - include dead fuzzer stats" 1>&2
echo " -m - just show minimal stats" 1>&2
echo " -n - no color output" 1>&2
echo " -s - skip details and output summary results only" 1>&2
echo 1>&2
exit 1
fi
if [ -z "$MINIMAL_ONLY" ]; then
echo "$0 status check tool for afl-fuzz by Michal Zalewski"
echo
fi
cd "$DIR" || exit 1
if [ -d queue ]; then
echo "[-] Error: parameter is an individual output directory, not a sync dir." 1>&2
exit 1
fi
BC=`which bc 2>/dev/null`
FUSER=`which fuser 2>/dev/null`
if [ -z "$NO_COLOR" ]; then
RED=`tput setaf 9 1 1 2>/dev/null`
GREEN=`tput setaf 2 1 1 2>/dev/null`
BLUE=`tput setaf 4 1 1 2>/dev/null`
YELLOW=`tput setaf 11 1 1 2>/dev/null`
NC=`tput sgr0`
RESET="$NC"
fi
CUR_TIME=`date +%s`
TMP=`mktemp -t .afl-whatsup-XXXXXXXX` || TMP=`mktemp -p /data/local/tmp .afl-whatsup-XXXXXXXX` || TMP=`mktemp -p /data/local/tmp .afl-whatsup-XXXXXXXX` || exit 1
ALIVE_CNT=0
DEAD_CNT=0
START_CNT=0
TOTAL_TIME=0
TOTAL_EXECS=0
TOTAL_EPS=0
TOTAL_CRASHES=0
TOTAL_HANGS=0
TOTAL_PFAV=0
TOTAL_PENDING=0
TOTAL_COVERAGE=
# Time since last find / crash / hang, formatted as string
FMT_TIME="0 days 0 hours"
FMT_FIND="${RED}none seen yet${NC}"
FMT_CRASH="none seen yet"
FMT_HANG="none seen yet"
if [ "$SUMMARY_ONLY" = "" ]; then
echo "Individual fuzzers"
echo "=================="
echo
fi
fmt_duration()
{
DUR_STRING=
if [ $1 -le 0 ]; then
return 1
fi
local duration=$((CUR_TIME - $1))
local days=$((duration / 60 / 60 / 24))
local hours=$(((duration / 60 / 60) % 24))
local minutes=$(((duration / 60) % 60))
local seconds=$((duration % 60))
if [ $duration -le 0 ]; then
DUR_STRING="0 seconds"
elif [ $duration -eq 1 ]; then
DUR_STRING="1 second"
elif [ $days -gt 0 ]; then
DUR_STRING="$days days, $hours hours"
elif [ $hours -gt 0 ]; then
DUR_STRING="$hours hours, $minutes minutes"
elif [ $minutes -gt 0 ]; then
DUR_STRING="$minutes minutes, $seconds seconds"
else
DUR_STRING="$seconds seconds"
fi
}
FIRST=true
TOTAL_WCOP=
TOTAL_LAST_FIND=0
for j in `find . -maxdepth 2 -iname fuzzer_setup | sort`; do
DIR=$(dirname "$j")
i=$DIR/fuzzer_stats
if [ -f "$i" ]; then
sed 's/^command_line.*$/_skip:1/;s/[ ]*:[ ]*/="/;s/$/"/' "$i" >"$TMP"
. "$TMP"
DIRECTORY=$DIR
DIR=${DIR##*/}
RUN_UNIX=$run_time
RUN_DAYS=$((RUN_UNIX / 60 / 60 / 24))
RUN_HRS=$(((RUN_UNIX / 60 / 60) % 24))
COVERAGE=$(echo $bitmap_cvg|tr -d %)
if [ -n "$TOTAL_COVERAGE" -a -n "$COVERAGE" -a -n "$BC" ]; then
if [ "$(echo "$TOTAL_COVERAGE < $COVERAGE" | bc)" -eq 1 ]; then
TOTAL_COVERAGE=$COVERAGE
fi
fi
if [ -z "$TOTAL_COVERAGE" ]; then TOTAL_COVERAGE=$COVERAGE ; fi
test -n "$cycles_wo_finds" && {
test -z "$FIRST" && TOTAL_WCOP="${TOTAL_WCOP}/"
TOTAL_WCOP="${TOTAL_WCOP}${cycles_wo_finds}"
FIRST=
}
if [ "$SUMMARY_ONLY" = "" ]; then
echo ">>> $afl_banner instance: $DIR ($RUN_DAYS days, $RUN_HRS hrs) fuzzer PID: $fuzzer_pid <<<"
echo
fi
if ! kill -0 "$fuzzer_pid" 2>/dev/null; then
IS_STARTING=
IS_DEAD=
if [ -e "$i" ] && [ -e "$j" ] && [ -n "$FUSER" ]; then
if [ "$i" -ot "$j" ]; then
# fuzzer_setup is newer than fuzzer_stats, maybe the instance is starting?
TMP_PID=`fuser -v "$DIRECTORY" 2>&1 | grep afl-fuzz`
if [ -n "$TMP_PID" ]; then
if [ "$SUMMARY_ONLY" = "" ]; then
echo " Instance is still starting up, skipping."
echo
fi
START_CNT=$((START_CNT + 1))
last_find=0
IS_STARTING=1
if [ "$PROCESS_DEAD" = "" ]; then
continue
fi
fi
fi
fi
if [ -z "$IS_STARTING" ]; then
if [ "$SUMMARY_ONLY" = "" ]; then
echo " Instance is dead or running remotely, skipping."
echo
fi
DEAD_CNT=$((DEAD_CNT + 1))
IS_DEAD=1
last_find=0
if [ "$PROCESS_DEAD" = "" ]; then
continue
fi
fi
fi
ALIVE_CNT=$((ALIVE_CNT + 1))
EXEC_SEC=0
test -z "$RUN_UNIX" -o "$RUN_UNIX" = 0 || EXEC_SEC=$((execs_done / RUN_UNIX))
PATH_PERC=$((cur_item * 100 / corpus_count))
TOTAL_TIME=$((TOTAL_TIME + RUN_UNIX))
TOTAL_EPS=$((TOTAL_EPS + EXEC_SEC))
TOTAL_EXECS=$((TOTAL_EXECS + execs_done))
TOTAL_CRASHES=$((TOTAL_CRASHES + saved_crashes))
TOTAL_HANGS=$((TOTAL_HANGS + saved_hangs))
TOTAL_PENDING=$((TOTAL_PENDING + pending_total))
TOTAL_PFAV=$((TOTAL_PFAV + pending_favs))
if [ "$last_find" -gt "$TOTAL_LAST_FIND" ]; then
TOTAL_LAST_FIND=$last_find
fi
if [ "$SUMMARY_ONLY" = "" ]; then
# Warnings in red
TIMEOUT_PERC=$((exec_timeout * 100 / execs_done))
if [ $TIMEOUT_PERC -ge 10 ]; then
echo " ${RED}timeout_ratio $TIMEOUT_PERC%${NC}"
fi
if [ $EXEC_SEC -eq 0 ]; then
echo " ${YELLOW}no data yet, 0 execs/sec${NC}"
elif [ $EXEC_SEC -lt 100 ]; then
echo " ${RED}slow execution, $EXEC_SEC execs/sec${NC}"
fi
fmt_duration $last_find && FMT_FIND=$DUR_STRING
fmt_duration $last_crash && FMT_CRASH=$DUR_STRING
fmt_duration $last_hang && FMT_HANG=$DUR_STRING
FMT_CWOP="not available"
test -n "$cycles_wo_finds" && {
test "$cycles_wo_finds" = 0 && FMT_CWOP="$cycles_wo_finds"
test "$cycles_wo_finds" -gt 10 && FMT_CWOP="${YELLOW}$cycles_wo_finds${NC}"
test "$cycles_wo_finds" -gt 50 && FMT_CWOP="${RED}$cycles_wo_finds${NC}"
}
echo " last_find : $FMT_FIND"
echo " last_crash : $FMT_CRASH"
if [ -z "$MINIMAL_ONLY" ]; then
echo " last_hang : $FMT_HANG"
echo " cycles_wo_finds : $FMT_CWOP"
fi
echo " coverage : $COVERAGE%"
if [ -z "$MINIMAL_ONLY" ]; then
CPU_USAGE=$(ps aux | grep -w $fuzzer_pid | grep -v grep | awk '{print $3}')
MEM_USAGE=$(ps aux | grep -w $fuzzer_pid | grep -v grep | awk '{print $4}')
echo " cpu usage $CPU_USAGE%, memory usage $MEM_USAGE%"
fi
echo " cycles $((cycles_done + 1)), lifetime speed $EXEC_SEC execs/sec, items $cur_item/$corpus_count (${PATH_PERC}%)"
if [ "$saved_crashes" = "0" ]; then
echo " pending $pending_favs/$pending_total, coverage $bitmap_cvg, no crashes yet"
else
echo " pending $pending_favs/$pending_total, coverage $bitmap_cvg, crashes saved $saved_crashes (!)"
fi
echo
fi
else
if [ ! -e "$i" -a -e "$j" ]; then
if [ '!' "$PROCESS_DEAD" = "" ]; then
ALIVE_CNT=$((ALIVE_CNT + 1))
fi
START_CNT=$((START_CNT + 1))
last_find=0
IS_STARTING=1
fi
fi
done
# Formatting for total time, time since last find, crash, and hang
fmt_duration $((CUR_TIME - TOTAL_TIME)) && FMT_TIME=$DUR_STRING
# Formatting for total execution
FMT_EXECS="0 millions"
EXECS_MILLION=$((TOTAL_EXECS / 1000 / 1000))
EXECS_THOUSAND=$((TOTAL_EXECS / 1000 % 1000))
if [ $EXECS_MILLION -gt 9 ]; then
FMT_EXECS="$EXECS_MILLION millions"
elif [ $EXECS_MILLION -gt 0 ]; then
FMT_EXECS="$EXECS_MILLION millions, $EXECS_THOUSAND thousands"
else
FMT_EXECS="$EXECS_THOUSAND thousands"
fi
rm -f "$TMP"
TOTAL_DAYS=$((TOTAL_TIME / 60 / 60 / 24))
TOTAL_HRS=$(((TOTAL_TIME / 60 / 60) % 24))
test -z "$TOTAL_WCOP" && TOTAL_WCOP="not available"
fmt_duration $TOTAL_LAST_FIND && TOTAL_LAST_FIND=$DUR_STRING
test "$TOTAL_TIME" = "0" && TOTAL_TIME=1
if [ "$PROCESS_DEAD" = "" ]; then
TXT="excluded from stats"
else
TXT="included in stats"
ALIVE_CNT=$(($ALIVE_CNT - $DEAD_CNT - $START_CNT))
fi
echo "Summary stats"
echo "============="
if [ -z "$SUMMARY_ONLY" -o -z "$MINIMAL_ONLY" ]; then
echo
fi
echo " Fuzzers alive : $ALIVE_CNT"
if [ ! "$START_CNT" = "0" ]; then
echo " Starting up : $START_CNT ($TXT)"
fi
if [ ! "$DEAD_CNT" = "0" ]; then
echo " Dead or remote : $DEAD_CNT ($TXT)"
fi
echo " Total run time : $FMT_TIME"
if [ -z "$MINIMAL_ONLY" ]; then
echo " Total execs : $FMT_EXECS"
echo " Cumulative speed : $TOTAL_EPS execs/sec"
fi
if [ "$ALIVE_CNT" -gt "0" ]; then
echo " Average speed : $((TOTAL_EPS / ALIVE_CNT)) execs/sec"
fi
if [ -z "$MINIMAL_ONLY" ]; then
echo " Pending items : $TOTAL_PFAV faves, $TOTAL_PENDING total"
fi
if [ "$ALIVE_CNT" -gt "1" -o -n "$MINIMAL_ONLY" ]; then
if [ "$ALIVE_CNT" -gt "0" ]; then
echo " Pending per fuzzer : $((TOTAL_PFAV/ALIVE_CNT)) faves, $((TOTAL_PENDING/ALIVE_CNT)) total (on average)"
fi
fi
echo " Coverage reached : ${TOTAL_COVERAGE}%"
echo " Crashes saved : $TOTAL_CRASHES"
if [ -z "$MINIMAL_ONLY" ]; then
echo " Hangs saved : $TOTAL_HANGS"
echo "Cycles without finds : $TOTAL_WCOP"
fi
echo " Time without finds : $TOTAL_LAST_FIND"
echo
exit 0