This adds a new persistent hook library `mipsel_read_into_a0.c`. With
it, you can test the persistent hook on the *mipsel* architecture.
I'm also updating the README in `utils/qemu_persistent_hook` and
Makefile and explain how to test the persistent hook on *mipsel*.
This all works thanks to qemuafl already having the correct CPU struct
for *mipsel* in `qemuafl/api.h`.
This patch also updates the root `.gitignore` file to ignore the two
test binaries `test` and `mipsel_test`.
qemuafl now supports persistent mode for *mipsel* targets. This patch
changes the `SYS` variable tests and runs the persistent qemu_mode
tests for *mipsel* as well.
This also adds an optional environment variable called `CPU_TARGET_CC`
that you can pass to the `./test-qemu-mode.sh` test script.
This allows you to specify a cross-compiler for the target platform. The
test script then it uses to compile `test-instr.c` and `test-compcov.c`.
Example usage:
```
CPU_TARGET_CC=mipsel-linux-gnu-cc CPU_TARGET=mipsel ./test-qemu-mode.sh
```
The output should look something like this:
```
[*] Using environment variable CPU_TARGET=mipsel for SYS
[*] starting AFL++ test framework ...
[*] Testing: qemu_mode
[*] Using mipsel-linux-gnu-cc as compiler for target
[*] running afl-fuzz for qemu_mode, this will take approx 10 seconds
[+] afl-fuzz is working correctly with qemu_mode
[*] running afl-fuzz for qemu_mode AFL_ENTRYPOINT, this will take approx 6 seconds
[+] afl-fuzz is working correctly with qemu_mode AFL_ENTRYPOINT
[-] not an intel or arm platform, cannot test qemu_mode compcov
[-] not an intel or arm platform, cannot test qemu_mode cmplog
[*] running afl-fuzz for persistent qemu_mode, this will take approx 10 seconds
[+] afl-fuzz is working correctly with persistent qemu_mode
[+] persistent qemu_mode was noticeable faster than standard qemu_mode
[*] running afl-fuzz for persistent qemu_mode with AFL_QEMU_PERSISTENT_EXITS, this will take approx 10 seconds
[+] afl-fuzz is working correctly with persistent qemu_mode and AFL_QEMU_PERSISTENT_EXITS
[+] persistent qemu_mode with AFL_QEMU_PERSISTENT_EXITS was noticeable faster than standard qemu_mode
[-] we cannot test qemu_mode unsigaction library (32 bit) because it is not present
[+] qemu_mode unsigaction library (64 bit) ignores signals
[*] 1 test cases completed.
[-] not all test cases were executed
[+] all tests were successful :-)
```
The target system might be different from the host system. For example,
you can fuzz Linux binaries compiled for *mipsel*, while your host is
*x86_64*.
Some of the tests depend on specific platforms to run correctly.
For example, the afl-fuzz qemu_mode cmplog test only works on Intel or
ARM systems. The `SYS` variable is populated using `uname -m` and the
test cases then consult this variable to decide whether to run the test
or not.
If you want to test afl-fuzz for qemu_mode on mipsel, you might
want to make sure that Intel or ARM tests don't run. With this
patch, you can supply your own `CPU_TARGET` environment variable and skip
platform specific tests. `SYS` then contains the value of `CPU_TARGET`.
This allows you to add tests for *mipsel* or other niche platforms in
the future as well.
Sample usage:
```
$ cd qemu_mode && env CPU_TARGET=mipsel ./build_qemu_support.sh
$ cd ../test && env CPU_TARGET=mipsel ./test-qemu-mode.sh
[*] Using environment variable CPU_TARGET=mipsel for SYS
[*] starting AFL++ test framework ...
[*] Testing: qemu_mode
...
```
The value of `classified`, `bits_new`, and `cksum`, were not always
correctly maintained.
1. In the past, `afl->queue_top->exec_cksum` was always assigned when
`add_to_queue`, however it became conditional since cd5764170595.
This doesn't change correctness because calibrate_case() will
calculate the checksum. However, this mean one calibration run is
wasted.
2. Sometimes `classified` is set incorrectly.
For example, this code snippet
```
new_bits = has_new_bits_unclassified(afl, afl->virgin_bits);
classified = 1;
```
should be changed to
```
new_bits = has_new_bits_unclassified(afl, afl->virgin_bits);
if (new_bits) classified = 1;
```
This commit fixed above issues and use macros to make the code easier to
understand. This should prevent to forget to set classified in the
future (like the bug fixed by 30c93d132166).
The macros also defers the calculations to where the values are really
needed. This could save cpu if the code returns earlier. For example,
if a case is timeout first and not timeout the second time, the current
code does classify_counts, which is not always needed.
Assume we have one main node and N secondary nodes in a parallel
fuzzing campaign. Every time the main node found a new case, the case
will be synced to all secondary nodes. Later when the main node sync,
the main node need to run the file again to see if the file is
interesting because they are "new" cases on the secondary nodes.
In other words, for one new case, the main node has to run the redundent
test N times. This is wasteful and slowed down the progress of main
node.
The wasteful issue on secondary nodes is acceptable because we can run
more secondary nodes to mitigate the inefficiency. OTOH, increasing the
number of secondary nodes slow down the main node further.
