This commit is contained in:
vanhauser-thc
2022-01-14 15:01:05 +01:00
parent 630ba07054
commit f42c0047c8
4 changed files with 4 additions and 250 deletions

View File

@ -15,6 +15,7 @@ AFL++ is maintained by:
* Heiko "hexcoder-" Eißfeldt <heiko.eissfeldt@hexco.de>, * Heiko "hexcoder-" Eißfeldt <heiko.eissfeldt@hexco.de>,
* Andrea Fioraldi <andreafioraldi@gmail.com> and * Andrea Fioraldi <andreafioraldi@gmail.com> and
* Dominik Maier <mail@dmnk.co>. * Dominik Maier <mail@dmnk.co>.
* Documentation: Jana Aydinbas
Originally developed by Michał "lcamtuf" Zalewski. Originally developed by Michał "lcamtuf" Zalewski.

View File

@ -1,122 +0,0 @@
# Restructure AFL++'s documentation
## About us
We are dedicated to everything around fuzzing, our main and most well known
contribution is the fuzzer `AFL++` which is part of all major Unix
distributions (e.g. Debian, Arch, FreeBSD, etc.) and is deployed on Google's
oss-fuzz and clusterfuzz. It is rated the top fuzzer on Google's fuzzbench.
We are four individuals from Europe supported by a large community.
All our tools are open source.
## About the AFL++ fuzzer project
AFL++ inherited it's documentation from the original Google AFL project.
Since then it has been massively improved - feature and performance wise -
and although the documenation has likewise been continued it has grown out
of proportion.
The documentation is done by non-natives to the English language, plus
none of us has a writer background.
We see questions on AFL++ usage on mailing lists (e.g. afl-users), discord
channels, web forums and as issues in our repository.
This only increases as AFL++ has been on the top of Google's fuzzbench
statistics (which measures the performance of fuzzers) and is now being
integrated in Google's oss-fuzz and clusterfuzz - and is in many Unix
packaging repositories, e.g. Debian, FreeBSD, etc.
AFL++ now has 44 (!) documentation files with 13k total lines of content.
This is way too much.
Hence AFL++ needs a complete overhaul of it's documentation, both on a
organisation/structural level as well as the content.
Overall the following actions have to be performed:
* Create a better structure of documentation so it is easier to find the
information that is being looked for, combining and/or splitting up the
existing documents as needed.
* Rewrite some documentation to remove duplication. Several information is
present several times in the documentation. These should be removed to
where needed so that we have as little bloat as possible.
* The documents have been written and modified by a lot of different people,
most of them non-native English speaker. Hence an overall review where
parts should be rewritten has to be performed and then the rewrite done.
* Create a cheat-sheet for a very short best-setup build and run of AFL++
* Pictures explain more than 1000 words. We need at least 4 images that
explain the workflow with AFL++:
- the build workflow
- the fuzzing workflow
- the fuzzing campaign management workflow
- the overall workflow that is an overview of the above
- maybe more? where the technical writes seems it necessary for
understanding.
Requirements:
* Documentation has to be in Markdown format
* Images have to be either in SVG or PNG format.
* All documentation should be (moved) in(to) docs/
The project does not require writing new documentation or tutorials beside the
cheat sheet. The technical information for the cheat sheet will be provided by
us.
## Metrics
AFL++ is a the highest performant fuzzer publicly available - but is also the
most feature rich and complex. With the publicity of AFL++' success and
deployment in Google projects internally and externally and availability as
a package on most Linux distributions we see more and more issues being
created and help requests on our Discord channel that would not be
necessary if people would have read through all our documentation - which
is unrealistic.
We expect the the new documenation after this project to be cleaner, easier
accessible and lighter to digest by our users, resulting in much less
help requests. On the other hand the amount of users using AFL++ should
increase as well as it will be more accessible which would also increase
questions again - but overall resulting in a reduction of help requests.
In numbers: we currently have per week on average 5 issues on Github,
10 questions on discord and 1 on mailing lists that would not be necessary
with perfect documentation and perfect people.
We would consider this project a success if afterwards we only have
2 issues on Github and 3 questions on discord anymore that would be answered
by reading the documentation. The mailing list is usually used by the most
novice users and we don't expect any less questions there.
## Project Budget
We have zero experience with technical writers, so this is very hard for us
to calculate. We expect it to be a lot of work though because of the amount
of documentation we have that needs to be restructured and partially rewritten
(44 documents with 13k total lines of content).
We assume the daily rate of a very good and experienced technical writer in
times of a pandemic to be ~500$ (according to web research), and calculate
the overall amout of work to be around 20 days for everything incl. the
graphics (but again - this is basically just guessing).
Technical Writer 10000$
Volunteer stipends 0$ (waved)
T-Shirts for the top 10 contributors and helpers to this documentation project:
10 AFL++ logo t-shirts 20$ each 200$
10 shipping cost of t-shirts 10$ each 100$
Total: 10.300$
(in the submission form 10.280$ was entered)
## Additional Information
We have participated in Google Summer of Code in 2020 and hope to be selected
again in 2021.
We have no experience with a technical writer, but we will support that person
with video calls, chats, emails and messaging, provide all necessary information
and write technical contents that is required for the success of this project.
It is clear to us that a technical writer knows how to write, but cannot know
the technical details in a complex tooling like in AFL++. This guidance, input,
etc. has to come from us.

View File

@ -1,124 +0,0 @@
# Restructure AFL++'s documentation - Case Study
## Problem statement
AFL++ inherited it's documentation from the original Google AFL project.
Since then it has been massively improved - feature and performance wise -
and although the documenation has likewise been continued it has grown out
of proportion.
The documentation is done by non-natives to the English language, plus
none of us has a writer background.
We see questions on AFL++ usage on mailing lists (e.g. afl-users), discord
channels, web forums and as issues in our repository.
Most of them could be answered if people would read through all the
documentation.
This only increases as AFL++ has been on the top of Google's fuzzbench
statistics (which measures the performance of fuzzers) and has been
integrated in Google's oss-fuzz and clusterfuzz - and is in many Unix
packaging repositories, e.g. Debian, FreeBSD, etc.
AFL++ had 44 (!) documentation files with 13k total lines of content.
This was way too much.
## Proposal abstract
AFL++'s documentatin needs a complete overhaul, both on a
organisation/structural level as well as the content.
Overall the following actions have to be performed:
* Create a better structure of documentation so it is easier to find the
information that is being looked for, combining and/or splitting up the
existing documents as needed.
* Rewrite some documentation to remove duplication. Several information is
present several times in the documentation. These should be removed to
where needed so that we have as little bloat as possible.
* The documents have been written and modified by a lot of different people,
most of them non-native English speaker. Hence an overall review where
parts should be rewritten has to be performed and then the rewrite done.
* Create a cheat-sheet for a very short best-setup build and run of AFL++
* Pictures explain more than 1000 words. We need at least 4 images that
explain the workflow with AFL++:
- the build workflow
- the fuzzing workflow
- the fuzzing campaign management workflow
- the overall workflow that is an overview of the above
- maybe more? where the technical writes seems it necessary for
understanding.
Requirements:
* Documentation has to be in Markdown format
* Images have to be either in SVG or PNG format.
* All documentation should be (moved) in(to) docs/
## Project description
We created our proposal by discussing in the team what the issues are and
what was needed to fix it.
This resulted in the [project proposal](https://github.com/AFLplusplus/AFLplusplus/blob/stable/docs/docs.md).
We did not want to be selected by a writer but select a writer ourselves, so
we combed through the list and reviewed every single one of them.
We were not looking for coders writing technical documentation, but rather
someone who is an experienced writer and has documented experience with
structuring documentation.
Few fit that profile and we sent out messages to 6 people.
We finally decided on Jana because she had a strong background in technical
documentation and structuring information.
She had no technical experience in fuzzing whatsoever, but we saw that as
a plus - of course this made the whole process longer to explain details,
but overall ensured that the documentation can be read by (mostly) everyone.
We communicated via video calls every few weeks and she kept a public kanban
board about her todos, additional we used a Signal channel.
Her changes were imported via PRs where we discussed details.
The project was off to a good start, but then Jana got pregnant with serious
side effects that made working impossible for her for a longer time, hence
the schedule was thrown back.
She offered to rescind the payment and we select a new writer, but we saw
little opportunity in that, as that would mean a new selection of a writer,
someone else with a different vision on how the result should look like so
basically a full restart of the project and a large impact on our own time.
So we agreed on - after discussion with the Google GSoD team - that she
continues the project after the GSoD completion deadline as best as she can.
End of November she took one week off from work and fully dedicated her time
for the documenation which brought the project a big step forward.
Originally the project should have been ended begin of October, but now - at
nearing the end of November, we are at about 85% completion, with the end
being expected around mid of December.
## Metrics
We merged most of the changes in our development branch and are getting
close to a state where the user documentation part is completed and we
can create a new release. Only then the new documentatin is actually visible
to users. Therefore no metrics could be collected so far.
We plan on a user-assisted QA review end of November/begin of December.
The documentation was reviewed by a few test users so far however who gave
it a thumbs up.
## Summary
The GSoD project itself is great. It helps to get the documentation back in
line.
It was and is a larger time investment from our side, but we expected that.
When the project is done, the documentation will be more accessible by users
and also need less maintenance by us.
There is still follow-up work to be done by us afterwards (web site for the
docs, etc.).
Not sure what we would do differently next time. I think we prepared best as
possible and reacted best as possible to the unexpected.
Recommendations for other organizations who would like to participate in GSoD:
- expect the process to take a larger part of your time. the writer needs
your full support.
- have someone dedicated from the dev/org side to support, educate and
supervice the writer
- set clear goals and expectations

View File

@ -769,8 +769,7 @@ void cull_queue(afl_state_t *afl) {
afl->top_rated[i]->favored = 1; afl->top_rated[i]->favored = 1;
++afl->queued_favored; ++afl->queued_favored;
if (afl->top_rated[i]->fuzz_level == 0 || if (!afl->top_rated[i]->was_fuzzed) {
!afl->top_rated[i]->was_fuzzed) {
++afl->pending_favored; ++afl->pending_favored;
@ -936,7 +935,7 @@ u32 calculate_score(afl_state_t *afl, struct queue_entry *q) {
n_items = 0; n_items = 0;
// Don't modify perf_score for unfuzzed seeds // Don't modify perf_score for unfuzzed seeds
if (q->fuzz_level == 0) break; if (!q->fuzz_level) break;
u32 i; u32 i;
for (i = 0; i < afl->queued_items; i++) { for (i = 0; i < afl->queued_items; i++) {
@ -967,7 +966,7 @@ u32 calculate_score(afl_state_t *afl, struct queue_entry *q) {
case FAST: case FAST:
// Don't modify unfuzzed seeds // Don't modify unfuzzed seeds
if (q->fuzz_level == 0) break; if (!q->fuzz_level) break;
switch ((u32)log2(afl->n_fuzz[q->n_fuzz_entry])) { switch ((u32)log2(afl->n_fuzz[q->n_fuzz_entry])) {