Objective is to avoid having to call system("servald rhizome import ...") to
handle a Rhizome direct POST /rhizome/bundle request. Antiquated code in and
around rhizome_import_bundle() needs much cleaning up, as indicated by some
TODO comments. Invocations must unnecessarily write the manifest into a file,
when they already have it in memory, ready to pass to the function.
All the 'rhizomeops' tests pass, but two 'rhizomeprotocol' tests are broken
by the changes in this commit.
Servald starts DNA helper, receives startup ACK, sends requests, receives
responses, handles malformed helper responses, echoes dnahelper stderr lines to
log, sends MDP reply packet, waits for dead helper process, all asynchronously.
Shuts down helper process during servald shutdown.
Remaining issues:
- Does not impose a timeout on helper responses.
- Only the first URI is reported by the "dna lookup" command.
Do not add 'filehash' var to manifest if filesize=0
Do not accept 'filehash' var when parsing manifest with filesize=0
When responding to a new rhizome advertisement, do not try to HTTP
request a payload if filesize=0, just import the manifest directly
Various operations, eg "rhizome file add", do not report 'filehash'
fields where 'filesize' is zero
Do not delete rows from MANIFESTS table which have empty filehash
Various related bug fixes
Remove need to nul-terminate the received buffers in HTTP fetch reply handling
and HTTP server request parsing.
Remove redundant copying of data.
More rigorous parsing code, probably less vulnerable to overrun exploits.
Better debug logging of requests and responses.
Rhizome manifest parser now parses and validates all known fields, informs
about unsupported fields, and unpacks fields into relevant struct manifest
elements where appropriate. Is also stricter about whitespace.
Rhizome fetch code now logs debug messages if DEBUG_RHIZOME_RX bit is on.
SET_NONBLOCKING(), SET_BLOCKING(), WRITE_STR() are now set_nonblock(),
set_block() and write_str() respectively, all of which log an error before
returning -1. There are other useful methods: write_all() treats anything less
than all bytes written as an error; write_nonblock() treats EAGAIN and EINTR as
zero bytes written, and a combination: write_all_nonblock().
Tests assert that stderr contains no ERROR: lines after a successful exit
Rewrote sqlite_exec_int64() to separate error outcomes from legitimate
result values
Changed several WHY() calls to DEBUG()
Improved test framework
