Revise sqlite_prepare() macro et al and underlying functions to use bind
varargs instead of sprintf(3)-style varargs.
Convert all SQL function calls in rhizome_database.c
OUT()s or where return() is used instead of RETURN().
Added OUT() to end of all functions using IN() that lacked it to
make it easier to statically analyse this invariant.
Fixed several return instead of RETURNs detected through use of
this tool. #49
sqlite_void_exec() and its ilk now return the count of changed rows, not
just zero, on success
sqlite_exec_prepared() and its ilk now return the count of rows (number
of step results SQLITE_ROW), instead of just zero, on success
rhizome_clean() function now produces an optional report of its changes
rhizome_fail_write() and rhizome_finish_write() now log WARNings not
ERRORs if the SQL DELETE FROM FILES or DELETE FROM FILEBLOBS statements
fail
Refactor rhizome_open_read() to use sqlite_exec_int64()
Ensure that 'rhizome extract' and 'rhizome dump' commands return exit
status of 1 in "not found" conditions, not 255, which is reserved for
errors
Test cases for four new commands: 'rhizome delete bundle', 'rhizome
delete manifest', 'rhizome delete payload' and 'rhizome delete file'
(no test case for 'rhizome clean' yet)
- made some debug output conditional.
- make it possible to debug slip decoding without getting swamped by other output.
- Improve reporting of unnamed schedule() calls.
- always collect performance statistics, and report on any excess
use (>1sec in every 3), even if debug.timing is not enabled.
- include file size information in Rhizome transfer messages.
- reduce output when debug.rhizome_rx is set.
- we no longer hit the database for every outgoing packet, attempting to announce bundles
- we no longer advertise manifests periodically
- when an interesting bar arrives, we ask for the manifest to be announced, which uses the existing packet format
Renamed rhizome extract file to rhizome dump file
Added rhizome extract file [manifest] [filepath] [pins]
Modified tests to use the appropriate command, assuming that MeshMS payloads will be encrypted
split files table into files and fileblobs to avoid pathological
delays when updating datavalid flag in files table (sqlite copies
the whole table row, including possibly large blob).
- close database after every command line operation
- don't cache rhizome enabled configuration
- don't send advertisements unless the database is open and the web server is running
- don't provess advertisements unless the database is open
Introduce __WHENCE__ macro and a block comment in log.h explaining it.
In "primitive" kinds of functions, rename 'whence' arguments to '__whence' and
use WHYF(), WARNF(), DEBUGF() macros instead of calling logMessage() directly.
In the case that the MANIFESTS 'author' column is not NULL, do not perform a
full bundle secret verification in order to clear the '.readonly' flag, just
check whether the author's SID is present in the keyring with a proper-size
rhizome secret.
Do not store negatives in the MANIFESTS 'author' column, just positives, so
that newly-unlocked identities are still checked.
Comment in rhizome_database.c about the semantics of the MANIFESTS.author
column.
Add ".readonly" output field, add logic to detect bundle author and update
MANIFESTS table 'author' column accordingly.
Add two 'rhizomeops' test cases to deal with authorless logic.
Rename some 'rhizomeops' test cases for consistency and to help filtering.
Replace ".selfsigned" column with ".author" and ".fromhere" columns in
output of "rhizome list" command. (Note that a "sender" column is
already present.)
Add 'author' field to struct rhizome_manifest.
Log all fully rendered SQL statements on DEBUG_RHIZOME.
Update 'rhizomeops' test cases and improve the assert_rhizome_list()
test function to be able to assert authorship of files.
It turns out that if the DB is locked, sqlite_prepare_v2() call can return
SQLITE_BUSY. The retry logic (implemented for issue #2) only provided for
sqlite_step() to return SQLITE_BUSY. It was a fairly straightforward matter to
extend the retry logic to cover statement preparation in an equally general
fashion.
The problem was observed while diagnosing failures in the rhizomeprotocol
DirectPush test case: the "servald rhizome list" command was failing due to a
locked database. See issue #9.
associated file before checking if it was already in the database.
Rhizome Direct can supply a manifest without associated file if
the file is already in the database, and so it was breaking.
Also removed "assert bundle_received_by"'s from rhizome direct
pull and sync tests because they are not needed, and were failing
because the same file contents was used for the files being
exchanged, and so file storage was not occurring, and thus the log
message being looked for was not being produced.
Push, pull and sync tests in rhizomeprotocol now pass, leaving
only two tests in error. #9
signatures were not being recorded. Also fixed separate issue
where rhizomeprotocols tests expected selfsigned to be 1 on
receiver end, when it will never be, because the BK doesn't match.
rhizome direct push test in rhizomeprotocols now passes. #9
Now the 'rhizomeprotocol' Push test case now passes. It should be renamed to
DirectPush.
Much refactoring of the Rhizome Direct HTTP request parsing. Now uses
strbuf_sprintf() instead of snprintf() in many places to check for buffer
overrun and ensure terminating nul. Still more of this kind of work is needed.
Improved debug that needs to be made conditional on DEBUG_RHIZOME_RX and
DEBUG_RHIZOME_TX. Some just needs removal.
Closes#2.
Rewrite all Rhizome db query code using new retry primitives defined in
"rhizome.h": sqlite_step_retry(), sqlite_retry(), sqlite_retry_done(), etc.
Replace all calls to sqlite3_prepare_v2() with sqlite_prepare() which does
proper error logging.
Fix bug: re-invoking sqlite3_blob_close() on SQLITE_BUSY return causes process
to abort. Use an explicit BEGIN...COMMIT around the blob writing code instead.
Tested using repeated invocations of batphone/tests/meshms1.
Delete deprecated Rhizome db code in rhizome_crypto.c that has been replaced
with keyring file.
