Refactor much of the new code in log.c
Add new functions: cf_load_strict() and cf_reload_strict() for use by the
server, which does not want to overwrite its in-memory config at all if there
is any defect in the config file. However, commands always want to overwrite
the default in-memory config, even with a defective config file, so that
logging has a chance to succeed.
Added `log_format.dump_config` boolean config option
Indepent configuration of show_pid, show_time and log level for each
destination
Update test scripts for new config options
Include xprintf.c in MDP client source files (now used by log.c)
All STRUCT cf_fmt_config_xxx() functions omit invalid values, silently
if the invalid value is the default, otherwise with a non-CFOK return
value.
The "config dump --full" option includes all options with valid values,
otherwise omits options having default values.
No more SEGV. Omits invalid (default) values. Logs unconditional DEBUG
output, to be removed before merging into development. Still missing
cf_cmp_ functions to prune out default values.
Improved config Object Model iterator logic to barf on internal NULL
nodes.
OUT()s or where return() is used instead of RETURN().
Added OUT() to end of all functions using IN() that lacked it to
make it easier to statically analyse this invariant.
Fixed several return instead of RETURNs detected through use of
this tool. #49
sqlite_void_exec() and its ilk now return the count of changed rows, not
just zero, on success
sqlite_exec_prepared() and its ilk now return the count of rows (number
of step results SQLITE_ROW), instead of just zero, on success
rhizome_clean() function now produces an optional report of its changes
rhizome_fail_write() and rhizome_finish_write() now log WARNings not
ERRORs if the SQL DELETE FROM FILES or DELETE FROM FILEBLOBS statements
fail
Refactor rhizome_open_read() to use sqlite_exec_int64()
Ensure that 'rhizome extract' and 'rhizome dump' commands return exit
status of 1 in "not found" conditions, not 255, which is reserved for
errors
Test cases for four new commands: 'rhizome delete bundle', 'rhizome
delete manifest', 'rhizome delete payload' and 'rhizome delete file'
(no test case for 'rhizome clean' yet)
- made some debug output conditional.
- make it possible to debug slip decoding without getting swamped by other output.
- Improve reporting of unnamed schedule() calls.
- always collect performance statistics, and report on any excess
use (>1sec in every 3), even if debug.timing is not enabled.
- include file size information in Rhizome transfer messages.
- reduce output when debug.rhizome_rx is set.
Output is same format as "keyring add" command
Fixed minor bug in rolling a random DID -- last byte was not random
Some code improvements, including better support for sid_t
Replace explicit [<pin,pin...>] command-line arguments with --entry-pin=<pin>
options. Update test scripts accordlingly.
Add --keyring-pin=<pin> option (with one test case).
Renamed rhizome extract file to rhizome dump file
Added rhizome extract file [manifest] [filepath] [pins]
Modified tests to use the appropriate command, assuming that MeshMS payloads will be encrypted
If configuration is bad, do not execute commands except those with the
PERMISSIVE_CONFIG property.
Flush log buffer immediately after clearing cf_limbo flag, in case there are no
further log messages that would cause the flush. (Fixes bug that an unrecognised
command produced no log output.)
More block comments in log.c.
Change a test case: configuration options are now case sensitive.
Fix config file load and parse logic in conf.c, always copy 'debug' flags
from config.debug.
The config schema 'interfaces' option is no longer MANDATORY.
Introduce new CLIFLAG_PERMISSIVE_CONFIG to supress bad-config ERROR messages
from the 'config set' and 'config get' commands.
Refactor cli_execute() into cli_parse() and cli_invoke(). Use *const* struct
command_line_option everywhere.
Instead of specifying a file path, if the file is given as '-' then the content
of the manifest is output in the 'manifest' key-value pair instead of written
to a file.
This allows the output fields of any command to contain nul characters, which
paves the way for extracting a manifest or other binary data directly instead
of having to write it into a temporary file.
Move lots of non-Serval-specific string functions from "serval.h"/dataformats.c
to "str.h"/str.c.
Add str_fromprint() function that reverses the effect of toprint(): interprets
escape sequences in the source string "\n\t\r\0\xHH\\" and replaces them with a
single char in the destination string.
- close database after every command line operation
- don't cache rhizome enabled configuration
- don't send advertisements unless the database is open and the web server is running
- don't provess advertisements unless the database is open
as recommended a while back by Dan Bernstein as offering the fastest
implementation of the crypto_sign() primitives for ARM.
Indeed this implementation IS faster. See comparison below for a
Rock 500 handset (800MHz(?) ARM6, no NEON):
Original ref/ implementation on an R500 stock rom (non-rooted)::
mean signature generation time = 96.80ms
mean signature verification time = 272.20ms
ref10/ implementations on an R500 stock rom (non-rooted):
mean signature generation time = 4.00ms
mean signature verification time = 13.00ms
Approximately 20x speed up, just like that :)
Add ".readonly" output field, add logic to detect bundle author and update
MANIFESTS table 'author' column accordingly.
Add two 'rhizomeops' test cases to deal with authorless logic.
Rename some 'rhizomeops' test cases for consistency and to help filtering.
Add test case for new feature of the "rhizome add" command: if the author SID
is not specified (empty arg) then it searches the keyring for the author.
Removed "authorSid" argument from several functions that also take a struct
rhizome_manifest * arg, since the author, if known, is now supplied in the
struct.
Improve return value handling and refactored some rhizome crypto code.
Replace ".selfsigned" column with ".author" and ".fromhere" columns in
output of "rhizome list" command. (Note that a "sender" column is
already present.)
Add 'author' field to struct rhizome_manifest.
Log all fully rendered SQL statements on DEBUG_RHIZOME.
Update 'rhizomeops' test cases and improve the assert_rhizome_list()
test function to be able to assert authorship of files.
Fixes#21. The problem was caused when the double-fork logic used in "servald
start" was clobbered in 5103176. This meant that the servald daemon process on
Android no longer had a PPID=1, but the PID of the long-lived
"org.servalproject" parent process which called the JNI entry point. Killing
the servald process then caused it to become a zombie process, since the
org.servalproject does not habitually call wait(2). That caused the "servald
stop" logic to send five SIGHUPs to the zombie without any error, making it
appear that the process was not dying.
Reinstated the double-fork logic and added a new test case to ensure that the
daemon process does not become a zombie on being killed prematurely.
Must be enabled by using rhizome.api.addfile.*
Certainly polishing to be done, including using filename supplied
during HTTP POST. Now to fix that, and make it all work with
final rhizomeprotocol test case.
rhizomeprotocol test cases 8 and 9 currently fail post-merge. #9
All test cases pass on Solaris (see issue #16) except the 'jni' tests because
there is no Java compiler available on Solaris, and the new 'rhizomeprotocol'
tests from the rhizomedirect branch, which have always failed.
serval_packetvisualise() is now replaced by DEBUG_packet_visualise() which uses
logging system not stderr (so now it will appear in Android log).
Replaced several fprintf(stderr,...) with DEBUGF(...).
Command line only prints a full help message on "help" command -- a command
parse failure simply informs the user about the "help" command.
Servald starts DNA helper, receives startup ACK, sends requests, receives
responses, handles malformed helper responses, echoes dnahelper stderr lines to
log, sends MDP reply packet, waits for dead helper process, all asynchronously.
Shuts down helper process during servald shutdown.
Remaining issues:
- Does not impose a timeout on helper responses.
- Only the first URI is reported by the "dna lookup" command.
Do not add 'filehash' var to manifest if filesize=0
Do not accept 'filehash' var when parsing manifest with filesize=0
When responding to a new rhizome advertisement, do not try to HTTP
request a payload if filesize=0, just import the manifest directly
Various operations, eg "rhizome file add", do not report 'filehash'
fields where 'filesize' is zero
Do not delete rows from MANIFESTS table which have empty filehash
Various related bug fixes
... as part of working on work on DNA helper callout facility.
(DNA replies now contain a token that can be used to match them
against DNA requests as a happy side-effect).
Debug bit mask is unsigned int everywhere.
Refactor DEBUG() macros a bit.
Add SERVALD_DEBUG_FILE env var to support test scripts.
Test defs put instance directories under $TFWVAR not $TFWTMP.
Do not use server_probe().
Use setsid() instead of setpgrp().
Still uses double-fork technique, as advised in setsid(2) man page.
Return exit status 10 if already running, instead of logging an error and exit
status 255.
results from each SID/servald instance.
Fixed bug in "keyring list" that was returning keyrings in
lower case instead of normalised form using overlay_render_sid()
Improve regular expressions for common data types in test scripts
Revert column count field delimiter in "rhizome list" from ":" to "\n"
Add a few more test cases
that is called only when needed, and marks a manifest as finalised
if the verifcation fails. reading a manifest now never sets
finalised flag, as either _finalise() or _verify() must be called.
Move rhizome_new_manifest() out of rhizome_read_manifest_file() so that the
out-of-manifest report shows the names of the functions where the manifests
were really allocated.
Replace with WHY() or WHYF() everywhere
Improve logMessage() to handle file/line/function printing internally
Mainly so that log messages from manifest alloc/free in rhizome_bundle.c
can make it into the Android log
This requires building with _GNU_SOURCE to get access to struct ucred on Linux. Previously the test failed on every platform except glibc <2.8 or so.
Building with _GNU_SOURCE causes TRUE to be defined so we can't use this as a variable anymore, and MIN/MAX so don't define our own.
Was calling exit() instead of returning -1 on error, which was causing
the serval app to exit in the JNI "start" command if rhizome could not
create its db file.
Use DEBUGF() instead of fprintf(stderr) in many places.
confValueGet() was not stripping \r\n off end of returned string.
rhizome_set_datastore_path() was keeping the pointer it was given (from
conValueGet()'s rotor) instead of making a strdup() copy.
