Commit Graph

73 Commits

Author SHA1 Message Date
Andrew Bettison
bacba19dc1 Refactor manifest: specific setter functions
Replace generic rhizome_manifest_set() and rhizome_manifest_set_ll()
with per-field setter functions, eg, rhizome_manifest_set_filesize().
Struct rhizome_manifest elements for all known fields, to replace the
use of rhizome_manifest_get() and rhizome_manifest_get_ll() everywhere:
sender, recipient, service, name, date, bundle_key.

Add boolean validity flags for binary blob types, to avoid having to compare
with many bytes of all-zero to detect presence, eg, has_sender, has_recipient,
has_author, has_bundle_key.  These maintained by the setter functions.

Rename existing manifest struct elements to be the same as their field
names: fileLength -> filesize, journalTail -> tail.

More use of unsigned int, size_t and uint64_t for payload sizes, offsets, byte
counts, etc. especially in rhizome_store.c and meshms.c.  More uniform use of
size_t to dimension memory buffers.  Fix some printf(3) style format strings
for 64-bit correctness on 32-bit systems.  Use new constant RHIZOME_SIZE_UNSET
instead of -1 to indicate unknown dimension, and explicitly assert its absence
before comparisons and arithmetic, for safety.

Replace some 'int' loop variables with 'unsigned' where appropriate.

Fix bugs discovered in MeshMS bundle private/public key generation and
bundle secret key handling for export/extract commands.

Instrument the first MeshMS test case to aid debugging.

New debug config flag: debug.manifest logs all modifications to all manifest
fields by setter functions.

Rename debug config flag: debug.rhizome_bind -> debug.rhizome_sql_bind.
2013-11-04 19:17:49 +10:30
Andrew Bettison
c2b78f7b29 Refactor manifest: replace int with unsigned 2013-10-30 14:14:38 +10:30
Andrew Bettison
416b82b6c0 Refactor manifest: change haveSecret to enum 2013-10-30 13:14:26 +10:30
Andrew Bettison
48921802f5 Use size_t in Rhizome store functions 2013-10-25 00:20:53 +10:30
Andrew Bettison
a95ef79139 Use sid_t everywhere
Remove stowSid() at last

Change API of tohex() and strbuf_tohex(), to pass string length instead of
binary byte count.  This allows odd numbers of hex digits to be produced.

Remove alloca_tohex_sid(); replace with alloca_tohex_sid_t()

New alloca_tohex_sid_t_trunc() macro
2013-10-09 18:54:21 +10:30
Andrew Bettison
7b427d48ee A few more uses of rhizome_bid_t
Now the TEXT_TOUPPER SQL bind action is only used for filehash hex
strings; all Bundle ID parameters are bound using RHIZOME_BID_T.
2013-10-04 04:50:37 +09:30
Andrew Bettison
ab31420faf rhizome_bid_t
Consistent type for internal binary representation of Rhizome Bundle ID
(aka Manifest ID)
2013-10-03 23:16:45 +09:30
Andrew Bettison
7e3a552011 Issue #69: refactor all Rhizome SQL queries to use binding
Remove all uses of sprintf(3) and its ilk to form SQL query commands.
Use the new sqlite_bind() and sqlite_prepare_bind() functions instead.
2013-10-03 15:15:30 +09:30
Andrew Bettison
06d7620314 Add "rhizome add file" --force-new option
Overrides the default behaviour of de-duplicating added manifests

New 'rhizomeops' test case: AddForceDuplicate
2013-09-30 16:48:18 +09:30
Jeremy Lakeman
5383b5e3be Fix signature verification cache 2013-09-24 14:34:51 +09:30
Jeremy Lakeman
e98592a7c1 Don't update the database author if we already know it 2013-08-27 15:11:56 +09:30
Jeremy Lakeman
94381bab49 Fix non-journal, non-block based payload encryption 2013-07-26 15:13:22 +09:30
Jeremy Lakeman
7634e1fcfb Add command to mark messages as read 2013-07-26 14:36:11 +09:30
Jeremy Lakeman
0c237db66a Add API to create bundles with deterministic ID's 2013-07-25 14:46:34 +09:30
Jeremy Lakeman
422671c906 Add API to generate public key from private key 2013-07-25 14:40:59 +09:30
Jeremy Lakeman
6dc5fde6b6 Fix xor encryption for journal bundles and unaligned read/write ops 2013-07-19 22:26:37 +09:30
Jeremy Lakeman
ef583bdfaa Fix meaning of rhizome crypto debug message 2013-06-27 16:50:58 +09:30
gardners
2252fdcaa7 created tool for automatically finding IN()s without matching
OUT()s or where return() is used instead of RETURN().
Added OUT() to end of all functions using IN() that lacked it to
make it easier to statically analyse this invariant.
Fixed several return instead of RETURNs detected through use of
this tool. #49
2013-02-20 16:18:56 +10:30
gardners
80168e3530 Improve debug logging
- add debug.trace to help track down segfaults
- warn about alarms without names (which are then hard to track down).
2013-02-20 15:36:30 +10:30
Jeremy Lakeman
b47d27dfe6 Log an error when a bundle secret can't be verified 2013-01-11 14:29:41 +10:30
Jeremy Lakeman
fd96c5bd76 Add bsk to correct command 2013-01-04 12:01:39 +10:30
Jeremy Lakeman
28a05baa54 Enable encryption of rhizome payloads 2013-01-03 16:12:24 +10:30
Jeremy Lakeman
dc43b83c80 Tidy up rhizome secret handling to improve reusability 2013-01-03 12:14:13 +10:30
Jeremy Lakeman
2967d1e00f Don't check for duplicates if an existing id has been supplied 2013-01-03 11:18:30 +10:30
Jeremy Lakeman
66deb575d3 Add offset to nonce instead of replacing byte values 2013-01-02 15:05:22 +10:30
Jeremy Lakeman
6c7ba438a3 Refactor rhizome extract manifest into more useful methods 2013-01-02 11:15:03 +10:30
Jeremy Lakeman
72bc597e92 Add encryption support to writing of rhizome content 2012-12-27 09:39:10 +10:30
Andrew Bettison
5985df751d Overhaul debug flags
Replace debugflags_t and DEBUG_XXX bit masks with config schema "debug.xxx"
entries.

No more support for "debug.all".
2012-12-11 15:59:46 +10:30
Jeremy Lakeman
6599835331 Squelch debug log message 2012-11-16 13:20:32 +10:30
Andrew Bettison
4450116472 Refactor string functions, add str_fromprint()
Move lots of non-Serval-specific string functions from "serval.h"/dataformats.c
to "str.h"/str.c.

Add str_fromprint() function that reverses the effect of toprint(): interprets
escape sequences in the source string "\n\t\r\0\xHH\\" and replaces them with a
single char in the destination string.
2012-11-07 16:42:45 +10:30
gardners
b23b28b16d Changed rhizome manifest signature block type code semantics to
allow signature blocks of 256 bytes, and to allow multiple
signature types with same length. #34
2012-10-29 16:07:42 +10:30
gardners
a40fcfec66 cleaned up NaCl build to just compile using make. #17 2012-10-18 17:26:52 +10:30
gardners
55c9cff8cd added code to replace old-format SAS with new crypto_sign implementation
type on next use (so that people upgrading automatically get a new valid
SAS instead of continuing to try to use an invalid one -- this is only
a reasonable course of action since we have made no formal releases of
the code). #17
2012-10-18 17:26:49 +10:30
gardners
499c4018bf Imported ref10/ implementation of crypto_sign from supercop-20120525
as recommended a while back by Dan Bernstein as offering the fastest
implementation of the crypto_sign() primitives for ARM.
Indeed this implementation IS faster. See comparison below for a
Rock 500 handset (800MHz(?) ARM6, no NEON):

Original ref/ implementation on an R500 stock rom (non-rooted)::
mean signature generation time = 96.80ms
mean signature verification time = 272.20ms

ref10/ implementations on an R500 stock rom (non-rooted):
mean signature generation time = 4.00ms
mean signature verification time = 13.00ms

Approximately 20x speed up, just like that :)
2012-10-18 17:26:48 +10:30
Andrew Bettison
52ccd07020 Issue #17, improve speed of "rhizome extract manifest"
In the case that the MANIFESTS 'author' column is not NULL, do not perform a
full bundle secret verification in order to clear the '.readonly' flag, just
check whether the author's SID is present in the keyring with a proper-size
rhizome secret.
2012-10-15 18:33:44 +10:30
Andrew Bettison
c43a7ba8e6 Issue #17, improve "rhizome extract manifest" command
Add ".readonly" output field, add logic to detect bundle author and update
MANIFESTS table 'author' column accordingly.

Add two 'rhizomeops' test cases to deal with authorless logic.

Rename some 'rhizomeops' test cases for consistency and to help filtering.
2012-10-15 16:48:04 +10:30
Andrew Bettison
eff3e9ec8f Issue #17, fix new 'rhizomeops' test
Was incorrectly supplying bundle secret to "rhizome add" command.
2012-10-15 14:01:48 +10:30
Andrew Bettison
3f1f495d68 Issue #17, overhaul manifest author crypto logic
Add test case for new feature of the "rhizome add" command: if the author SID
is not specified (empty arg) then it searches the keyring for the author.

Removed "authorSid" argument from several functions that also take a struct
rhizome_manifest * arg, since the author, if known, is now supplied in the
struct.

Improve return value handling and refactored some rhizome crypto code.
2012-10-11 17:58:24 +10:30
Andrew Bettison
3678522872 Issue #17, add AUTHOR column to Rhizome MANIFESTS table
Replace ".selfsigned" column with ".author" and ".fromhere" columns in
output of "rhizome list" command.  (Note that a "sender" column is
already present.)

Add 'author' field to struct rhizome_manifest.

Log all fully rendered SQL statements on DEBUG_RHIZOME.

Update 'rhizomeops' test cases and improve the assert_rhizome_list()
test function to be able to assert authorship of files.
2012-10-09 17:48:06 +10:30
Andrew Bettison
cc9cb8d827 Issue #9, fix old 'rhizomeprotocol' test failures
Now only the five new Rhizome Direct tests fail.
2012-10-02 16:32:48 +09:30
Andrew Bettison
3f1e0dd763 Finish sleep-retry on locked Rhizome db
Closes #2.

Rewrite all Rhizome db query code using new retry primitives defined in
"rhizome.h": sqlite_step_retry(), sqlite_retry(), sqlite_retry_done(), etc.
Replace all calls to sqlite3_prepare_v2() with sqlite_prepare() which does
proper error logging.

Fix bug: re-invoking sqlite3_blob_close() on SQLITE_BUSY return causes process
to abort.  Use an explicit BEGIN...COMMIT around the blob writing code instead.

Tested using repeated invocations of batphone/tests/meshms1.

Delete deprecated Rhizome db code in rhizome_crypto.c that has been replaced
with keyring file.
2012-08-24 15:38:39 +09:30
Andrew Bettison
f37ec5af09 Reduce use of stderr in many places
serval_packetvisualise() is now replaced by DEBUG_packet_visualise() which uses
logging system not stderr (so now it will appear in Android log).

Replaced several fprintf(stderr,...) with DEBUGF(...).

Command line only prints a full help message on "help" command -- a command
parse failure simply informs the user about the "help" command.
2012-08-01 17:54:02 +09:30
gardners
baf8543def added cache for positive and negative rhizome manifest signature
verifications to avoid repeating expensive lookups.
not perfect, and not tested.
2012-06-26 21:21:46 +09:30
gardners
e542b41746 instrumented various rhizome functions. 2012-06-26 16:03:34 +09:30
Andrew Bettison
208b9c15fd Factor out rhizome_hex_to_bytes() and chartonybl() 2012-06-25 14:51:21 +09:30
Andrew Bettison
362f4b4ed8 Remove sensitive data from debug 2012-06-21 18:38:52 +09:30
Andrew Bettison
0997909053 Fix import bundle test and bug
Add lots of debug to track down cause of rhizome list .selfsigned column bug
2012-06-15 18:10:10 +09:30
Andrew Bettison
637ef02f30 Add _selfsigned column to "rhizome list" output
Add <pin,pin...> arg1 to "rhizome list" command, so it can open the keyring

Some tests in tests/rhizome fail
2012-06-08 18:25:43 +09:30
Andrew Bettison
7a71a521f8 Pass author SID parameters in binary, not hex 2012-06-08 15:27:15 +09:30
Andrew Bettison
86c7819f97 Remove spurious ERROR messsages
Tests assert that stderr contains no ERROR: lines after a successful exit

Rewrote sqlite_exec_int64() to separate error outcomes from legitimate
result values

Changed several WHY() calls to DEBUG()

Improved test framework
2012-06-08 13:13:26 +09:30