Commit Graph

123 Commits

Author SHA1 Message Date
Andrew Bettison
7d9a5faa4e Move if(config.debug.xxx) tests into DEBUGF()
Original DEBUG() and DEBUGF() macros renamed to _DEBUG() and _DEBUGF()
New DEBUG() and DEBUGF() macros, first argument is flagname
New DEBUGF2(foo, bar, ...) macro does if(config.debug.foo||config.debug.bar) test
Replace almost all config.debug.xxx references to IF_DEBUG(xxx)
2015-07-13 16:00:05 +09:30
Andrew Bettison
5120e50dd7 Suppress Rhizome add de-duplication for journals 2015-04-13 16:24:05 +09:30
Andrew Bettison
dbe0cf862f Remove 'debug.manifests' option
Use existing 'debug.rhizome_manifest' instead
Improve format of manifest alloc/free log messages
2015-03-30 18:19:26 +10:30
Andrew Bettison
7f564e4dcf Make strn_to_xxx() functions consistent
All strn_to_xxx() functions should take a 'size_t len' parameter

All functions that take a 'const char **endp' parameter should
be named parse_xxx(), and should also have a 'ssize_t len' parameter
which can be -1 to indicate a null-terminated string
2015-03-28 04:53:09 +10:30
Andrew Bettison
015b4a0b07 Refactor and improve "journal append"
Distinguish between fatal and user-supplied-input errors, return
error descriptive text in a strbuf to allow dynamic content, apply
user-supplied field assgnments/deletions _after_ copying existing
manifest fields, use exit status 4 (invalid manifest) when applying
journal append to a non-journal or vice versa
2015-03-23 17:38:46 +10:30
Andrew Bettison
73a4191547 Improve manifest field copy/unset logging
Add __whence arguments to all field copy and unset functions
2015-03-23 17:32:57 +10:30
Andrew Bettison
655b94eb3c Fix failing 'rhizomeops' tests 2015-03-19 03:21:27 +10:30
Andrew Bettison
4d5cded047 Change return type of rhizome_fill_manifest()
To help improve diagnostic feedback through APIs
2015-03-16 22:40:08 +10:30
Andrew Bettison
016bb32b89 Add rhizome_manifest_overwrite() function 2015-03-16 22:36:38 +10:30
Andrew Bettison
a23c56626d Update some copyright notices 2014-11-20 00:01:12 +10:30
Andrew Bettison
e633c0fa0a "rhizome add file" manifest field=value varargs
Now a new manifest can be formed, and an existing manifest modified, by
giving "field=value" or "!field" command-line arguments at the end of
the "rhizome add file" command (after the <bsk> argument).

This makes it easier to script rhizome operations.
2014-11-19 13:26:50 +10:30
Andrew Bettison
c95807b002 Refactor manifest parsing
Consolidate tests for valid field names and values into
one place
2014-11-19 13:17:40 +10:30
Andrew Bettison
ff339011b4 Refactor Rhizome manifest parsing
Now the parsing code uses the set/del functions for all supported
manifest fields, so no duplicate logic any more
2014-11-19 10:49:13 +10:30
Andrew Bettison
2f1843858d Support unrecognised manifest fields
Refactor Rhizome manifest parsing in preparation for command-line
field assignment
2014-11-19 10:47:31 +10:30
Andrew Bettison
65d6005cbf Create authored bundle from bundle secret
Improve the logic in rhizome_fill_manifest() to handle the case where
the secret and author are both supplied, by setting the BK field so the
secret can be recovered from the manifest later
2014-11-14 13:11:44 +10:30
Jeremy Lakeman
851144ea0a Refactor keyring identity iteration 2014-10-31 15:19:59 +10:30
Andrew Bettison
cf43635789 Improve Rhizome HTTP RESTful interface
Add RHIZOME_BUNDLE_STATUS_READONLY enum value

Tighten up switch statements on bundle and payload status enums (no
default labels)

Rename some recently added enum entries

Return bundle status and payload status in HTTP responses

Add test for failing to decrypt a foreign encrypted bundle payload,
fix bug that caused an assertion failure

Add tests for fetching a non-existent manifest and fetching bundles
whose payload blob is not in the store
2014-07-10 06:29:51 +09:30
Andrew Bettison
61023287b1 Refactor Rhizome manifest validation
New function returns text describing the failed validation
2014-07-10 06:29:09 +09:30
Andrew Bettison
00dc3bf27e Fix Rhizome manifest parsing bug
If an over-large manifest was supplied, signature extraction went
into a tight loop
2014-04-29 15:08:02 +09:30
Andrew Bettison
dc588e964d Fix "name=/file" bug in Rhizome add
Improved Rhizome test scripts and added new assert_manifest_fields()
function to Rhizome test defs
2014-03-12 15:48:00 +10:30
Andrew Bettison
42e6168998 Improve Rhizome manifest setter functions
Remove interactions between rhizome_manifest_set_filesize()
and rhizome_manifest_set_filehash(), since any inconsistency
is caught by rhizome_manifest_validate() in any case
2013-12-30 16:30:43 +10:30
Andrew Bettison
dd5048bd43 rhizome_manifest_set_name_from_path()
Refactor rhizome_fill_manifest() to use it
2013-12-30 16:30:43 +10:30
Andrew Bettison
72040517e1 New enum rhizome_payload_status
Refactor a lot of Rhizome bundle storage code to use the new "enum
rhizome_payload_status" instead of mysterious int values to represent
the outcome of the operation.
2013-12-30 16:30:35 +10:30
Andrew Bettison
9ebef81a49 Formalise "rhizome add file" exit status
Formalise add-bundle result in "enum rhizome_bundle_status"

Rewrite rhizome_manifest_finalise(), rhizome_find_duplicate() and
rhizome_add_manifest() to return enum rhizome_bundle_status

New function rhizome_manifest_check_stored() that compares a manifest
with its stored counterpart and returns enum rhizome_bundle_status

Remove redundant rhizome_manifest_check_sanity(), consolidating all
manifest validation rules in rhizome_manifest_validate(), which now
checks the 'id' field is present, and that 'sender' and 'recipient' are
both present for MeshMS

Correct manifest finalisation logic: set the 'finalised' flag in
rhizome_manifest_validate(), not in rhizome_manifest_verify() (which
sets 'selfSigned'), and consistently clear 'finalised' flag in all
attribute setter functions

Remove manifest 'ttl' field and all references thereof (leaving unused
space in Rhizome BAR)

Rename some payload functions for clarity
2013-12-21 14:37:18 +10:30
Andrew Bettison
273c5f2689 Make rhizome_manifest_pack_variables() static 2013-12-20 11:29:26 +10:30
Andrew Bettison
8a1ce7f373 Use manifest 'has_id' consistently 2013-12-20 11:29:26 +10:30
Andrew Bettison
d52ba4c871 Remove rhizome_read_manifest_file()
Replace with memcpy() followed by rhizome_manifest_parse()
for memory buffers

Replace with rhizome_read_manifest_from_file() for files
2013-12-20 11:29:17 +10:30
Andrew Bettison
21328e2888 Improve read_whole_file()
Move into os.c, declare and document in os.h
2013-12-19 19:14:52 +10:30
Andrew Bettison
b5f7a088b2 Fix rhizome_fill_manifest() 'name' semantics
Only set 'name' field if the given pathname is valid

Do not set 'name' field if no pathname supplied
2013-12-18 18:02:46 +10:30
Andrew Bettison
075f9c7c27 Use uint64_t for Rhizome manifest version
Instead of int64_t.  Fixes some -Wsign-compare warnings.

Replace sqlite_exec_int64() with sqlite_exec_uint64().

Also store rowid as uint64_t, and use 0 not -1 to indicate
unset.
2013-12-11 11:11:34 +10:30
Andrew Bettison
476a538ef7 Fix -Wsign-compare warnings in keyring: unsigned cn,in,kp 2013-12-10 16:34:35 +10:30
Andrew Bettison
68555ce241 Add some Rhizome manifest DEBUG that was missing
Revealed by -Wunused-parameter
2013-12-09 18:21:32 +10:30
Jeremy Lakeman
b63f1f0114 Fix formatting issues 2013-12-06 12:10:08 +10:30
Andrew Bettison
2bb0d2047a Enforce valid Rhizome 'service' and 'name' strings 2013-12-02 11:12:31 +10:30
Andrew Bettison
8db5f9c14a Merge branch 'anyservice' into 'development'
Allows any valid "service" manifest field in the "rhizome add file"
command

Many improvements in Rhizome manifest parsing; stricter manifest syntax
rules (no comment or blank lines, field names must be alphanumeric
identifiers), faster preliminary manifest inspection when receiving
manifest advertisements or syncing manifests

The 'development' branch introduces "struct socket_address" which
coincidentally fixed the recently encountered Linux kernel 3.12
recvmsg(2) EINVAL problem, so that 'rhizomeprotocol' tests which fail on
the 'anyservice' branch will pass after this merge
2013-12-01 05:44:01 +10:30
Andrew Bettison
0dd1b302b5 Rewrite Rhizome manifest parsing
Move validation checks into new function rhizome_manifest_validate()

Remove rhizome_manifest 'errors' field

Replace rhizome_manifest 'warnings' with 'malformed'

Replace rhizome_manifest 'manifest_bytes' with 'manifest_body_bytes' and
refactor to use 'manifest_all_bytes' in all manifest i/o

Refactor rhizome_manifest_verify() and reverse sense of return value to
match rhizome_manifest_validate()

New function rhizome_manifest_inspect() -- lightweight manifest parser
used when receiving Rhizome advertisements

New 'rhizomeops' test case for invalid manifest "service" field values,
now passes
2013-12-01 05:13:34 +10:30
Jeremy Lakeman
99d5d9fa1f Refactor dataformats into separate header 2013-11-27 13:11:42 +10:30
Andrew Bettison
973bb9c897 Merge branch 'naf4' into 'development' 2013-11-13 17:59:01 +10:30
Jeremy Lakeman
4434b1b65d Create keyring.h and move definitions there 2013-11-13 11:55:12 +10:30
Andrew Bettison
13634f8748 Add ROWID field to struct rhizome_manifest
New ".rowid" output field from rhizome add, import, extract, export
operations.  (Also added missing ".inserttime" and "date" fields to
some operations.)

Use new "rhizome add file" .rowid output field to check output of of
/restful/rhizome/bundlelist.json
2013-11-11 18:18:08 +10:30
Andrew Bettison
45442d3eb4 Rewrite bundle author authentication
Replaced 'int has_author' manifest element with new 'enum authorship'
element to record the result of author authentication, to avoid
repeating expensive crypto operations.

Separated the handling of bundle secret arguments from author lookup and
authentication.  The new rhizome_apply_bundle_secret(m,bsk) is now
called at the top level to set the manifest secret key (if it
validates), and thereafter there is no need to pass the 'bsk' argument
to any other functions, as they can simply check the 'haveSecret' field
of the manifest.

Removed rhizome_extract_privatekey() which combined author lookup and
bundle secret validation, and replaced it with functions that only deal
with the author: rhizome_lookup_author() and rhizome_authenticate_author().

Renamed other functions to make their purpose and effect clearer.

Formalised the semantics of only storing AUTHENTICATED author SIDs in
the 'author' column of the MANIFESTS table, which necessitated a change
to a 'rhizomeops' test case: when adding a file using a BK-less
manifest, the author column is set to null, so the Rhizome list output
does not show the bundle as ".fromhere" and does not give an author for
that bundle.
2013-11-06 23:58:17 +10:30
Andrew Bettison
3aa24f7407 Refactor "rhizome list" main loop
Preparing for re-use in HTTP /restful/rhizome/bundlelist.json
2013-11-04 23:47:09 +10:30
Andrew Bettison
bacba19dc1 Refactor manifest: specific setter functions
Replace generic rhizome_manifest_set() and rhizome_manifest_set_ll()
with per-field setter functions, eg, rhizome_manifest_set_filesize().
Struct rhizome_manifest elements for all known fields, to replace the
use of rhizome_manifest_get() and rhizome_manifest_get_ll() everywhere:
sender, recipient, service, name, date, bundle_key.

Add boolean validity flags for binary blob types, to avoid having to compare
with many bytes of all-zero to detect presence, eg, has_sender, has_recipient,
has_author, has_bundle_key.  These maintained by the setter functions.

Rename existing manifest struct elements to be the same as their field
names: fileLength -> filesize, journalTail -> tail.

More use of unsigned int, size_t and uint64_t for payload sizes, offsets, byte
counts, etc. especially in rhizome_store.c and meshms.c.  More uniform use of
size_t to dimension memory buffers.  Fix some printf(3) style format strings
for 64-bit correctness on 32-bit systems.  Use new constant RHIZOME_SIZE_UNSET
instead of -1 to indicate unknown dimension, and explicitly assert its absence
before comparisons and arithmetic, for safety.

Replace some 'int' loop variables with 'unsigned' where appropriate.

Fix bugs discovered in MeshMS bundle private/public key generation and
bundle secret key handling for export/extract commands.

Instrument the first MeshMS test case to aid debugging.

New debug config flag: debug.manifest logs all modifications to all manifest
fields by setter functions.

Rename debug config flag: debug.rhizome_bind -> debug.rhizome_sql_bind.
2013-11-04 19:17:49 +10:30
Andrew Bettison
c2b78f7b29 Refactor manifest: replace int with unsigned 2013-10-30 14:14:38 +10:30
Jeremy Lakeman
c7fa395968 Fix format patterns for 64-bit compilation 2013-10-14 14:45:43 +10:30
Andrew Bettison
0437e4adbb Add writev_all() function
Use it in rhizome_write_manifest_file() instead of two separate write(2)
system calls or buffered stdio.
2013-10-12 03:54:18 +10:30
Andrew Bettison
97cbebc91e Issue #11: improve types in file i/o functions
Use open(2)/read(2)/write(2) instead of fopen(3)/fread(3)/fwrite(3) in
several places to avoid unnecessary buffering

Fix a bug in Rhizome HTTP add's handling of unconfigured manifest
template file

Improve some debug and error logging for file i/o
2013-10-11 16:24:51 +10:30
Andrew Bettison
49729cc768 Remove default service=file for new manifests 2013-10-11 15:39:10 +10:30
Andrew Bettison
974c7a56a0 Issue #11: Use rhizome_filehash_t everywhere 2013-10-11 15:35:27 +10:30
Andrew Bettison
a95ef79139 Use sid_t everywhere
Remove stowSid() at last

Change API of tohex() and strbuf_tohex(), to pass string length instead of
binary byte count.  This allows odd numbers of hex digits to be produced.

Remove alloca_tohex_sid(); replace with alloca_tohex_sid_t()

New alloca_tohex_sid_t_trunc() macro
2013-10-09 18:54:21 +10:30