Commit Graph

223 Commits

Author SHA1 Message Date
Jeremy Lakeman
56e91518b8 Fix fetch insert assertion 2013-11-21 16:16:12 +10:30
Andrew Bettison
45442d3eb4 Rewrite bundle author authentication
Replaced 'int has_author' manifest element with new 'enum authorship'
element to record the result of author authentication, to avoid
repeating expensive crypto operations.

Separated the handling of bundle secret arguments from author lookup and
authentication.  The new rhizome_apply_bundle_secret(m,bsk) is now
called at the top level to set the manifest secret key (if it
validates), and thereafter there is no need to pass the 'bsk' argument
to any other functions, as they can simply check the 'haveSecret' field
of the manifest.

Removed rhizome_extract_privatekey() which combined author lookup and
bundle secret validation, and replaced it with functions that only deal
with the author: rhizome_lookup_author() and rhizome_authenticate_author().

Renamed other functions to make their purpose and effect clearer.

Formalised the semantics of only storing AUTHENTICATED author SIDs in
the 'author' column of the MANIFESTS table, which necessitated a change
to a 'rhizomeops' test case: when adding a file using a BK-less
manifest, the author column is set to null, so the Rhizome list output
does not show the bundle as ".fromhere" and does not give an author for
that bundle.
2013-11-06 23:58:17 +10:30
Andrew Bettison
bacba19dc1 Refactor manifest: specific setter functions
Replace generic rhizome_manifest_set() and rhizome_manifest_set_ll()
with per-field setter functions, eg, rhizome_manifest_set_filesize().
Struct rhizome_manifest elements for all known fields, to replace the
use of rhizome_manifest_get() and rhizome_manifest_get_ll() everywhere:
sender, recipient, service, name, date, bundle_key.

Add boolean validity flags for binary blob types, to avoid having to compare
with many bytes of all-zero to detect presence, eg, has_sender, has_recipient,
has_author, has_bundle_key.  These maintained by the setter functions.

Rename existing manifest struct elements to be the same as their field
names: fileLength -> filesize, journalTail -> tail.

More use of unsigned int, size_t and uint64_t for payload sizes, offsets, byte
counts, etc. especially in rhizome_store.c and meshms.c.  More uniform use of
size_t to dimension memory buffers.  Fix some printf(3) style format strings
for 64-bit correctness on 32-bit systems.  Use new constant RHIZOME_SIZE_UNSET
instead of -1 to indicate unknown dimension, and explicitly assert its absence
before comparisons and arithmetic, for safety.

Replace some 'int' loop variables with 'unsigned' where appropriate.

Fix bugs discovered in MeshMS bundle private/public key generation and
bundle secret key handling for export/extract commands.

Instrument the first MeshMS test case to aid debugging.

New debug config flag: debug.manifest logs all modifications to all manifest
fields by setter functions.

Rename debug config flag: debug.rhizome_bind -> debug.rhizome_sql_bind.
2013-11-04 19:17:49 +10:30
Andrew Bettison
c2b78f7b29 Refactor manifest: replace int with unsigned 2013-10-30 14:14:38 +10:30
Andrew Bettison
2a9329c0c8 Fix bugs in new HTTP server MIME body parsing code
Fixes 'rhizomeprotocol' test 24 HttpImport.  Five tests still fail.
2013-10-27 13:19:23 +10:30
Andrew Bettison
0397a47753 Fix some test failures in new HTTP server code 2013-10-26 20:28:47 +10:30
Andrew Bettison
6488f7ad65 Fix basic bugs in new HTTP server code
All 'rhizomeprotocol' HTTP tests still fail
2013-10-25 17:38:51 +10:30
Andrew Bettison
00cf61721d Rename http_header_complete() to is_http_header_complete() 2013-10-16 11:22:02 +10:30
Andrew Bettison
97cbebc91e Issue #11: improve types in file i/o functions
Use open(2)/read(2)/write(2) instead of fopen(3)/fread(3)/fwrite(3) in
several places to avoid unnecessary buffering

Fix a bug in Rhizome HTTP add's handling of unconfigured manifest
template file

Improve some debug and error logging for file i/o
2013-10-11 16:24:51 +10:30
Andrew Bettison
974c7a56a0 Issue #11: Use rhizome_filehash_t everywhere 2013-10-11 15:35:27 +10:30
Andrew Bettison
a95ef79139 Use sid_t everywhere
Remove stowSid() at last

Change API of tohex() and strbuf_tohex(), to pass string length instead of
binary byte count.  This allows odd numbers of hex digits to be produced.

Remove alloca_tohex_sid(); replace with alloca_tohex_sid_t()

New alloca_tohex_sid_t_trunc() macro
2013-10-09 18:54:21 +10:30
Jeremy Lakeman
60e3f5a3fc Fix printf formats for 64bit compilation 2013-10-09 10:31:52 +10:30
Andrew Bettison
ffafae86b1 Issue #11: replace long long with int64_t 2013-10-07 05:54:46 +10:30
Andrew Bettison
f5e7db089c Fix bugs revealed by test failures
Introduced by the recent change to rhizome_bid_t everywhere
2013-10-04 03:12:52 +09:30
Andrew Bettison
ab31420faf rhizome_bid_t
Consistent type for internal binary representation of Rhizome Bundle ID
(aka Manifest ID)
2013-10-03 23:16:45 +09:30
Andrew Bettison
7e3a552011 Issue #69: refactor all Rhizome SQL queries to use binding
Remove all uses of sprintf(3) and its ilk to form SQL query commands.
Use the new sqlite_bind() and sqlite_prepare_bind() functions instead.
2013-10-03 15:15:30 +09:30
Jeremy Lakeman
79bdb5f993 Add more html debugging for rhizome sync and fetch queues 2013-09-24 14:34:51 +09:30
Jeremy Lakeman
57577abca6 Merge branch 'mavlink' into development 2013-09-20 14:58:29 +09:30
Jeremy Lakeman
9f1407043a Reduce rhizome MDP debug messages 2013-09-20 14:41:49 +09:30
Andrew Bettison
2b3119b49b Issue #20: merge branch 'sockets' into 'development'
Daniel's improvements to the local file/abstract socket code, with many
improvements to bring it up to date.
2013-09-19 04:30:14 +09:30
Jeremy Lakeman
2a5ba97b48 Add rhizome test over fakeradio 2013-09-13 14:02:35 +09:30
Jeremy Lakeman
8accabfcac Always store payloads with a temporary id 2013-08-27 16:15:51 +09:30
Jeremy Lakeman
f2e7391672 Fix format warnings on 64bit machines 2013-08-26 11:54:46 +09:30
gardners
65cc17b7e1 add bidVersion reporting to rhizome_tx debug. 2013-08-25 22:32:13 +09:30
gardners
39785313f3 fix rhizome/mdp debugging for when slot is NULL. 2013-08-25 21:57:41 +09:30
gardners
24f4b789f3 add some more rhizome/mdp debugging. 2013-08-25 21:49:31 +09:30
gardners
ed9971e8c1 increase Rhizome over MDP debug output to track down inability of
MEs to do rhizome over mdp on ad-hoc wifi
2013-08-25 21:30:17 +09:30
gardners
53c9a406e1 don't give up on an HTTP rhizome transfer on an empty read if
the error is EAGAIN.
2013-08-25 17:35:34 +09:30
gardners
53ce40a053 fix http->mdp rhizome fetch failover regression. 2013-08-25 16:38:24 +09:30
Jeremy Lakeman
643b21d6a7 Fetch file payloads if they are missing, even if we already have the manifest 2013-08-22 15:14:21 +09:30
Jeremy Lakeman
1468d2deb0 Pre-emptively send small payloads via MDP 2013-08-21 09:21:40 +09:30
Jeremy Lakeman
771cb4151b Fetch journal range via HTTP 2013-08-15 17:00:43 +09:30
Jeremy Lakeman
0b40ef655c Log format warnings for strbuf_sprintf 2013-08-15 16:31:26 +09:30
Jeremy Lakeman
e322114850 Only transfer new journal content via MDP 2013-08-15 14:36:56 +09:30
Jeremy Lakeman
68bf04f69d Change manifest service to MeshMS2 2013-08-01 11:37:35 +09:30
Jeremy Lakeman
15ad333195 Implicitly buffer all writes to database blobs 2013-07-18 17:04:12 +09:30
Jeremy Lakeman
a4bf4edbfc Write out of order packets with MDP transport
- add random order writing to rhizome store API
- remove rhizome flush, force all callers to provide buffer
2013-07-17 16:57:59 +09:30
Jeremy Lakeman
ea2e55c62c Simplify state code for swapping to MDP transfer 2013-07-16 15:38:00 +09:30
Jeremy Lakeman
3855ff35c4 Clean up string formatting for 64bit compilation 2013-07-15 09:59:24 +09:30
Jeremy Lakeman
8ac359e550 Use consistent integer types for 64bit compilation 2013-07-13 14:47:06 +09:30
Jeremy Lakeman
add895fd97 Close manifest leak 2013-07-11 13:52:38 +09:30
Jeremy Lakeman
a1ac7febe2 Add rhizome fetch status html page 2013-07-11 12:32:06 +09:30
Jeremy Lakeman
8249f65f8f Don't create unused import folder 2013-07-03 10:54:05 +09:30
Jeremy Lakeman
96f6453a43 Don't attempt to read unless the descriptor is ready 2013-06-20 11:18:20 +09:30
Jeremy Lakeman
ceb3c17b70 Perform cheap in-memory tests first, before starting fetch 2013-06-20 11:10:02 +09:30
Jeremy Lakeman
3e57add9ca Fix, initialise read / write handles so we don't close fd=0 2013-06-19 12:05:56 +09:30
Jeremy Lakeman
96c0889f9a Don't ask for manifests if we don't have room in transfer queues 2013-04-11 15:24:41 +09:30
Andrew Bettison
157fe3f1d6 Fix Solaris SEGV bug: null pointer passed to sprintf() 2013-03-06 12:56:41 +10:30
Jeremy Lakeman
4cdd3da77f Add config option to disable fetching via rhizome 2013-03-05 10:56:20 +10:30
Jeremy Lakeman
73786bcb5d Refactor all rhizome reading and writing
- The API in rhizome_store.c is used for all reading and writing
- external storage is now usable for all transport options
2013-02-20 16:18:59 +10:30