Commit Graph

221 Commits

Author SHA1 Message Date
Andrew Bettison
bacba19dc1 Refactor manifest: specific setter functions
Replace generic rhizome_manifest_set() and rhizome_manifest_set_ll()
with per-field setter functions, eg, rhizome_manifest_set_filesize().
Struct rhizome_manifest elements for all known fields, to replace the
use of rhizome_manifest_get() and rhizome_manifest_get_ll() everywhere:
sender, recipient, service, name, date, bundle_key.

Add boolean validity flags for binary blob types, to avoid having to compare
with many bytes of all-zero to detect presence, eg, has_sender, has_recipient,
has_author, has_bundle_key.  These maintained by the setter functions.

Rename existing manifest struct elements to be the same as their field
names: fileLength -> filesize, journalTail -> tail.

More use of unsigned int, size_t and uint64_t for payload sizes, offsets, byte
counts, etc. especially in rhizome_store.c and meshms.c.  More uniform use of
size_t to dimension memory buffers.  Fix some printf(3) style format strings
for 64-bit correctness on 32-bit systems.  Use new constant RHIZOME_SIZE_UNSET
instead of -1 to indicate unknown dimension, and explicitly assert its absence
before comparisons and arithmetic, for safety.

Replace some 'int' loop variables with 'unsigned' where appropriate.

Fix bugs discovered in MeshMS bundle private/public key generation and
bundle secret key handling for export/extract commands.

Instrument the first MeshMS test case to aid debugging.

New debug config flag: debug.manifest logs all modifications to all manifest
fields by setter functions.

Rename debug config flag: debug.rhizome_bind -> debug.rhizome_sql_bind.
2013-11-04 19:17:49 +10:30
Andrew Bettison
c2b78f7b29 Refactor manifest: replace int with unsigned 2013-10-30 14:14:38 +10:30
Andrew Bettison
2a9329c0c8 Fix bugs in new HTTP server MIME body parsing code
Fixes 'rhizomeprotocol' test 24 HttpImport.  Five tests still fail.
2013-10-27 13:19:23 +10:30
Andrew Bettison
0397a47753 Fix some test failures in new HTTP server code 2013-10-26 20:28:47 +10:30
Andrew Bettison
6488f7ad65 Fix basic bugs in new HTTP server code
All 'rhizomeprotocol' HTTP tests still fail
2013-10-25 17:38:51 +10:30
Andrew Bettison
00cf61721d Rename http_header_complete() to is_http_header_complete() 2013-10-16 11:22:02 +10:30
Andrew Bettison
97cbebc91e Issue #11: improve types in file i/o functions
Use open(2)/read(2)/write(2) instead of fopen(3)/fread(3)/fwrite(3) in
several places to avoid unnecessary buffering

Fix a bug in Rhizome HTTP add's handling of unconfigured manifest
template file

Improve some debug and error logging for file i/o
2013-10-11 16:24:51 +10:30
Andrew Bettison
974c7a56a0 Issue #11: Use rhizome_filehash_t everywhere 2013-10-11 15:35:27 +10:30
Andrew Bettison
a95ef79139 Use sid_t everywhere
Remove stowSid() at last

Change API of tohex() and strbuf_tohex(), to pass string length instead of
binary byte count.  This allows odd numbers of hex digits to be produced.

Remove alloca_tohex_sid(); replace with alloca_tohex_sid_t()

New alloca_tohex_sid_t_trunc() macro
2013-10-09 18:54:21 +10:30
Jeremy Lakeman
60e3f5a3fc Fix printf formats for 64bit compilation 2013-10-09 10:31:52 +10:30
Andrew Bettison
ffafae86b1 Issue #11: replace long long with int64_t 2013-10-07 05:54:46 +10:30
Andrew Bettison
f5e7db089c Fix bugs revealed by test failures
Introduced by the recent change to rhizome_bid_t everywhere
2013-10-04 03:12:52 +09:30
Andrew Bettison
ab31420faf rhizome_bid_t
Consistent type for internal binary representation of Rhizome Bundle ID
(aka Manifest ID)
2013-10-03 23:16:45 +09:30
Andrew Bettison
7e3a552011 Issue #69: refactor all Rhizome SQL queries to use binding
Remove all uses of sprintf(3) and its ilk to form SQL query commands.
Use the new sqlite_bind() and sqlite_prepare_bind() functions instead.
2013-10-03 15:15:30 +09:30
Jeremy Lakeman
79bdb5f993 Add more html debugging for rhizome sync and fetch queues 2013-09-24 14:34:51 +09:30
Jeremy Lakeman
57577abca6 Merge branch 'mavlink' into development 2013-09-20 14:58:29 +09:30
Jeremy Lakeman
9f1407043a Reduce rhizome MDP debug messages 2013-09-20 14:41:49 +09:30
Andrew Bettison
2b3119b49b Issue #20: merge branch 'sockets' into 'development'
Daniel's improvements to the local file/abstract socket code, with many
improvements to bring it up to date.
2013-09-19 04:30:14 +09:30
Jeremy Lakeman
2a5ba97b48 Add rhizome test over fakeradio 2013-09-13 14:02:35 +09:30
Jeremy Lakeman
8accabfcac Always store payloads with a temporary id 2013-08-27 16:15:51 +09:30
Jeremy Lakeman
f2e7391672 Fix format warnings on 64bit machines 2013-08-26 11:54:46 +09:30
gardners
65cc17b7e1 add bidVersion reporting to rhizome_tx debug. 2013-08-25 22:32:13 +09:30
gardners
39785313f3 fix rhizome/mdp debugging for when slot is NULL. 2013-08-25 21:57:41 +09:30
gardners
24f4b789f3 add some more rhizome/mdp debugging. 2013-08-25 21:49:31 +09:30
gardners
ed9971e8c1 increase Rhizome over MDP debug output to track down inability of
MEs to do rhizome over mdp on ad-hoc wifi
2013-08-25 21:30:17 +09:30
gardners
53c9a406e1 don't give up on an HTTP rhizome transfer on an empty read if
the error is EAGAIN.
2013-08-25 17:35:34 +09:30
gardners
53ce40a053 fix http->mdp rhizome fetch failover regression. 2013-08-25 16:38:24 +09:30
Jeremy Lakeman
643b21d6a7 Fetch file payloads if they are missing, even if we already have the manifest 2013-08-22 15:14:21 +09:30
Jeremy Lakeman
1468d2deb0 Pre-emptively send small payloads via MDP 2013-08-21 09:21:40 +09:30
Jeremy Lakeman
771cb4151b Fetch journal range via HTTP 2013-08-15 17:00:43 +09:30
Jeremy Lakeman
0b40ef655c Log format warnings for strbuf_sprintf 2013-08-15 16:31:26 +09:30
Jeremy Lakeman
e322114850 Only transfer new journal content via MDP 2013-08-15 14:36:56 +09:30
Jeremy Lakeman
68bf04f69d Change manifest service to MeshMS2 2013-08-01 11:37:35 +09:30
Jeremy Lakeman
15ad333195 Implicitly buffer all writes to database blobs 2013-07-18 17:04:12 +09:30
Jeremy Lakeman
a4bf4edbfc Write out of order packets with MDP transport
- add random order writing to rhizome store API
- remove rhizome flush, force all callers to provide buffer
2013-07-17 16:57:59 +09:30
Jeremy Lakeman
ea2e55c62c Simplify state code for swapping to MDP transfer 2013-07-16 15:38:00 +09:30
Jeremy Lakeman
3855ff35c4 Clean up string formatting for 64bit compilation 2013-07-15 09:59:24 +09:30
Jeremy Lakeman
8ac359e550 Use consistent integer types for 64bit compilation 2013-07-13 14:47:06 +09:30
Jeremy Lakeman
add895fd97 Close manifest leak 2013-07-11 13:52:38 +09:30
Jeremy Lakeman
a1ac7febe2 Add rhizome fetch status html page 2013-07-11 12:32:06 +09:30
Jeremy Lakeman
8249f65f8f Don't create unused import folder 2013-07-03 10:54:05 +09:30
Jeremy Lakeman
96f6453a43 Don't attempt to read unless the descriptor is ready 2013-06-20 11:18:20 +09:30
Jeremy Lakeman
ceb3c17b70 Perform cheap in-memory tests first, before starting fetch 2013-06-20 11:10:02 +09:30
Jeremy Lakeman
3e57add9ca Fix, initialise read / write handles so we don't close fd=0 2013-06-19 12:05:56 +09:30
Jeremy Lakeman
96c0889f9a Don't ask for manifests if we don't have room in transfer queues 2013-04-11 15:24:41 +09:30
Andrew Bettison
157fe3f1d6 Fix Solaris SEGV bug: null pointer passed to sprintf() 2013-03-06 12:56:41 +10:30
Jeremy Lakeman
4cdd3da77f Add config option to disable fetching via rhizome 2013-03-05 10:56:20 +10:30
Jeremy Lakeman
73786bcb5d Refactor all rhizome reading and writing
- The API in rhizome_store.c is used for all reading and writing
- external storage is now usable for all transport options
2013-02-20 16:18:59 +10:30
gardners
5a3dec28d9 make rhizome over mdp idle timeout configurable at runtime. 2013-02-20 16:18:57 +10:30
gardners
a23612428b make rhizome over mdp block size configurable.
1K seems too big to be reliable on our prototype UHF radio links,
as CRCs fail almost all the time. #48
2013-02-20 16:18:57 +10:30