Commit Graph

60 Commits

Author SHA1 Message Date
gardners
2252fdcaa7 created tool for automatically finding IN()s without matching
OUT()s or where return() is used instead of RETURN().
Added OUT() to end of all functions using IN() that lacked it to
make it easier to statically analyse this invariant.
Fixed several return instead of RETURNs detected through use of
this tool. #49
2013-02-20 16:18:56 +10:30
gardners
d593b569c1 improve reporting of broken manifests, and make conditional on
debug.rejecteddata
2013-02-20 15:36:30 +10:30
Jeremy Lakeman
b25ff45438 Importing the same bundle twice returns duplicate flag 2013-01-15 10:32:48 +10:30
Jeremy Lakeman
5a82bfcf31 Add support for importing and exporting a file with an appended manifest 2013-01-11 14:30:52 +10:30
Jeremy Lakeman
28a05baa54 Enable encryption of rhizome payloads 2013-01-03 16:12:24 +10:30
Jeremy Lakeman
e26522bdc6 Minor improvements to command line time measurements 2013-01-03 14:16:33 +10:30
Jeremy Lakeman
dc43b83c80 Tidy up rhizome secret handling to improve reusability 2013-01-03 12:14:13 +10:30
Jeremy Lakeman
cf6ecbc57d Generate version id before reading the file contents 2013-01-03 11:22:01 +10:30
Jeremy Lakeman
2967d1e00f Don't check for duplicates if an existing id has been supplied 2013-01-03 11:18:30 +10:30
Jeremy Lakeman
6c7ba438a3 Refactor rhizome extract manifest into more useful methods 2013-01-02 11:15:03 +10:30
Jeremy Lakeman
a492c05d64 Re-add database cleanup 2012-12-31 15:09:12 +10:30
Jeremy Lakeman
f64de66b34 Refactor add via HTTP to be equivalent to command line add 2012-12-20 15:36:07 +10:30
Andrew Bettison
5985df751d Overhaul debug flags
Replace debugflags_t and DEBUG_XXX bit masks with config schema "debug.xxx"
entries.

No more support for "debug.all".
2012-12-11 15:59:46 +10:30
Andrew Bettison
65ea612e27 Fix bugs revealed by Rhizome stress test
Overhauled the file fetch queue logic in rhizome_fetch.c.

Now the 'rhizomeprotocol' stress test passes in approximately 5 minutes on my
2009-vintage Dell laptop.

Added a call to rhizome_enqueue_suggestions() in rhizome_fetch_close() so that
a new Rhizome GET request is sent as soon as a fetch slot becomes free, instead
of waiting for the (default 5 second) timer to trigger the next GET.
2012-10-18 17:30:22 +10:30
Andrew Bettison
259b9a9d24 Logging/debug improvements: __WHENCE__, __whence
Introduce __WHENCE__ macro and a block comment in log.h explaining it.

In "primitive" kinds of functions, rename 'whence' arguments to '__whence' and
use WHYF(), WARNF(), DEBUGF() macros instead of calling logMessage() directly.
2012-10-16 17:00:03 +10:30
Andrew Bettison
3f1f495d68 Issue #17, overhaul manifest author crypto logic
Add test case for new feature of the "rhizome add" command: if the author SID
is not specified (empty arg) then it searches the keyring for the author.

Removed "authorSid" argument from several functions that also take a struct
rhizome_manifest * arg, since the author, if known, is now supplied in the
struct.

Improve return value handling and refactored some rhizome crypto code.
2012-10-11 17:58:24 +10:30
Andrew Bettison
89343c69cd Merge branch 'rhizomedirect' into 'master' 2012-10-05 17:45:30 +09:30
Andrew Bettison
cc9cb8d827 Issue #9, fix old 'rhizomeprotocol' test failures
Now only the five new Rhizome Direct tests fail.
2012-10-02 16:32:48 +09:30
Jeremy Lakeman
ac3864ff20 Reuse command line parsing framework with monitor interface 2012-09-25 13:31:34 +09:30
Andrew Bettison
27a0a6eeb5 Introduce struct __sourceloc
Replaces (const char *file, unsigned int line, const char *function) arguments
to all logging functions, simplifies malloc/free tracking code in
overlay_buffer.c and Rhizome manifest alloc/free tracking in rhizome_bundle.c.

Use __HERE__ macro instead of (__FILE__, __LINE__, __FUNCTION__) everywhere.

Special __NOWHERE__ macro is equivalent to (NULL, 0, NULL).

Declare net.c functions in new "net.h" header, so log.c doesn't have to pull
in the entire "serval.h" just to use write_str().

Facilitates progress on issue #2.
2012-08-23 12:31:07 +09:30
Andrew Bettison
0e5c5e0e98 More work on dnahelper, almost done
Servald starts DNA helper, receives startup ACK, sends requests, receives
responses, handles malformed helper responses, echoes dnahelper stderr lines to
log, sends MDP reply packet, waits for dead helper process, all asynchronously.
Shuts down helper process during servald shutdown.

Remaining issues:
 - Does not impose a timeout on helper responses.
 - Only the first URI is reported by the "dna lookup" command.
2012-07-20 18:17:43 +09:30
Andrew Bettison
b50943d09c Fix handling of empty payload (filesize = 0)
Do not add 'filehash' var to manifest if filesize=0
Do not accept 'filehash' var when parsing manifest with filesize=0
When responding to a new rhizome advertisement, do not try to HTTP
request a payload if filesize=0, just import the manifest directly
Various operations, eg "rhizome file add", do not report 'filehash'
fields where 'filesize' is zero
Do not delete rows from MANIFESTS table which have empty filehash
Various related bug fixes
2012-07-16 18:40:30 +09:30
Andrew Bettison
49ee4d13f0 Re-write manifest parsing, improve rhizome_fetch.c logging
Rhizome manifest parser now parses and validates all known fields, informs
about unsupported fields, and unpacks fields into relevant struct manifest
elements where appropriate.  Is also stricter about whitespace.

Rhizome fetch code now logs debug messages if DEBUG_RHIZOME_RX bit is on.
2012-07-12 16:39:01 +09:30
Jeremy Lakeman
495de9e0ec Add more profiling instrumentation 2012-07-03 15:50:04 +09:30
gardners
aab400164f push manifest verification to as late as possible to avoid
unwarranted 400ms CPU hits for every manifest seen.
2012-06-26 16:54:40 +09:30
Andrew Bettison
6a941cbc4d Convert two #warnings to //TODO comments 2012-06-25 15:46:22 +09:30
Andrew Bettison
208b9c15fd Factor out rhizome_hex_to_bytes() and chartonybl() 2012-06-25 14:51:21 +09:30
Andrew Bettison
7a71a521f8 Pass author SID parameters in binary, not hex 2012-06-08 15:27:15 +09:30
Andrew Bettison
b14db5c28b Improve "rhizome add file" for empty payloads
Accept the empty string for a payload pathname to mean a zero-length
payload.
2012-06-05 15:45:53 +09:30
gardners
884e427fbb work towards making manifests only accept UPPERCASE hex strings
(mixing them between versions of manifests causes confusion).
2012-05-27 17:56:11 +02:00
gardners
596e25c16e cleanups for extracting files from rhizome. 2012-05-26 11:36:18 +02:00
gardners
5c40edb55d Fixed bugs with importing files using user-supplied manifest
(was not decoding BKs)
2012-05-26 11:36:18 +02:00
gardners
c2dfefe764 fixed bug in duplicate detection 2012-05-26 11:36:18 +02:00
gardners
04cc05d6e1 refactored manifest verification into a separate function
that is called only when needed, and marks a manifest as finalised
if the verifcation fails.  reading a manifest now never sets
finalised flag, as either _finalise() or _verify() must be called.
2012-05-26 11:36:18 +02:00
gardners
b0dca7e03f merged in latest changes from team. 2012-05-26 11:36:18 +02:00
Andrew Bettison
bea9188cd5 Fix another manifest leak
Add manifest alloc/free debug logging to help, 'debug.manifests' option
2012-05-25 17:01:56 +09:30
Andrew Bettison
49aec4d331 Improve rhizome manifest debugging
Move rhizome_new_manifest() out of rhizome_read_manifest_file() so that the
out-of-manifest report shows the names of the functions where the manifests
were really allocated.
2012-05-25 15:38:13 +09:30
Andrew Bettison
0ec3feaa29 Expunge setReason()
Replace with WHY() or WHYF() everywhere

Improve logMessage() to handle file/line/function printing internally

Mainly so that log messages from manifest alloc/free in rhizome_bundle.c
can make it into the Android log
2012-05-24 17:11:55 +09:30
Andrew Bettison
fa26b53294 Fix SEGV bug
rhizome_write_manifest_file() was not checking for NULL fopen() result
Standardise rhizome "import" directory pathname handling
2012-05-24 11:28:32 +09:30
Andrew Bettison
1af9125392 All rhizome database keys are uppercase hex
FILES.id, MANIFESTS.id, FILEMANIFESTS.fileid, FILEMANIFESTS.manifestid
Named constants for hex and binary ID sizes
2012-05-23 16:04:00 +09:30
gardners
0abc7ff96e added code to create and maintain priority download list for
rhizome, so that we get things in a sensible order.
2012-05-22 15:57:18 +09:30
gardners
3866352340 further improvements to supressing broken manifest time wasting
in rhizome.
2012-05-22 14:20:01 +09:30
Andrew Bettison
f4293e8ab3 Add "rhizome hash file" command
Also fixed a file descriptor leak in rhizome_hash_file() -- missing fclose()
2012-05-21 14:42:53 +09:30
Andrew Bettison
ddd17db1c4 Make "rhizome add file" work for MeshMS 2012-05-21 00:09:14 +09:30
gardners
0eafc34970 BK encoding and decoding complete, but untested. 2012-05-15 20:04:41 +09:30
gardners
525d3c4154 substantial work towards implementing BK field in rhizome manifests. 2012-05-15 17:24:25 +09:30
gardners
17370b3464 added comment for starting work on handling BK fields in manifests. 2012-05-15 15:44:49 +09:30
gardners
ae44826d7e improved debug message when rejecting an unsigned or incorrectly
signed manifest.
2012-05-12 07:36:39 +09:30
Andrew Bettison
e746557a08 Fix bug in overlay_gettime_ms()
The sequence start time was only getting set in overlay_interface_init(), which
was not always called, or was called after overlay_gettime_ms() had already
been used.

Added FATALF(), FATAL() and FATAL_perror() macros.

Removed a bunch of debug statements no longer needed in monitor.c and server.c.
2012-05-11 10:38:46 +09:30
Andrew Bettison
7ba15ccdd7 Add "rhizome extract file" command with tests 2012-05-02 17:57:35 +09:30