Commit Graph

132 Commits

Author SHA1 Message Date
Andrew Bettison
d4320f20da Merge branch 'naf4' into 'development'
New HTTP RESTful Rhizome API, plus tests, and lots of Rhizome
improvements and refactoring
2013-12-30 18:14:34 +10:30
Andrew Bettison
9ebef81a49 Formalise "rhizome add file" exit status
Formalise add-bundle result in "enum rhizome_bundle_status"

Rewrite rhizome_manifest_finalise(), rhizome_find_duplicate() and
rhizome_add_manifest() to return enum rhizome_bundle_status

New function rhizome_manifest_check_stored() that compares a manifest
with its stored counterpart and returns enum rhizome_bundle_status

Remove redundant rhizome_manifest_check_sanity(), consolidating all
manifest validation rules in rhizome_manifest_validate(), which now
checks the 'id' field is present, and that 'sender' and 'recipient' are
both present for MeshMS

Correct manifest finalisation logic: set the 'finalised' flag in
rhizome_manifest_validate(), not in rhizome_manifest_verify() (which
sets 'selfSigned'), and consistently clear 'finalised' flag in all
attribute setter functions

Remove manifest 'ttl' field and all references thereof (leaving unused
space in Rhizome BAR)

Rename some payload functions for clarity
2013-12-21 14:37:18 +10:30
Andrew Bettison
d52ba4c871 Remove rhizome_read_manifest_file()
Replace with memcpy() followed by rhizome_manifest_parse()
for memory buffers

Replace with rhizome_read_manifest_from_file() for files
2013-12-20 11:29:17 +10:30
Jeremy Lakeman
8d80cf55f2 Fix printf formats 2013-12-16 15:19:15 +10:30
Andrew Bettison
075f9c7c27 Use uint64_t for Rhizome manifest version
Instead of int64_t.  Fixes some -Wsign-compare warnings.

Replace sqlite_exec_int64() with sqlite_exec_uint64().

Also store rowid as uint64_t, and use 0 not -1 to indicate
unset.
2013-12-11 11:11:34 +10:30
Andrew Bettison
ebc3133f5c Change overlay_buffer fields from int to size_t
Fixes many -Wsign-compare warnings
2013-12-10 17:03:30 +10:30
Andrew Bettison
30b2059608 Fix -Wunused-parameter warnings
New UNUSED(param) macro suppresses the warning for a specific parameter

In some cases, removed the unused parameters.  In others, used the
parameter in a DEBUGF() or assert() statement to document the intent.
2013-12-09 18:22:18 +10:30
Andrew Bettison
8db5f9c14a Merge branch 'anyservice' into 'development'
Allows any valid "service" manifest field in the "rhizome add file"
command

Many improvements in Rhizome manifest parsing; stricter manifest syntax
rules (no comment or blank lines, field names must be alphanumeric
identifiers), faster preliminary manifest inspection when receiving
manifest advertisements or syncing manifests

The 'development' branch introduces "struct socket_address" which
coincidentally fixed the recently encountered Linux kernel 3.12
recvmsg(2) EINVAL problem, so that 'rhizomeprotocol' tests which fail on
the 'anyservice' branch will pass after this merge
2013-12-01 05:44:01 +10:30
Andrew Bettison
0dd1b302b5 Rewrite Rhizome manifest parsing
Move validation checks into new function rhizome_manifest_validate()

Remove rhizome_manifest 'errors' field

Replace rhizome_manifest 'warnings' with 'malformed'

Replace rhizome_manifest 'manifest_bytes' with 'manifest_body_bytes' and
refactor to use 'manifest_all_bytes' in all manifest i/o

Refactor rhizome_manifest_verify() and reverse sense of return value to
match rhizome_manifest_validate()

New function rhizome_manifest_inspect() -- lightweight manifest parser
used when receiving Rhizome advertisements

New 'rhizomeops' test case for invalid manifest "service" field values,
now passes
2013-12-01 05:13:34 +10:30
Jeremy Lakeman
0d5d878521 Refactor socket name and address length handling 2013-11-27 13:08:37 +10:30
Andrew Bettison
a9ccd38adc Improve overlay buffer ob_xxx() primitives
All ob_append_xxx(b,...) functions return void

ob_makespace() returns 1 if successful, 0 if not

Add ob_overrun(b) predicate to check for overrun after any number of
appends
2013-11-26 15:33:54 +10:30
Andrew Bettison
ba0ab14c69 Clean up inclusion of <netinet/in.h>
Make all #include <netinet/in.h> conditional upon HAVE_NETINET_IN_H

Remove unnecessary #include <netinet/in.h> from source files

Reformat include block in "serval.h" for readability
2013-11-12 11:39:06 +10:30
Andrew Bettison
bacba19dc1 Refactor manifest: specific setter functions
Replace generic rhizome_manifest_set() and rhizome_manifest_set_ll()
with per-field setter functions, eg, rhizome_manifest_set_filesize().
Struct rhizome_manifest elements for all known fields, to replace the
use of rhizome_manifest_get() and rhizome_manifest_get_ll() everywhere:
sender, recipient, service, name, date, bundle_key.

Add boolean validity flags for binary blob types, to avoid having to compare
with many bytes of all-zero to detect presence, eg, has_sender, has_recipient,
has_author, has_bundle_key.  These maintained by the setter functions.

Rename existing manifest struct elements to be the same as their field
names: fileLength -> filesize, journalTail -> tail.

More use of unsigned int, size_t and uint64_t for payload sizes, offsets, byte
counts, etc. especially in rhizome_store.c and meshms.c.  More uniform use of
size_t to dimension memory buffers.  Fix some printf(3) style format strings
for 64-bit correctness on 32-bit systems.  Use new constant RHIZOME_SIZE_UNSET
instead of -1 to indicate unknown dimension, and explicitly assert its absence
before comparisons and arithmetic, for safety.

Replace some 'int' loop variables with 'unsigned' where appropriate.

Fix bugs discovered in MeshMS bundle private/public key generation and
bundle secret key handling for export/extract commands.

Instrument the first MeshMS test case to aid debugging.

New debug config flag: debug.manifest logs all modifications to all manifest
fields by setter functions.

Rename debug config flag: debug.rhizome_bind -> debug.rhizome_sql_bind.
2013-11-04 19:17:49 +10:30
Andrew Bettison
97cbebc91e Issue #11: improve types in file i/o functions
Use open(2)/read(2)/write(2) instead of fopen(3)/fread(3)/fwrite(3) in
several places to avoid unnecessary buffering

Fix a bug in Rhizome HTTP add's handling of unconfigured manifest
template file

Improve some debug and error logging for file i/o
2013-10-11 16:24:51 +10:30
Andrew Bettison
a95ef79139 Use sid_t everywhere
Remove stowSid() at last

Change API of tohex() and strbuf_tohex(), to pass string length instead of
binary byte count.  This allows odd numbers of hex digits to be produced.

Remove alloca_tohex_sid(); replace with alloca_tohex_sid_t()

New alloca_tohex_sid_t_trunc() macro
2013-10-09 18:54:21 +10:30
Andrew Bettison
ffafae86b1 Issue #11: replace long long with int64_t 2013-10-07 05:54:46 +10:30
Andrew Bettison
ab31420faf rhizome_bid_t
Consistent type for internal binary representation of Rhizome Bundle ID
(aka Manifest ID)
2013-10-03 23:16:45 +09:30
Andrew Bettison
7e3a552011 Issue #69: refactor all Rhizome SQL queries to use binding
Remove all uses of sprintf(3) and its ilk to form SQL query commands.
Use the new sqlite_bind() and sqlite_prepare_bind() functions instead.
2013-10-03 15:15:30 +09:30
Jeremy Lakeman
b38b4720cc Add html stats for http connection and mdp transfer counts 2013-09-24 14:34:51 +09:30
Jeremy Lakeman
c13c7799f6 Send mavlink heartbeat every second 2013-09-13 14:37:33 +09:30
Jeremy Lakeman
2a5ba97b48 Add rhizome test over fakeradio 2013-09-13 14:02:35 +09:30
Jeremy Lakeman
91fcf8b1b3 Stop announcing BARs via legacy process 2013-09-10 14:53:12 +09:30
Jeremy Lakeman
643b21d6a7 Fetch file payloads if they are missing, even if we already have the manifest 2013-08-22 15:14:21 +09:30
Jeremy Lakeman
1468d2deb0 Pre-emptively send small payloads via MDP 2013-08-21 09:21:40 +09:30
Jeremy Lakeman
81afc42d8b Major refactor to packet queuing and delivery in preparation for unicast link tracking.
Unicast tests broken
2013-08-09 09:37:08 +09:30
Jeremy Lakeman
8ac359e550 Use consistent integer types for 64bit compilation 2013-07-13 14:47:06 +09:30
Jeremy Lakeman
2f32635b87 Minor code cleanup / refactoring 2013-07-12 10:56:24 +09:30
Jeremy Lakeman
9372200dfc Improve readability of manifest decision making process 2013-07-11 13:48:18 +09:30
Jeremy Lakeman
adbbed5275 Add new rhizome sync protocol 2013-06-19 12:05:57 +09:30
Jeremy Lakeman
a18230f591 Reduce latency impact of BAR lookups based on aggregate time 2013-06-14 14:09:58 +09:30
Jeremy Lakeman
4cdd3da77f Add config option to disable fetching via rhizome 2013-03-05 10:56:20 +10:30
Jeremy Lakeman
a6405fe387 Don't advertise http server if it is disabled 2013-02-20 16:18:59 +10:30
gardners
2252fdcaa7 created tool for automatically finding IN()s without matching
OUT()s or where return() is used instead of RETURN().
Added OUT() to end of all functions using IN() that lacked it to
make it easier to statically analyse this invariant.
Fixed several return instead of RETURNs detected through use of
this tool. #49
2013-02-20 16:18:56 +10:30
gardners
d49649ed96 add stored bundle count to /rssi web page display. 2013-02-20 15:36:28 +10:30
Jeremy Lakeman
f2025cd601 Configure rhizome announce interval 2013-02-20 15:36:24 +10:30
Jeremy Lakeman
45efb3f899 Add more raw options to serial interface 2013-02-20 15:36:24 +10:30
Jeremy Lakeman
ba1800012e Only advertise BAR's on an alarm
- we no longer hit the database for every outgoing packet, attempting to announce bundles
- we no longer advertise manifests periodically
- when an interesting bar arrives, we ask for the manifest to be announced, which uses the existing packet format
2013-02-20 15:36:23 +10:30
Jeremy Lakeman
c7caec488a Don't aggregate packets together on radio links 2013-02-20 15:36:23 +10:30
Jeremy Lakeman
b8737125eb Fix RETURN macro usage 2013-01-23 15:37:17 +10:30
Jeremy Lakeman
6de54bc0f1 Restart announcements when new bundles added, use more efficient sql, remember position per interface 2013-01-11 14:30:51 +10:30
Jeremy Lakeman
0f81c1f641 Minor code cleanup 2012-12-14 17:07:28 +10:30
Andrew Bettison
5985df751d Overhaul debug flags
Replace debugflags_t and DEBUG_XXX bit masks with config schema "debug.xxx"
entries.

No more support for "debug.all".
2012-12-11 15:59:46 +10:30
Jeremy Lakeman
7e59df86fe Rewind announcements on sql failures 2012-12-06 12:33:44 +10:30
Jeremy Lakeman
2b21a691ca Merge remote-tracking branch 'origin/master' into packet_format
Conflicts:
	constants.h
	packetformats.c
2012-12-03 14:10:32 +10:30
Jeremy Lakeman
a67e4114a8 Allow for payloads with a pre-determined destination 2012-12-03 13:44:31 +10:30
gardners
b91e832ad7 added improved control of rhizome http and mdp servers via
separate config options with their own predicate functions.
2012-11-30 14:47:27 +10:30
gardners
faad1f26b1 Pass SID as well as ip:port through rhizome fetch request logic.
This is in preparation for rhizome over MDP.
2012-11-29 15:38:04 +10:30
Jeremy Lakeman
aeaeac8a79 No broadcast id required for single hop 2012-11-28 15:44:39 +10:30
Jeremy Lakeman
627e86fb73 Shift all payload header writing to one place 2012-11-28 15:43:58 +10:30
Jeremy Lakeman
f159e15901 Simplify payload length storage 2012-11-28 15:43:57 +10:30