Commit Graph

139 Commits

Author SHA1 Message Date
Jeremy Lakeman
2b21a691ca Merge remote-tracking branch 'origin/master' into packet_format
Conflicts:
	constants.h
	packetformats.c
2012-12-03 14:10:32 +10:30
Jeremy Lakeman
a67e4114a8 Allow for payloads with a pre-determined destination 2012-12-03 13:44:31 +10:30
gardners
b91e832ad7 added improved control of rhizome http and mdp servers via
separate config options with their own predicate functions.
2012-11-30 14:47:27 +10:30
gardners
faad1f26b1 Pass SID as well as ip:port through rhizome fetch request logic.
This is in preparation for rhizome over MDP.
2012-11-29 15:38:04 +10:30
Jeremy Lakeman
aeaeac8a79 No broadcast id required for single hop 2012-11-28 15:44:39 +10:30
Jeremy Lakeman
627e86fb73 Shift all payload header writing to one place 2012-11-28 15:43:58 +10:30
Jeremy Lakeman
f159e15901 Simplify payload length storage 2012-11-28 15:43:57 +10:30
Jeremy Lakeman
ce63e96db2 Use decode structure for tracking previous sid 2012-11-28 15:43:57 +10:30
Jeremy Lakeman
4ca6db34d6 Encapsulate payload header details 2012-11-28 15:40:39 +10:30
Jeremy Lakeman
a4b392d693 Don't include rhizome adverts in packets with voice frames 2012-10-29 13:10:20 +10:30
Jeremy Lakeman
de95bb3971 Gracefully disable rhizome if the database cannot be opened
- close database after every command line operation
- don't cache rhizome enabled configuration
- don't send advertisements unless the database is open and the web server is running
- don't provess advertisements unless the database is open
2012-10-29 13:05:03 +10:30
Andrew Bettison
2a8cfb0404 Quieten Rhizome advertisement debug logging
Introduce DEBUG_RHIZOME_ADS and sqlite_set_debugmask(), use in
rhizome_packetformats.c advertisement mustering and processing code.
2012-10-10 13:22:30 +10:30
gardners
164e9a7b87 removed dependency on -lm log2 to make Android building easier. 2012-10-08 10:46:56 +10:30
Andrew Bettison
89343c69cd Merge branch 'rhizomedirect' into 'master' 2012-10-05 17:45:30 +09:30
Andrew Bettison
3ce57c8955 Fix SQL prepare retry code missed by 3040d2b665 2012-10-04 16:21:08 +09:30
gardners
a81dafa180 work towards making rhizome direct http pull work. #9 2012-10-03 15:49:40 +02:00
gardners
e015f0670b Switched to new BAR format with 15 bytes of BID prefix, and TTL
at the end, and log2(filesize) instead of filesize.  Equally
importantly BAR construction and parsing now uses #defines for
field sizes and offsets instead of it being hardwired without
meaningful documentation.
WILL BREAK BACKWARD COMPATIBILITY WITH PREVIOUS BUILDS.
YOU MUST DELETE AND REBUILD YOUR RHIZOME DATABASE AS OLD-FORMAT
BIDs WILL BE IN THERE AND GET SENT, AND STRANGE THINGS WILL HAPPEN.
This break with backwards compatibility is only reasonable to
consider because we have not yet had an official build using the
new Rhizome with old BAR format.  0.08 uses old Rhizome.  #9
2012-10-03 12:22:59 +02:00
Andrew Bettison
cc9cb8d827 Issue #9, fix old 'rhizomeprotocol' test failures
Now only the five new Rhizome Direct tests fail.
2012-10-02 16:32:48 +09:30
Jeremy Lakeman
66e0711d6b Only send one audio block per packet, but request that it be sent multiple times
If we receive a large buffer of audio, we want to stuff the packet with multiple frames and send them together.
And we want to send redundant copies of the audio to help recover from packet loss.
But if all our redundant copies end up in the same packet, we're screwed anyway.

This is a temporary hack until the network layer implements NACK / retry for resilient multi-hop delivery
2012-09-28 15:23:49 +09:30
Andrew Bettison
14721fab19 Merge branch 'solaris' into rhizomedirect
All test cases pass on Solaris (see issue #16) except the 'jni' tests because
there is no Java compiler available on Solaris, and the new 'rhizomeprotocol'
tests from the rhizomedirect branch, which have always failed.
2012-09-27 17:38:16 +09:30
gardners
f6bab96846 fixed bug when extracting BID prefix. #9 2012-09-21 13:47:58 +02:00
gardners
f7e75d5347 debugging and abstraction improvements in rhizome direct
synchronisation code. #9
2012-09-21 13:41:27 +02:00
gardners
692eb2cdc9 added BAR list comparison code for Rhizome direct.
Now returns series of "I have [newer]"'s and "Please send me"'s,
consisting of a 1 byte ID (0x01 or 0x02 respectively), followed
by the 64bit BID prefix from the BAR.  As with all of Rhizome
Direct at present, the geo bounding box is ignored for now.
2012-09-14 12:10:14 +08:00
gardners
446adaa868 Added note regarding new architecture to existing comment. #1 2012-09-08 07:55:32 +09:30
Jeremy Lakeman
3b55643b08 Send my full sid periodically for any outgoing payload header 2012-09-07 10:01:34 +09:30
Paul Gardner-Stephen
bba6839656 Initial stab at porting to Solaris.
It compiles without warning (with CC=gcc) but doesn't link
because NaCL doesn't build yet.
2012-09-05 20:42:50 +09:30
Jeremy Lakeman
e05806f25e Separate routing calculation from resulting rules, refactor more code to use subscriber structure 2012-08-31 15:11:18 +09:30
Jeremy Lakeman
f6c899e107 Refactor overlay buffer interface to support simple parsing 2012-08-31 15:03:07 +09:30
Daniel O'Connor
19dd10b1cc Log the address & port of the sender of a broken Rhizome packet. 2012-08-27 12:11:06 +09:30
Andrew Bettison
3f1e0dd763 Finish sleep-retry on locked Rhizome db
Closes #2.

Rewrite all Rhizome db query code using new retry primitives defined in
"rhizome.h": sqlite_step_retry(), sqlite_retry(), sqlite_retry_done(), etc.
Replace all calls to sqlite3_prepare_v2() with sqlite_prepare() which does
proper error logging.

Fix bug: re-invoking sqlite3_blob_close() on SQLITE_BUSY return causes process
to abort.  Use an explicit BEGIN...COMMIT around the blob writing code instead.

Tested using repeated invocations of batphone/tests/meshms1.

Delete deprecated Rhizome db code in rhizome_crypto.c that has been replaced
with keyring file.
2012-08-24 15:38:39 +09:30
Jeremy Lakeman
152d46ddb6 Reduce and improve logging 2012-08-09 13:30:52 +09:30
Andrew Bettison
12d8bc0c12 Use new typedef time_ms_t everywhere 2012-08-09 12:14:32 +09:30
Andrew Bettison
240be25793 Replace overlay_gettime_ms() with gettime_ms() 2012-07-30 17:22:38 +09:30
Jeremy Lakeman
ddba8033e4 Refactor packet envelope and sid abbreviation reading and writing 2012-07-17 15:30:50 +09:30
Jeremy Lakeman
e9566de0af Fix payload length fields 2012-07-13 12:18:49 +09:30
Andrew Bettison
41f3228300 Refactor rhizome HTTP request parsing
Replace DEBUG_RHIZOMESYNC with DEBUG_RHIZOME_TX
2012-07-13 11:06:10 +09:30
Andrew Bettison
d111f763c7 Fix bugs revealed by 'rhizomeprotocol' test
Was not transmitting actual HTTP server port in rhizome announcements, was
always transmitting port 4110.

When trying for a free HTTP server port, sometimes bind() succeeds but listen()
fails with EADDRINUSE, so new logic to deal with that.
2012-07-12 12:10:59 +09:30
Andrew Bettison
4eb3910dc1 Fix bug in Rhizome fetch - byte order of sin_port
Also improve some debug logging
2012-07-11 14:21:49 +09:30
Andrew Bettison
72769e162b Replace overlay_render_sid_prefix() with alloca_tohex()
To eliminate timebomb bugs caused by rotor wraparound
2012-07-04 10:30:46 +09:30
Andrew Bettison
8020ea3b74 Merge branch 'eventscheduler' into master
Conflicts:
	commandline.c
	monitor.c
	overlay.c
	overlay_interface.c
	overlay_packetformats.c
	rhizome_fetch.c
	rhizome_http.c
	rhizome_packetformats.c
	serval.h
	server.c
	testdefs.sh
	testframework.sh
	tests/dnaprotocol
	tests/server
2012-07-03 10:26:22 +09:30
Andrew Bettison
d689e96454 Fix rhizome HTTP port number in packet 2012-07-02 16:07:35 +09:30
gardners
aab400164f push manifest verification to as late as possible to avoid
unwarranted 400ms CPU hits for every manifest seen.
2012-06-26 16:54:40 +09:30
gardners
e542b41746 instrumented various rhizome functions. 2012-06-26 16:03:34 +09:30
Andrew Bettison
3d0038754b Add HTTP port number to rhizome advertise packet 2012-06-25 18:03:00 +09:30
Andrew Bettison
86c7819f97 Remove spurious ERROR messsages
Tests assert that stderr contains no ERROR: lines after a successful exit

Rewrote sqlite_exec_int64() to separate error outcomes from legitimate
result values

Changed several WHY() calls to DEBUG()

Improved test framework
2012-06-08 13:13:26 +09:30
gardners
0cccb5bf83 debug cleanup 2012-05-28 13:51:23 +02:00
gardners
40ed766880 debugging fiddles. 2012-05-27 08:52:36 +02:00
gardners
30e53f963c fixed various manifest synchronisation bugs, including probably
the main one that was stopping meshms (large manifests would
practically never get advertised under certain (common) conditions).
2012-05-27 08:31:48 +02:00
gardners
04cc05d6e1 refactored manifest verification into a separate function
that is called only when needed, and marks a manifest as finalised
if the verifcation fails.  reading a manifest now never sets
finalised flag, as either _finalise() or _verify() must be called.
2012-05-26 11:36:18 +02:00
Andrew Bettison
d215d90b40 Refactored some rhizome db SQL code
Added sqlite_exec_void() function, so that sqlite_exec_int64() can return
an error if no rows are found instead of 0, which can be misleading.
2012-05-25 19:42:45 +09:30
Andrew Bettison
fdb34593ed Improve Rhizome saw-advertisement debug logging 2012-05-25 18:07:19 +09:30
Andrew Bettison
bea9188cd5 Fix another manifest leak
Add manifest alloc/free debug logging to help, 'debug.manifests' option
2012-05-25 17:01:56 +09:30
Andrew Bettison
49aec4d331 Improve rhizome manifest debugging
Move rhizome_new_manifest() out of rhizome_read_manifest_file() so that the
out-of-manifest report shows the names of the functions where the manifests
were really allocated.
2012-05-25 15:38:13 +09:30
Andrew Bettison
e74b9e1d6b Maybe fix cause of SEGV bug
Was caused when we ran out of manifest structs
2012-05-24 17:27:59 +09:30
Andrew Bettison
0ec3feaa29 Expunge setReason()
Replace with WHY() or WHYF() everywhere

Improve logMessage() to handle file/line/function printing internally

Mainly so that log messages from manifest alloc/free in rhizome_bundle.c
can make it into the Android log
2012-05-24 17:11:55 +09:30
Andrew Bettison
60e4209b88 Diagnose and fix rhizome MeshMS 2012-05-23 19:01:07 +09:30
gardners
115cb942dc included fix for suppressing bad behaviour due to signature-less manifests. 2012-05-22 17:33:33 +09:30
gardners
a9be08b0cc supress bad rhizome manifest messages. 2012-05-22 17:24:06 +09:30
gardners
6a83f2434f cleaned up some debug output, among other things. 2012-05-22 17:04:24 +09:30
gardners
e319b146a6 cleaned up some debugging. 2012-05-22 15:57:19 +09:30
gardners
e3387fbecf periodic sucking in from rhizome prioritised list now works. 2012-05-22 15:57:18 +09:30
gardners
f528110257 almost have priority pre-listing of bundles for rhizome working. 2012-05-22 15:57:18 +09:30
gardners
0abc7ff96e added code to create and maintain priority download list for
rhizome, so that we get things in a sensible order.
2012-05-22 15:57:18 +09:30
gardners
3866352340 further improvements to supressing broken manifest time wasting
in rhizome.
2012-05-22 14:20:01 +09:30
gardners
8e9396419c adding ignored manifest list to let us ignore repeated offerings
of broken manifests without wasting effort.
2012-05-22 14:15:35 +09:30
gardners
29bb86d9e7 first cut at supressing rhizome activity during calls. 2012-05-22 13:05:29 +09:30
gardners
4a3fc65385 back out undefined function bug. 2012-05-22 12:11:42 +09:30
gardners
fd343ca5b0 debug fiddles etc. 2012-05-22 11:45:54 +09:30
gardners
c3350bb493 fix bug introduced in using snprintf instead of strbuf_*
use WHY() macros for all output.
2012-05-22 10:02:24 +09:30
gardners
f562a6361d completed working around strbuf_* bugs so that rhizome works again. 2012-05-21 22:45:08 +09:30
gardners
1349c045b2 fixed a null pointer dereference. 2012-05-21 14:09:53 +09:30
gardners
76663d1b9e fixed sqlite handle leaks when advertising rhizome manifests. 2012-05-17 11:14:10 +09:30
gardners
a811dbcab8 fixed some (but not all) rhizome database handle leak bugs in
rhizome advertisement code.
2012-05-17 11:14:10 +09:30
gardners
1b9f706209 fixed bug in dispatch of rhizome advertisements.
hopefully no more mal-formed packets.
2012-05-01 05:25:10 +09:30
gardners
5cc3283646 Cleaned up some debug output. 2012-03-27 19:50:31 +10:30
gardners
6759a26720 Cleaned out some debugging output after tracking down and fixing
memory corruption bug.  Next challenge is to find out why broadcast
MDP packets are not getting dispatched properly (is trying to treat
broadcast address as unicast address it seems).
2012-03-23 07:06:57 +10:30
gardners
bd999138cf Added note about inscrutable frame construction. 2012-03-22 17:16:04 +10:30
gardners
2b42f77ccf Fixed some bounds checking, and added some fairly rigorous memory
handling debug aids. No known memory corruption bugs remain, I
think.
2012-03-22 17:10:27 +10:30
gardners
13d1d3084e Various fixes to track down memory handling bugs.
Fixed one free-before-time bug with queuing MDP frames.
Some heap corruption bug seems to remain.
2012-03-22 16:33:25 +10:30
gardners
76ef948455 Added back rather vital line that was mistakenly removed during a debugging
cleanout.
2012-03-21 15:26:07 +10:30
gardners
70497df7b5 More work on MDP. Removed some debug fluff.
Added framework for MDP ping, and some work towards MDP port binding
and sending packets with option to wait for reply.  MDP server
doesn't yet support port binding, and client doesn't yet support
reading replies.
2012-03-19 16:06:34 +10:30
gardners
1eb4c34015 Renamed mphlr.h to serval.h
(also some temporary debugging has been added in some places)
2012-02-23 12:45:42 +10:30
gardners
ce2afbe2ab Various fixes and addition of memory corruption framework (not yet active).
Rhizome now advertises all manifests, instead of accidently skipping some.
2012-02-15 23:38:23 +10:30
gardners
720ab89309 Debugging fiddles and fixed bug where only the first few manifests
would get advertised.  But now there is a nasty crash bug somewhere.
2012-02-09 17:10:52 +10:30
gardners
8a24fb95b1 Converted manifest storage to using static set of manifest structures
and added manifest leak detection and some other memory corruption
protections.
2012-01-28 11:45:45 +10:30
gardners
423fb589b9 Bug fixes to rhizome manifest:version cache.
Lookup now really happens before database query.
2012-01-27 17:18:14 +10:30
gardners
41daec5c96 Fixed all zeroes manifestid problem.
Removed incorrectly pasted piece of code.
Various debug cleanups along the way.
2012-01-27 15:38:06 +10:30
gardners
92968394e5 oops, missed a file on last commit. 2012-01-12 16:48:19 +10:30
gardners
abb3021986 Added refactored rhizome code in separate files. 2012-01-12 14:08:24 +10:30