Commit Graph

1349 Commits

Author SHA1 Message Date
gardners
9d7507c8b5 Fixed byte munching bug in multi-part post parser. Also removed
need for any parser look-ahead, i.e., it is now LALR(1), and well
suited to parsing streaming data. #9
2012-09-06 05:43:29 +09:30
gardners
35ec3bae0d revamped multi-part form parser to tokenise into lines. This
feels like a much simpler and cleaner design. Still more to do. #9
2012-09-06 05:34:25 +09:30
gardners
db79190c81 more work on multi-part POST form data parser. #9 2012-09-06 05:15:25 +09:30
gardners
ecbe8e4f8c improve HTTP server response code handling, including fixing
iching calculator bug - unknown response codes not more than four
no longer result in "a suffusion of yellow".
Also stopped dropping HTTP requests on empty reads when parsing
a multi-part POST.
2012-09-05 16:35:08 +09:30
gardners
95f59ab7b0 Fixed handling of multi-part form POST requests (removed HTTP
response stub that was causing instakill of such requests).
2012-09-05 16:02:39 +09:30
gardners
4cab16598c added code to rhizome http request client poll routine to call
multi-part form data parser.
2012-09-05 15:53:16 +09:30
gardners
e73f0262c7 Added prototype for HTTP multi-part post parser for rhizome direct. 2012-09-05 15:51:22 +09:30
gardners
ed7edd3865 Added boundarystring field to rhizome_http_request structure.
Added call to new (currently stub) function for processing bytes
received from a HTTP POST multipart encoded form. #9
2012-09-01 14:11:23 +09:30
gardners
8170df7447 added extra field to rhizome http request structure and update
rhizome direct code to remember the path in a multipart form post
operation, and also the boundary string that will separate the
parts.
2012-08-31 13:15:05 +09:30
gardners
211f75cb53 Added initial parsing of POST requests, extracting content-length
and content-type, and complaining if they aren't in the format we
expect. #9
2012-08-31 13:01:29 +09:30
gardners
0a7c633977 swapped arg order to what seemed a more sensible order. #9 2012-08-31 13:01:29 +09:30
gardners
fb82043246 added str_str() to help parse http POST requests for rhizome direct. #9 2012-08-31 13:01:29 +09:30
gardners
389ae8e989 added skeleton HTTP request parser for rhizome direct.
Doesn't know how to do POST yet, which is needed for all rhizome
direct operations. #9
2012-08-31 13:01:29 +09:30
gardners
9d2aa61792 Refactoring of Rhizome http server code so that we can supply a
different request parser, but otherwise share code between
rhizome transfers and rhizome direct. #9
2012-08-31 13:01:28 +09:30
gardners
bde9d1c56a Rhizome http server now allows specification of client socket callback
when starting, so that same server code can be shared for rhizome transfers
and rhizome direct. #9
2012-08-31 13:01:28 +09:30
gardners
2b29893a02 Initial work towards rhizome direct syncrhonisation functions #9 2012-08-31 12:58:37 +09:30
Andrew Bettison
420add0f4b Merge branch 'master' into rhizomedirect 2012-08-30 09:38:19 +09:30
Andrew Bettison
9149c80297 Improve test framework: assert '!' command...
The assert command now supports leading '!' args in its command, each negating
the sense of the following command.
2012-08-30 09:33:15 +09:30
Andrew Bettison
b3f9e437b3 Issue #9, add test case for local HTTP add 2012-08-27 17:57:41 +09:30
Andrew Bettison
41680ff663 Move functions from testdefs.sh to testdefs_rhizome.sh
... where they should have been in the first place.

Also, clean up the RHIZOME HTTP SERVER START log message.

Issue #9.
2012-08-27 17:07:44 +09:30
gardners
bae2e2e4c2 Updated non-Android build process to work again (sqlite3 include path fixed,
also improved so that sqlite3.c doesn't need to exist as a symlink to the
real file).
2012-08-27 17:03:50 +09:30
Andrew Bettison
f93e32b4bc Add 'rhizomeprotocol' tests for push, pull, sync
See issue #9.

These commands are not yet implemented, so of course the new
tests fail.
2012-08-27 16:29:46 +09:30
Andrew Bettison
00c9c540f5 DNA helper treat poll(2) POLLNVAL like POLLERR
Fixes #5, test cases failing on Mac OS X
2012-08-27 13:03:13 +09:30
Andrew Bettison
60d84817bf Add WARNF_perror() macro 2012-08-27 13:03:13 +09:30
Daniel O'Connor
19dd10b1cc Log the address & port of the sender of a broken Rhizome packet. 2012-08-27 12:11:06 +09:30
Daniel O'Connor
00b350aeb1 Add a note about why we aren't using SIPS/ZRTP. 2012-08-27 11:28:06 +09:30
Andrew Bettison
30e21810e4 Improve dnahelper diagnostics to investigate issue #5 2012-08-27 11:19:59 +09:30
Andrew Bettison
836a6d9b6a Issue #5, fix one failing test, two to go
On Max OS X, /bin/sh does not support echo -n option, but /bin/bash does
2012-08-24 18:40:30 +09:30
Andrew Bettison
25fbd6ffe0 Improve test framework: colours on Max OS X
Fixes #7.
2012-08-24 16:30:41 +09:30
Andrew Bettison
3a6b424c2d Fix test framework: job tracking on Mac OS X
Caused by difference in basic regular expressions supported by sed(1) on Mac
and on Linux: Mac does not support the \+ repeater, have to use \{1,\} instead.
2012-08-24 16:02:21 +09:30
Andrew Bettison
3f1e0dd763 Finish sleep-retry on locked Rhizome db
Closes #2.

Rewrite all Rhizome db query code using new retry primitives defined in
"rhizome.h": sqlite_step_retry(), sqlite_retry(), sqlite_retry_done(), etc.
Replace all calls to sqlite3_prepare_v2() with sqlite_prepare() which does
proper error logging.

Fix bug: re-invoking sqlite3_blob_close() on SQLITE_BUSY return causes process
to abort.  Use an explicit BEGIN...COMMIT around the blob writing code instead.

Tested using repeated invocations of batphone/tests/meshms1.

Delete deprecated Rhizome db code in rhizome_crypto.c that has been replaced
with keyring file.
2012-08-24 15:38:39 +09:30
Daniel O'Connor
d793dbc948 Restore testframework.sh that I previously rewound. 2012-08-24 15:36:17 +09:30
Daniel O'Connor
0dd2145ed4 assert that the port number matches between calls to overlay_interface_init_any() 2012-08-24 15:21:23 +09:30
Daniel O'Connor
e98ec36391 Set MYSID to my SID before starting the dna helper so it can return it in a SID URI. 2012-08-24 11:43:30 +09:30
Andrew Bettison
bb4819aa83 Issue #2, sleep-retry logic in all Rhizome operations
Much refactoring and removal of cruft.

SQL query errors are now logged with the filename, line number and function
where they were invoked, not of the low-level function that discovered the
error.  This makes use of the new __HERE__ notation introduced last commit.
2012-08-23 17:43:35 +09:30
Andrew Bettison
27a0a6eeb5 Introduce struct __sourceloc
Replaces (const char *file, unsigned int line, const char *function) arguments
to all logging functions, simplifies malloc/free tracking code in
overlay_buffer.c and Rhizome manifest alloc/free tracking in rhizome_bundle.c.

Use __HERE__ macro instead of (__FILE__, __LINE__, __FUNCTION__) everywhere.

Special __NOWHERE__ macro is equivalent to (NULL, 0, NULL).

Declare net.c functions in new "net.h" header, so log.c doesn't have to pull
in the entire "serval.h" just to use write_str().

Facilitates progress on issue #2.
2012-08-23 12:31:07 +09:30
Andrew Bettison
fd3da58a7c Issue #2, add sleep-retry logic for most database queries
All the queries that used sqlite_exec_void() and sqlite_exec_int64() and
sqlite_exec_strbuf() now do a sleep-retry while the Rhizome db is locked.

There are other queries that still need conversion, and some old infinite
retry logic that needs replacing.
2012-08-22 19:09:30 +09:30
Andrew Bettison
65d6bf191a Fix strbuf API comment 2012-08-22 19:03:00 +09:30
Andrew Bettison
d8fd9fa411 Refactor nanosleep(2) calls into sleep_ms() function 2012-08-21 13:35:33 +09:30
Andrew Bettison
83cb027484 Add FATALF_perror() macro 2012-08-21 13:35:33 +09:30
Andrew Bettison
e534d93a84 Fix tests broken by execpath change
Commit 5cbc2167ec changed the servald 'start'
command to use a different argv[0], which meant that the testdefs.sh
get_servald_pids() function no longer detected running servald processes.
This broke most of the 'server' test cases.

testdefs.sh setup_servald() now makes a full copy of servald once per test
case, not just a symbolic link, so that the logic in get_servald_pids() works
again.
2012-08-21 13:35:33 +09:30
Jeremy Lakeman
48c899df39 Pass remote sequence number through monitor interface 2012-08-21 13:32:55 +09:30
Jeremy Lakeman
00b1609deb Fix audio rotor buffer 2012-08-21 13:32:55 +09:30
Jeremy Lakeman
890feaadc9 Fix snprintf arguments 2012-08-21 13:32:55 +09:30
Andrew Bettison
5cbc2167ec Start on problem of database lock errors
Add sqlite_exec_void_retry() function, use it in
rhizome_update_file_priority().  This should be reviewed to ensure that the
server process never sleeps.

The general problem remains of what the servald process should do if the
database is locked when it tries to update.  Simplest solution is to sleep and
retry, but that blocks all other services and would hurt VoMP.  A better
solution would be for each Rhizome operation to collect its database updates
into a single transaction and place that in a work queue that gets called using
schedule() (or even watch() if a file-descriptor event can somehow be used when
the database becomes available).  Another solution is perhaps to perform all
Rhizome operations in a dedicated process that can block indefinitely on the
database without affecting servald responsiveness.
2012-08-20 18:43:48 +09:30
Andrew Bettison
1c1e1a8ad0 Add WHYFNULL() macro 2012-08-20 18:43:48 +09:30
Andrew Bettison
3d82983888 Improve test framework: better log messages from assertGrep 2012-08-20 18:43:47 +09:30
Jeremy Lakeman
b9874af3b9 Really fix initialisation of monitor client 2012-08-20 16:11:33 +09:30
Daniel O'Connor
fc20a4768d Use get_self_executable_path to resolve our name when calling
exec otherwise it will fail when given a relative path as we
chdir to / before trying it.
2012-08-20 15:02:00 +09:30
Andrew Bettison
a14c8427cb Improve test framework: kill residual processes 2012-08-20 11:28:03 +09:30