Commit Graph

77 Commits

Author SHA1 Message Date
Andrew Bettison
5ff5a02bb9 Fix a couple more bugs in new HTTP server code 2013-10-25 23:57:23 +10:30
Andrew Bettison
fa21bec880 Rewrite HTTP server 2013-10-25 00:19:37 +10:30
Andrew Bettison
640a61cbe5 Add 'debug.rhizome_httpd' config option 2013-10-16 11:23:45 +10:30
Andrew Bettison
00cf61721d Rename http_header_complete() to is_http_header_complete() 2013-10-16 11:22:02 +10:30
Andrew Bettison
97cbebc91e Issue #11: improve types in file i/o functions
Use open(2)/read(2)/write(2) instead of fopen(3)/fread(3)/fwrite(3) in
several places to avoid unnecessary buffering

Fix a bug in Rhizome HTTP add's handling of unconfigured manifest
template file

Improve some debug and error logging for file i/o
2013-10-11 16:24:51 +10:30
Andrew Bettison
49729cc768 Remove default service=file for new manifests 2013-10-11 15:39:10 +10:30
Andrew Bettison
974c7a56a0 Issue #11: Use rhizome_filehash_t everywhere 2013-10-11 15:35:27 +10:30
Andrew Bettison
a95ef79139 Use sid_t everywhere
Remove stowSid() at last

Change API of tohex() and strbuf_tohex(), to pass string length instead of
binary byte count.  This allows odd numbers of hex digits to be produced.

Remove alloca_tohex_sid(); replace with alloca_tohex_sid_t()

New alloca_tohex_sid_t_trunc() macro
2013-10-09 18:54:21 +10:30
Jeremy Lakeman
60e3f5a3fc Fix printf formats for 64bit compilation 2013-10-09 10:31:52 +10:30
Andrew Bettison
ffafae86b1 Issue #11: replace long long with int64_t 2013-10-07 05:54:46 +10:30
Andrew Bettison
7b427d48ee A few more uses of rhizome_bid_t
Now the TEXT_TOUPPER SQL bind action is only used for filehash hex
strings; all Bundle ID parameters are bound using RHIZOME_BID_T.
2013-10-04 04:50:37 +09:30
Andrew Bettison
ab31420faf rhizome_bid_t
Consistent type for internal binary representation of Rhizome Bundle ID
(aka Manifest ID)
2013-10-03 23:16:45 +09:30
Andrew Bettison
5e164607aa Expand output of rhizome add and import
Rename 'secret:' field to '.secret:' (non-manifest-fields start with '.'
using the same convention as rhizome list)

Add '.author' and 'BK' fields to "rhizome add"

Add 'BK' field to "rhizome import"

Fix 'rhizomeops' tests to assert no 'author' and 'BK' output fields from
"rhizome add" with no author

Fiz testdefs.sh and testdefs_rhizome.sh to support new output fields
2013-09-27 22:34:22 +09:30
Jeremy Lakeman
4838a529bd Don't busy wait while testing for MDP failover 2013-09-24 15:15:32 +09:30
Jeremy Lakeman
931ca60554 Refactor and simplify http server
- Define a function per page that is responsible for parsing requests
- Define a "generator" callback function for filling the buffer for a response
- Remove features that have never been used
- Remove functions that are no longer used
2013-09-24 14:34:50 +09:30
Jeremy Lakeman
f9b828c3dd Remove payload if hash doesn't match when reading back 2013-08-21 15:45:18 +09:30
Jeremy Lakeman
21132466f1 Serve a range of a file payload via HTTP 2013-08-15 16:31:26 +09:30
Jeremy Lakeman
3855ff35c4 Clean up string formatting for 64bit compilation 2013-07-15 09:59:24 +09:30
Andrew Bettison
300ec986f1 Fix warnings from gcc 4.6.3 2013-03-18 16:48:15 +10:30
Jeremy Lakeman
73786bcb5d Refactor all rhizome reading and writing
- The API in rhizome_store.c is used for all reading and writing
- external storage is now usable for all transport options
2013-02-20 16:18:59 +10:30
gardners
641d749ab4 Create files for large rhizome bundles
- configurable size threashold
2013-02-20 15:36:32 +10:30
Jeremy Lakeman
84ad4debfa Throttle serial radio devices when tx buffer is in use 2013-02-20 15:36:22 +10:30
Jeremy Lakeman
6ddf840541 Read the entire response before processing it. Issue #45. 2013-02-08 13:46:15 +10:30
Jeremy Lakeman
f64de66b34 Refactor add via HTTP to be equivalent to command line add 2012-12-20 15:36:07 +10:30
Andrew Bettison
5985df751d Overhaul debug flags
Replace debugflags_t and DEBUG_XXX bit masks with config schema "debug.xxx"
entries.

No more support for "debug.all".
2012-12-11 15:59:46 +10:30
Andrew Bettison
8425882ffc Merge branch 'newconfig' into 'master'
Conflicts:
	commandline.c
	conf.h
	dataformats.c
	log.h
	overlay_address.c
	overlay_interface.c
	packetformats.c
	rhizome.c
	serval.h
	tests/directory_service
	vomp_console.c
2012-12-07 14:09:55 +10:30
gardners
f827c7c1c8 fixed various problems following split of files into fileblobs 2012-12-04 16:47:45 +10:30
Andrew Bettison
caa209fc1d Integrate new config into servald
Not passing any tests yet, but compiles and links and simple uses do not
SEGV.
2012-12-04 14:12:28 +10:30
gardners
faad1f26b1 Pass SID as well as ip:port through rhizome fetch request logic.
This is in preparation for rhizome over MDP.
2012-11-29 15:38:04 +10:30
Andrew Bettison
c84b7e5db4 Improve str.h functions
Add strn_startswith() and strncase_startswith().

Make all str*_startswith() functions take const char * arguments, to make it
possible to do safe programming with consts.
2012-11-20 18:10:08 +10:30
Andrew Bettison
b66d8d6c5b Fix bug in 'rhizome direct pull'
Was not fetching payload of remote bundle, just manifest.  The problem was
caused by a change of logic recently to not activate any queued fetch
candidates immediately, but wait until the next fd_poll(), so that parsing a
single packetful of rhizome advertisments would start fetching the most
important one first, instead of the first one parsed.
2012-11-12 16:24:38 +10:30
Andrew Bettison
3c7eb7b058 Add missing sqlite3_blob_close()
Fixes #33.
2012-10-30 16:12:40 +10:30
Andrew Bettison
a9d3a1387c Issue #30, rewrite Rhizome fetch queues and slots
Causes SEGV in rhizomestress test.
2012-10-23 18:10:20 +10:30
Andrew Bettison
839de7557c Merge branch 'rhizomestress' into 'master' 2012-10-18 17:57:10 +10:30
Andrew Bettison
65ea612e27 Fix bugs revealed by Rhizome stress test
Overhauled the file fetch queue logic in rhizome_fetch.c.

Now the 'rhizomeprotocol' stress test passes in approximately 5 minutes on my
2009-vintage Dell laptop.

Added a call to rhizome_enqueue_suggestions() in rhizome_fetch_close() so that
a new Rhizome GET request is sent as soon as a fetch slot becomes free, instead
of waiting for the (default 5 second) timer to trigger the next GET.
2012-10-18 17:30:22 +10:30
gardners
499c4018bf Imported ref10/ implementation of crypto_sign from supercop-20120525
as recommended a while back by Dan Bernstein as offering the fastest
implementation of the crypto_sign() primitives for ARM.
Indeed this implementation IS faster. See comparison below for a
Rock 500 handset (800MHz(?) ARM6, no NEON):

Original ref/ implementation on an R500 stock rom (non-rooted)::
mean signature generation time = 96.80ms
mean signature verification time = 272.20ms

ref10/ implementations on an R500 stock rom (non-rooted):
mean signature generation time = 4.00ms
mean signature verification time = 13.00ms

Approximately 20x speed up, just like that :)
2012-10-18 17:26:48 +10:30
Andrew Bettison
3678522872 Issue #17, add AUTHOR column to Rhizome MANIFESTS table
Replace ".selfsigned" column with ".author" and ".fromhere" columns in
output of "rhizome list" command.  (Note that a "sender" column is
already present.)

Add 'author' field to struct rhizome_manifest.

Log all fully rendered SQL statements on DEBUG_RHIZOME.

Update 'rhizomeops' test cases and improve the assert_rhizome_list()
test function to be able to assert authorship of files.
2012-10-09 17:48:06 +10:30
Andrew Bettison
6954325b04 Server process no longer becomes a zombie on Android
Fixes #21.  The problem was caused when the double-fork logic used in "servald
start" was clobbered in 5103176.  This meant that the servald daemon process on
Android no longer had a PPID=1, but the PID of the long-lived
"org.servalproject" parent process which called the JNI entry point.  Killing
the servald process then caused it to become a zombie process, since the
org.servalproject does not habitually call wait(2).  That caused the "servald
stop" logic to send five SIGHUPs to the zombie without any error, making it
appear that the process was not dying.

Reinstated the double-fork logic and added a new test case to ensure that the
daemon process does not become a zombie on being killed prematurely.
2012-10-08 17:20:19 +10:30
Andrew Bettison
b4c92a289e Fix SEGV bugs on Solaris
Revealed by 'rhizomeprotocol' tests FileTransferDelete, DirectPush and
DirectSync.
2012-10-05 15:49:36 +09:30
Andrew Bettison
063fe6d467 Fix compile warning on Solaris 2012-10-05 15:09:10 +09:30
Andrew Bettison
2e7d19a3f7 Issue #9, fix rhizome direct push and sync tests
The "rhizome direct push" command (and also sync) was not waiting for the
server's HTTP response, so it was exiting before the server had finished
storing the bundle, which led to a race with the subsequent "assert
bundle_received_by" test.  Fixed by adding the missing code to receive the HTTP
response.

Refactored the code used for parsing HTTP responses in rhizome_fetch.c, and
used it in rhizome_direct_http.c.
2012-10-04 17:38:33 +09:30
Andrew Bettison
9614d4f89c Issue #9, a little bit of refactoring 2012-10-04 14:35:09 +09:30
gardners
90fd7e3fd8 fixed bug where <1KB manifests would get found in 0-1K and 1K-2K
buckets.  Also removed some debugging. #9
2012-10-03 18:22:48 +02:00
gardners
99f8e9b86d progress on tracking down post-merge bugs that are stopping
rhizome direct push/pull/sync tests from working. #9
2012-10-03 17:55:55 +02:00
gardners
661f369296 Reintroduce fix for returning full buffer of responses to rhizome
direct sync enquiry request.  Still something else broken
post-merge, but we are getting closer. #9
2012-10-03 16:51:08 +02:00
gardners
33107e795f Reduced debug output. #9 2012-10-03 16:26:41 +02:00
gardners
4e0c0064ba added rhizome.api.addfile.author config option to supply author SID
as hex if not supplying a manifest template.
modified rhizomeprotocol test of manifest-less HTTP import to set
that option.  That test now passes.
ALL rhizomeprotocol tests now pass. #9
2012-10-03 16:24:06 +02:00
gardners
636cba8363 rhizome direct http manifest-less file import now uses the name
supplied in the HTTP POST form. #9
2012-10-03 16:13:41 +02:00
gardners
dc3137707f rhizome import bundle from file sans-manifest via http now works.
Must be enabled by using rhizome.api.addfile.*
Certainly polishing to be done, including using filename supplied
during HTTP POST.  Now to fix that, and make it all work with
final rhizomeprotocol test case.
rhizomeprotocol test cases 8 and 9 currently fail post-merge. #9
2012-10-03 16:13:06 +02:00
gardners
a81dafa180 work towards making rhizome direct http pull work. #9 2012-10-03 15:49:40 +02:00