Commit Graph

110 Commits

Author SHA1 Message Date
Jeremy Lakeman
3855ff35c4 Clean up string formatting for 64bit compilation 2013-07-15 09:59:24 +09:30
Andrew Bettison
300ec986f1 Fix warnings from gcc 4.6.3 2013-03-18 16:48:15 +10:30
Jeremy Lakeman
73786bcb5d Refactor all rhizome reading and writing
- The API in rhizome_store.c is used for all reading and writing
- external storage is now usable for all transport options
2013-02-20 16:18:59 +10:30
gardners
641d749ab4 Create files for large rhizome bundles
- configurable size threashold
2013-02-20 15:36:32 +10:30
Jeremy Lakeman
84ad4debfa Throttle serial radio devices when tx buffer is in use 2013-02-20 15:36:22 +10:30
Jeremy Lakeman
6ddf840541 Read the entire response before processing it. Issue #45. 2013-02-08 13:46:15 +10:30
Jeremy Lakeman
f64de66b34 Refactor add via HTTP to be equivalent to command line add 2012-12-20 15:36:07 +10:30
Andrew Bettison
5985df751d Overhaul debug flags
Replace debugflags_t and DEBUG_XXX bit masks with config schema "debug.xxx"
entries.

No more support for "debug.all".
2012-12-11 15:59:46 +10:30
Andrew Bettison
8425882ffc Merge branch 'newconfig' into 'master'
Conflicts:
	commandline.c
	conf.h
	dataformats.c
	log.h
	overlay_address.c
	overlay_interface.c
	packetformats.c
	rhizome.c
	serval.h
	tests/directory_service
	vomp_console.c
2012-12-07 14:09:55 +10:30
gardners
f827c7c1c8 fixed various problems following split of files into fileblobs 2012-12-04 16:47:45 +10:30
Andrew Bettison
caa209fc1d Integrate new config into servald
Not passing any tests yet, but compiles and links and simple uses do not
SEGV.
2012-12-04 14:12:28 +10:30
gardners
faad1f26b1 Pass SID as well as ip:port through rhizome fetch request logic.
This is in preparation for rhizome over MDP.
2012-11-29 15:38:04 +10:30
Andrew Bettison
c84b7e5db4 Improve str.h functions
Add strn_startswith() and strncase_startswith().

Make all str*_startswith() functions take const char * arguments, to make it
possible to do safe programming with consts.
2012-11-20 18:10:08 +10:30
Andrew Bettison
b66d8d6c5b Fix bug in 'rhizome direct pull'
Was not fetching payload of remote bundle, just manifest.  The problem was
caused by a change of logic recently to not activate any queued fetch
candidates immediately, but wait until the next fd_poll(), so that parsing a
single packetful of rhizome advertisments would start fetching the most
important one first, instead of the first one parsed.
2012-11-12 16:24:38 +10:30
Andrew Bettison
3c7eb7b058 Add missing sqlite3_blob_close()
Fixes #33.
2012-10-30 16:12:40 +10:30
Andrew Bettison
a9d3a1387c Issue #30, rewrite Rhizome fetch queues and slots
Causes SEGV in rhizomestress test.
2012-10-23 18:10:20 +10:30
Andrew Bettison
839de7557c Merge branch 'rhizomestress' into 'master' 2012-10-18 17:57:10 +10:30
Andrew Bettison
65ea612e27 Fix bugs revealed by Rhizome stress test
Overhauled the file fetch queue logic in rhizome_fetch.c.

Now the 'rhizomeprotocol' stress test passes in approximately 5 minutes on my
2009-vintage Dell laptop.

Added a call to rhizome_enqueue_suggestions() in rhizome_fetch_close() so that
a new Rhizome GET request is sent as soon as a fetch slot becomes free, instead
of waiting for the (default 5 second) timer to trigger the next GET.
2012-10-18 17:30:22 +10:30
gardners
499c4018bf Imported ref10/ implementation of crypto_sign from supercop-20120525
as recommended a while back by Dan Bernstein as offering the fastest
implementation of the crypto_sign() primitives for ARM.
Indeed this implementation IS faster. See comparison below for a
Rock 500 handset (800MHz(?) ARM6, no NEON):

Original ref/ implementation on an R500 stock rom (non-rooted)::
mean signature generation time = 96.80ms
mean signature verification time = 272.20ms

ref10/ implementations on an R500 stock rom (non-rooted):
mean signature generation time = 4.00ms
mean signature verification time = 13.00ms

Approximately 20x speed up, just like that :)
2012-10-18 17:26:48 +10:30
Andrew Bettison
3678522872 Issue #17, add AUTHOR column to Rhizome MANIFESTS table
Replace ".selfsigned" column with ".author" and ".fromhere" columns in
output of "rhizome list" command.  (Note that a "sender" column is
already present.)

Add 'author' field to struct rhizome_manifest.

Log all fully rendered SQL statements on DEBUG_RHIZOME.

Update 'rhizomeops' test cases and improve the assert_rhizome_list()
test function to be able to assert authorship of files.
2012-10-09 17:48:06 +10:30
Andrew Bettison
6954325b04 Server process no longer becomes a zombie on Android
Fixes #21.  The problem was caused when the double-fork logic used in "servald
start" was clobbered in 5103176.  This meant that the servald daemon process on
Android no longer had a PPID=1, but the PID of the long-lived
"org.servalproject" parent process which called the JNI entry point.  Killing
the servald process then caused it to become a zombie process, since the
org.servalproject does not habitually call wait(2).  That caused the "servald
stop" logic to send five SIGHUPs to the zombie without any error, making it
appear that the process was not dying.

Reinstated the double-fork logic and added a new test case to ensure that the
daemon process does not become a zombie on being killed prematurely.
2012-10-08 17:20:19 +10:30
Andrew Bettison
b4c92a289e Fix SEGV bugs on Solaris
Revealed by 'rhizomeprotocol' tests FileTransferDelete, DirectPush and
DirectSync.
2012-10-05 15:49:36 +09:30
Andrew Bettison
063fe6d467 Fix compile warning on Solaris 2012-10-05 15:09:10 +09:30
Andrew Bettison
2e7d19a3f7 Issue #9, fix rhizome direct push and sync tests
The "rhizome direct push" command (and also sync) was not waiting for the
server's HTTP response, so it was exiting before the server had finished
storing the bundle, which led to a race with the subsequent "assert
bundle_received_by" test.  Fixed by adding the missing code to receive the HTTP
response.

Refactored the code used for parsing HTTP responses in rhizome_fetch.c, and
used it in rhizome_direct_http.c.
2012-10-04 17:38:33 +09:30
Andrew Bettison
9614d4f89c Issue #9, a little bit of refactoring 2012-10-04 14:35:09 +09:30
gardners
90fd7e3fd8 fixed bug where <1KB manifests would get found in 0-1K and 1K-2K
buckets.  Also removed some debugging. #9
2012-10-03 18:22:48 +02:00
gardners
99f8e9b86d progress on tracking down post-merge bugs that are stopping
rhizome direct push/pull/sync tests from working. #9
2012-10-03 17:55:55 +02:00
gardners
661f369296 Reintroduce fix for returning full buffer of responses to rhizome
direct sync enquiry request.  Still something else broken
post-merge, but we are getting closer. #9
2012-10-03 16:51:08 +02:00
gardners
33107e795f Reduced debug output. #9 2012-10-03 16:26:41 +02:00
gardners
4e0c0064ba added rhizome.api.addfile.author config option to supply author SID
as hex if not supplying a manifest template.
modified rhizomeprotocol test of manifest-less HTTP import to set
that option.  That test now passes.
ALL rhizomeprotocol tests now pass. #9
2012-10-03 16:24:06 +02:00
gardners
636cba8363 rhizome direct http manifest-less file import now uses the name
supplied in the HTTP POST form. #9
2012-10-03 16:13:41 +02:00
gardners
dc3137707f rhizome import bundle from file sans-manifest via http now works.
Must be enabled by using rhizome.api.addfile.*
Certainly polishing to be done, including using filename supplied
during HTTP POST.  Now to fix that, and make it all work with
final rhizomeprotocol test case.
rhizomeprotocol test cases 8 and 9 currently fail post-merge. #9
2012-10-03 16:13:06 +02:00
gardners
a81dafa180 work towards making rhizome direct http pull work. #9 2012-10-03 15:49:40 +02:00
gardners
92a5423b17 Moved rhizome direct imports to be in-process instead of
out-of-process. rhizomeprotocol push test still fails.
Investigating. #9
2012-10-03 12:28:21 +02:00
gardners
e015f0670b Switched to new BAR format with 15 bytes of BID prefix, and TTL
at the end, and log2(filesize) instead of filesize.  Equally
importantly BAR construction and parsing now uses #defines for
field sizes and offsets instead of it being hardwired without
meaningful documentation.
WILL BREAK BACKWARD COMPATIBILITY WITH PREVIOUS BUILDS.
YOU MUST DELETE AND REBUILD YOUR RHIZOME DATABASE AS OLD-FORMAT
BIDs WILL BE IN THERE AND GET SENT, AND STRANGE THINGS WILL HAPPEN.
This break with backwards compatibility is only reasonable to
consider because we have not yet had an official build using the
new Rhizome with old BAR format.  0.08 uses old Rhizome.  #9
2012-10-03 12:22:59 +02:00
gardners
c356ead09c fixed bug in rhizome direct http push code.
Rhizome direct push via HTTP now works. #9
2012-10-03 12:22:52 +02:00
Andrew Bettison
bbe6d12693 Issue #9, fix 'rhizome direct push' command
Now the 'rhizomeprotocol' Push test case now passes.  It should be renamed to
DirectPush.

Much refactoring of the Rhizome Direct HTTP request parsing.  Now uses
strbuf_sprintf() instead of snprintf() in many places to check for buffer
overrun and ensure terminating nul.  Still more of this kind of work is needed.

Improved debug that needs to be made conditional on DEBUG_RHIZOME_RX and
DEBUG_RHIZOME_TX.  Some just needs removal.
2012-10-03 17:51:37 +09:30
Andrew Bettison
5fed0460e1 Issue #9, 'rhizomeprotocol' HttpImport test now passes
Any HTTP client can import a bundle into Rhizome using a PUSH request.
2012-10-02 17:15:14 +09:30
Andrew Bettison
e51745e39d Issue #9, fix HTTP server "POST /rhizome/bundle"
Handle failure cases properly.  The HttpImport test now fails because the
"servald" executable is not found by the system(3) call -- need to give the
full path name, or avoid the use of system(3) altogether.
2012-09-28 17:53:20 +09:30
Andrew Bettison
04abc7db0f Fix Solaris compile error and some warnings
The mmap(2) MAP_FILE flag is not defined on Solaris, and is optional or
ignored on Mac OS X and Linux.

Two "unused variable" warnings.

Issue #9.
2012-09-28 11:07:34 +09:30
gardners
59a969cfa3 fixed one of the outstanding issues with rhizome direct http push. 2012-09-27 22:01:07 +09:30
gardners
9ecf6499f8 now tries to write bundle file to server. server seems to hang.
Getting very close now to rhizome direct http push working, though.
2012-09-26 17:12:48 +09:30
gardners
5c2b61758e now tries to connect to rhizome direct http server to push bundles.
(manifest is written, but file is not.  manifest also seems to be
missing signature block.) #9
2012-09-26 16:47:55 +09:30
gardners
5787dd860b Rhizome direct http forms first part of POST request to push bundles
to far side. #9
2012-09-26 16:01:27 +09:30
gardners
c23d424d4d more work towards rhizome direct http actually pushing new/updated
bundles to remote party.
2012-09-22 15:21:02 +09:30
gardners
f7ef416091 bug fixes for rhizome direct response handling.
Can now generate a full response buffer instead of only using 9/32
of the space. #9
2012-09-21 15:45:10 +02:00
gardners
ee89f283e0 add initial parsing of rhizome direct response. #9 2012-09-21 14:40:36 +02:00
gardners
a7d42b3c73 added note contemplating about where parsing of rhizome direct
responses should occur. #9
2012-09-21 14:31:14 +02:00
gardners
3caf82ad15 fixed bug in rhizome direct http response formation. #9 2012-09-21 14:27:02 +02:00
gardners
cae11bd444 fixed rhizome direct response generation to allow request and
responses to be of different sizes. #9
2012-09-21 14:04:53 +02:00
gardners
f7e75d5347 debugging and abstraction improvements in rhizome direct
synchronisation code. #9
2012-09-21 13:41:27 +02:00
gardners
6553050566 rhizome direct enquiry request now gets back enquiry response,
and realises that it needs to process it.  It does not yet
process it. #9
2012-09-11 14:34:58 +09:30
gardners
9a5dbbd14f rhizome direct http transport now asks for response to enquiry when
enquiry received by server.  Generating responses to enquiries not
yet implemented. #9
2012-09-11 13:09:40 +09:30
gardners
bb0be86db3 fixed content-disposition problems. Request now gets accepted.
Need to write code to handle it now, and send cursor fill back
the otherway. #9
2012-09-11 09:34:41 +09:30
gardners
0e04dde28c changed rhizome direct POST URI paths to /rhizome/import and
/rhizome/enquiry.  Also fixed bug with content-length header
line when forming /rhizome/enquiry request.  post multipart
data is still malformed in some way. #9
2012-09-11 09:28:11 +09:30
gardners
e8009ac94a rhizome direct http transport sends request, and receives http
reply (including if error).  Doesn't capture the reply body yet. #9
2012-09-11 09:04:38 +09:30
gardners
e4896cd028 now tries to open connection to rhizome direct server using
http transport.  #9
2012-09-11 06:56:04 +09:30
gardners
4f5f2c2444 work towards http transport working for rhizome direct. #9 2012-09-10 20:22:14 +09:30
gardners
dd01f7939d further work on rhizome direct dispatching etc.
For some reason finds the same manifest several times (size bin
filtering seems to not be working right).
Also sync doesn't realise it has finished, and so doesn't return
when done.
2012-09-10 09:16:27 +09:30
gardners
d796a482b7 refactored rhizome direct code to separate http transport specific
code from general rhizome direct BAR buffer gathering and processing.
2012-09-09 13:50:09 +09:30