ExternalVendorCode/serval-dna
1
0
Fork 0
mirror of https://github.com/servalproject/serval-dna.git synced 2024-12-19 13:17:56 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,855 Commits 22 Branches 0 Tags 33 MiB
b7d4fa478e
Commit Graph

57 Commits

Author SHA1 Message Date
Andrew Bettison
71ed78e058 Make all 'config' tests pass 
Change a test case: configuration options are now case sensitive.

Fix config file load and parse logic in conf.c, always copy 'debug' flags
from config.debug.

The config schema 'interfaces' option is no longer MANDATORY.

Introduce new CLIFLAG_PERMISSIVE_CONFIG to supress bad-config ERROR messages
from the 'config set' and 'config get' commands.

Refactor cli_execute() into cli_parse() and cli_invoke().  Use *const* struct
command_line_option everywhere.
 2012-12-04 16:52:49 +10:30
Andrew Bettison
caa209fc1d Integrate new config into servald 
Not passing any tests yet, but compiles and links and simple uses do not
SEGV.
 2012-12-04 14:12:28 +10:30
Andrew Bettison
89343c69cd Merge branch 'rhizomedirect' into 'master' 2012-10-05 17:45:30 +09:30
Andrew Bettison
3040d2b665 Add Rhizome DB retries on SQL statement preparation 
It turns out that if the DB is locked, sqlite_prepare_v2() call can return
SQLITE_BUSY.  The retry logic (implemented for issue #2) only provided for
sqlite_step() to return SQLITE_BUSY.  It was a fairly straightforward matter to
extend the retry logic to cover statement preparation in an equally general
fashion.

The problem was observed while diagnosing failures in the rhizomeprotocol
DirectPush test case: the "servald rhizome list" command was failing due to a
locked database.  See issue #9.
 2012-10-04 14:30:20 +09:30
gardners
90fd7e3fd8 fixed bug where <1KB manifests would get found in 0-1K and 1K-2K 
buckets.  Also removed some debugging. #9
 2012-10-03 18:22:48 +02:00
gardners
99f8e9b86d progress on tracking down post-merge bugs that are stopping 
rhizome direct push/pull/sync tests from working. #9
 2012-10-03 17:55:55 +02:00
gardners
4fd832f4ba fixed bug which was preventing files <1024 bytes from being 
synched by rhizome direct. #9
 2012-10-03 12:26:13 +02:00
gardners
13679ecb23 Fixed bug in new rhizome direct sync command line parsing. 
Now works from command line.
Updated rhizomeprotocol rhizome direct tests, but these still
fail -- under investigation. #9
 2012-10-03 12:26:10 +02:00
gardners
d11109bca5 Improve rhizome direct push/pull/sync command line parsing and 
use of configuration. Not yet finished, but allows multiple
rhizome direct peers and non-http URLs. #9
 2012-10-03 12:24:22 +02:00
gardners
e015f0670b Switched to new BAR format with 15 bytes of BID prefix, and TTL 
at the end, and log2(filesize) instead of filesize.  Equally
importantly BAR construction and parsing now uses #defines for
field sizes and offsets instead of it being hardwired without
meaningful documentation.
WILL BREAK BACKWARD COMPATIBILITY WITH PREVIOUS BUILDS.
YOU MUST DELETE AND REBUILD YOUR RHIZOME DATABASE AS OLD-FORMAT
BIDs WILL BE IN THERE AND GET SENT, AND STRANGE THINGS WILL HAPPEN.
This break with backwards compatibility is only reasonable to
consider because we have not yet had an official build using the
new Rhizome with old BAR format.  0.08 uses old Rhizome.  #9
 2012-10-03 12:22:59 +02:00
gardners
d4fe995206 added note about sqlite3 query results weirdness. #9 2012-10-03 12:22:58 +02:00
gardners
0d988921ac added debug message for when sqlite returns results that it shouldn't. 
(FILESIZE BETWEEN X and Y is not behaving correctly -- it sometimes
returns results with the out of bounds FILESIZE values). #9
 2012-10-03 12:22:58 +02:00
gardners
5787dd860b Rhizome direct http forms first part of POST request to push bundles 
to far side. #9
 2012-09-26 16:01:27 +09:30
gardners
7ef4d942b2 rhizome direct now reads manifest data given a prefix. 2012-09-26 15:15:08 +09:30
gardners
8e4496f397 fixed bug in rhizome_direct_get_manifest() #9 2012-09-26 15:03:15 +09:30
gardners
c23d424d4d more work towards rhizome direct http actually pushing new/updated 
bundles to remote party.
 2012-09-22 15:21:02 +09:30
gardners
f7ef416091 bug fixes for rhizome direct response handling. 
Can now generate a full response buffer instead of only using 9/32
of the space. #9
 2012-09-21 15:45:10 +02:00
gardners
5d881226c1 fixed bug in comparison of IHAVE lists. Now can generate list of 
BARs that rhizome direct sync requestor has not seen. #9
 2012-09-21 14:10:11 +02:00
gardners
cae11bd444 fixed rhizome direct response generation to allow request and 
responses to be of different sizes. #9
 2012-09-21 14:04:53 +02:00
gardners
f7e75d5347 debugging and abstraction improvements in rhizome direct 
synchronisation code. #9
 2012-09-21 13:41:27 +02:00
gardners
692eb2cdc9 added BAR list comparison code for Rhizome direct. 
Now returns series of "I have [newer]"'s and "Please send me"'s,
consisting of a 1 byte ID (0x01 or 0x02 respectively), followed
by the 64bit BID prefix from the BAR.  As with all of Rhizome
Direct at present, the geo bounding box is ignored for now.
 2012-09-14 12:10:14 +08:00
gardners
6553050566 rhizome direct enquiry request now gets back enquiry response, 
and realises that it needs to process it.  It does not yet
process it. #9
 2012-09-11 14:34:58 +09:30
gardners
9a5dbbd14f rhizome direct http transport now asks for response to enquiry when 
enquiry received by server.  Generating responses to enquiries not
yet implemented. #9
 2012-09-11 13:09:40 +09:30
gardners
4f5f2c2444 work towards http transport working for rhizome direct. #9 2012-09-10 20:22:14 +09:30
gardners
872b88f744 fixed hanging bug. still gets bundles showing up in query 
results even though size bin is wrong. That bug doesn't stop
us for now, but should be sorted out.
rhizome_direct_http_dispatch() can now be implemented
(synchronously for now will be fine).
 2012-09-10 09:22:44 +09:30
gardners
dd01f7939d further work on rhizome direct dispatching etc. 
For some reason finds the same manifest several times (size bin
filtering seems to not be working right).
Also sync doesn't realise it has finished, and so doesn't return
when done.
 2012-09-10 09:16:27 +09:30
gardners
d796a482b7 refactored rhizome direct code to separate http transport specific 
code from general rhizome direct BAR buffer gathering and processing.
 2012-09-09 13:50:09 +09:30
gardners
1d4c865a35 fixed bugs in cursor range pickling. #9 2012-09-09 09:15:52 +09:30
gardners
f0c4bfd7bf fixed cursor range reporting bugs. #9 2012-09-09 09:06:38 +09:30
gardners
27fc7a44d0 rhizome direct cursor operations now include pickled cursor range 
in output buffer (but doesn't yet report the range correctly). #9
 2012-09-09 08:58:47 +09:30
gardners
0207d8e541 added cursor limit pickling, restoration functions. cursor 
functions now try to honour these. Not yet tested. #9
 2012-09-09 08:30:38 +09:30
gardners
53228f078e rhizome direct now has a cursor function to iterate through all 
bundles in the rhizome database in an appropriate priority order
(currently small before big). #9
 2012-09-08 20:26:18 +09:30
gardners
3e4b20f5df rhizome_direct_get_bars() now allows selections to be based on 
size of associated data in a bundle, so that we can synchronise
small things first.  Also preliminary work on making a general
cursor-type wrapper function for get_bars() so that it is easy
for any rhizome direct transport driver to iterate over the
known bundles in a rhizome datastore. #9
 2012-09-08 20:00:14 +09:30
gardners
b13c23de6a rhizome_direct_get_bars() now sets bid_high correctly. #9 2012-09-08 09:35:58 +09:30
gardners
89b09563a3 Wrote function to get a range of BARs from the Rhizome database 
for use in Rhizome Direct synchronisation requests. #9
 2012-09-08 07:55:57 +09:30
gardners
6d725535f9 starting work on the actual synchronisation code for rhizome direct. 2012-09-07 22:53:16 +09:30
gardners
d3f5c6e598 Merged rhizome direct httpd back into main rhizome httpd server. 
bundles can be imported via HTTP POST multi-part form. #9
 2012-09-06 13:21:13 +09:30
gardners
f9a29645a8 prevent double-processing of form when fully received. 
add actual call to servald rhizome import bundle.
seems to work, though yet to be tested with a genuine bundle. #9
 2012-09-06 07:52:12 +09:30
gardners
d342e7b0d2 Added multipart form end processing stage. Now has place holder 
for calling rhizome import. #9
 2012-09-06 07:34:53 +09:30
gardners
edf99e9be3 implemented removal of extraneous CRLF from end of each form field 
file. #9
 2012-09-06 07:15:52 +09:30
gardners
2d9a620352 removed fclose() for form field file from other code paths, so 
that there is only one path.  This is where we will add the 2
byte rewind that our parser requires.
 2012-09-06 07:11:09 +09:30
gardners
95f2bc273e fixed bug with boundary line handling that prevented end of form 
from being detected. #9
 2012-09-06 07:08:10 +09:30
gardners
b51a7b2dbf more work on parser. Now writes field files, but fails to detect 
the boundary string at the end of the form. #9
 2012-09-06 06:59:26 +09:30
gardners
3dad0798a4 further work on multipart POST form parser. Now does almost 
everything except actually write manifest and data to files
for importing. #9
 2012-09-06 06:29:05 +09:30
gardners
6aca69a54f Further work on multi-part POST parser. Simple state machine 
partially implemented that recognises the field elements being
submitted and complains if they are wrong. #9
 2012-09-06 06:09:38 +09:30
gardners
f3662c0d04 fixed CRLF handling in multi-part POST parser so that CRLF 
presence is not forgotten. #9
 2012-09-06 05:49:43 +09:30
gardners
9d7507c8b5 Fixed byte munching bug in multi-part post parser. Also removed 
need for any parser look-ahead, i.e., it is now LALR(1), and well
suited to parsing streaming data. #9
 2012-09-06 05:43:29 +09:30
gardners
35ec3bae0d revamped multi-part form parser to tokenise into lines. This 
feels like a much simpler and cleaner design. Still more to do. #9
 2012-09-06 05:34:25 +09:30
gardners
db79190c81 more work on multi-part POST form data parser. #9 2012-09-06 05:15:25 +09:30
gardners
95f59ab7b0 Fixed handling of multi-part form POST requests (removed HTTP 
response stub that was causing instakill of such requests).
 2012-09-05 16:02:39 +09:30