ExternalVendorCode/serval-dna
1
0
Fork 0
mirror of https://github.com/servalproject/serval-dna.git synced 2024-12-20 21:53:12 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
1,729 Commits 22 Branches 0 Tags 33 MiB
abcac03981
Commit Graph

133 Commits

Author SHA1 Message Date
gardners
abcac03981 work on tracking down hash calculation bug for files > one 
window long.
 2012-12-04 06:26:44 +10:30
gardners
43316ade3a added buffer when streaming to database so that we don't waste 
most of our time opening and closing blobs, and don't have to keep
the blob open all the time.
 2012-12-03 21:47:02 +10:30
gardners
de7a4ce2e3 fixed bugs with rhizome streaming direct to database. 
that now works, but rhizome direct pull now fails.
 2012-12-03 20:51:14 +10:30
gardners
e108fe0822 fixed bug in handling of sqlite3_blob_write(). Still more to fix. 2012-12-03 17:10:39 +10:30
gardners
5d24ba7db9 first cut at progressive hashing and checking of hash when streaming 
file into database via rhizome.
 2012-12-03 17:05:12 +10:30
gardners
e08593b466 stream manifest to buffer, and files to blob in database instead 
of via files.
NOT COMPLETE - needs to calculate progressive hash and decide
if file is ok at end of import.
 2012-12-03 16:52:08 +10:30
gardners
7ec695c940 feedback from code review by Jeremy. 
Some preparation for out-of-order handling.
Consolidated redundantly redundant fields in fetch slot structure.
Probably fixed spurious alarm bug.
 2012-12-03 11:50:35 +10:30
gardners
115736f0a4 cleaned out excess debugging messages and upped block size from 
200 bytes to 1KB to greatly improve throughput.  Will eventually
want to make block size selection be based on network interface.
 2012-12-01 10:52:08 +10:30
gardners
b7ae55c143 receiving multiple bundles via rhizome over mdp works now, although 
there is a bug with alarms being called on fetch slots after they
complete, even though they apparently get unscheduled.
 2012-12-01 10:47:19 +10:30
gardners
55df66f6d6 culled some debug output. 
mostly works, but fetch slots can get muddled.
 2012-12-01 07:43:18 +10:30
gardners
ac734ff00d fixed bugs with rhizome over mdp receive for files >32*200 bytes 
long.  also we request next block of data immediately after
receiving all of the previous block.
 2012-12-01 07:37:22 +10:30
gardners
7e6d836627 remember to update timeout when storing data via rhizome mdp. 2012-11-30 22:55:10 +10:30
gardners
41261bf122 gadzooks, rhizome over mdp appears to work. 2012-11-30 22:45:27 +10:30
gardners
dbb1fe8d1e rhizome over mdp now writes content for in-order packets, and 
import gets triggered, but file hash currently doesn't match.
 2012-11-30 22:42:28 +10:30
gardners
79c90dde6e now rhizome mdp replies get parsed and the appropriate slot, if 
any, is located.
 2012-11-30 22:26:31 +10:30
gardners
1be28246e4 we now pass bid+version in Rhizome over MDP block request. 2012-11-30 21:13:29 +10:30
gardners
0c7419da5d changed bar to bid in rhizome mdp fetch elements, because that 
is probably the better way to address the content.  Should add
version to make sure.
 2012-11-30 21:08:06 +10:30
gardners
e5d5f3f750 fixed bugs with Rhizome over MDP requests. Now seems to output 
a series of MDP block requests as it should.
 2012-11-30 20:20:33 +10:30
gardners
11cce162dc fixed bugs with preparing Rhizome MDP packets (source sid was not 
being set properly).
 2012-11-30 20:06:47 +10:30
gardners
05745d9b9a same fix as previous commit, but in another place. 2012-11-30 20:03:53 +10:30
gardners
5becae0136 fixed segfault bug with obtaining own SID. 2012-11-30 20:03:18 +10:30
gardners
7a4e3d20f8 Actually send Rhizome over MDP request packets. 
Actually call appropriate callback when receiving Rhizome over
MDP packets.
 2012-11-30 14:17:05 +10:30
gardners
eb7524e068 factored out internal MDP services into a separate file, and made 
core case statement in that brief and clear.
 2012-11-30 11:48:00 +10:30
gardners
a665750f4c prepare and dispatch mdp frames for rhizome over mdp. 
still need to set bodies of frames, and schedule call backs,
and handle the requests when received.
 2012-11-29 21:08:11 +10:30
gardners
c05ebada8c more work on rhizome over MDP. 2012-11-29 20:39:30 +10:30
gardners
629a36f041 call switch to mdp fetch if http request setup fails. 2012-11-29 16:43:19 +10:30
gardners
a07e5761c4 added code to try switching to mdp if http fetching fails for 
any reason.
 2012-11-29 16:15:51 +10:30
gardners
faad1f26b1 Pass SID as well as ip:port through rhizome fetch request logic. 
This is in preparation for rhizome over MDP.
 2012-11-29 15:38:04 +10:30
Andrew Bettison
c84b7e5db4 Improve str.h functions 
Add strn_startswith() and strncase_startswith().

Make all str*_startswith() functions take const char * arguments, to make it
possible to do safe programming with consts.
 2012-11-20 18:10:08 +10:30
Andrew Bettison
0e435683f7 Move toprint functions from log.h to str.h 
Move alloca_sockaddr() to strbuf_helpers.h

So that stand-alone executables can use them without pulling in the entire
logging framework.
 2012-11-20 18:10:08 +10:30
Andrew Bettison
b66d8d6c5b Fix bug in 'rhizome direct pull' 
Was not fetching payload of remote bundle, just manifest.  The problem was
caused by a change of logic recently to not activate any queued fetch
candidates immediately, but wait until the next fd_poll(), so that parsing a
single packetful of rhizome advertisments would start fetching the most
important one first, instead of the first one parsed.
 2012-11-12 16:24:38 +10:30
Jeremy Lakeman
2993ee46c7 Move rhizome fetch info message to end of transfer 2012-11-12 15:35:53 +10:30
Jeremy Lakeman
c79c5d8968 Fix fetch process for manifests with the same payload, make sure failures don't get stuck in a loop 2012-11-09 15:57:06 +10:30
Andrew Bettison
a29f685bdd Fix assertion failure in new Rhizome fetch queue logic 
Revealed by improved Rhizome stress test that uses wide range of file sizes.
 2012-11-05 14:46:16 +10:30
Andrew Bettison
39fc4ce6de Issue #30, remove periodic Rhizome fetch alarm 
Replace the main-loop scheduled periodic alarm with an "activate" alarm that is
scheduled whenever a fetch candidate is added to any queue, unless the alarm is
already scheduled.

Replace the "rhizome.fetch_interval_ms" config item with
"rhizome.fetch_delay_ms" [default 50], which is the number of milliseconds
between adding a fetch candidate and firing the "activate" alarm.  This allows
time for a few more Rhizome advertisment packets to arrive after the first one,
before deciding which fetches to start first.

Add new `is_scheduled()` alarm primitive.
 2012-10-24 15:13:50 +10:30
Andrew Bettison
935a545ac7 Issue #30, fix Rhizome fetch slot allocation logic 
Also improve the block comment documentation of many functions.
 2012-10-24 12:08:41 +10:30
Andrew Bettison
9ee9b63aad Issue #30, fox SEGV bug in new Rhizome fetch queue code 
The rhizomestress test now passes, also all rhizomeprotocol tests.
 2012-10-24 09:59:36 +10:30
Andrew Bettison
a9d3a1387c Issue #30, rewrite Rhizome fetch queues and slots 
Causes SEGV in rhizomestress test.
 2012-10-23 18:10:20 +10:30
Andrew Bettison
65ea612e27 Fix bugs revealed by Rhizome stress test 
Overhauled the file fetch queue logic in rhizome_fetch.c.

Now the 'rhizomeprotocol' stress test passes in approximately 5 minutes on my
2009-vintage Dell laptop.

Added a call to rhizome_enqueue_suggestions() in rhizome_fetch_close() so that
a new Rhizome GET request is sent as soon as a fetch slot becomes free, instead
of waiting for the (default 5 second) timer to trigger the next GET.
 2012-10-18 17:30:22 +10:30
Andrew Bettison
5ee82689d2 Start fixing Rhizome stress bugs 
This commit does not compile.
 2012-10-18 11:56:02 +10:30
Jeremy Lakeman
8b9505670d Fix handling of socket closure on OSX 2012-10-08 16:53:52 +10:30
Andrew Bettison
89343c69cd Merge branch 'rhizomedirect' into 'master' 2012-10-05 17:45:30 +09:30
Andrew Bettison
2e7d19a3f7 Issue #9, fix rhizome direct push and sync tests 
The "rhizome direct push" command (and also sync) was not waiting for the
server's HTTP response, so it was exiting before the server had finished
storing the bundle, which led to a race with the subsequent "assert
bundle_received_by" test.  Fixed by adding the missing code to receive the HTTP
response.

Refactored the code used for parsing HTTP responses in rhizome_fetch.c, and
used it in rhizome_direct_http.c.
 2012-10-04 17:38:33 +09:30
gardners
d43db3335c Looks like rhizome direct http pull works now. 
But tests don't yet pass ... #9
 2012-10-03 15:50:43 +02:00
gardners
25a64b5485 fixed some more bugs in rhizome direct http pull. 
Now actually triggers the http request to obtain the file, but
the peer address is 0.0.0.0:0.  Investigating. #9
 2012-10-03 15:50:43 +02:00
gardners
81cc9ddb45 more work on rhizome direct http pull. Now passing all the right 
data we need to allow the prefix2manifest reception to launch
the file body request via normal rhizome http fetch process. #9
 2012-10-03 15:50:43 +02:00
gardners
17de56a04c fixed some problems with the rhizome direct http pull setup code. 
still working on remaining issues. #9
 2012-10-03 15:50:43 +02:00
gardners
a81dafa180 work towards making rhizome direct http pull work. #9 2012-10-03 15:49:40 +02:00
Andrew Bettison
5fed0460e1 Issue #9, 'rhizomeprotocol' HttpImport test now passes 
Any HTTP client can import a bundle into Rhizome using a PUSH request.
 2012-10-02 17:15:14 +09:30
Andrew Bettison
cc9cb8d827 Issue #9, fix old 'rhizomeprotocol' test failures 
Now only the five new Rhizome Direct tests fail.
 2012-10-02 16:32:48 +09:30