Commit Graph

1745 Commits

Author SHA1 Message Date
gardners
e2cf0ff506 fixed bug with rhizome direct sync manifest fetching. 2012-12-06 09:31:41 +10:30
gardners
5416e4baca fixed bugs that were preventing writing to database blob from
buffer.  Increased maximum buffer size to useful size.
2012-12-04 17:26:47 +10:30
gardners
39f99da3f2 fixing various bugs with rhizome streaming to database. 2012-12-04 17:18:31 +10:30
gardners
f827c7c1c8 fixed various problems following split of files into fileblobs 2012-12-04 16:47:45 +10:30
gardners
6dfe654ab0 adjusted tuning of Rhizome over MDP to use 32x1KB packets for
better throughput.  Still sub-optimal for wired ethernet type
interfaces, but should be ok for WiFi where max broadcast throughput
is 100kb/sec.
2012-12-04 16:19:22 +10:30
gardners
3ace113be1 fixed error in orphan file cleanup sql statement. 2012-12-04 16:19:01 +10:30
gardners
de7ff09930 fixed bug where disabling Rhizome over MDP server also disabled
client functionality.
2012-12-04 16:10:17 +10:30
gardners
fc0c134cbf added speed indication on completion of rhizome transfer.
split files table into files and fileblobs to avoid pathological
delays when updating datavalid flag in files table (sqlite copies
the whole table row, including possibly large blob).
2012-12-04 16:03:56 +10:30
gardners
23cace8455 removed writing temporary file for debugging. 2012-12-04 13:49:47 +10:30
gardners
1a12f81474 Merge branch 'master' into rhizomeovermdp 2012-12-04 11:23:31 +10:30
gardners
6179240874 enable overriding default instance path at configure time. 2012-12-04 11:23:03 +10:30
gardners
2e4e111764 fixed bug that prevented remaining rhizome tests from passing. 2012-12-04 08:50:00 +10:30
gardners
a7e09aac2c fixed scheduling bug that stopped rhizomeprotocol tests 1 and 3. 2012-12-04 08:30:58 +10:30
gardners
254b0a4fd6 fixed problem with rhizome over http (alarm without callback
specified).
2012-12-04 07:12:03 +10:30
gardners
fcd5142aaf fixed bug with streaming to database of rhizome fetches. 2012-12-04 06:56:10 +10:30
gardners
a433f06530 fixed typo in updated DELETE statement. 2012-12-04 06:27:39 +10:30
gardners
abcac03981 work on tracking down hash calculation bug for files > one
window long.
2012-12-04 06:26:44 +10:30
gardners
43316ade3a added buffer when streaming to database so that we don't waste
most of our time opening and closing blobs, and don't have to keep
the blob open all the time.
2012-12-03 21:47:02 +10:30
gardners
de7a4ce2e3 fixed bugs with rhizome streaming direct to database.
that now works, but rhizome direct pull now fails.
2012-12-03 20:51:14 +10:30
gardners
e108fe0822 fixed bug in handling of sqlite3_blob_write(). Still more to fix. 2012-12-03 17:10:39 +10:30
gardners
adda5b5571 fixed bug when dropping incomplete file fetch row. 2012-12-03 17:10:21 +10:30
gardners
5d24ba7db9 first cut at progressive hashing and checking of hash when streaming
file into database via rhizome.
2012-12-03 17:05:12 +10:30
gardners
e08593b466 stream manifest to buffer, and files to blob in database instead
of via files.
NOT COMPLETE - needs to calculate progressive hash and decide
if file is ok at end of import.
2012-12-03 16:52:08 +10:30
gardners
b39c71b206 factor out code for creating empty blob for inserting file into. 2012-12-03 16:19:01 +10:30
gardners
3e7e29f5f5 improve handling of incomplete files in rhizome database (clean
all out on start up, but only clean out specific ones when inserting).
2012-12-03 12:20:50 +10:30
gardners
7ec695c940 feedback from code review by Jeremy.
Some preparation for out-of-order handling.
Consolidated redundantly redundant fields in fetch slot structure.
Probably fixed spurious alarm bug.
2012-12-03 11:50:35 +10:30
gardners
7b4b9dbf81 improved test coverage for Rhizome over MDP, and made testing
of MDP vs HTTP more explicit for an existing test case.
2012-12-01 10:56:48 +10:30
gardners
115736f0a4 cleaned out excess debugging messages and upped block size from
200 bytes to 1KB to greatly improve throughput.  Will eventually
want to make block size selection be based on network interface.
2012-12-01 10:52:08 +10:30
gardners
b7ae55c143 receiving multiple bundles via rhizome over mdp works now, although
there is a bug with alarms being called on fetch slots after they
complete, even though they apparently get unscheduled.
2012-12-01 10:47:19 +10:30
gardners
55df66f6d6 culled some debug output.
mostly works, but fetch slots can get muddled.
2012-12-01 07:43:18 +10:30
gardners
ac734ff00d fixed bugs with rhizome over mdp receive for files >32*200 bytes
long.  also we request next block of data immediately after
receiving all of the previous block.
2012-12-01 07:37:22 +10:30
gardners
21f562122e added test for rhizome over mdp for a big file, to make sure that
timeout handling works properly.
2012-11-30 22:57:34 +10:30
gardners
7e6d836627 remember to update timeout when storing data via rhizome mdp. 2012-11-30 22:55:10 +10:30
gardners
41261bf122 gadzooks, rhizome over mdp appears to work. 2012-11-30 22:45:27 +10:30
gardners
dbb1fe8d1e rhizome over mdp now writes content for in-order packets, and
import gets triggered, but file hash currently doesn't match.
2012-11-30 22:42:28 +10:30
gardners
79c90dde6e now rhizome mdp replies get parsed and the appropriate slot, if
any, is located.
2012-11-30 22:26:31 +10:30
gardners
ebbc8001cf fixed bug with sending data blocks (was sending packets for beyond
end of file).
2012-11-30 22:02:10 +10:30
gardners
94e8a4b676 fixed composition of rhizome over mdp reply packets. 2012-11-30 21:52:44 +10:30
gardners
1912f24da4 sending MDP Rhizome replies in response to requests now occurs,
and replies are received, but not yet parsed.
2012-11-30 21:50:17 +10:30
gardners
1be28246e4 we now pass bid+version in Rhizome over MDP block request. 2012-11-30 21:13:29 +10:30
gardners
0c7419da5d changed bar to bid in rhizome mdp fetch elements, because that
is probably the better way to address the content.  Should add
version to make sure.
2012-11-30 21:08:06 +10:30
gardners
979ca36ba6 fixed byte order bug in read routines. 2012-11-30 21:07:57 +10:30
gardners
692817ffba added functions for reading values from byte streams. 2012-11-30 21:02:43 +10:30
gardners
e5d5f3f750 fixed bugs with Rhizome over MDP requests. Now seems to output
a series of MDP block requests as it should.
2012-11-30 20:20:33 +10:30
gardners
11cce162dc fixed bugs with preparing Rhizome MDP packets (source sid was not
being set properly).
2012-11-30 20:06:47 +10:30
gardners
05745d9b9a same fix as previous commit, but in another place. 2012-11-30 20:03:53 +10:30
gardners
5becae0136 fixed segfault bug with obtaining own SID. 2012-11-30 20:03:18 +10:30
gardners
6c0e6ef1c2 added extra tests for Rhizome over HTTP and MDP transports.
HTTP transport works, MDP transport segfaults.  To be examined.
2012-11-30 17:42:17 +10:30
gardners
3c4abfb566 improved and simplified test for rhizome http not running 2012-11-30 15:06:31 +10:30
gardners
9e29db827f added test for correct function of rhizome.http.enable config
option.
2012-11-30 15:02:34 +10:30