Commit Graph

1744 Commits

Author SHA1 Message Date
Andrew Bettison
e1cacdb687 Issue #9, improve 'rhizomeprotocol' tests
Rename Push, Pull, Sync to DirectPush, DirectPull, DirectSync.

Move HttpAdd test to just after HttpImport test.

Tighten up the assertions in Rhizome Direct test cases.

Factor the "assert_rhizome_list" test out of assert_received() to avoid having
to water down assert_rhizome_list(), which currently takes an exact list of all
files that must be present in the list, and is more valuable that way.
2012-10-04 14:25:08 +09:30
Andrew Bettison
f18d7ba9af Re-instate .selfsigned check in assert_rhizome_list()
Also add checks for the "version", "date" and ".inserttime" columns, that they
match the corresponding manifest or are at least well formed.

Issue #9.
2012-10-04 14:22:06 +09:30
Andrew Bettison
37d4b4fa3a Issue #9, fixed a compiler warning and bug
The warning was that 'fd' could be used uninitialised in rhizome_store_file(),
and it was true!
2012-10-04 11:42:13 +09:30
Andrew Bettison
f527ae9425 Improve dump() debugging function
Now dump() accepts any type of pointer to the data.

Also tidied up some other debugging macro definitions to reduce duplication of
code.
2012-10-04 11:40:40 +09:30
gardners
90fd7e3fd8 fixed bug where <1KB manifests would get found in 0-1K and 1K-2K
buckets.  Also removed some debugging. #9
2012-10-03 18:22:48 +02:00
gardners
8aa18db913 ... and same fix for sync case, and that passes now as well.
All rhizomeprotocol tests now PASS with merged changes. #9
2012-10-03 18:17:12 +02:00
gardners
ab61ee698e switched rhizome direct push/pull/sync target from instance A to B
and made corresponding adjustments.  Oddly push and pull work again.
Maybe there is some subtle bug in the test framework?
Anyway, now to fix the sync test. #9
2012-10-03 18:15:09 +02:00
gardners
99f8e9b86d progress on tracking down post-merge bugs that are stopping
rhizome direct push/pull/sync tests from working. #9
2012-10-03 17:55:55 +02:00
gardners
295f7c68ec some rhizome schema cleanups, including importantly making filesize
an integer not text column, so that FILESIZE BETWEEN clauses behave
numerically rather than lexographically.  Added an index for good
measure. #9
2012-10-03 17:54:39 +02:00
gardners
661f369296 Reintroduce fix for returning full buffer of responses to rhizome
direct sync enquiry request.  Still something else broken
post-merge, but we are getting closer. #9
2012-10-03 16:51:08 +02:00
gardners
33107e795f Reduced debug output. #9 2012-10-03 16:26:41 +02:00
gardners
4e0c0064ba added rhizome.api.addfile.author config option to supply author SID
as hex if not supplying a manifest template.
modified rhizomeprotocol test of manifest-less HTTP import to set
that option.  That test now passes.
ALL rhizomeprotocol tests now pass. #9
2012-10-03 16:24:06 +02:00
gardners
2e907e6fff fixed some bugs with rhizomeprotocol tests and manifest-less
file import. It now rungs in rhizomeprotocol test suite, but fails
because the BK= field is missing (but the file is imported).
Also removed the assertions about receiving bundles from the push
test as it was non-deterministic.  The bundle reception is already
tested, so there is no loss.
Investigating missing BK=.
Also test 8 (sync) fails post-merge, although push and pull work
intepdendently. Will investigate that post-merge. #9
2012-10-03 16:23:27 +02:00
gardners
636cba8363 rhizome direct http manifest-less file import now uses the name
supplied in the HTTP POST form. #9
2012-10-03 16:13:41 +02:00
gardners
dc3137707f rhizome import bundle from file sans-manifest via http now works.
Must be enabled by using rhizome.api.addfile.*
Certainly polishing to be done, including using filename supplied
during HTTP POST.  Now to fix that, and make it all work with
final rhizomeprotocol test case.
rhizomeprotocol test cases 8 and 9 currently fail post-merge. #9
2012-10-03 16:13:06 +02:00
gardners
40dff25814 test 8 (rhizome import via http post using curl) almost works
(something broken in merge of AB and PGS changes -- will investigate
once merge complete). #9
2012-10-03 15:55:49 +02:00
gardners
132cfd6fec Fixed bug in rhizome_store_file(), where it tried to open the
associated file before checking if it was already in the database.
Rhizome Direct can supply a manifest without associated file if
the file is already in the database, and so it was breaking.
Also removed "assert bundle_received_by"'s from rhizome direct
pull and sync tests because they are not needed, and were failing
because the same file contents was used for the files being
exchanged, and so file storage was not occurring, and thus the log
message being looked for was not being produced.
Push, pull and sync tests in rhizomeprotocol now pass, leaving
only two tests in error.  #9
2012-10-03 15:50:43 +02:00
gardners
d43db3335c Looks like rhizome direct http pull works now.
But tests don't yet pass ... #9
2012-10-03 15:50:43 +02:00
gardners
25a64b5485 fixed some more bugs in rhizome direct http pull.
Now actually triggers the http request to obtain the file, but
the peer address is 0.0.0.0:0.  Investigating. #9
2012-10-03 15:50:43 +02:00
gardners
81cc9ddb45 more work on rhizome direct http pull. Now passing all the right
data we need to allow the prefix2manifest reception to launch
the file body request via normal rhizome http fetch process. #9
2012-10-03 15:50:43 +02:00
gardners
e9136ec7ee first cut at code to lookup manifest by prefix and deliver the
manifest in full. #9
2012-10-03 15:50:43 +02:00
gardners
17de56a04c fixed some problems with the rhizome direct http pull setup code.
still working on remaining issues. #9
2012-10-03 15:50:43 +02:00
gardners
a81dafa180 work towards making rhizome direct http pull work. #9 2012-10-03 15:49:40 +02:00
gardners
7c544b25f2 applied similar fixes to rhizome direct pull and sync tests as
were recently made to rhizome direct push test. Now to implement
the rhizome direct http pull functionality so that they work... #9
2012-10-03 12:28:23 +02:00
gardners
1c72778c50 fixed problem with rhizome direct push / manifest import where
signatures were not being recorded.  Also fixed separate issue
where rhizomeprotocols tests expected selfsigned to be 1 on
receiver end, when it will never be, because the BK doesn't match.
rhizome direct push test in rhizomeprotocols now passes. #9
2012-10-03 12:28:23 +02:00
gardners
92a5423b17 Moved rhizome direct imports to be in-process instead of
out-of-process. rhizomeprotocol push test still fails.
Investigating. #9
2012-10-03 12:28:21 +02:00
gardners
4fd832f4ba fixed bug which was preventing files <1024 bytes from being
synched by rhizome direct. #9
2012-10-03 12:26:13 +02:00
gardners
13679ecb23 Fixed bug in new rhizome direct sync command line parsing.
Now works from command line.
Updated rhizomeprotocol rhizome direct tests, but these still
fail -- under investigation. #9
2012-10-03 12:26:10 +02:00
gardners
d11109bca5 Improve rhizome direct push/pull/sync command line parsing and
use of configuration. Not yet finished, but allows multiple
rhizome direct peers and non-http URLs. #9
2012-10-03 12:24:22 +02:00
gardners
e015f0670b Switched to new BAR format with 15 bytes of BID prefix, and TTL
at the end, and log2(filesize) instead of filesize.  Equally
importantly BAR construction and parsing now uses #defines for
field sizes and offsets instead of it being hardwired without
meaningful documentation.
WILL BREAK BACKWARD COMPATIBILITY WITH PREVIOUS BUILDS.
YOU MUST DELETE AND REBUILD YOUR RHIZOME DATABASE AS OLD-FORMAT
BIDs WILL BE IN THERE AND GET SENT, AND STRANGE THINGS WILL HAPPEN.
This break with backwards compatibility is only reasonable to
consider because we have not yet had an official build using the
new Rhizome with old BAR format.  0.08 uses old Rhizome.  #9
2012-10-03 12:22:59 +02:00
gardners
d4fe995206 added note about sqlite3 query results weirdness. #9 2012-10-03 12:22:58 +02:00
gardners
0d988921ac added debug message for when sqlite returns results that it shouldn't.
(FILESIZE BETWEEN X and Y is not behaving correctly -- it sometimes
returns results with the out of bounds FILESIZE values). #9
2012-10-03 12:22:58 +02:00
gardners
c356ead09c fixed bug in rhizome direct http push code.
Rhizome direct push via HTTP now works. #9
2012-10-03 12:22:52 +02:00
Andrew Bettison
bbe6d12693 Issue #9, fix 'rhizome direct push' command
Now the 'rhizomeprotocol' Push test case now passes.  It should be renamed to
DirectPush.

Much refactoring of the Rhizome Direct HTTP request parsing.  Now uses
strbuf_sprintf() instead of snprintf() in many places to check for buffer
overrun and ensure terminating nul.  Still more of this kind of work is needed.

Improved debug that needs to be made conditional on DEBUG_RHIZOME_RX and
DEBUG_RHIZOME_TX.  Some just needs removal.
2012-10-03 17:51:37 +09:30
Andrew Bettison
245c7e7f49 Change str_str() to return (char*) not (const char*)
Also rewrite implementation to handle edge cases, rename arguments to be more
self documenting.
2012-10-03 17:13:45 +09:30
Andrew Bettison
46f81d9d2e Fix formatting bug in strbuf_toprint() et al
Was printing chars above \x7f as \xffffffhh.
2012-10-03 17:10:57 +09:30
Jeremy Lakeman
4340d97475 Add simple peer count command 2012-10-03 16:21:47 +09:30
Jeremy Lakeman
32cd46c165 Simplify id peers command, hide mdp implementation details from node info 2012-10-03 16:21:06 +09:30
Jeremy Lakeman
b72c01518b Move SAS key storage into subscriber structure 2012-10-03 13:59:46 +09:30
Andrew Bettison
ebf2aa83ac Issue #9, re-order 'rhizomeprotocol' HttpImport test 2012-10-02 17:16:44 +09:30
Andrew Bettison
5fed0460e1 Issue #9, 'rhizomeprotocol' HttpImport test now passes
Any HTTP client can import a bundle into Rhizome using a PUSH request.
2012-10-02 17:15:14 +09:30
Andrew Bettison
cc9cb8d827 Issue #9, fix old 'rhizomeprotocol' test failures
Now only the five new Rhizome Direct tests fail.
2012-10-02 16:32:48 +09:30
Jeremy Lakeman
9a7c18c9f6 Only parse incoming packets with port numbers matching our expected responses 2012-10-02 16:15:30 +09:30
Jeremy Lakeman
8cc5f8152e Use common poll / alarm framework for console operation 2012-10-02 15:53:09 +09:30
Andrew Bettison
4daaa8f164 Fix SEGV in serval_packetvisualise() 2012-10-02 15:47:45 +09:30
Andrew Bettison
17ef2249e4 Issue #9, rewrite rhizome_import_bundle()
Objective is to avoid having to call system("servald rhizome import ...") to
handle a Rhizome direct POST /rhizome/bundle request.  Antiquated code in and
around rhizome_import_bundle() needs much cleaning up, as indicated by some
TODO comments.  Invocations must unnecessarily write the manifest into a file,
when they already have it in memory, ready to pass to the function.

All the 'rhizomeops' tests pass, but two 'rhizomeprotocol' tests are broken
by the changes in this commit.
2012-09-28 17:55:43 +09:30
Andrew Bettison
e51745e39d Issue #9, fix HTTP server "POST /rhizome/bundle"
Handle failure cases properly.  The HttpImport test now fails because the
"servald" executable is not found by the system(3) call -- need to give the
full path name, or avoid the use of system(3) altogether.
2012-09-28 17:53:20 +09:30
Andrew Bettison
4379be5c50 Issue #9, improve rhizome HttpImport test
Disable any inherited HTTP proxy settings that could interfere with curl(1).

Better invocation of curl(1) to fail on error response.
2012-09-28 17:50:49 +09:30
Andrew Bettison
62f8d223ea Improve alloca_toprint() functions
Replace "..." quoting with `...` quoting in output, to avoid slosh-escaping the
common double-quote character (") in log output.

Introduce alloca_str_toprint() function that produces an entire null-terminated
string in printable form.

Change various toprint strbuf helper functions to take two optional quote chars
instead of one mandatory.
2012-09-28 17:46:40 +09:30
Jeremy Lakeman
a182d1e53a Improve vomp console messages 2012-09-28 16:09:25 +09:30