Commit Graph

1444 Commits

Author SHA1 Message Date
gardners
d11109bca5 Improve rhizome direct push/pull/sync command line parsing and
use of configuration. Not yet finished, but allows multiple
rhizome direct peers and non-http URLs. #9
2012-10-03 12:24:22 +02:00
gardners
e015f0670b Switched to new BAR format with 15 bytes of BID prefix, and TTL
at the end, and log2(filesize) instead of filesize.  Equally
importantly BAR construction and parsing now uses #defines for
field sizes and offsets instead of it being hardwired without
meaningful documentation.
WILL BREAK BACKWARD COMPATIBILITY WITH PREVIOUS BUILDS.
YOU MUST DELETE AND REBUILD YOUR RHIZOME DATABASE AS OLD-FORMAT
BIDs WILL BE IN THERE AND GET SENT, AND STRANGE THINGS WILL HAPPEN.
This break with backwards compatibility is only reasonable to
consider because we have not yet had an official build using the
new Rhizome with old BAR format.  0.08 uses old Rhizome.  #9
2012-10-03 12:22:59 +02:00
gardners
d4fe995206 added note about sqlite3 query results weirdness. #9 2012-10-03 12:22:58 +02:00
gardners
0d988921ac added debug message for when sqlite returns results that it shouldn't.
(FILESIZE BETWEEN X and Y is not behaving correctly -- it sometimes
returns results with the out of bounds FILESIZE values). #9
2012-10-03 12:22:58 +02:00
gardners
c356ead09c fixed bug in rhizome direct http push code.
Rhizome direct push via HTTP now works. #9
2012-10-03 12:22:52 +02:00
Andrew Bettison
bbe6d12693 Issue #9, fix 'rhizome direct push' command
Now the 'rhizomeprotocol' Push test case now passes.  It should be renamed to
DirectPush.

Much refactoring of the Rhizome Direct HTTP request parsing.  Now uses
strbuf_sprintf() instead of snprintf() in many places to check for buffer
overrun and ensure terminating nul.  Still more of this kind of work is needed.

Improved debug that needs to be made conditional on DEBUG_RHIZOME_RX and
DEBUG_RHIZOME_TX.  Some just needs removal.
2012-10-03 17:51:37 +09:30
Andrew Bettison
245c7e7f49 Change str_str() to return (char*) not (const char*)
Also rewrite implementation to handle edge cases, rename arguments to be more
self documenting.
2012-10-03 17:13:45 +09:30
Andrew Bettison
46f81d9d2e Fix formatting bug in strbuf_toprint() et al
Was printing chars above \x7f as \xffffffhh.
2012-10-03 17:10:57 +09:30
Andrew Bettison
ebf2aa83ac Issue #9, re-order 'rhizomeprotocol' HttpImport test 2012-10-02 17:16:44 +09:30
Andrew Bettison
5fed0460e1 Issue #9, 'rhizomeprotocol' HttpImport test now passes
Any HTTP client can import a bundle into Rhizome using a PUSH request.
2012-10-02 17:15:14 +09:30
Andrew Bettison
cc9cb8d827 Issue #9, fix old 'rhizomeprotocol' test failures
Now only the five new Rhizome Direct tests fail.
2012-10-02 16:32:48 +09:30
Andrew Bettison
4daaa8f164 Fix SEGV in serval_packetvisualise() 2012-10-02 15:47:45 +09:30
Andrew Bettison
17ef2249e4 Issue #9, rewrite rhizome_import_bundle()
Objective is to avoid having to call system("servald rhizome import ...") to
handle a Rhizome direct POST /rhizome/bundle request.  Antiquated code in and
around rhizome_import_bundle() needs much cleaning up, as indicated by some
TODO comments.  Invocations must unnecessarily write the manifest into a file,
when they already have it in memory, ready to pass to the function.

All the 'rhizomeops' tests pass, but two 'rhizomeprotocol' tests are broken
by the changes in this commit.
2012-09-28 17:55:43 +09:30
Andrew Bettison
e51745e39d Issue #9, fix HTTP server "POST /rhizome/bundle"
Handle failure cases properly.  The HttpImport test now fails because the
"servald" executable is not found by the system(3) call -- need to give the
full path name, or avoid the use of system(3) altogether.
2012-09-28 17:53:20 +09:30
Andrew Bettison
4379be5c50 Issue #9, improve rhizome HttpImport test
Disable any inherited HTTP proxy settings that could interfere with curl(1).

Better invocation of curl(1) to fail on error response.
2012-09-28 17:50:49 +09:30
Andrew Bettison
62f8d223ea Improve alloca_toprint() functions
Replace "..." quoting with `...` quoting in output, to avoid slosh-escaping the
common double-quote character (") in log output.

Introduce alloca_str_toprint() function that produces an entire null-terminated
string in printable form.

Change various toprint strbuf helper functions to take two optional quote chars
instead of one mandatory.
2012-09-28 17:46:40 +09:30
Andrew Bettison
d8f06d0582 Improve test framework: tfw_cat and shell backtrace
tfw_cat() now behaves better if the file is missing a final newline.

Shell backtrace lines no longer prefixed with log prefix (timestamp).
2012-09-28 15:30:47 +09:30
Andrew Bettison
d26c3f67e8 Fix broken 'dnaprotocol' tests
Explicitly create identities in 'dnaprotocol' fixtures because
start_servald_instances() no longer does it.

The create_single_identity() function was not setting NAMEA, NAMEB, etc.  when
they contained a space, because of shell misquoting.
(cherry picked from commit 9d8707a0ad)
2012-09-28 12:16:55 +09:30
Andrew Bettison
c1695a67b2 Fix bug in 'keyring list' command
Revealed by recent rhizomeprotocol test improvements.  Was always printing
empty DID and name fields.
(cherry picked from commit 7452c215e2)
2012-09-28 12:16:55 +09:30
Andrew Bettison
c2b515347f Improve testdefs: foreach_instance() function
Cherry picked from commit 87133cb2de, conflicts
in tests/rhizomeprotocol resolved by hand: bundle_received_by() now takes two
args before list of instances.

The 'rhizomeprotocol' tests now use create_single_identity() in fixtures (which
tests the 'keyring list' command).

Use foreach_instance() function in various places, and replace functions that
used to loop over instances with functions that work on the current instance.
2012-09-28 12:15:10 +09:30
Andrew Bettison
5b8194eae0 Refactor test definitions
Cherry picked from commit e994626429, conflicts
in testdefs.sh resolved manually (start_servald_instances() sets DUMMYx
variable).

Move create_rhizome_identities() from testdefs_rhizome.sh to
create_identities() in testdefs.sh

Rename create_identity() in testdefs.sh to create_single_identity(), uses
create_identities() to do its work.
2012-09-28 12:05:22 +09:30
Andrew Bettison
04abc7db0f Fix Solaris compile error and some warnings
The mmap(2) MAP_FILE flag is not defined on Solaris, and is optional or
ignored on Mac OS X and Linux.

Two "unused variable" warnings.

Issue #9.
2012-09-28 11:07:34 +09:30
gardners
59a969cfa3 fixed one of the outstanding issues with rhizome direct http push. 2012-09-27 22:01:07 +09:30
Andrew Bettison
14721fab19 Merge branch 'solaris' into rhizomedirect
All test cases pass on Solaris (see issue #16) except the 'jni' tests because
there is no Java compiler available on Solaris, and the new 'rhizomeprotocol'
tests from the rhizomedirect branch, which have always failed.
2012-09-27 17:38:16 +09:30
Andrew Bettison
b91fc4e5b5 Refactor some 'rhizomeprotocol' tests
Replace explicit inline binary data from HttpImport test, so that Git stops
thinking the shell script is a binary file.

Put curl(1) detection into common fixture function.
2012-09-27 17:36:22 +09:30
Andrew Bettison
b44053a480 Issue #16, fix 'dnaprotocol' tests broken on Solaris
Same problem that broke most of the 'dnahelper' test fixtures; the dnahelper
script was using #!/bin/sh, which on Solaris is not Bash.
2012-09-27 16:50:29 +09:30
Andrew Bettison
76401bd31b Improve some Rhizome debug logging code 2012-09-27 15:34:13 +09:30
Andrew Bettison
ca6d8f487d Fix broken 'rhizomeprotocol' tests on Solaris 2012-09-27 15:14:43 +09:30
gardners
9ecf6499f8 now tries to write bundle file to server. server seems to hang.
Getting very close now to rhizome direct http push working, though.
2012-09-26 17:12:48 +09:30
gardners
5c2b61758e now tries to connect to rhizome direct http server to push bundles.
(manifest is written, but file is not.  manifest also seems to be
missing signature block.) #9
2012-09-26 16:47:55 +09:30
gardners
5787dd860b Rhizome direct http forms first part of POST request to push bundles
to far side. #9
2012-09-26 16:01:27 +09:30
gardners
7ef4d942b2 rhizome direct now reads manifest data given a prefix. 2012-09-26 15:15:08 +09:30
gardners
8e4496f397 fixed bug in rhizome_direct_get_manifest() #9 2012-09-26 15:03:15 +09:30
gardners
c23d424d4d more work towards rhizome direct http actually pushing new/updated
bundles to remote party.
2012-09-22 15:21:02 +09:30
gardners
f7ef416091 bug fixes for rhizome direct response handling.
Can now generate a full response buffer instead of only using 9/32
of the space. #9
2012-09-21 15:45:10 +02:00
gardners
ee89f283e0 add initial parsing of rhizome direct response. #9 2012-09-21 14:40:36 +02:00
gardners
a7d42b3c73 added note contemplating about where parsing of rhizome direct
responses should occur. #9
2012-09-21 14:31:14 +02:00
gardners
3caf82ad15 fixed bug in rhizome direct http response formation. #9 2012-09-21 14:27:02 +02:00
gardners
5d881226c1 fixed bug in comparison of IHAVE lists. Now can generate list of
BARs that rhizome direct sync requestor has not seen. #9
2012-09-21 14:10:11 +02:00
gardners
cae11bd444 fixed rhizome direct response generation to allow request and
responses to be of different sizes. #9
2012-09-21 14:04:53 +02:00
gardners
f6bab96846 fixed bug when extracting BID prefix. #9 2012-09-21 13:47:58 +02:00
gardners
f7e75d5347 debugging and abstraction improvements in rhizome direct
synchronisation code. #9
2012-09-21 13:41:27 +02:00
gardners
f0472abf43 added new rhizome direct sources to android build process. 2012-09-14 11:26:40 +02:00
gardners
692eb2cdc9 added BAR list comparison code for Rhizome direct.
Now returns series of "I have [newer]"'s and "Please send me"'s,
consisting of a 1 byte ID (0x01 or 0x02 respectively), followed
by the 64bit BID prefix from the BAR.  As with all of Rhizome
Direct at present, the geo bounding box is ignored for now.
2012-09-14 12:10:14 +08:00
Andrew Bettison
45e3d1d078 Add more debug to Rhizome fetch code
To help investigate issue #16, 'rhizomeprotocol' test case 1 failing on
Solaris.
2012-09-13 14:43:47 +09:30
Andrew Bettison
1f4688db99 #16 Fix 'rhizomeops' tests that fail on Solaris 2012-09-13 14:08:52 +09:30
Andrew Bettison
3f4ee701ec Artefacts created by libtoolize -c -i
Replaces symbolic links created from previous libtoolize -i (without -c) which
of course do not work across all platforms.
2012-09-12 15:21:50 +09:30
gardners
6553050566 rhizome direct enquiry request now gets back enquiry response,
and realises that it needs to process it.  It does not yet
process it. #9
2012-09-11 14:34:58 +09:30
gardners
9a5dbbd14f rhizome direct http transport now asks for response to enquiry when
enquiry received by server.  Generating responses to enquiries not
yet implemented. #9
2012-09-11 13:09:40 +09:30
gardners
bb0be86db3 fixed content-disposition problems. Request now gets accepted.
Need to write code to handle it now, and send cursor fill back
the otherway. #9
2012-09-11 09:34:41 +09:30