Commit Graph

231 Commits

Author SHA1 Message Date
Andrew Bettison
a9d3a1387c Issue #30, rewrite Rhizome fetch queues and slots
Causes SEGV in rhizomestress test.
2012-10-23 18:10:20 +10:30
Jeremy Lakeman
f1b0374b97 Extract crypto signing primitives from mdp processing 2012-10-19 14:54:41 +10:30
Andrew Bettison
839de7557c Merge branch 'rhizomestress' into 'master' 2012-10-18 17:57:10 +10:30
Andrew Bettison
65ea612e27 Fix bugs revealed by Rhizome stress test
Overhauled the file fetch queue logic in rhizome_fetch.c.

Now the 'rhizomeprotocol' stress test passes in approximately 5 minutes on my
2009-vintage Dell laptop.

Added a call to rhizome_enqueue_suggestions() in rhizome_fetch_close() so that
a new Rhizome GET request is sent as soon as a fetch slot becomes free, instead
of waiting for the (default 5 second) timer to trigger the next GET.
2012-10-18 17:30:22 +10:30
gardners
499c4018bf Imported ref10/ implementation of crypto_sign from supercop-20120525
as recommended a while back by Dan Bernstein as offering the fastest
implementation of the crypto_sign() primitives for ARM.
Indeed this implementation IS faster. See comparison below for a
Rock 500 handset (800MHz(?) ARM6, no NEON):

Original ref/ implementation on an R500 stock rom (non-rooted)::
mean signature generation time = 96.80ms
mean signature verification time = 272.20ms

ref10/ implementations on an R500 stock rom (non-rooted):
mean signature generation time = 4.00ms
mean signature verification time = 13.00ms

Approximately 20x speed up, just like that :)
2012-10-18 17:26:48 +10:30
Andrew Bettison
259b9a9d24 Logging/debug improvements: __WHENCE__, __whence
Introduce __WHENCE__ macro and a block comment in log.h explaining it.

In "primitive" kinds of functions, rename 'whence' arguments to '__whence' and
use WHYF(), WARNF(), DEBUGF() macros instead of calling logMessage() directly.
2012-10-16 17:00:03 +10:30
Andrew Bettison
52ccd07020 Issue #17, improve speed of "rhizome extract manifest"
In the case that the MANIFESTS 'author' column is not NULL, do not perform a
full bundle secret verification in order to clear the '.readonly' flag, just
check whether the author's SID is present in the keyring with a proper-size
rhizome secret.
2012-10-15 18:33:44 +10:30
Andrew Bettison
3f1f495d68 Issue #17, overhaul manifest author crypto logic
Add test case for new feature of the "rhizome add" command: if the author SID
is not specified (empty arg) then it searches the keyring for the author.

Removed "authorSid" argument from several functions that also take a struct
rhizome_manifest * arg, since the author, if known, is now supplied in the
struct.

Improve return value handling and refactored some rhizome crypto code.
2012-10-11 17:58:24 +10:30
Andrew Bettison
2a8cfb0404 Quieten Rhizome advertisement debug logging
Introduce DEBUG_RHIZOME_ADS and sqlite_set_debugmask(), use in
rhizome_packetformats.c advertisement mustering and processing code.
2012-10-10 13:22:30 +10:30
Andrew Bettison
3678522872 Issue #17, add AUTHOR column to Rhizome MANIFESTS table
Replace ".selfsigned" column with ".author" and ".fromhere" columns in
output of "rhizome list" command.  (Note that a "sender" column is
already present.)

Add 'author' field to struct rhizome_manifest.

Log all fully rendered SQL statements on DEBUG_RHIZOME.

Update 'rhizomeops' test cases and improve the assert_rhizome_list()
test function to be able to assert authorship of files.
2012-10-09 17:48:06 +10:30
Andrew Bettison
89343c69cd Merge branch 'rhizomedirect' into 'master' 2012-10-05 17:45:30 +09:30
Andrew Bettison
d470ac730b Cosmetic improvement for better self-documentation 2012-10-04 18:11:16 +09:30
Andrew Bettison
2e7d19a3f7 Issue #9, fix rhizome direct push and sync tests
The "rhizome direct push" command (and also sync) was not waiting for the
server's HTTP response, so it was exiting before the server had finished
storing the bundle, which led to a race with the subsequent "assert
bundle_received_by" test.  Fixed by adding the missing code to receive the HTTP
response.

Refactored the code used for parsing HTTP responses in rhizome_fetch.c, and
used it in rhizome_direct_http.c.
2012-10-04 17:38:33 +09:30
Andrew Bettison
3040d2b665 Add Rhizome DB retries on SQL statement preparation
It turns out that if the DB is locked, sqlite_prepare_v2() call can return
SQLITE_BUSY.  The retry logic (implemented for issue #2) only provided for
sqlite_step() to return SQLITE_BUSY.  It was a fairly straightforward matter to
extend the retry logic to cover statement preparation in an equally general
fashion.

The problem was observed while diagnosing failures in the rhizomeprotocol
DirectPush test case: the "servald rhizome list" command was failing due to a
locked database.  See issue #9.
2012-10-04 14:30:20 +09:30
gardners
dc3137707f rhizome import bundle from file sans-manifest via http now works.
Must be enabled by using rhizome.api.addfile.*
Certainly polishing to be done, including using filename supplied
during HTTP POST.  Now to fix that, and make it all work with
final rhizomeprotocol test case.
rhizomeprotocol test cases 8 and 9 currently fail post-merge. #9
2012-10-03 16:13:06 +02:00
gardners
a81dafa180 work towards making rhizome direct http pull work. #9 2012-10-03 15:49:40 +02:00
gardners
92a5423b17 Moved rhizome direct imports to be in-process instead of
out-of-process. rhizomeprotocol push test still fails.
Investigating. #9
2012-10-03 12:28:21 +02:00
gardners
e015f0670b Switched to new BAR format with 15 bytes of BID prefix, and TTL
at the end, and log2(filesize) instead of filesize.  Equally
importantly BAR construction and parsing now uses #defines for
field sizes and offsets instead of it being hardwired without
meaningful documentation.
WILL BREAK BACKWARD COMPATIBILITY WITH PREVIOUS BUILDS.
YOU MUST DELETE AND REBUILD YOUR RHIZOME DATABASE AS OLD-FORMAT
BIDs WILL BE IN THERE AND GET SENT, AND STRANGE THINGS WILL HAPPEN.
This break with backwards compatibility is only reasonable to
consider because we have not yet had an official build using the
new Rhizome with old BAR format.  0.08 uses old Rhizome.  #9
2012-10-03 12:22:59 +02:00
gardners
c356ead09c fixed bug in rhizome direct http push code.
Rhizome direct push via HTTP now works. #9
2012-10-03 12:22:52 +02:00
Andrew Bettison
5fed0460e1 Issue #9, 'rhizomeprotocol' HttpImport test now passes
Any HTTP client can import a bundle into Rhizome using a PUSH request.
2012-10-02 17:15:14 +09:30
Andrew Bettison
17ef2249e4 Issue #9, rewrite rhizome_import_bundle()
Objective is to avoid having to call system("servald rhizome import ...") to
handle a Rhizome direct POST /rhizome/bundle request.  Antiquated code in and
around rhizome_import_bundle() needs much cleaning up, as indicated by some
TODO comments.  Invocations must unnecessarily write the manifest into a file,
when they already have it in memory, ready to pass to the function.

All the 'rhizomeops' tests pass, but two 'rhizomeprotocol' tests are broken
by the changes in this commit.
2012-09-28 17:55:43 +09:30
Jeremy Lakeman
ac3864ff20 Reuse command line parsing framework with monitor interface 2012-09-25 13:31:34 +09:30
gardners
c23d424d4d more work towards rhizome direct http actually pushing new/updated
bundles to remote party.
2012-09-22 15:21:02 +09:30
gardners
cae11bd444 fixed rhizome direct response generation to allow request and
responses to be of different sizes. #9
2012-09-21 14:04:53 +02:00
gardners
f7e75d5347 debugging and abstraction improvements in rhizome direct
synchronisation code. #9
2012-09-21 13:41:27 +02:00
gardners
692eb2cdc9 added BAR list comparison code for Rhizome direct.
Now returns series of "I have [newer]"'s and "Please send me"'s,
consisting of a 1 byte ID (0x01 or 0x02 respectively), followed
by the 64bit BID prefix from the BAR.  As with all of Rhizome
Direct at present, the geo bounding box is ignored for now.
2012-09-14 12:10:14 +08:00
gardners
6553050566 rhizome direct enquiry request now gets back enquiry response,
and realises that it needs to process it.  It does not yet
process it. #9
2012-09-11 14:34:58 +09:30
gardners
9a5dbbd14f rhizome direct http transport now asks for response to enquiry when
enquiry received by server.  Generating responses to enquiries not
yet implemented. #9
2012-09-11 13:09:40 +09:30
gardners
4f5f2c2444 work towards http transport working for rhizome direct. #9 2012-09-10 20:22:14 +09:30
gardners
dd01f7939d further work on rhizome direct dispatching etc.
For some reason finds the same manifest several times (size bin
filtering seems to not be working right).
Also sync doesn't realise it has finished, and so doesn't return
when done.
2012-09-10 09:16:27 +09:30
gardners
27fc7a44d0 rhizome direct cursor operations now include pickled cursor range
in output buffer (but doesn't yet report the range correctly). #9
2012-09-09 08:58:47 +09:30
gardners
0207d8e541 added cursor limit pickling, restoration functions. cursor
functions now try to honour these. Not yet tested. #9
2012-09-09 08:30:38 +09:30
gardners
53228f078e rhizome direct now has a cursor function to iterate through all
bundles in the rhizome database in an appropriate priority order
(currently small before big). #9
2012-09-08 20:26:18 +09:30
gardners
3e4b20f5df rhizome_direct_get_bars() now allows selections to be based on
size of associated data in a bundle, so that we can synchronise
small things first.  Also preliminary work on making a general
cursor-type wrapper function for get_bars() so that it is easy
for any rhizome direct transport driver to iterate over the
known bundles in a rhizome datastore. #9
2012-09-08 20:00:14 +09:30
gardners
b13c23de6a rhizome_direct_get_bars() now sets bid_high correctly. #9 2012-09-08 09:35:58 +09:30
gardners
89b09563a3 Wrote function to get a range of BARs from the Rhizome database
for use in Rhizome Direct synchronisation requests. #9
2012-09-08 07:55:57 +09:30
gardners
d3f5c6e598 Merged rhizome direct httpd back into main rhizome httpd server.
bundles can be imported via HTTP POST multi-part form. #9
2012-09-06 13:21:13 +09:30
gardners
3dad0798a4 further work on multipart POST form parser. Now does almost
everything except actually write manifest and data to files
for importing. #9
2012-09-06 06:29:05 +09:30
gardners
db79190c81 more work on multi-part POST form data parser. #9 2012-09-06 05:15:25 +09:30
gardners
e73f0262c7 Added prototype for HTTP multi-part post parser for rhizome direct. 2012-09-05 15:51:22 +09:30
gardners
ed7edd3865 Added boundarystring field to rhizome_http_request structure.
Added call to new (currently stub) function for processing bytes
received from a HTTP POST multipart encoded form. #9
2012-09-01 14:11:23 +09:30
gardners
8170df7447 added extra field to rhizome http request structure and update
rhizome direct code to remember the path in a multipart form post
operation, and also the boundary string that will separate the
parts.
2012-08-31 13:15:05 +09:30
gardners
389ae8e989 added skeleton HTTP request parser for rhizome direct.
Doesn't know how to do POST yet, which is needed for all rhizome
direct operations. #9
2012-08-31 13:01:29 +09:30
gardners
9d2aa61792 Refactoring of Rhizome http server code so that we can supply a
different request parser, but otherwise share code between
rhizome transfers and rhizome direct. #9
2012-08-31 13:01:28 +09:30
gardners
bde9d1c56a Rhizome http server now allows specification of client socket callback
when starting, so that same server code can be shared for rhizome transfers
and rhizome direct. #9
2012-08-31 13:01:28 +09:30
Andrew Bettison
3f1e0dd763 Finish sleep-retry on locked Rhizome db
Closes #2.

Rewrite all Rhizome db query code using new retry primitives defined in
"rhizome.h": sqlite_step_retry(), sqlite_retry(), sqlite_retry_done(), etc.
Replace all calls to sqlite3_prepare_v2() with sqlite_prepare() which does
proper error logging.

Fix bug: re-invoking sqlite3_blob_close() on SQLITE_BUSY return causes process
to abort.  Use an explicit BEGIN...COMMIT around the blob writing code instead.

Tested using repeated invocations of batphone/tests/meshms1.

Delete deprecated Rhizome db code in rhizome_crypto.c that has been replaced
with keyring file.
2012-08-24 15:38:39 +09:30
Andrew Bettison
bb4819aa83 Issue #2, sleep-retry logic in all Rhizome operations
Much refactoring and removal of cruft.

SQL query errors are now logged with the filename, line number and function
where they were invoked, not of the low-level function that discovered the
error.  This makes use of the new __HERE__ notation introduced last commit.
2012-08-23 17:43:35 +09:30
Andrew Bettison
27a0a6eeb5 Introduce struct __sourceloc
Replaces (const char *file, unsigned int line, const char *function) arguments
to all logging functions, simplifies malloc/free tracking code in
overlay_buffer.c and Rhizome manifest alloc/free tracking in rhizome_bundle.c.

Use __HERE__ macro instead of (__FILE__, __LINE__, __FUNCTION__) everywhere.

Special __NOWHERE__ macro is equivalent to (NULL, 0, NULL).

Declare net.c functions in new "net.h" header, so log.c doesn't have to pull
in the entire "serval.h" just to use write_str().

Facilitates progress on issue #2.
2012-08-23 12:31:07 +09:30
Andrew Bettison
fd3da58a7c Issue #2, add sleep-retry logic for most database queries
All the queries that used sqlite_exec_void() and sqlite_exec_int64() and
sqlite_exec_strbuf() now do a sleep-retry while the Rhizome db is locked.

There are other queries that still need conversion, and some old infinite
retry logic that needs replacing.
2012-08-22 19:09:30 +09:30
Andrew Bettison
5cbc2167ec Start on problem of database lock errors
Add sqlite_exec_void_retry() function, use it in
rhizome_update_file_priority().  This should be reviewed to ensure that the
server process never sleeps.

The general problem remains of what the servald process should do if the
database is locked when it tries to update.  Simplest solution is to sleep and
retry, but that blocks all other services and would hurt VoMP.  A better
solution would be for each Rhizome operation to collect its database updates
into a single transaction and place that in a work queue that gets called using
schedule() (or even watch() if a file-descriptor event can somehow be used when
the database becomes available).  Another solution is perhaps to perform all
Rhizome operations in a dedicated process that can block indefinitely on the
database without affecting servald responsiveness.
2012-08-20 18:43:48 +09:30
Andrew Bettison
12d8bc0c12 Use new typedef time_ms_t everywhere 2012-08-09 12:14:32 +09:30
Andrew Bettison
b50943d09c Fix handling of empty payload (filesize = 0)
Do not add 'filehash' var to manifest if filesize=0
Do not accept 'filehash' var when parsing manifest with filesize=0
When responding to a new rhizome advertisement, do not try to HTTP
request a payload if filesize=0, just import the manifest directly
Various operations, eg "rhizome file add", do not report 'filehash'
fields where 'filesize' is zero
Do not delete rows from MANIFESTS table which have empty filehash
Various related bug fixes
2012-07-16 18:40:30 +09:30
Andrew Bettison
c791ba94d0 Refactor HTTP response parsing
Remove need to nul-terminate the received buffers in HTTP fetch reply handling
and HTTP server request parsing.

Remove redundant copying of data.

More rigorous parsing code, probably less vulnerable to overrun exploits.

Better debug logging of requests and responses.
2012-07-13 18:06:55 +09:30
Andrew Bettison
49ee4d13f0 Re-write manifest parsing, improve rhizome_fetch.c logging
Rhizome manifest parser now parses and validates all known fields, informs
about unsupported fields, and unpacks fields into relevant struct manifest
elements where appropriate.  Is also stricter about whitespace.

Rhizome fetch code now logs debug messages if DEBUG_RHIZOME_RX bit is on.
2012-07-12 16:39:01 +09:30
Andrew Bettison
d111f763c7 Fix bugs revealed by 'rhizomeprotocol' test
Was not transmitting actual HTTP server port in rhizome announcements, was
always transmitting port 4110.

When trying for a free HTTP server port, sometimes bind() succeeds but listen()
fails with EADDRINUSE, so new logic to deal with that.
2012-07-12 12:10:59 +09:30
Andrew Bettison
ff2c98afb3 Add "rhizome.fetch_interval_ms" config option 2012-07-11 16:51:30 +09:30
Andrew Bettison
8020ea3b74 Merge branch 'eventscheduler' into master
Conflicts:
	commandline.c
	monitor.c
	overlay.c
	overlay_interface.c
	overlay_packetformats.c
	rhizome_fetch.c
	rhizome_http.c
	rhizome_packetformats.c
	serval.h
	server.c
	testdefs.sh
	testframework.sh
	tests/dnaprotocol
	tests/server
2012-07-03 10:26:22 +09:30
Andrew Bettison
08a8ec13e8 Start rhizome HTTP server on first available port 2012-07-02 16:24:07 +09:30
Jeremy Lakeman
89566e4d3d Refactor how functions are scheduled or file handes are watched 2012-07-02 13:19:54 +09:30
Jeremy Lakeman
29cba17891 Fix rhizome transfers 2012-06-27 16:54:42 +09:30
gardners
b40a468276 Merge branch 'eventscheduler'
Conflicts:
	monitor.c
	serval.h
2012-06-25 16:50:23 +09:30
Andrew Bettison
208b9c15fd Factor out rhizome_hex_to_bytes() and chartonybl() 2012-06-25 14:51:21 +09:30
gardners
5dc6d122a5 significant progress towards clean callback scheduler for poll()
events and timed callbacks.
2012-06-22 16:05:49 +09:30
Andrew Bettison
56cb12f507 Refactor rhizome db creation and execution functions 2012-06-12 18:12:36 +09:30
Andrew Bettison
637ef02f30 Add _selfsigned column to "rhizome list" output
Add <pin,pin...> arg1 to "rhizome list" command, so it can open the keyring

Some tests in tests/rhizome fail
2012-06-08 18:25:43 +09:30
Andrew Bettison
7a71a521f8 Pass author SID parameters in binary, not hex 2012-06-08 15:27:15 +09:30
Andrew Bettison
86c7819f97 Remove spurious ERROR messsages
Tests assert that stderr contains no ERROR: lines after a successful exit

Rewrote sqlite_exec_int64() to separate error outcomes from legitimate
result values

Changed several WHY() calls to DEBUG()

Improved test framework
2012-06-08 13:13:26 +09:30
Andrew Bettison
a9ad1b6afc Add <bsk> optional arg to "rhizome add file"
Improve regular expressions for common data types in test scripts

Revert column count field delimiter in "rhizome list" from ":" to "\n"

Add a few more test cases
2012-06-05 13:58:59 +09:30
Andrew Bettison
304db6e5f4 Fix SEGV bug in "rhizome extract file" command
Add optional <key> argument, validate and parse it properly.
2012-06-01 18:12:59 +09:30
gardners
2aaea99bfa added function for computing bundle shared secret, e.g., for
MeshMS encryption.  Currently just returns NULL (no encryption),
and not yet hooked in.
2012-05-28 13:51:24 +02:00
Jeremy Lakeman
36389d2b78 Insert manifest within a transaction 2012-05-28 14:08:01 +09:30
gardners
30e53f963c fixed various manifest synchronisation bugs, including probably
the main one that was stopping meshms (large manifests would
practically never get advertised under certain (common) conditions).
2012-05-27 08:31:48 +02:00
gardners
94d17a9260 more work on encrypting bundles. code passes all 23 rhizome tests. 2012-05-26 11:36:19 +02:00
gardners
58cc45043b added code to encrypt files when storing into the database. 2012-05-26 11:36:19 +02:00
gardners
2de6bb025a rhizome_extract_file can now decrypt a file, and also uses
progressive blob operations so that we can extract files of
unlimited size.
2012-05-26 11:36:19 +02:00
gardners
04cc05d6e1 refactored manifest verification into a separate function
that is called only when needed, and marks a manifest as finalised
if the verifcation fails.  reading a manifest now never sets
finalised flag, as either _finalise() or _verify() must be called.
2012-05-26 11:36:18 +02:00
gardners
e213a9eae8 renamed sanity_check to check_sanity for consistency.
Put missing check for illegal service type into check_sanity.
2012-05-26 11:36:18 +02:00
gardners
8161709c4e work on fixing handling of duplicate rhizome bundle debouncing.
(seems to work from command line, but not from the test framework).
2012-05-26 11:36:18 +02:00
gardners
b0dca7e03f merged in latest changes from team. 2012-05-26 11:36:18 +02:00
Andrew Bettison
d215d90b40 Refactored some rhizome db SQL code
Added sqlite_exec_void() function, so that sqlite_exec_int64() can return
an error if no rows are found instead of 0, which can be misleading.
2012-05-25 19:42:45 +09:30
Andrew Bettison
bea9188cd5 Fix another manifest leak
Add manifest alloc/free debug logging to help, 'debug.manifests' option
2012-05-25 17:01:56 +09:30
Andrew Bettison
49aec4d331 Improve rhizome manifest debugging
Move rhizome_new_manifest() out of rhizome_read_manifest_file() so that the
out-of-manifest report shows the names of the functions where the manifests
were really allocated.
2012-05-25 15:38:13 +09:30
Andrew Bettison
c0ac693957 Remove old payload when updating a manifest with a new payload 2012-05-25 14:29:55 +09:30
Andrew Bettison
fa26b53294 Fix SEGV bug
rhizome_write_manifest_file() was not checking for NULL fopen() result
Standardise rhizome "import" directory pathname handling
2012-05-24 11:28:32 +09:30
Andrew Bettison
1af9125392 All rhizome database keys are uppercase hex
FILES.id, MANIFESTS.id, FILEMANIFESTS.fileid, FILEMANIFESTS.manifestid
Named constants for hex and binary ID sizes
2012-05-23 16:04:00 +09:30
gardners
e3387fbecf periodic sucking in from rhizome prioritised list now works. 2012-05-22 15:57:18 +09:30
gardners
f528110257 almost have priority pre-listing of bundles for rhizome working. 2012-05-22 15:57:18 +09:30
gardners
0abc7ff96e added code to create and maintain priority download list for
rhizome, so that we get things in a sensible order.
2012-05-22 15:57:18 +09:30
gardners
8e9396419c adding ignored manifest list to let us ignore repeated offerings
of broken manifests without wasting effort.
2012-05-22 14:15:35 +09:30
gardners
29bb86d9e7 first cut at supressing rhizome activity during calls. 2012-05-22 13:05:29 +09:30
Andrew Bettison
e6575cb369 Introduce macro constants for Rhizome service names
Use "MeshMS1" instead of "MeshMS"... version 2 won't be far away
2012-05-20 16:07:22 +09:30
Andrew Bettison
33f7ea7103 Add three optional args to "rhizome list" command
service, sender_sid, recipient_sid - not yet implemented
2012-05-16 13:47:58 +09:30
gardners
0eafc34970 BK encoding and decoding complete, but untested. 2012-05-15 20:04:41 +09:30
gardners
525d3c4154 substantial work towards implementing BK field in rhizome manifests. 2012-05-15 17:24:25 +09:30
Andrew Bettison
f90b21ec78 Implement rhizome config options
rhizome.datastore_path - if not set, reverts to serval instance path
rhizome.enable - if not set, defaults to true
2012-05-15 12:56:10 +09:30
Andrew Bettison
b02e9b9ac4 Overhaul rhizome datastore path code
Create datastore directory and all parent dirs if it does not exist
More robust error checking and buffer overflow protection
New mkdirs() and mkdirsn() functions
2012-05-14 18:37:32 +09:30
gardners
0b5a9665b2 first cut code for sending monitor socket announcements when
rhizome bundles are received.
2012-05-12 07:24:52 +09:30
Andrew Bettison
7ba15ccdd7 Add "rhizome extract file" command with tests 2012-05-02 17:57:35 +09:30
Andrew Bettison
146eb7cdfd Add "rhizome extract manifest" command 2012-05-02 16:03:09 +09:30
Andrew Bettison
a73916e85a Implement JNI command-line interface (untested)
- argv is (const char *const *) everywhere, to avoid having to strdup() all the
  Java arg strings
- changed (char*) to (const char*) in lots of places to fix cascading compiler
  warnings as a result of argv constness
- fixed a bug in "config get" command when used without 'variable' arg
2012-04-23 17:18:15 +09:30
Andrew Bettison
de88d3db21 Make "dna rhizome add" work for more test cases
- adding a bundle using an existing manifest with an incorrect payload should
   fail with an error
 - adding a bundle using an existing manifest to update the payload to a new
   version should succeed
 - improve format of "dna rhizome list" output to one bundle per line
2012-04-13 18:03:19 +09:30
Andrew Bettison
7b53fd6782 Rhizome add does not add duplicate files
- If a bundle has the same payload hash and manifest.name as one already in
   the rhizome store, then the "dna rhizome add" command will not add it
 - Now passes fourth test case in tests/dna_rhizome
2012-04-12 18:33:43 +09:30
Andrew Bettison
f52db7f406 Fix bugs revealed by rhizome "add manifest" test case 2012-04-10 18:13:05 +09:30
Andrew Bettison
74986a0c30 Implement "rhizome add file" and "rhizome list" 2012-04-02 17:42:40 +09:30
Daniel O'Connor
a8411edf3f - Use system wide sqlite3 lib via pkg-config
- Remove nacl-gcc-prep, it's now in nacl
- Sort sources and have 1 per line.
- Add missing source files.
- Use naclinc.txt and nacllib.txt generated by nacl/nacl-gcc-prep
- Update ignore file now nacl files don't pollute this directory.
2012-02-27 12:41:48 +10:30
gardners
8a24fb95b1 Converted manifest storage to using static set of manifest structures
and added manifest leak detection and some other memory corruption
protections.
2012-01-28 11:45:45 +10:30
gardners
45e8d68522 Added a manifest:version cache to avoid database lookups when being
offered manifests, thus saving cpu/energy.
2012-01-27 17:11:18 +10:30
gardners
c0676fd530 Fixed various bugs and memory leaks in rhizome file fetching.
Now successfully fetches manifests for files we already have in
the database, using the existing database copy of the file.
Also added filter to stop same file being fetched more than once at
same time..
2012-01-27 16:21:48 +10:30
gardners
cdf1c45da4 Rhizome fetching over http now receives files and imports them.
Still some wrinkles to work out.
2012-01-13 21:13:17 +10:30
gardners
f862ba0af6 Rhizome file transfer via http getting close
(send side needs to send file body)
2012-01-13 17:21:06 +10:30
gardners
bf9f8559f1 Various debug fiddles and conversion from select() to poll()
(although poll() still doesn't let us monitor ordinary files
which is silly.)
2012-01-12 16:47:24 +10:30
gardners
42b5c08321 Refactored rhizome.c into several smaller files.
Manifests are now gleaned from advertisements, and the ones that
are interesting are scheduled for importing into database.
(actual importing is not yet done -- requires fetching of file).
2012-01-12 14:05:05 +10:30
gardners
4e6ef2e69e Various fixes to Rhizome web server and BAR generation.
Can now present a set of BARs via HTTP.
2012-01-03 16:35:02 +10:30
gardners
6231e8784c Added ability to de-hex result rows when streaming results of sql
query.
2012-01-03 14:54:32 +10:30
gardners
c380ab0450 More work on rhizome web server.
Can now serve lists of manifest/file IDs.
2012-01-03 14:45:50 +10:30
gardners
9c8ea5908c Various fixes towards working internal web server. 2012-01-03 08:57:52 +10:30
gardners
d29efac21f More work on getting Rhizome web server working.
Can now produce parsable http responses as well as parse requests.
Now to actually implement some of the requests ...
2011-12-29 10:11:03 +10:30
gardners
ccf26e1fe0 Rhizome web server can now serve a 400 error message, which means
that lots of the underlying stuff is now in place.
2011-12-23 04:25:18 +10:30
gardners
16fb3a3f61 Added missing GPL license statements. 2011-12-21 20:25:05 +10:30
gardners
cbd0cf6117 Began work on rhizome integrated web server. 2011-12-21 16:28:08 +10:30
gardners
b7b795d4fc manifest reading now reads and can verify signatures.
Still incomplete. Doesn't yet count verify errors etc.
2011-12-21 07:46:12 +10:30
gardners
a159edf152 updating manifests now correctly removes old files and old
versions of the manifest.
group subscriptions not yet implemented, and reading of signatures
not yet implemented.  Controls on group membership and pushing files
to groups not yet implemented.  Group ciphers not yet implemented.
2011-12-20 22:09:49 +10:30
gardners
745cade469 Further work on getting signatures and signed manifests working.
Some work on updating existing manifests, although some cascading
consequences need addressing (in particular checking if the file
associated with the old version of the manifest still needs to hang
about, and if so, whether its highest priority has changed).
2011-12-20 17:27:24 +10:30
gardners
cc09a0f994 Some database schema tweaks and additional checks on manifests when
importing (we now need to have either the key for the manifest, or to
be creating the manifest).
2011-12-20 15:48:26 +10:30
gardners
a50c56ac92 Importing of a bundle into the database now seems to work.
(group membership not currently handled, signing not currently
handled).
2011-12-20 13:24:09 +10:30
gardners
312eaa397b Significant progress on Rhizome.
Manifest reading and writing to disk (not database) seems okay now.
Working on database side of things next.
2011-12-20 11:25:52 +10:30
gardners
17f0b5da10 Moved BYTE_ORDER definition to Android.mk 2011-12-19 09:01:03 +10:30
gardners
d5004e15d2 Fix various Android NDK build issues
(specify endianness, copy libsqlite.so from an Android phone
for link templating)
2011-12-19 08:56:25 +10:30
gardners
16d22d41a3 Fixed crypto structure size specifications to specify the exact
crypto implementation (wise anyway, but required for our Android
cross-compile of NaCl).
2011-12-19 08:22:34 +10:30
gardners
c110e46a08 fixes to previous commit. 2011-12-19 08:10:02 +10:30
gardners
000dd2db55 Moved rhizome prototypes into separate header file.
(needed for android building where undefined functions are
an error, not a warning).
2011-12-19 08:04:31 +10:30