Commit Graph

2891 Commits

Author SHA1 Message Date
Andrew Bettison
6e4acb6ab9 HTTP /restful/rhizome/newsince/... token validation
Ensure that token has the same UUID as the Rhizome database.
2013-11-19 16:56:24 +10:30
Andrew Bettison
fce0893173 Implement HTTP /restful/rhizome/newsince/.../bundlelist.json
Now functions as per requirements, blocks for 60 seconds sending
new bundles as they appear, sends new token with each new bundle.
Tested manually.
2013-11-19 15:43:51 +10:30
Andrew Bettison
21b10b2cb7 Rhizome list cursor 'rowid_since' semantics
Setting 'rowid_since' now causes the cursor to iterate over bundles
in chronological order (oldest first) from that rowid upward.
2013-11-19 15:41:12 +10:30
Andrew Bettison
1acfff6ab5 Add HTTP server pause response function 2013-11-19 15:38:45 +10:30
Andrew Bettison
2c07f7888e Improve 'debug.io' fdqueue.c debug logging 2013-11-19 15:37:18 +10:30
Andrew Bettison
cb7b465888 Reduce Rhizome SQL trace logging to once per prepared statement 2013-11-18 16:24:03 +10:30
Andrew Bettison
c1f0c0cb07 Implement HTTP /restful/rhizome/newsince/.../bundlelist.json
Not tested
2013-11-18 16:23:27 +10:30
Andrew Bettison
27052384ed Add Base64URL encode/decode functions 2013-11-18 16:22:55 +10:30
Andrew Bettison
f8f7716544 Improve HTTP /restful/rhizome/bundlelist.json token
Now tokens are included in each row instead of a single token in the
enclosing JSON object.  Only puts tokens on the first row, and
thereafter all rows with a rowid higher than for the previous row that
had a token.
2013-11-18 15:10:28 +10:30
Andrew Bettison
fb46ae0289 Encode HTTP /restful/rhizome/bundlelist.json token in Base64 2013-11-18 11:40:34 +10:30
Andrew Bettison
e59a62115b Add base64_encodev() multi-buffer Base64 encoder
Also replace base64_encode_len() inline function with macro
BASE64_ENCODED_LEN()
2013-11-18 11:39:23 +10:30
Andrew Bettison
b8e0859880 Fix off-by-one buffer size errors when using tohex() 2013-11-15 17:10:33 +10:30
Andrew Bettison
f2b652c094 Refactor Base64 decoding from HTTP server to str.c
Also add a Base64 encoding function, not tested yet
2013-11-15 17:09:55 +10:30
Andrew Bettison
4fd94783d3 Refactor URI character class functions
To use array lookup instead of boolean OR expressions
2013-11-15 11:19:51 +10:30
Andrew Bettison
9418f9d65d Refactor HTTP string utils into str.[hc]
Improve "str.h" layout and commenting.  Re-implement hexvalue().
2013-11-14 23:13:37 +10:30
Andrew Bettison
973bb9c897 Merge branch 'naf4' into 'development' 2013-11-13 17:59:01 +10:30
Andrew Bettison
4380fdcccd Token in /restful/rhizome/bundlelist.json output 2013-11-13 16:58:28 +10:30
Andrew Bettison
a14326deeb Log SQLite PRAGMA statements as well 2013-11-13 12:45:32 +10:30
Andrew Bettison
1634d68dd0 Format UUID strings as lower case hex 2013-11-13 12:45:02 +10:30
Andrew Bettison
64db53a092 Add random UUID to Rhizome database 2013-11-13 12:44:31 +10:30
Jeremy Lakeman
21b123212c Merge branch 'naf5' into development 2013-11-13 12:32:55 +10:30
Jeremy Lakeman
0b404b0260 Implement id list command 2013-11-13 12:00:46 +10:30
Jeremy Lakeman
ff1f1ff093 Allow tag primitives to be used outside of keyring entries 2013-11-13 12:00:05 +10:30
Jeremy Lakeman
c618c724e1 Ensure that invalid subscriber id arguments fail validation 2013-11-13 12:00:02 +10:30
Jeremy Lakeman
9680b24f23 Add support for adding and updating tags in the keyring 2013-11-13 11:59:21 +10:30
Jeremy Lakeman
e5e95f00f8 Pass key length into unpack function 2013-11-13 11:59:18 +10:30
Jeremy Lakeman
8c03f18e5e Refactor keyring searching 2013-11-13 11:59:13 +10:30
Jeremy Lakeman
4434b1b65d Create keyring.h and move definitions there 2013-11-13 11:55:12 +10:30
Jeremy Lakeman
534b01ba2a Add explicit set did test 2013-11-13 11:52:46 +10:30
Andrew Bettison
50bbb722d0 Add uuid_t and UUID primitive functions 2013-11-12 17:44:03 +10:30
Andrew Bettison
b44046d612 Forbid HTTP /restful/rhizome/bundlelist.json except from loopback 2013-11-12 11:40:47 +10:30
Andrew Bettison
ba0ab14c69 Clean up inclusion of <netinet/in.h>
Make all #include <netinet/in.h> conditional upon HAVE_NETINET_IN_H

Remove unnecessary #include <netinet/in.h> from source files

Reformat include block in "serval.h" for readability
2013-11-12 11:39:06 +10:30
Andrew Bettison
13634f8748 Add ROWID field to struct rhizome_manifest
New ".rowid" output field from rhizome add, import, extract, export
operations.  (Also added missing ".inserttime" and "date" fields to
some operations.)

Use new "rhizome add file" .rowid output field to check output of of
/restful/rhizome/bundlelist.json
2013-11-11 18:18:08 +10:30
Andrew Bettison
6b961c56ce Fix HTTP /restful/rhizome/bundlelist.json
Write test case assertions using jq(1) utility, increase from four
bundles to 100.

Fix bugs in HTTP server content generation logic.

Make payload content generator read payload 4KiB at a time, to
always read on filesystem block boundaries for performance.  Increase
size of payload in relevant test case.
2013-11-11 16:21:26 +10:30
Andrew Bettison
701f14fdf9 Improve test defs: extract_stdout_keyvalue_optional()
Logs its variable assignment
2013-11-11 16:17:16 +10:30
Andrew Bettison
ca66a387dd Add "date" output field to Rhizome commands
To wit: "add file" and "import bundle"
2013-11-11 16:16:04 +10:30
Andrew Bettison
d5b48f5a9e Improve HTTP server generated content logic
Content generator functions now take arguments describing the buffer
they are to fill, and respond with a struct containing the number of
bytes filled, and the number of free bytes needed before being called
again.

The HTTP response logic now fills the buffer as much as possible before
calling write(2) by topping it up instead of waiting for it to be
completely emptied before generating more content.
2013-11-09 11:31:21 +10:30
Andrew Bettison
a62b6f9250 Improve Rhizome list cursor
Separate function rhizome_list_commit() which, if not called, causes the
next list re-query to include the last row that was fetched with
rhizome_list_next(), rather than excluding it.
2013-11-08 18:29:49 +10:30
Andrew Bettison
32ce7f5ed9 Do not catch SIGQUIT
So that it can be used to make a core dump during manual debugging
2013-11-08 18:25:33 +10:30
Andrew Bettison
1b906f3f11 Implement HTTP /restful/rhizome/bundlelist.json
Only tested for one bundle.
2013-11-07 23:40:56 +10:30
Andrew Bettison
9ecf9891fb Add strbuf JSON helper functions 2013-11-07 23:39:53 +10:30
Andrew Bettison
d337542067 Improve HTTP server generated content logic
Support generated content with an unspecified Content-Length.  Generator
functions return 1 if there is more content to come, and 0 if they have
just produced the last piece of content.
2013-11-07 23:39:24 +10:30
Andrew Bettison
051eca4775 Improve Rhizome list cursor
Remove 'rowcount' element

Order by descending ROWID, which is functionally the same as descending
inserttime but more reliable

Replace '_offset' cursor element with rowid first-last range to record
the expanding window of rows already returned, which allows release and
re-open of cursor mid-listing without missing rows or producing duplicates
2013-11-07 17:22:06 +10:30
Andrew Bettison
47988bfe0f Make Content-Length optional in HTTP responses
To eventually support the /restful/rhizome/bundlelist.json query which
will not compute in advance the length of its response.
2013-11-07 17:18:38 +10:30
Andrew Bettison
4f9cbeab20 Remove unused elements from struct rhizome_http_request
Also improve comments
2013-11-07 17:14:09 +10:30
Andrew Bettison
9606b4b1b8 Refactor "rhizome list" command
Move output formatting into commandline.c, leave database query in
rhizome_database.c
2013-11-07 10:21:11 +10:30
Andrew Bettison
e00c945fd1 Merge branch 'naf4' into 'development'
Big refactor of Rhizome author and bundle secret handling.
2013-11-07 00:42:42 +10:30
Andrew Bettison
45442d3eb4 Rewrite bundle author authentication
Replaced 'int has_author' manifest element with new 'enum authorship'
element to record the result of author authentication, to avoid
repeating expensive crypto operations.

Separated the handling of bundle secret arguments from author lookup and
authentication.  The new rhizome_apply_bundle_secret(m,bsk) is now
called at the top level to set the manifest secret key (if it
validates), and thereafter there is no need to pass the 'bsk' argument
to any other functions, as they can simply check the 'haveSecret' field
of the manifest.

Removed rhizome_extract_privatekey() which combined author lookup and
bundle secret validation, and replaced it with functions that only deal
with the author: rhizome_lookup_author() and rhizome_authenticate_author().

Renamed other functions to make their purpose and effect clearer.

Formalised the semantics of only storing AUTHENTICATED author SIDs in
the 'author' column of the MANIFESTS table, which necessitated a change
to a 'rhizomeops' test case: when adding a file using a BK-less
manifest, the author column is set to null, so the Rhizome list output
does not show the bundle as ".fromhere" and does not give an author for
that bundle.
2013-11-06 23:58:17 +10:30
Andrew Bettison
4aea05f445 Fix prototype of bcopy()
Add const to void * src argument
2013-11-06 23:56:33 +10:30
Andrew Bettison
a90eced640 Fix test framework: bug in assertStdoutIs() et al
Could overwrite the latest stdout or stderr file produced by execute()
2013-11-06 23:56:33 +10:30