Commit Graph

2671 Commits

Author SHA1 Message Date
Andrew Bettison
a882b8ebc6 Fix Rhizome Direct stress test - longer command timeouts
Since a default timeout was added to the execute() primitive
(b1b5a79) the Rhizome Direct test fails.
2013-11-20 15:06:07 +10:30
Andrew Bettison
874a3a9ef7 Improve test framework: shell backtraces
When in a the finalise part of a test (possibly as a result of a
failure or error), make sure that shell backtraces do not list the
test case functions that failed, since they have already been
reported in a backtrace at the point of failure
2013-11-20 15:03:48 +10:30
Andrew Bettison
c5014baf62 Improve test framework: better fork functions
Replace forkKillAll with fork_terminate_all

Replace forkWaitAll with fork_wait_all

Add fork_terminate and fork_wait functions

Introduce fork labels '%<alphanumeric>' as arguments to all fork
functions

Treat a forked process exit status 143 (killed with SIGTERM) as a
failure or error -- it happens when a forked process is terminated
using fork_terminate or fork_terminate_all
2013-11-20 15:01:37 +10:30
Andrew Bettison
29fab6dcb7 Add 'rhizome.api.restful.newsince_poll_ms' config option 2013-11-20 14:50:57 +10:30
Andrew Bettison
833466411c Test case for HTTP /restful/rhizome/newsince/...
Not passing yet
2013-11-19 17:28:04 +10:30
Andrew Bettison
8ca34749de Improve test framework: slightly clearer logging 2013-11-19 17:27:25 +10:30
Andrew Bettison
6e4acb6ab9 HTTP /restful/rhizome/newsince/... token validation
Ensure that token has the same UUID as the Rhizome database.
2013-11-19 16:56:24 +10:30
Andrew Bettison
fce0893173 Implement HTTP /restful/rhizome/newsince/.../bundlelist.json
Now functions as per requirements, blocks for 60 seconds sending
new bundles as they appear, sends new token with each new bundle.
Tested manually.
2013-11-19 15:43:51 +10:30
Andrew Bettison
21b10b2cb7 Rhizome list cursor 'rowid_since' semantics
Setting 'rowid_since' now causes the cursor to iterate over bundles
in chronological order (oldest first) from that rowid upward.
2013-11-19 15:41:12 +10:30
Andrew Bettison
1acfff6ab5 Add HTTP server pause response function 2013-11-19 15:38:45 +10:30
Andrew Bettison
2c07f7888e Improve 'debug.io' fdqueue.c debug logging 2013-11-19 15:37:18 +10:30
Andrew Bettison
cb7b465888 Reduce Rhizome SQL trace logging to once per prepared statement 2013-11-18 16:24:03 +10:30
Andrew Bettison
c1f0c0cb07 Implement HTTP /restful/rhizome/newsince/.../bundlelist.json
Not tested
2013-11-18 16:23:27 +10:30
Andrew Bettison
27052384ed Add Base64URL encode/decode functions 2013-11-18 16:22:55 +10:30
Andrew Bettison
f8f7716544 Improve HTTP /restful/rhizome/bundlelist.json token
Now tokens are included in each row instead of a single token in the
enclosing JSON object.  Only puts tokens on the first row, and
thereafter all rows with a rowid higher than for the previous row that
had a token.
2013-11-18 15:10:28 +10:30
Andrew Bettison
fb46ae0289 Encode HTTP /restful/rhizome/bundlelist.json token in Base64 2013-11-18 11:40:34 +10:30
Andrew Bettison
e59a62115b Add base64_encodev() multi-buffer Base64 encoder
Also replace base64_encode_len() inline function with macro
BASE64_ENCODED_LEN()
2013-11-18 11:39:23 +10:30
Andrew Bettison
b8e0859880 Fix off-by-one buffer size errors when using tohex() 2013-11-15 17:10:33 +10:30
Andrew Bettison
f2b652c094 Refactor Base64 decoding from HTTP server to str.c
Also add a Base64 encoding function, not tested yet
2013-11-15 17:09:55 +10:30
Andrew Bettison
4fd94783d3 Refactor URI character class functions
To use array lookup instead of boolean OR expressions
2013-11-15 11:19:51 +10:30
Andrew Bettison
9418f9d65d Refactor HTTP string utils into str.[hc]
Improve "str.h" layout and commenting.  Re-implement hexvalue().
2013-11-14 23:13:37 +10:30
Andrew Bettison
973bb9c897 Merge branch 'naf4' into 'development' 2013-11-13 17:59:01 +10:30
Andrew Bettison
4380fdcccd Token in /restful/rhizome/bundlelist.json output 2013-11-13 16:58:28 +10:30
Andrew Bettison
a14326deeb Log SQLite PRAGMA statements as well 2013-11-13 12:45:32 +10:30
Andrew Bettison
1634d68dd0 Format UUID strings as lower case hex 2013-11-13 12:45:02 +10:30
Andrew Bettison
64db53a092 Add random UUID to Rhizome database 2013-11-13 12:44:31 +10:30
Jeremy Lakeman
21b123212c Merge branch 'naf5' into development 2013-11-13 12:32:55 +10:30
Jeremy Lakeman
0b404b0260 Implement id list command 2013-11-13 12:00:46 +10:30
Jeremy Lakeman
ff1f1ff093 Allow tag primitives to be used outside of keyring entries 2013-11-13 12:00:05 +10:30
Jeremy Lakeman
c618c724e1 Ensure that invalid subscriber id arguments fail validation 2013-11-13 12:00:02 +10:30
Jeremy Lakeman
9680b24f23 Add support for adding and updating tags in the keyring 2013-11-13 11:59:21 +10:30
Jeremy Lakeman
e5e95f00f8 Pass key length into unpack function 2013-11-13 11:59:18 +10:30
Jeremy Lakeman
8c03f18e5e Refactor keyring searching 2013-11-13 11:59:13 +10:30
Jeremy Lakeman
4434b1b65d Create keyring.h and move definitions there 2013-11-13 11:55:12 +10:30
Jeremy Lakeman
534b01ba2a Add explicit set did test 2013-11-13 11:52:46 +10:30
Andrew Bettison
50bbb722d0 Add uuid_t and UUID primitive functions 2013-11-12 17:44:03 +10:30
Andrew Bettison
b44046d612 Forbid HTTP /restful/rhizome/bundlelist.json except from loopback 2013-11-12 11:40:47 +10:30
Andrew Bettison
ba0ab14c69 Clean up inclusion of <netinet/in.h>
Make all #include <netinet/in.h> conditional upon HAVE_NETINET_IN_H

Remove unnecessary #include <netinet/in.h> from source files

Reformat include block in "serval.h" for readability
2013-11-12 11:39:06 +10:30
Andrew Bettison
13634f8748 Add ROWID field to struct rhizome_manifest
New ".rowid" output field from rhizome add, import, extract, export
operations.  (Also added missing ".inserttime" and "date" fields to
some operations.)

Use new "rhizome add file" .rowid output field to check output of of
/restful/rhizome/bundlelist.json
2013-11-11 18:18:08 +10:30
Andrew Bettison
6b961c56ce Fix HTTP /restful/rhizome/bundlelist.json
Write test case assertions using jq(1) utility, increase from four
bundles to 100.

Fix bugs in HTTP server content generation logic.

Make payload content generator read payload 4KiB at a time, to
always read on filesystem block boundaries for performance.  Increase
size of payload in relevant test case.
2013-11-11 16:21:26 +10:30
Andrew Bettison
701f14fdf9 Improve test defs: extract_stdout_keyvalue_optional()
Logs its variable assignment
2013-11-11 16:17:16 +10:30
Andrew Bettison
ca66a387dd Add "date" output field to Rhizome commands
To wit: "add file" and "import bundle"
2013-11-11 16:16:04 +10:30
Andrew Bettison
d5b48f5a9e Improve HTTP server generated content logic
Content generator functions now take arguments describing the buffer
they are to fill, and respond with a struct containing the number of
bytes filled, and the number of free bytes needed before being called
again.

The HTTP response logic now fills the buffer as much as possible before
calling write(2) by topping it up instead of waiting for it to be
completely emptied before generating more content.
2013-11-09 11:31:21 +10:30
Andrew Bettison
a62b6f9250 Improve Rhizome list cursor
Separate function rhizome_list_commit() which, if not called, causes the
next list re-query to include the last row that was fetched with
rhizome_list_next(), rather than excluding it.
2013-11-08 18:29:49 +10:30
Andrew Bettison
32ce7f5ed9 Do not catch SIGQUIT
So that it can be used to make a core dump during manual debugging
2013-11-08 18:25:33 +10:30
Andrew Bettison
1b906f3f11 Implement HTTP /restful/rhizome/bundlelist.json
Only tested for one bundle.
2013-11-07 23:40:56 +10:30
Andrew Bettison
9ecf9891fb Add strbuf JSON helper functions 2013-11-07 23:39:53 +10:30
Andrew Bettison
d337542067 Improve HTTP server generated content logic
Support generated content with an unspecified Content-Length.  Generator
functions return 1 if there is more content to come, and 0 if they have
just produced the last piece of content.
2013-11-07 23:39:24 +10:30
Andrew Bettison
051eca4775 Improve Rhizome list cursor
Remove 'rowcount' element

Order by descending ROWID, which is functionally the same as descending
inserttime but more reliable

Replace '_offset' cursor element with rowid first-last range to record
the expanding window of rows already returned, which allows release and
re-open of cursor mid-listing without missing rows or producing duplicates
2013-11-07 17:22:06 +10:30
Andrew Bettison
47988bfe0f Make Content-Length optional in HTTP responses
To eventually support the /restful/rhizome/bundlelist.json query which
will not compute in advance the length of its response.
2013-11-07 17:18:38 +10:30