Commit Graph

466 Commits

Author SHA1 Message Date
Andrew Bettison
711364ad8d Merge branch 'meshextender' into 'development' 2013-11-21 17:05:03 +10:30
Andrew Bettison
513aa15968 Finish HTTP /restful/rhizome/newsince/... test case
Uses new test framework fork support
2013-11-20 15:06:58 +10:30
Andrew Bettison
a882b8ebc6 Fix Rhizome Direct stress test - longer command timeouts
Since a default timeout was added to the execute() primitive
(b1b5a79) the Rhizome Direct test fails.
2013-11-20 15:06:07 +10:30
Andrew Bettison
c5014baf62 Improve test framework: better fork functions
Replace forkKillAll with fork_terminate_all

Replace forkWaitAll with fork_wait_all

Add fork_terminate and fork_wait functions

Introduce fork labels '%<alphanumeric>' as arguments to all fork
functions

Treat a forked process exit status 143 (killed with SIGTERM) as a
failure or error -- it happens when a forked process is terminated
using fork_terminate or fork_terminate_all
2013-11-20 15:01:37 +10:30
Andrew Bettison
29fab6dcb7 Add 'rhizome.api.restful.newsince_poll_ms' config option 2013-11-20 14:50:57 +10:30
Andrew Bettison
833466411c Test case for HTTP /restful/rhizome/newsince/...
Not passing yet
2013-11-19 17:28:04 +10:30
Andrew Bettison
f8f7716544 Improve HTTP /restful/rhizome/bundlelist.json token
Now tokens are included in each row instead of a single token in the
enclosing JSON object.  Only puts tokens on the first row, and
thereafter all rows with a rowid higher than for the previous row that
had a token.
2013-11-18 15:10:28 +10:30
Andrew Bettison
973bb9c897 Merge branch 'naf4' into 'development' 2013-11-13 17:59:01 +10:30
Andrew Bettison
4380fdcccd Token in /restful/rhizome/bundlelist.json output 2013-11-13 16:58:28 +10:30
Jeremy Lakeman
0b404b0260 Implement id list command 2013-11-13 12:00:46 +10:30
Jeremy Lakeman
ff1f1ff093 Allow tag primitives to be used outside of keyring entries 2013-11-13 12:00:05 +10:30
Jeremy Lakeman
9680b24f23 Add support for adding and updating tags in the keyring 2013-11-13 11:59:21 +10:30
Jeremy Lakeman
534b01ba2a Add explicit set did test 2013-11-13 11:52:46 +10:30
Andrew Bettison
13634f8748 Add ROWID field to struct rhizome_manifest
New ".rowid" output field from rhizome add, import, extract, export
operations.  (Also added missing ".inserttime" and "date" fields to
some operations.)

Use new "rhizome add file" .rowid output field to check output of of
/restful/rhizome/bundlelist.json
2013-11-11 18:18:08 +10:30
Andrew Bettison
6b961c56ce Fix HTTP /restful/rhizome/bundlelist.json
Write test case assertions using jq(1) utility, increase from four
bundles to 100.

Fix bugs in HTTP server content generation logic.

Make payload content generator read payload 4KiB at a time, to
always read on filesystem block boundaries for performance.  Increase
size of payload in relevant test case.
2013-11-11 16:21:26 +10:30
Andrew Bettison
45442d3eb4 Rewrite bundle author authentication
Replaced 'int has_author' manifest element with new 'enum authorship'
element to record the result of author authentication, to avoid
repeating expensive crypto operations.

Separated the handling of bundle secret arguments from author lookup and
authentication.  The new rhizome_apply_bundle_secret(m,bsk) is now
called at the top level to set the manifest secret key (if it
validates), and thereafter there is no need to pass the 'bsk' argument
to any other functions, as they can simply check the 'haveSecret' field
of the manifest.

Removed rhizome_extract_privatekey() which combined author lookup and
bundle secret validation, and replaced it with functions that only deal
with the author: rhizome_lookup_author() and rhizome_authenticate_author().

Renamed other functions to make their purpose and effect clearer.

Formalised the semantics of only storing AUTHENTICATED author SIDs in
the 'author' column of the MANIFESTS table, which necessitated a change
to a 'rhizomeops' test case: when adding a file using a BK-less
manifest, the author column is set to null, so the Rhizome list output
does not show the bundle as ".fromhere" and does not give an author for
that bundle.
2013-11-06 23:58:17 +10:30
Andrew Bettison
016fbe0244 Merge branch 'development' into 'naf4' 2013-11-04 20:01:58 +10:30
Andrew Bettison
bacba19dc1 Refactor manifest: specific setter functions
Replace generic rhizome_manifest_set() and rhizome_manifest_set_ll()
with per-field setter functions, eg, rhizome_manifest_set_filesize().
Struct rhizome_manifest elements for all known fields, to replace the
use of rhizome_manifest_get() and rhizome_manifest_get_ll() everywhere:
sender, recipient, service, name, date, bundle_key.

Add boolean validity flags for binary blob types, to avoid having to compare
with many bytes of all-zero to detect presence, eg, has_sender, has_recipient,
has_author, has_bundle_key.  These maintained by the setter functions.

Rename existing manifest struct elements to be the same as their field
names: fileLength -> filesize, journalTail -> tail.

More use of unsigned int, size_t and uint64_t for payload sizes, offsets, byte
counts, etc. especially in rhizome_store.c and meshms.c.  More uniform use of
size_t to dimension memory buffers.  Fix some printf(3) style format strings
for 64-bit correctness on 32-bit systems.  Use new constant RHIZOME_SIZE_UNSET
instead of -1 to indicate unknown dimension, and explicitly assert its absence
before comparisons and arithmetic, for safety.

Replace some 'int' loop variables with 'unsigned' where appropriate.

Fix bugs discovered in MeshMS bundle private/public key generation and
bundle secret key handling for export/extract commands.

Instrument the first MeshMS test case to aid debugging.

New debug config flag: debug.manifest logs all modifications to all manifest
fields by setter functions.

Rename debug config flag: debug.rhizome_bind -> debug.rhizome_sql_bind.
2013-11-04 19:17:49 +10:30
Andrew Bettison
675e2021cc Add ".secret" output field to all rhizome export/extract commands 2013-11-04 17:42:22 +10:30
Andrew Bettison
2ac1bc3240 Get HTTP Basic authentication working
Second 'rhizomehttp' test now passes
2013-10-30 11:07:45 +10:30
Andrew Bettison
21fe12859f Implement HTTP basic authentication
Use it in /restful/rhizome/bundlelist.json -- first 'rhizomehttp' test
passes
2013-10-29 17:32:04 +10:30
Andrew Bettison
9d54c629b2 Merge branch 'naf4' into 'development'
Rewritten HTTP request parsing and response buffering
2013-10-28 18:20:40 +10:30
Andrew Bettison
6ee691b161 Fix bugs in new HTTP server MIME body parsing code
Fixes all remaining 'rhizomeprotocol' test failures.
2013-10-28 14:25:16 +10:30
Andrew Bettison
2a9329c0c8 Fix bugs in new HTTP server MIME body parsing code
Fixes 'rhizomeprotocol' test 24 HttpImport.  Five tests still fail.
2013-10-27 13:19:23 +10:30
Andrew Bettison
0397a47753 Fix some test failures in new HTTP server code 2013-10-26 20:28:47 +10:30
gardners
e9e0a1b4a4 allow setting of % of 252 byte RS protected packets that we want
to have delivered.  Adjust tests accordingly.  Create new test
for ~50% packet delivery on fakeradio in preparation for network
coding.
2013-10-23 19:45:50 -07:00
Andrew Bettison
640a61cbe5 Add 'debug.rhizome_httpd' config option 2013-10-16 11:23:45 +10:30
Jeremy Lakeman
d5d5737f1c Rename 'id revoke' to 'id relinquish' to match documentation 2013-10-15 14:52:07 +10:30
Andrew Bettison
017286b158 Merge branch 'development' into 'naf4' 2013-10-15 11:16:56 +10:30
Andrew Bettison
3d3e900e72 First HTTP RESTful interface tests 2013-10-14 17:46:09 +10:30
Andrew Bettison
70af3289f6 Refactor setup for 'rhizomeops' test script 2013-10-14 16:49:06 +10:30
Andrew Bettison
eb46cc99b3 Improve test defs: setup_curl()
Replace setup_curl_7() function in separate test scripts with a single,
general setup_curl() function in testdefs.sh which takes the minimum
version number as its argument.
2013-10-14 16:48:20 +10:30
Andrew Bettison
ef2cc1404c Improve test framework: tfw_cmp_version()
Also add unit tests for test framework itself
2013-10-14 16:45:56 +10:30
Jeremy Lakeman
b8ec568711 Send 3-way request to stop routing an identity 2013-10-14 14:40:19 +10:30
Jeremy Lakeman
0c1c767af0 Wait until sid from local keyring is unreachable before claiming ownership 2013-10-10 15:59:44 +10:30
Jeremy Lakeman
ef7351bddc Unload identities from a running daemon 2013-10-10 14:27:00 +10:30
Jeremy Lakeman
ae7e120ed5 Pass keyring entry pin to running daemon and unlock identities 2013-10-09 15:34:41 +10:30
Andrew Bettison
c9f9025581 Fix intermittent 'routing' test failures
Caused by tests running so slowly that nodes become unreachable due to
timeout before the assertion of reachability is made.  The timeout was
hard coded at five times the tick interval (2.5 seconds).

Introduced new config options: mdp.iftype.*.reachable_timeout_ms and
interface.*.mdp.reachable_timeout_ms to allow the timeout to be
configured.  Configure the timeout to 60 seconds in 'routing' tests.

Changed the config schema to move options interfaces.*.mdp_tick_ms and
interfaces.*.packet_interval into interfaces.*.mdp.tick_ms and
interfaces.*.mdp.packet_interval (introduced new sub-struct 'mdp').
2013-10-08 16:36:07 +10:30
Andrew Bettison
55edc74482 Fix intermittent 'rhizomeops' test failure 2013-10-07 04:47:14 +10:30
Andrew Bettison
c3a4ab1f25 Fix two Rhizome Direct test cases 2013-10-04 03:11:54 +09:30
Andrew Bettison
41c862ea30 Fix 'rhizomeops' test case
Was broken by 919455608f
2013-10-03 15:13:28 +09:30
Andrew Bettison
06d7620314 Add "rhizome add file" --force-new option
Overrides the default behaviour of de-duplicating added manifests

New 'rhizomeops' test case: AddForceDuplicate
2013-09-30 16:48:18 +09:30
Andrew Bettison
2868cf0adf Improve Rhizome test functions
Options to allow manifest name to be given explicitly to override the
conventional "<filename>.manifest"
2013-09-30 16:39:58 +09:30
Andrew Bettison
919455608f Rename 'rhizomeops' test case
AddDuplicate -> AddDeDuplicate
2013-09-30 16:34:17 +09:30
Andrew Bettison
5e164607aa Expand output of rhizome add and import
Rename 'secret:' field to '.secret:' (non-manifest-fields start with '.'
using the same convention as rhizome list)

Add '.author' and 'BK' fields to "rhizome add"

Add 'BK' field to "rhizome import"

Fix 'rhizomeops' tests to assert no 'author' and 'BK' output fields from
"rhizome add" with no author

Fiz testdefs.sh and testdefs_rhizome.sh to support new output fields
2013-09-27 22:34:22 +09:30
Jeremy Lakeman
dab07dd4dc Remove monitor.socket & mdp.socket from tests 2013-09-24 16:27:34 +09:30
Jeremy Lakeman
57577abca6 Merge branch 'mavlink' into development 2013-09-20 14:58:29 +09:30
Jeremy Lakeman
e3a5e8c353 Use heartbeat packets to control serial buffering
- Improve simulation of fakeradio
- Backtrack on partial RS errors in case of truncation / heartbeat insertion
2013-09-20 14:41:50 +09:30
Andrew Bettison
4851eaf536 Issue #20: remove unique socket names from tests
The recent work on abstract local socket names means that every instance has
its own unique abstract socket name anyway, so there is no need for test
scripts to use $TFWUNIQUE in MDP and Monitor socket names to avoid
intereference when run concurrently.
2013-09-20 14:24:55 +09:30
Jeremy Lakeman
2a5ba97b48 Add rhizome test over fakeradio 2013-09-13 14:02:35 +09:30
Jeremy Lakeman
3f123e1875 Send the next packet via stream interface ASAP 2013-09-12 16:14:01 +09:30
Jeremy Lakeman
6f26594447 Simplify mavlink link frame creation
- build one frame at a time directly from the prepared overlay buffer
2013-09-11 17:15:43 +09:30
Jeremy Lakeman
e29564bc6c Refactor fakeradio
- completely event driven and non-blocking
- modelling of tx & rx serial buffers that will truncate on overfilling
- each radio takes turns to send a packet, with approx TDMA overhead
2013-09-11 15:03:43 +09:30
gardners
0cbebedc77 Add tx throttling for packet radios
avoids missing data due to lack of flow control.
2013-09-10 11:53:38 +09:30
Andrew Bettison
7add45214c Add [keyring-pin] arg to keyring load command 2013-09-10 11:28:55 +09:30
Andrew Bettison
e74aec9354 Tests for starting daemon with keyring entry PINs 2013-09-09 14:47:02 +09:30
Andrew Bettison
d177512746 Add test for server start with --keyring-pin option 2013-09-09 14:47:02 +09:30
Andrew Bettison
18e2916cec Add more keyring load tests, fix duplicate identity bugs 2013-09-07 04:03:28 +09:30
Andrew Bettison
8a300c2520 Get new keyring dump command and test working 2013-09-05 16:34:01 +09:30
Andrew Bettison
4b0550afd1 Fix new keyring load test 2013-09-05 09:19:45 +09:30
Andrew Bettison
f95e41374b Implement keyring load command, not working yet
Write new keyring test, fails with ERROR because assertGrep -F option is
not supported.
2013-09-04 23:47:17 +09:30
Andrew Bettison
9d1c3e0cba Fix test case doc comment 2013-09-03 17:05:50 +09:30
Jeremy Lakeman
e3b616421f Remember outgoing links 2013-08-30 17:38:29 +09:30
Andrew Bettison
572d7c2b5d Add tests for new keyring environment vars 2013-08-30 11:20:19 +09:30
Jeremy Lakeman
44ab51e2ab Don't clean the database for every command line operation by default 2013-08-27 13:44:05 +09:30
gardners
9b09021605 add test for falling over to MDP when HTTP rhizome fetching fails. 2013-08-25 16:37:44 +09:30
Jeremy Lakeman
643b21d6a7 Fetch file payloads if they are missing, even if we already have the manifest 2013-08-22 15:14:21 +09:30
Jeremy Lakeman
f9b828c3dd Remove payload if hash doesn't match when reading back 2013-08-21 15:45:18 +09:30
Jeremy Lakeman
1468d2deb0 Pre-emptively send small payloads via MDP 2013-08-21 09:21:40 +09:30
Jeremy Lakeman
5c374e9e6e Basic routing test with simulated serial modem interface 2013-08-19 16:59:10 +09:30
Jeremy Lakeman
771cb4151b Fetch journal range via HTTP 2013-08-15 17:00:43 +09:30
Jeremy Lakeman
21132466f1 Serve a range of a file payload via HTTP 2013-08-15 16:31:26 +09:30
Jeremy Lakeman
e322114850 Only transfer new journal content via MDP 2013-08-15 14:36:56 +09:30
Jeremy Lakeman
e6ce14af45 Test rhizome over unicast links 2013-08-13 12:06:05 +09:30
Jeremy Lakeman
36b0fe51ae Fix ping via directory service test 2013-08-13 10:55:48 +09:30
Jeremy Lakeman
a516af616e Improve reliability
- don't drop packets that haven't been resolved yet on incoming acks
- statistically drop unicast packets during tests
2013-08-12 16:38:37 +09:30
Jeremy Lakeman
9a50d8a9ef Route mulit-hop via unicast links 2013-08-09 16:37:54 +09:30
Jeremy Lakeman
55657623aa Detect unicast links 2013-08-09 12:18:14 +09:30
Jeremy Lakeman
81afc42d8b Major refactor to packet queuing and delivery in preparation for unicast link tracking.
Unicast tests broken
2013-08-09 09:37:08 +09:30
Jeremy Lakeman
0f2d082a0d Add stress test that sends lots of messages while transferring 2013-08-05 11:46:30 +09:30
Jeremy Lakeman
91a600fba7 Add separate rows for delivered and read status
- add explicit transaction around blob writes so the commit can be retried
- remove status columns
- remove SID's from message log output to reduce size and complexity
2013-08-02 14:02:56 +09:30
Jeremy Lakeman
68bf04f69d Change manifest service to MeshMS2 2013-08-01 11:37:35 +09:30
Jeremy Lakeman
e4882f175b Output message read offsets in conversation list 2013-07-31 10:33:01 +09:30
Jeremy Lakeman
88c986ebb5 Use sid_t consistently, tweak command output 2013-07-30 14:05:05 +09:30
Jeremy Lakeman
94381bab49 Fix non-journal, non-block based payload encryption 2013-07-26 15:13:22 +09:30
Jeremy Lakeman
fa012347a8 Improve test reliability 2013-07-26 14:40:06 +09:30
Jeremy Lakeman
7634e1fcfb Add command to mark messages as read 2013-07-26 14:36:11 +09:30
Jeremy Lakeman
a1caa4bf68 ACK conversations when listed, track conversations in a bundle 2013-07-25 14:49:06 +09:30
Jeremy Lakeman
f1139d4c0e Reduce meshms entry encoding size 2013-07-23 16:21:07 +09:30
Jeremy Lakeman
ddfb7d9417 First cut of new meshms API, unfinished 2013-07-22 17:55:02 +09:30
Jeremy Lakeman
2c1a995275 Don't encrypt by default when the recipient is broadcast 2013-07-19 10:28:15 +09:30
Jeremy Lakeman
a4bf4edbfc Write out of order packets with MDP transport
- add random order writing to rhizome store API
- remove rhizome flush, force all callers to provide buffer
2013-07-17 16:57:59 +09:30
Jeremy Lakeman
49e0286b43 Reduce packet header size for point-to-point links
- new interface.x.point_to_point config
- disable local echo for point to point dummy interfaces
- add ME and YOU sid abbreviation codes after learning other sid
2013-07-16 15:38:00 +09:30
Jeremy Lakeman
98dcd973ec Add help / version output jni test 2013-07-09 16:24:47 +09:30
Jeremy Lakeman
d6c9dbc1a3 Improve reliability of multiple identity test 2013-07-08 17:12:56 +09:30
Jeremy Lakeman
88d88e685c Improve backward compatibility for broadcasts & packet retries 2013-07-08 16:01:58 +09:30
Jeremy Lakeman
62a4cd407d Identify a journal bundle by the presense of a tail field 2013-07-08 14:57:47 +09:30
Jeremy Lakeman
e758e0130f Create journal append command 2013-07-08 14:29:08 +09:30
Jeremy Lakeman
6254e0d81a Reduce IO strain from stress test while increasing bundle count 2013-06-20 11:10:02 +09:30
Jeremy Lakeman
3e57add9ca Fix, initialise read / write handles so we don't close fd=0 2013-06-19 12:05:56 +09:30