Commit Graph

624 Commits

Author SHA1 Message Date
Andrew Bettison
0dd1b302b5 Rewrite Rhizome manifest parsing
Move validation checks into new function rhizome_manifest_validate()

Remove rhizome_manifest 'errors' field

Replace rhizome_manifest 'warnings' with 'malformed'

Replace rhizome_manifest 'manifest_bytes' with 'manifest_body_bytes' and
refactor to use 'manifest_all_bytes' in all manifest i/o

Refactor rhizome_manifest_verify() and reverse sense of return value to
match rhizome_manifest_validate()

New function rhizome_manifest_inspect() -- lightweight manifest parser
used when receiving Rhizome advertisements

New 'rhizomeops' test case for invalid manifest "service" field values,
now passes
2013-12-01 05:13:34 +10:30
Andrew Bettison
5e5e097d7f Accept any well formed Rhizome service field
Do not limit service field to only recognised supported values 'file',
'MeshMS1' and 'MeshMS2'

New test case checks for malformed service field, fails
2013-12-01 05:13:33 +10:30
Jeremy Lakeman
1681abd620 Increase default mdp timeout & make it configurable 2013-11-27 13:10:11 +10:30
Jeremy Lakeman
1b34c2dc13 Improve test reliability 2013-11-27 13:10:09 +10:30
Jeremy Lakeman
ad33264834 Rename mavlink to radio_link
- split link state from slip structures and encapsulate it completely
- allocate interface specific buffers on demand
2013-11-27 13:09:26 +10:30
Jeremy Lakeman
5ea19ab538 Better approximation of TDMA timing 2013-11-27 13:00:02 +10:30
Andrew Bettison
851dbb4d64 Add 'debug.subscriber' config option
Add debug logging to find_subscriber() function
2013-11-26 15:33:54 +10:30
Andrew Bettison
59a6ded10c Update 'dnaprotocol' debug flags 2013-11-26 15:33:54 +10:30
Andrew Bettison
711364ad8d Merge branch 'meshextender' into 'development' 2013-11-21 17:05:03 +10:30
Andrew Bettison
513aa15968 Finish HTTP /restful/rhizome/newsince/... test case
Uses new test framework fork support
2013-11-20 15:06:58 +10:30
Andrew Bettison
a882b8ebc6 Fix Rhizome Direct stress test - longer command timeouts
Since a default timeout was added to the execute() primitive
(b1b5a79) the Rhizome Direct test fails.
2013-11-20 15:06:07 +10:30
Andrew Bettison
c5014baf62 Improve test framework: better fork functions
Replace forkKillAll with fork_terminate_all

Replace forkWaitAll with fork_wait_all

Add fork_terminate and fork_wait functions

Introduce fork labels '%<alphanumeric>' as arguments to all fork
functions

Treat a forked process exit status 143 (killed with SIGTERM) as a
failure or error -- it happens when a forked process is terminated
using fork_terminate or fork_terminate_all
2013-11-20 15:01:37 +10:30
Andrew Bettison
29fab6dcb7 Add 'rhizome.api.restful.newsince_poll_ms' config option 2013-11-20 14:50:57 +10:30
Andrew Bettison
833466411c Test case for HTTP /restful/rhizome/newsince/...
Not passing yet
2013-11-19 17:28:04 +10:30
Andrew Bettison
f8f7716544 Improve HTTP /restful/rhizome/bundlelist.json token
Now tokens are included in each row instead of a single token in the
enclosing JSON object.  Only puts tokens on the first row, and
thereafter all rows with a rowid higher than for the previous row that
had a token.
2013-11-18 15:10:28 +10:30
Andrew Bettison
973bb9c897 Merge branch 'naf4' into 'development' 2013-11-13 17:59:01 +10:30
Andrew Bettison
4380fdcccd Token in /restful/rhizome/bundlelist.json output 2013-11-13 16:58:28 +10:30
Jeremy Lakeman
0b404b0260 Implement id list command 2013-11-13 12:00:46 +10:30
Jeremy Lakeman
ff1f1ff093 Allow tag primitives to be used outside of keyring entries 2013-11-13 12:00:05 +10:30
Jeremy Lakeman
9680b24f23 Add support for adding and updating tags in the keyring 2013-11-13 11:59:21 +10:30
Jeremy Lakeman
534b01ba2a Add explicit set did test 2013-11-13 11:52:46 +10:30
Andrew Bettison
13634f8748 Add ROWID field to struct rhizome_manifest
New ".rowid" output field from rhizome add, import, extract, export
operations.  (Also added missing ".inserttime" and "date" fields to
some operations.)

Use new "rhizome add file" .rowid output field to check output of of
/restful/rhizome/bundlelist.json
2013-11-11 18:18:08 +10:30
Andrew Bettison
6b961c56ce Fix HTTP /restful/rhizome/bundlelist.json
Write test case assertions using jq(1) utility, increase from four
bundles to 100.

Fix bugs in HTTP server content generation logic.

Make payload content generator read payload 4KiB at a time, to
always read on filesystem block boundaries for performance.  Increase
size of payload in relevant test case.
2013-11-11 16:21:26 +10:30
Andrew Bettison
45442d3eb4 Rewrite bundle author authentication
Replaced 'int has_author' manifest element with new 'enum authorship'
element to record the result of author authentication, to avoid
repeating expensive crypto operations.

Separated the handling of bundle secret arguments from author lookup and
authentication.  The new rhizome_apply_bundle_secret(m,bsk) is now
called at the top level to set the manifest secret key (if it
validates), and thereafter there is no need to pass the 'bsk' argument
to any other functions, as they can simply check the 'haveSecret' field
of the manifest.

Removed rhizome_extract_privatekey() which combined author lookup and
bundle secret validation, and replaced it with functions that only deal
with the author: rhizome_lookup_author() and rhizome_authenticate_author().

Renamed other functions to make their purpose and effect clearer.

Formalised the semantics of only storing AUTHENTICATED author SIDs in
the 'author' column of the MANIFESTS table, which necessitated a change
to a 'rhizomeops' test case: when adding a file using a BK-less
manifest, the author column is set to null, so the Rhizome list output
does not show the bundle as ".fromhere" and does not give an author for
that bundle.
2013-11-06 23:58:17 +10:30
Andrew Bettison
016fbe0244 Merge branch 'development' into 'naf4' 2013-11-04 20:01:58 +10:30
Andrew Bettison
bacba19dc1 Refactor manifest: specific setter functions
Replace generic rhizome_manifest_set() and rhizome_manifest_set_ll()
with per-field setter functions, eg, rhizome_manifest_set_filesize().
Struct rhizome_manifest elements for all known fields, to replace the
use of rhizome_manifest_get() and rhizome_manifest_get_ll() everywhere:
sender, recipient, service, name, date, bundle_key.

Add boolean validity flags for binary blob types, to avoid having to compare
with many bytes of all-zero to detect presence, eg, has_sender, has_recipient,
has_author, has_bundle_key.  These maintained by the setter functions.

Rename existing manifest struct elements to be the same as their field
names: fileLength -> filesize, journalTail -> tail.

More use of unsigned int, size_t and uint64_t for payload sizes, offsets, byte
counts, etc. especially in rhizome_store.c and meshms.c.  More uniform use of
size_t to dimension memory buffers.  Fix some printf(3) style format strings
for 64-bit correctness on 32-bit systems.  Use new constant RHIZOME_SIZE_UNSET
instead of -1 to indicate unknown dimension, and explicitly assert its absence
before comparisons and arithmetic, for safety.

Replace some 'int' loop variables with 'unsigned' where appropriate.

Fix bugs discovered in MeshMS bundle private/public key generation and
bundle secret key handling for export/extract commands.

Instrument the first MeshMS test case to aid debugging.

New debug config flag: debug.manifest logs all modifications to all manifest
fields by setter functions.

Rename debug config flag: debug.rhizome_bind -> debug.rhizome_sql_bind.
2013-11-04 19:17:49 +10:30
Andrew Bettison
675e2021cc Add ".secret" output field to all rhizome export/extract commands 2013-11-04 17:42:22 +10:30
Andrew Bettison
2ac1bc3240 Get HTTP Basic authentication working
Second 'rhizomehttp' test now passes
2013-10-30 11:07:45 +10:30
Andrew Bettison
21fe12859f Implement HTTP basic authentication
Use it in /restful/rhizome/bundlelist.json -- first 'rhizomehttp' test
passes
2013-10-29 17:32:04 +10:30
Andrew Bettison
9d54c629b2 Merge branch 'naf4' into 'development'
Rewritten HTTP request parsing and response buffering
2013-10-28 18:20:40 +10:30
Andrew Bettison
6ee691b161 Fix bugs in new HTTP server MIME body parsing code
Fixes all remaining 'rhizomeprotocol' test failures.
2013-10-28 14:25:16 +10:30
Andrew Bettison
2a9329c0c8 Fix bugs in new HTTP server MIME body parsing code
Fixes 'rhizomeprotocol' test 24 HttpImport.  Five tests still fail.
2013-10-27 13:19:23 +10:30
Andrew Bettison
0397a47753 Fix some test failures in new HTTP server code 2013-10-26 20:28:47 +10:30
gardners
e9e0a1b4a4 allow setting of % of 252 byte RS protected packets that we want
to have delivered.  Adjust tests accordingly.  Create new test
for ~50% packet delivery on fakeradio in preparation for network
coding.
2013-10-23 19:45:50 -07:00
Andrew Bettison
640a61cbe5 Add 'debug.rhizome_httpd' config option 2013-10-16 11:23:45 +10:30
Jeremy Lakeman
d5d5737f1c Rename 'id revoke' to 'id relinquish' to match documentation 2013-10-15 14:52:07 +10:30
Andrew Bettison
017286b158 Merge branch 'development' into 'naf4' 2013-10-15 11:16:56 +10:30
Andrew Bettison
3d3e900e72 First HTTP RESTful interface tests 2013-10-14 17:46:09 +10:30
Andrew Bettison
70af3289f6 Refactor setup for 'rhizomeops' test script 2013-10-14 16:49:06 +10:30
Andrew Bettison
eb46cc99b3 Improve test defs: setup_curl()
Replace setup_curl_7() function in separate test scripts with a single,
general setup_curl() function in testdefs.sh which takes the minimum
version number as its argument.
2013-10-14 16:48:20 +10:30
Andrew Bettison
ef2cc1404c Improve test framework: tfw_cmp_version()
Also add unit tests for test framework itself
2013-10-14 16:45:56 +10:30
Jeremy Lakeman
b8ec568711 Send 3-way request to stop routing an identity 2013-10-14 14:40:19 +10:30
Jeremy Lakeman
0c1c767af0 Wait until sid from local keyring is unreachable before claiming ownership 2013-10-10 15:59:44 +10:30
Jeremy Lakeman
ef7351bddc Unload identities from a running daemon 2013-10-10 14:27:00 +10:30
Jeremy Lakeman
ae7e120ed5 Pass keyring entry pin to running daemon and unlock identities 2013-10-09 15:34:41 +10:30
Andrew Bettison
c9f9025581 Fix intermittent 'routing' test failures
Caused by tests running so slowly that nodes become unreachable due to
timeout before the assertion of reachability is made.  The timeout was
hard coded at five times the tick interval (2.5 seconds).

Introduced new config options: mdp.iftype.*.reachable_timeout_ms and
interface.*.mdp.reachable_timeout_ms to allow the timeout to be
configured.  Configure the timeout to 60 seconds in 'routing' tests.

Changed the config schema to move options interfaces.*.mdp_tick_ms and
interfaces.*.packet_interval into interfaces.*.mdp.tick_ms and
interfaces.*.mdp.packet_interval (introduced new sub-struct 'mdp').
2013-10-08 16:36:07 +10:30
Andrew Bettison
55edc74482 Fix intermittent 'rhizomeops' test failure 2013-10-07 04:47:14 +10:30
Andrew Bettison
c3a4ab1f25 Fix two Rhizome Direct test cases 2013-10-04 03:11:54 +09:30
Andrew Bettison
41c862ea30 Fix 'rhizomeops' test case
Was broken by 919455608f
2013-10-03 15:13:28 +09:30
Andrew Bettison
06d7620314 Add "rhizome add file" --force-new option
Overrides the default behaviour of de-duplicating added manifests

New 'rhizomeops' test case: AddForceDuplicate
2013-09-30 16:48:18 +09:30
Andrew Bettison
2868cf0adf Improve Rhizome test functions
Options to allow manifest name to be given explicitly to override the
conventional "<filename>.manifest"
2013-09-30 16:39:58 +09:30
Andrew Bettison
919455608f Rename 'rhizomeops' test case
AddDuplicate -> AddDeDuplicate
2013-09-30 16:34:17 +09:30
Andrew Bettison
5e164607aa Expand output of rhizome add and import
Rename 'secret:' field to '.secret:' (non-manifest-fields start with '.'
using the same convention as rhizome list)

Add '.author' and 'BK' fields to "rhizome add"

Add 'BK' field to "rhizome import"

Fix 'rhizomeops' tests to assert no 'author' and 'BK' output fields from
"rhizome add" with no author

Fiz testdefs.sh and testdefs_rhizome.sh to support new output fields
2013-09-27 22:34:22 +09:30
Jeremy Lakeman
dab07dd4dc Remove monitor.socket & mdp.socket from tests 2013-09-24 16:27:34 +09:30
Jeremy Lakeman
57577abca6 Merge branch 'mavlink' into development 2013-09-20 14:58:29 +09:30
Jeremy Lakeman
e3a5e8c353 Use heartbeat packets to control serial buffering
- Improve simulation of fakeradio
- Backtrack on partial RS errors in case of truncation / heartbeat insertion
2013-09-20 14:41:50 +09:30
Andrew Bettison
4851eaf536 Issue #20: remove unique socket names from tests
The recent work on abstract local socket names means that every instance has
its own unique abstract socket name anyway, so there is no need for test
scripts to use $TFWUNIQUE in MDP and Monitor socket names to avoid
intereference when run concurrently.
2013-09-20 14:24:55 +09:30
Jeremy Lakeman
2a5ba97b48 Add rhizome test over fakeradio 2013-09-13 14:02:35 +09:30
Jeremy Lakeman
3f123e1875 Send the next packet via stream interface ASAP 2013-09-12 16:14:01 +09:30
Jeremy Lakeman
6f26594447 Simplify mavlink link frame creation
- build one frame at a time directly from the prepared overlay buffer
2013-09-11 17:15:43 +09:30
Jeremy Lakeman
e29564bc6c Refactor fakeradio
- completely event driven and non-blocking
- modelling of tx & rx serial buffers that will truncate on overfilling
- each radio takes turns to send a packet, with approx TDMA overhead
2013-09-11 15:03:43 +09:30
gardners
0cbebedc77 Add tx throttling for packet radios
avoids missing data due to lack of flow control.
2013-09-10 11:53:38 +09:30
Andrew Bettison
7add45214c Add [keyring-pin] arg to keyring load command 2013-09-10 11:28:55 +09:30
Andrew Bettison
e74aec9354 Tests for starting daemon with keyring entry PINs 2013-09-09 14:47:02 +09:30
Andrew Bettison
d177512746 Add test for server start with --keyring-pin option 2013-09-09 14:47:02 +09:30
Andrew Bettison
18e2916cec Add more keyring load tests, fix duplicate identity bugs 2013-09-07 04:03:28 +09:30
Andrew Bettison
8a300c2520 Get new keyring dump command and test working 2013-09-05 16:34:01 +09:30
Andrew Bettison
4b0550afd1 Fix new keyring load test 2013-09-05 09:19:45 +09:30
Andrew Bettison
f95e41374b Implement keyring load command, not working yet
Write new keyring test, fails with ERROR because assertGrep -F option is
not supported.
2013-09-04 23:47:17 +09:30
Andrew Bettison
9d1c3e0cba Fix test case doc comment 2013-09-03 17:05:50 +09:30
Jeremy Lakeman
e3b616421f Remember outgoing links 2013-08-30 17:38:29 +09:30
Andrew Bettison
572d7c2b5d Add tests for new keyring environment vars 2013-08-30 11:20:19 +09:30
Jeremy Lakeman
44ab51e2ab Don't clean the database for every command line operation by default 2013-08-27 13:44:05 +09:30
gardners
9b09021605 add test for falling over to MDP when HTTP rhizome fetching fails. 2013-08-25 16:37:44 +09:30
Jeremy Lakeman
643b21d6a7 Fetch file payloads if they are missing, even if we already have the manifest 2013-08-22 15:14:21 +09:30
Jeremy Lakeman
f9b828c3dd Remove payload if hash doesn't match when reading back 2013-08-21 15:45:18 +09:30
Jeremy Lakeman
1468d2deb0 Pre-emptively send small payloads via MDP 2013-08-21 09:21:40 +09:30
Jeremy Lakeman
5c374e9e6e Basic routing test with simulated serial modem interface 2013-08-19 16:59:10 +09:30
Jeremy Lakeman
771cb4151b Fetch journal range via HTTP 2013-08-15 17:00:43 +09:30
Jeremy Lakeman
21132466f1 Serve a range of a file payload via HTTP 2013-08-15 16:31:26 +09:30
Jeremy Lakeman
e322114850 Only transfer new journal content via MDP 2013-08-15 14:36:56 +09:30
Jeremy Lakeman
e6ce14af45 Test rhizome over unicast links 2013-08-13 12:06:05 +09:30
Jeremy Lakeman
36b0fe51ae Fix ping via directory service test 2013-08-13 10:55:48 +09:30
Jeremy Lakeman
a516af616e Improve reliability
- don't drop packets that haven't been resolved yet on incoming acks
- statistically drop unicast packets during tests
2013-08-12 16:38:37 +09:30
Jeremy Lakeman
9a50d8a9ef Route mulit-hop via unicast links 2013-08-09 16:37:54 +09:30
Jeremy Lakeman
55657623aa Detect unicast links 2013-08-09 12:18:14 +09:30
Jeremy Lakeman
81afc42d8b Major refactor to packet queuing and delivery in preparation for unicast link tracking.
Unicast tests broken
2013-08-09 09:37:08 +09:30
Jeremy Lakeman
0f2d082a0d Add stress test that sends lots of messages while transferring 2013-08-05 11:46:30 +09:30
Jeremy Lakeman
91a600fba7 Add separate rows for delivered and read status
- add explicit transaction around blob writes so the commit can be retried
- remove status columns
- remove SID's from message log output to reduce size and complexity
2013-08-02 14:02:56 +09:30
Jeremy Lakeman
68bf04f69d Change manifest service to MeshMS2 2013-08-01 11:37:35 +09:30
Jeremy Lakeman
e4882f175b Output message read offsets in conversation list 2013-07-31 10:33:01 +09:30
Jeremy Lakeman
88c986ebb5 Use sid_t consistently, tweak command output 2013-07-30 14:05:05 +09:30
Jeremy Lakeman
94381bab49 Fix non-journal, non-block based payload encryption 2013-07-26 15:13:22 +09:30
Jeremy Lakeman
fa012347a8 Improve test reliability 2013-07-26 14:40:06 +09:30
Jeremy Lakeman
7634e1fcfb Add command to mark messages as read 2013-07-26 14:36:11 +09:30
Jeremy Lakeman
a1caa4bf68 ACK conversations when listed, track conversations in a bundle 2013-07-25 14:49:06 +09:30
Jeremy Lakeman
f1139d4c0e Reduce meshms entry encoding size 2013-07-23 16:21:07 +09:30
Jeremy Lakeman
ddfb7d9417 First cut of new meshms API, unfinished 2013-07-22 17:55:02 +09:30
Jeremy Lakeman
2c1a995275 Don't encrypt by default when the recipient is broadcast 2013-07-19 10:28:15 +09:30
Jeremy Lakeman
a4bf4edbfc Write out of order packets with MDP transport
- add random order writing to rhizome store API
- remove rhizome flush, force all callers to provide buffer
2013-07-17 16:57:59 +09:30
Jeremy Lakeman
49e0286b43 Reduce packet header size for point-to-point links
- new interface.x.point_to_point config
- disable local echo for point to point dummy interfaces
- add ME and YOU sid abbreviation codes after learning other sid
2013-07-16 15:38:00 +09:30
Jeremy Lakeman
98dcd973ec Add help / version output jni test 2013-07-09 16:24:47 +09:30
Jeremy Lakeman
d6c9dbc1a3 Improve reliability of multiple identity test 2013-07-08 17:12:56 +09:30
Jeremy Lakeman
88d88e685c Improve backward compatibility for broadcasts & packet retries 2013-07-08 16:01:58 +09:30
Jeremy Lakeman
62a4cd407d Identify a journal bundle by the presense of a tail field 2013-07-08 14:57:47 +09:30
Jeremy Lakeman
e758e0130f Create journal append command 2013-07-08 14:29:08 +09:30
Jeremy Lakeman
6254e0d81a Reduce IO strain from stress test while increasing bundle count 2013-06-20 11:10:02 +09:30
Jeremy Lakeman
3e57add9ca Fix, initialise read / write handles so we don't close fd=0 2013-06-19 12:05:56 +09:30
Jeremy Lakeman
40364be92f Fix and improve rhizome stress test
- reduce dependance on routing
- fail immediately if a servald instance crashes
2013-06-18 16:33:41 +09:30
Jeremy Lakeman
c6241c6634 Ensure routing table is updated when indirect nodes go offline 2013-06-11 10:18:28 +09:30
Jeremy Lakeman
b1f384cd63 Swap between broadcast & unicast links as they become available 2013-05-31 11:32:31 +09:30
Jeremy Lakeman
bc50d3e184 Establish a unicast link to nodes who are filtering broadcasts 2013-05-29 11:58:06 +09:30
Jeremy Lakeman
d6d3c1f413 Add ping backward compatibility test 2013-05-29 11:58:06 +09:30
Jeremy Lakeman
37ef97268b Disable rhizome when testing dnahelper to reduce IO load 2013-05-29 11:58:06 +09:30
Jeremy Lakeman
aa75e519f3 Rename 2-hop test 2013-05-29 11:58:06 +09:30
Jeremy Lakeman
8f575262ff Reduce packet storm to reduce I/O required for running tests 2013-05-29 11:58:06 +09:30
Jeremy Lakeman
8ab5073869 Adjust packet format for better duplicate detection 2013-05-29 11:58:05 +09:30
Jeremy Lakeman
02be4c2979 Add support for retransmitting broadcast mdp packets 2013-05-29 11:58:05 +09:30
Jeremy Lakeman
a213872f09 Don't ack if we aren't being used in a route 2013-05-29 11:58:05 +09:30
Jeremy Lakeman
5b4e4919cf Mark resent packets so they can be dropped 2013-05-29 11:58:05 +09:30
Jeremy Lakeman
e519633f7a Resend last ACK if it is explicitly NACKed 2013-05-29 11:58:05 +09:30
Jeremy Lakeman
e5856225cf Add packet retransmissions due to missed ack's 2013-05-29 11:58:05 +09:30
Jeremy Lakeman
b10746b3a5 Add stub functions for retransmission / throttling 2013-05-29 11:58:04 +09:30
Jeremy Lakeman
abfe6963d2 Wait for servald daemon to enter main loop for more reliable tests 2013-05-27 14:24:33 +09:30
Jeremy Lakeman
118dcca40b Improve quality and readability of some routing tests 2013-05-10 12:05:28 +09:30
Jeremy Lakeman
ef1e93bf52 Establish neighbour links with nodes running our legacy routing engine 2013-05-09 16:57:33 +09:30
Jeremy Lakeman
a59b306173 Allow for out of order packets, reduce link state spam 2013-05-08 11:37:31 +09:30
Jeremy Lakeman
c0b31a2774 Report packet acks, make routing decisions based on dropped packets 2013-05-08 11:37:02 +09:30
Jeremy Lakeman
a29c1d3468 Don't override subscriber interface due to unicast probe packets 2013-05-02 14:29:31 +09:30
Jeremy Lakeman
dd03014b44 Choose the best link from a neighbour 2013-05-02 14:29:31 +09:30
Jeremy Lakeman
9ee09e36a0 Replace old routing engine with new link state engine 2013-05-02 14:29:28 +09:30
Andrew Bettison
4e48725c93 Merge branch 'new-keyring' into 'development' 2013-04-30 17:57:59 +09:30
Andrew Bettison
cf26a77019 Use "keyring dump" command in back-compatible keyring test 2013-04-30 17:56:10 +09:30
Andrew Bettison
deed4665fe Fix keyring test copyright and header comments 2013-04-29 17:38:01 +09:30
Jeremy Lakeman
559b270966 Only compare routes that have all links via the same neighbour 2013-04-29 17:07:36 +09:30
Andrew Bettison
78b3a3a8c4 Add keyring test case (combined PINs) 2013-04-29 16:00:17 +09:30
Andrew Bettison
ae69da284f Add keyring back-compatibility test case 2013-04-29 15:11:04 +09:30
Andrew Bettison
6fd49099cb Rewrite keyring pack/unpack for extensibility
Use new rotbuf primitives to handle rotated buffers
2013-04-29 14:31:50 +09:30
Jeremy Lakeman
d7caaf4973 Add length of link record for forward compatibility 2013-04-29 10:06:22 +09:30
Jeremy Lakeman
85b885dde2 Add basic link state routing 2013-04-26 16:54:01 +09:30
Andrew Bettison
121ae2d609 Improve timeout control in routing tests 2013-04-22 16:01:38 +09:30
Andrew Bettison
81af4460f3 Fix broken "ping" command arg parsing 2013-04-12 17:18:25 +09:30
Jeremy Lakeman
73342a9659 Announce routes to secondary keyring entries 2013-04-08 10:03:51 +09:30
Andrew Bettison
2c379fadb1 Merge branch 'naf2-logging' into 'development'
Conflicts:
	commandline.c
	tests/rhizomeprotocol
2013-04-05 17:39:23 +10:30
Andrew Bettison
ec481e5f0f Rename config option 'log.stderr' to 'log.console'
Because Android <stdio.h> defines 'stderr' as a macro...
2013-04-05 16:55:14 +10:30
Andrew Bettison
f7fc63c452 Improve logging tests, fix failures, add 'log.file.path' config option 2013-04-05 16:23:20 +10:30
Andrew Bettison
f06613c35f Fix test failures from latest logging changes 2013-04-05 15:28:54 +10:30
Andrew Bettison
21991726ae Add logging tests, add "log" command
Refactor log.c ready for log file rotation
2013-04-02 17:01:48 +10:30
Andrew Bettison
0f83b9a5fc Fix #60 - "rhizome direct" command line URL
Add test case for "rhizome push <url>" command
2013-04-02 14:54:46 +10:30
Andrew Bettison
95b0c028f2 Log to file, Android and stderr simultaneously
Indepent configuration of show_pid, show_time and log level for each
destination

Update test scripts for new config options

Include xprintf.c in MDP client source files (now used by log.c)
2013-03-27 16:17:38 +10:30
gardners
3a0a4df216 add place-holder tests for adding channel labels to Rhizome. #58 2013-03-18 13:27:15 +10:30
Andrew Bettison
5dd6a53615 Fix 'rhizomeops' tests
Rename duplicate AddNoAuthor test to AddNoAuthorEncrypted

Fix broken AddNoAuthor test

Add new AddInvalidAuthor test
2013-03-15 15:58:17 +10:30
Andrew Bettison
bc7c58ee3c Improve keyring test case 2013-03-06 15:29:27 +10:30
Andrew Bettison
03f38d2031 Improve keyring debugging 2013-03-06 14:15:27 +10:30
Jeremy Lakeman
4cdd3da77f Add config option to disable fetching via rhizome 2013-03-05 10:56:20 +10:30
Jeremy Lakeman
9e2eb1aea4 Add keyring test for implicitly created identity 2013-02-28 16:27:53 +10:30
Jeremy Lakeman
ee95e47d1f Add rhizome export commands that don't try to decrypt file contents
export [bundle/manifest/file]
extract [bundle/file]

"dump file" has been replaced with "export file".
2013-02-27 11:39:23 +10:30
Jeremy Lakeman
b20e714ffb Test that changing config.rhizome.external_blobs still allows existing blobs to be read (FAILs) 2013-02-22 13:37:01 +10:30
Jeremy Lakeman
90386ce1b1 Merge remote-tracking branch 'origin/development' into serial
Conflicts:
	rhizome.h
	rhizome_database.c
	rhizome_fetch.c
	rhizome_store.c
2013-02-20 17:23:03 +10:30
Jeremy Lakeman
73786bcb5d Refactor all rhizome reading and writing
- The API in rhizome_store.c is used for all reading and writing
- external storage is now usable for all transport options
2013-02-20 16:18:59 +10:30
Jeremy Lakeman
e5fa357f07 Add slip test to automated testing 2013-02-20 16:18:59 +10:30
gardners
944486c048 simplify external/internal blob storage to a boolean flag instead
of a threshold.
2013-02-20 16:18:59 +10:30
Jeremy Lakeman
bb8b3d6376 Add new mdp trace command 2013-02-20 16:18:58 +10:30
gardners
a23612428b make rhizome over mdp block size configurable.
1K seems too big to be reliable on our prototype UHF radio links,
as CRCs fail almost all the time. #48
2013-02-20 16:18:57 +10:30
gardners
57013c3a03 removed temporary verbose output from test. 2013-02-20 16:18:57 +10:30
Andrew Bettison
390655580d Add 'rhizome delete' and 'rhizome clean' commands
sqlite_void_exec() and its ilk now return the count of changed rows, not
just zero, on success

sqlite_exec_prepared() and its ilk now return the count of rows (number
of step results SQLITE_ROW), instead of just zero, on success

rhizome_clean() function now produces an optional report of its changes

rhizome_fail_write() and rhizome_finish_write() now log WARNings not
ERRORs if the SQL DELETE FROM FILES or DELETE FROM FILEBLOBS statements
fail

Refactor rhizome_open_read() to use sqlite_exec_int64()

Ensure that 'rhizome extract' and 'rhizome dump' commands return exit
status of 1 in "not found" conditions, not 255, which is reserved for
errors

Test cases for four new commands: 'rhizome delete bundle', 'rhizome
delete manifest', 'rhizome delete payload' and 'rhizome delete file'
(no test case for 'rhizome clean' yet)
2013-02-20 15:37:09 +10:30
gardners
641d749ab4 Create files for large rhizome bundles
- configurable size threashold
2013-02-20 15:36:32 +10:30
Jeremy Lakeman
6d9bbe2e2c Refactor interface handling to separate encapsulation from stream type 2013-02-20 15:36:23 +10:30
Andrew Bettison
788b3e37b2 Fix JNI argv parsing: throw exception on NULL
Add new 'jni' test case to ensure this works
2013-02-15 19:33:12 +10:30
Andrew Bettison
c42260f5f9 Replace "add did" command with improved "keyring add did"
Output is same format as "keyring add" command

Fixed minor bug in rolling a random DID -- last byte was not random

Some code improvements, including better support for sid_t
2013-02-14 16:06:01 +10:30
Andrew Bettison
433f49551a Cosmetic improvements to 'routing' test script 2013-02-14 16:02:09 +10:30
Andrew Bettison
f66a894a63 Improve keyring PIN handling
Replace explicit [<pin,pin...>] command-line arguments with --entry-pin=<pin>
options.  Update test scripts accordlingly.

Add --keyring-pin=<pin> option (with one test case).
2013-02-13 16:49:52 +10:30
Andrew Bettison
46513bf192 Add keyring tests to 'all' test script 2013-02-11 18:43:12 +10:30
Andrew Bettison
b2923744b0 Add 'keyring' test script with three basic tests 2013-02-11 18:41:36 +10:30
Andrew Bettison
2eb930b94c Improve Rhizome Direct stress test
Reduce log verbosity, report extra files received by nodes, re-order
push/pull/sync process start order to increase chance of cross-fetching.
2013-02-11 18:39:46 +10:30
Andrew Bettison
cb288d8b9c Fix new Rhizome Direct stress test
Now PASSES with Jeremy's recent fix, but FAILED before
2013-02-08 19:24:03 +10:30
Andrew Bettison
b5a55c8c08 Add Rhizome Direct stress test
Use new test framework fork() support to run several rhizome direct
push/pull/sync processes at once to a single server.

Speed up setup for Rhizome Transfer stress test.
2013-02-08 18:46:30 +10:30
Andrew Bettison
7a8323b31d Improve Rhizome stress tests
Rename tests to start with "Stress" prefix

Refactor StressRhizomeAdd test to use conventional utilities and be more
readable.  Remove trailing "sleep 10".
2013-02-08 12:08:24 +10:30
Andrew Bettison
a6b6209c6c Improve Rhizome Direct tests
Push/pull three bundles of varying sizes each way instead of one 64 byte
bundle each way
2013-02-08 10:39:41 +10:30
Andrew Bettison
c804b9d5ce Fix copyright messages in test scripts 2013-02-06 18:55:31 +10:30
Andrew Bettison
248e2b7096 Add new 'all' test script
Runs all tests except Rhizome stress tests.
2013-02-06 18:51:01 +10:30
Jeremy Lakeman
b35c1aca43 Specify that unicast traffic should use unicast UDP packets 2013-01-29 11:57:13 +11:00
Andrew Bettison
546fccc794 Support relative log.file config option
As already documented in doc/Servald-Configuration.md
2013-01-23 14:54:25 +10:30
Jeremy Lakeman
dbd4cb1771 Fix jni tests on osx & with new interface 2013-01-23 14:43:50 +10:30
Jeremy Lakeman
ad01836d0e Use database query to respond to rhizome list, add name argument 2013-01-17 11:45:45 +10:30
Jeremy Lakeman
b25ff45438 Importing the same bundle twice returns duplicate flag 2013-01-15 10:32:48 +10:30
Jeremy Lakeman
5a82bfcf31 Add support for importing and exporting a file with an appended manifest 2013-01-11 14:30:52 +10:30
Jeremy Lakeman
7825f9dbd2 Combine manifest and file extraction into one method that can also do both 2013-01-11 14:30:51 +10:30
Andrew Bettison
5ac394a3c1 Maybe fix tests/server:NoZombie occasional failure
Introduced a two-second wait for the KILLed process to die.
2013-01-10 12:12:51 +10:30
Jeremy Lakeman
c7cf80b352 Use sqlite user schema version to remember state 2013-01-06 13:04:49 +10:30
Jeremy Lakeman
2b480c1d1a Reinstate filesize and hash file extract output 2013-01-04 12:01:40 +10:30
Jeremy Lakeman
fd96c5bd76 Add bsk to correct command 2013-01-04 12:01:39 +10:30
Jeremy Lakeman
28a05baa54 Enable encryption of rhizome payloads 2013-01-03 16:12:24 +10:30
Jeremy Lakeman
8b045dd1a6 Extract files based on the manifest id
Renamed rhizome extract file to rhizome dump file
Added rhizome extract file [manifest] [filepath] [pins]
Modified tests to use the appropriate command, assuming that MeshMS payloads will be encrypted
2013-01-02 12:58:05 +10:30
Jeremy Lakeman
f906d5a46c Test each file immediately after adding 2012-12-28 12:00:43 +10:30
Jeremy Lakeman
3f45623d04 New stress test, rhizome add while transferring 2012-12-27 16:51:31 +10:30
Jeremy Lakeman
50b216da96 Make directory service test fail if the dna helper doesn't start 2012-12-20 16:50:50 +10:30
Jeremy Lakeman
38589f293c Interface speed is no longer validated 2012-12-20 16:20:12 +10:30
Jeremy Lakeman
f64de66b34 Refactor add via HTTP to be equivalent to command line add 2012-12-20 15:36:07 +10:30
Jeremy Lakeman
d76b3a25f9 Fail duplicate manifest test if secret is different 2012-12-17 16:09:22 +10:30
Jeremy Lakeman
55be5208a9 Remove packet queue delays, rely on network limits 2012-12-14 17:11:57 +10:30
Andrew Bettison
7d970d8f92 Merge branch 'newconfig' into 'master' 2012-12-11 17:21:56 +10:30
Andrew Bettison
5985df751d Overhaul debug flags
Replace debugflags_t and DEBUG_XXX bit masks with config schema "debug.xxx"
entries.

No more support for "debug.all".
2012-12-11 15:59:46 +10:30
Jeremy Lakeman
5798e5acb5 Always send scan packets with full sid 2012-12-11 13:32:02 +10:30
Jeremy Lakeman
dc6d0da07e Add explicit route tests and improve test speed 2012-12-11 11:27:18 +10:30
Jeremy Lakeman
c3f573ea7f Add more messages, wait for directory service to start first
And make sure gaps in sockaddr_in are zeroed.
2012-12-10 14:42:57 +10:30
Jeremy Lakeman
2932544eb8 Add IPv4 addresses to dummy interface 2012-12-10 14:06:25 +10:30
Andrew Bettison
8425882ffc Merge branch 'newconfig' into 'master'
Conflicts:
	commandline.c
	conf.h
	dataformats.c
	log.h
	overlay_address.c
	overlay_interface.c
	packetformats.c
	rhizome.c
	serval.h
	tests/directory_service
	vomp_console.c
2012-12-07 14:09:55 +10:30
Andrew Bettison
fe042bfa64 Servald 'config [ set|del ... ] ... ' command
Update all test scripts to avoid repetitive executeOk_servald invocations.
2012-12-06 12:31:19 +10:30
Jeremy Lakeman
4557c82b1f Increase delay before failing tests 2012-12-06 11:54:09 +10:30
Andrew Bettison
4985359158 Fix 'rhizomestress' test failures 2012-12-05 16:11:23 +10:30
Andrew Bettison
9e6059e473 Fix 'routing' test failures 2012-12-05 15:50:34 +10:30
Andrew Bettison
b497c88d9d Fix 'directory_service' test failures
New configuration schema.
2012-12-05 15:47:14 +10:30
Andrew Bettison
250fad71e8 Fix 'dnaprotocol' test failures
Config option "mdp.TYPE.tick_ms" is now "mdp.iftype.TYPE.tick_ms"
2012-12-05 15:26:25 +10:30
Andrew Bettison
1a3f916cfb Fix 'rhizomeprotocol' test failures
Config option rhizome.direct.peer.count is no longer required.

Bug in new URI parsing code.
2012-12-05 15:23:59 +10:30
Andrew Bettison
8384fed9c5 Fix failing 'dnahelper' test 2012-12-05 14:51:38 +10:30
Andrew Bettison
aa638a9bfd Fix new 'interfaces' config option
Introduce CFINCOMPATIBLE config parse result flag.

Sort interface rules by unsigned integer key.

Legacy and modern 'interfaces' config styles are now incompatible.

Validate config_network_interface struct to enforce that only exactly one of
'match' and 'dummy' options are set.

Add test cases for 'interface' config option.
2012-12-05 12:58:07 +10:30
Andrew Bettison
c53789d764 Legacy 'interfaces' config option improvements
Legacy form is now incompatible with modern form.

Test case to check legacy parsing.
2012-12-04 18:22:26 +10:30
Andrew Bettison
93c38a764d Improve servald 'config' 'set' and 'del' commands
Return exit status 2 if the new config is invalid in any way.
2012-12-04 18:20:53 +10:30
Andrew Bettison
71ed78e058 Make all 'config' tests pass
Change a test case: configuration options are now case sensitive.

Fix config file load and parse logic in conf.c, always copy 'debug' flags
from config.debug.

The config schema 'interfaces' option is no longer MANDATORY.

Introduce new CLIFLAG_PERMISSIVE_CONFIG to supress bad-config ERROR messages
from the 'config set' and 'config get' commands.

Refactor cli_execute() into cli_parse() and cli_invoke().  Use *const* struct
command_line_option everywhere.
2012-12-04 16:52:49 +10:30
Jeremy Lakeman
5c7eb4a594 Add unicast probe packets
Probe directory service instead of assuming that it is always reachable
Fix directory service and routing tests
2012-12-04 14:47:57 +10:30
Jeremy Lakeman
2b21a691ca Merge remote-tracking branch 'origin/master' into packet_format
Conflicts:
	constants.h
	packetformats.c
2012-12-03 14:10:32 +10:30
gardners
7b4b9dbf81 improved test coverage for Rhizome over MDP, and made testing
of MDP vs HTTP more explicit for an existing test case.
2012-12-01 10:56:48 +10:30
gardners
21f562122e added test for rhizome over mdp for a big file, to make sure that
timeout handling works properly.
2012-11-30 22:57:34 +10:30
gardners
6c0e6ef1c2 added extra tests for Rhizome over HTTP and MDP transports.
HTTP transport works, MDP transport segfaults.  To be examined.
2012-11-30 17:42:17 +10:30
gardners
3c4abfb566 improved and simplified test for rhizome http not running 2012-11-30 15:06:31 +10:30
gardners
9e29db827f added test for correct function of rhizome.http.enable config
option.
2012-11-30 15:02:34 +10:30
Jeremy Lakeman
d5f78bcffe Rework reachable link flags for simpler bitmask tests 2012-11-30 14:38:09 +10:30
Jeremy Lakeman
e0878644fe Add command to dump routing table 2012-11-28 15:43:58 +10:30
Andrew Bettison
21a0f31ae4 Merge branch 'whacktmps' into 'master' 2012-11-15 13:41:45 +10:30
Jeremy Lakeman
10bde11918 Remove resolution of peer details from node info command 2012-11-14 12:17:41 +10:30
Jeremy Lakeman
dd93f201e3 Add separate reverse lookup command 2012-11-14 12:17:14 +10:30
Andrew Bettison
5751569b59 Rhizome extract manifest to '-' writes to output
Instead of specifying a file path, if the file is given as '-' then the content
of the manifest is output in the 'manifest' key-value pair instead of written
to a file.
2012-11-07 20:29:12 +10:30
Andrew Bettison
5185c944c9 JNI interface returns byte[] instead of String
This allows the output fields of any command to contain nul characters, which
paves the way for extracting a manifest or other binary data directly instead
of having to write it into a temporary file.
2012-11-07 17:12:04 +10:30
Andrew Bettison
548b837d2a Rhizome stress test: increase timeout from 3 to 10 minutes 2012-11-05 14:45:23 +10:30
Andrew Bettison
03f71dcccf Improve rhizome stress test
Generate test files from 2KiB to 32MiB in size, instead of all about 13 bytes
in size.  Guess what?  Found a bug!
2012-10-31 18:17:06 +10:30
Andrew Bettison
39fc4ce6de Issue #30, remove periodic Rhizome fetch alarm
Replace the main-loop scheduled periodic alarm with an "activate" alarm that is
scheduled whenever a fetch candidate is added to any queue, unless the alarm is
already scheduled.

Replace the "rhizome.fetch_interval_ms" config item with
"rhizome.fetch_delay_ms" [default 50], which is the number of milliseconds
between adding a fetch candidate and firing the "activate" alarm.  This allows
time for a few more Rhizome advertisment packets to arrive after the first one,
before deciding which fetches to start first.

Add new `is_scheduled()` alarm primitive.
2012-10-24 15:13:50 +10:30
Andrew Bettison
af1a6c4297 Split Rhizome stress test into separate file 2012-10-18 17:46:17 +10:30
Andrew Bettison
877f76557a Fix 'rhizomeprotocol' stress test 2012-10-18 17:29:58 +10:30
Andrew Bettison
1eeb040602 Improve new Rhizome stress test
Now it fails for the right reasons, and does not produce ERROR.
2012-10-18 11:51:10 +10:30
Andrew Bettison
381fc7cad8 Use finally() function in test suites
Mainly to call stop_all_servald_servers(), which asserts that the server is
still running and responding to orderly shutdown.
2012-10-18 11:46:32 +10:30
Andrew Bettison
7317f9b56c Add a new 'rhizomeprotocol' test case: stress test
Refactored some testdefs.sh and testdefs_rhizome.sh, not yet complete.
2012-10-17 18:10:39 +10:30
Andrew Bettison
c43a7ba8e6 Issue #17, improve "rhizome extract manifest" command
Add ".readonly" output field, add logic to detect bundle author and update
MANIFESTS table 'author' column accordingly.

Add two 'rhizomeops' test cases to deal with authorless logic.

Rename some 'rhizomeops' test cases for consistency and to help filtering.
2012-10-15 16:48:04 +10:30
Andrew Bettison
eff3e9ec8f Issue #17, fix new 'rhizomeops' test
Was incorrectly supplying bundle secret to "rhizome add" command.
2012-10-15 14:01:48 +10:30
Andrew Bettison
3f1f495d68 Issue #17, overhaul manifest author crypto logic
Add test case for new feature of the "rhizome add" command: if the author SID
is not specified (empty arg) then it searches the keyring for the author.

Removed "authorSid" argument from several functions that also take a struct
rhizome_manifest * arg, since the author, if known, is now supplied in the
struct.

Improve return value handling and refactored some rhizome crypto code.
2012-10-11 17:58:24 +10:30
Jeremy Lakeman
b22e3f1964 Add hop count to mdp ping
- Allow mdp client to specify ttl
- Copy ttl into and out of overlay frame
2012-10-10 16:22:31 +10:30
Andrew Bettison
3678522872 Issue #17, add AUTHOR column to Rhizome MANIFESTS table
Replace ".selfsigned" column with ".author" and ".fromhere" columns in
output of "rhizome list" command.  (Note that a "sender" column is
already present.)

Add 'author' field to struct rhizome_manifest.

Log all fully rendered SQL statements on DEBUG_RHIZOME.

Update 'rhizomeops' test cases and improve the assert_rhizome_list()
test function to be able to assert authorship of files.
2012-10-09 17:48:06 +10:30
Jeremy Lakeman
1f379ea209 Add and preserve QOS bit flags in packet header 2012-10-09 16:17:07 +10:30
Jeremy Lakeman
9842e22353 Add multi-hop routing test 2012-10-09 14:49:24 +10:30
Andrew Bettison
5a6a42f36d Improve test defs: extract_stdout_keyvalue()
The extract_stdout_keyvalue() function now accepts an optional delimiter
argument, to override its default of ':'.  This makes it useful for extracting
the output of "servald config get" commands.
2012-10-09 13:35:00 +10:30