Commit Graph

753 Commits

Author SHA1 Message Date
Andrew Bettison
04b2a20e54 Rhizome Java API: improved form-data headers
Change manifest Content-Type from rhizome-manifest/text to
rhizome/manifest; format="text+binarysig"

Add "Content-Transfer-Encoding" form-part headers to Java API
sent form parts, although not currently checked by Rhizome
RESTful interface
2014-07-10 21:12:40 +09:30
Andrew Bettison
a81d05b4f6 Rhizome Java API: insert bundle 2014-07-10 06:43:17 +09:30
Andrew Bettison
3715c5bf0b Rhizome Java API: negative fetch tests 2014-07-10 06:30:50 +09:30
Andrew Bettison
2aec8f31a4 Rhizome Java API: get decrypted payload 2014-07-10 06:30:50 +09:30
Andrew Bettison
d16be8f42d Improve MeshMS HTTP RESTful interface
Provide "meshms_status_message" in returned JSON status content
2014-07-10 06:30:50 +09:30
Andrew Bettison
cf43635789 Improve Rhizome HTTP RESTful interface
Add RHIZOME_BUNDLE_STATUS_READONLY enum value

Tighten up switch statements on bundle and payload status enums (no
default labels)

Rename some recently added enum entries

Return bundle status and payload status in HTTP responses

Add test for failing to decrypt a foreign encrypted bundle payload,
fix bug that caused an assertion failure

Add tests for fetching a non-existent manifest and fetching bundles
whose payload blob is not in the store
2014-07-10 06:29:51 +09:30
Andrew Bettison
34b6ff48bf Rhizome Java API: get raw payload
Added more "Serval-Rhizome-Bundle-xxx" headers to /restful/rhizome/
responses, so that a more complete manifest can be constructed from
them
2014-07-10 06:29:09 +09:30
Andrew Bettison
93e67ede63 Rhizome Java API: get manifest
Fixes assertion violation in GET /restful/rhizome/<BID>.rhm when <BID>
not found
2014-07-10 06:29:09 +09:30
Andrew Bettison
db8ee79a4e Rhizome Java API: list bundles 2014-07-10 06:29:09 +09:30
Andrew Bettison
a9ec5dcf69 Refactor Rhizome test script ready for re-use
Move add_bundles() function from 'rhizomerestful' test script into
testdefs_rhizome.sh, rename to rhizome_add_bundles() and add a new
SID argument
2014-06-30 11:11:01 +09:30
Andrew Bettison
067340bbba Change HTTP Authorization realm to "Serval RESTful API"
Was "Serval Rhizome", which is not accurate
2014-06-30 11:11:01 +09:30
Andrew Bettison
11e9d38299 Rename config 'rhizome.api.restful' to 'api.restful' 2014-06-30 11:11:01 +09:30
Jeremy Lakeman
63f02607e6 Fix duplicate payload detection over unicast links 2014-06-26 17:20:06 +09:30
Andrew Bettison
776d4de894 Split up 'rhizomehttp' test script
New scripts 'rhizomerestful' and 'meshmsrestful'
New testdefs_json.sh
Improved testdefs.sh functions for discovering HTTP server port
2014-06-26 15:08:48 +09:30
Andrew Bettison
660daf0438 Merge latest MeshMS Java API additions into 'development' 2014-06-24 15:10:16 +09:30
Andrew Bettison
c79a382a27 MeshMS Java API: mark messages as read 2014-06-24 12:12:48 +09:30
Andrew Bettison
7736a4ceb1 New HTTP RESTful requests: MeshMS read message(s) 2014-06-24 12:11:58 +09:30
Jeremy Lakeman
9f6ace518a Test that msp listen --service_name shuts down 2014-06-23 14:23:30 +09:30
Jeremy Lakeman
ca8f651b8d Cleanup the rhizome database every 30 minutes or so 2014-06-23 13:49:58 +09:30
Andrew Bettison
eba7f6555f MeshMS Java API: send message 2014-06-23 13:20:57 +09:30
Jeremy Lakeman
2ef73884a1 Add configurable limit to ensure a minimum amount of free space is preserved
- default to 100MB of free space
- currently disabled on android due to missing sys/statvfs.h
2014-06-23 13:06:21 +09:30
Andrew Bettison
819b8dc9e7 MeshMS RESTful API stricter on incoming form data
/restful/meshms/SID/SID/sendmessage now enforces Content-Disposition:
form-data (was ignoring), and enforces Content-Type: text/plain;
charset=utf-8 (was ignoring)

New negative test cases to ensure that these are treated strictly
2014-06-23 12:27:09 +09:30
Jeremy Lakeman
68dbaef38d Refactor Java API
Add stateful class to represent the state of the serval daemon, shifting some functions from BatPhone
Add convenience functions to get pre-configured MDP and HTTP Restful client classes
2014-06-20 10:55:09 +09:30
Andrew Bettison
f948f8e12e Include Java tests in 'tests/all' if Java compiler available 2014-06-19 11:38:39 +09:30
Andrew Bettison
b14c58bdf6 Merge NAF4 MeshMS Java API into development
Currently implements "list conversations" and "list messages"
2014-06-19 11:21:37 +09:30
Jeremy Lakeman
7ff89afcf4 Reinstate rhizome database storage limit
- old / large payloads should be evicted to fit more payloads
- if there isn't enough space, new payloads will not be added
2014-06-19 11:09:09 +09:30
Andrew Bettison
e9437e9a61 MeshMS Java API: list messages 2014-06-19 10:37:33 +09:30
Andrew Bettison
9cbd7c365c MeshMS Java API: list conversations 2014-06-19 09:55:12 +09:30
Andrew Bettison
f2772b0ce8 Refactor tests: testdefs_java.sh, testdefs_meshms.sh
meshms_add_messages() now puts message texts into TEXT[$n] array
instead of "text$n" files
2014-06-19 09:54:29 +09:30
Jeremy Lakeman
f4e6841ed7 Ensure directory service tests aren't creating a global routing table 2014-06-16 17:22:46 +09:30
Jeremy Lakeman
80e139c2d9 Ensure orphan payloads or manifests are cleaned up 2014-06-16 15:56:31 +09:30
Jeremy Lakeman
ee37385ec7 Recalculate routing table after last neighbours is lost, or interface is down 2014-06-12 17:15:29 +09:30
Jeremy Lakeman
52da7778cd Allow closing a monitor connection to force the server to quit 2014-06-11 20:39:26 +09:30
Andrew Bettison
11dcc3e972 Fix minor bugs in server start/kill tests 2014-06-11 11:15:28 +09:30
Jeremy Lakeman
afd31fe12c Ensure a race condition while starting servald only starts one process 2014-06-11 09:01:56 +09:30
Jeremy Lakeman
0b0e4cc8b4 Delay rhizome open database using an alarm, close on config change 2014-06-11 09:01:55 +09:30
Jeremy Lakeman
f09e6c38d2 Test that server quits after deleting pid file 2014-06-05 14:55:50 +09:30
Jeremy Lakeman
6d4ad0e150 Rework scheduler to allow for lazy but important alarms 2014-06-04 17:13:08 +09:30
Jeremy Lakeman
2a572d19bf Rework rhizome store to write content more lazily and cleanup more carefully 2014-06-04 17:13:08 +09:30
Jeremy Lakeman
77be2c7ef3 Fix mdp trace assert 2014-06-04 17:13:08 +09:30
Jeremy Lakeman
b6de3fcd34 Only send rhizome sync packets when we have neighbours 2014-05-30 15:03:22 +09:30
Jeremy Lakeman
0c294079af Improve reliability of routing tests 2014-05-30 14:56:31 +09:30
Andrew Bettison
33c61d64b9 Fix a faulty MSP test 2014-05-26 16:15:52 +09:30
Andrew Bettison
757e6c62ab Merge network simulator into development 2014-05-26 15:52:40 +09:30
Jeremy Lakeman
9f23b9bdcd Fix remove obsolete debug flags from test scripts 2014-05-26 14:16:35 +09:30
Jeremy Lakeman
3373d64079 Routing tests shouldn't care about prefer_unicast default value 2014-05-26 13:57:27 +09:30
Jeremy Lakeman
06f7eed260 Add msp test over unreliable link 2014-05-23 10:47:22 +09:30
Jeremy Lakeman
4dd2d2991e Use simulator for existing msp tests 2014-05-23 10:47:22 +09:30
Jeremy Lakeman
7420f46653 Create new AF_UNIX based network simulator 2014-05-23 10:47:22 +09:30
Jeremy Lakeman
c0c5412931 Add support for STOPping a connection before shutdown 2014-05-21 17:17:07 +09:30
Jeremy Lakeman
ca9faeef9b Remove respawn_on_crash behaviour, we aren't using or testing it 2014-05-12 13:44:51 +09:30
Jeremy Lakeman
672104bdd8 Create extensible service discovery protocol, with initial basic implementation 2014-05-09 11:21:18 +09:30
Jeremy Lakeman
4204973052 Allow for more duplicates in 2hop test 2014-05-09 11:21:16 +09:30
Andrew Bettison
df1814309b Improve MDP filter rules file syntax 2014-05-05 14:45:33 +09:30
Jeremy Lakeman
8ddd256b5c Add more asserts for ping stats, with some slack 2014-05-05 14:35:31 +09:30
Jeremy Lakeman
ab915a13fc Use routing table to check if a node is unreachable 2014-05-05 13:22:36 +09:30
Andrew Bettison
206aa936c0 Refactor 'vomp' test fifo handling
Use new execute --stdout-file=PATH test framework feature
Remove unnecessary shell variable indirection
2014-04-30 17:21:58 +09:30
Andrew Bettison
5035abde34 Improve test framework: assert_fork_is_running
Also assert_fork_is_not_running
Added 'framework' test case for fork primitives
2014-04-30 11:10:20 +09:30
Andrew Bettison
525aee88c5 Fix test framework bug: needed shopt -s extglob 2014-04-30 11:08:53 +09:30
Andrew Bettison
ea9de737a2 MDP packet filter rules configuration
Configurable rules file path: mdp.filter_rules_path

Add filter rules parsing code, add filtering to outgoing as well as
incoming packets

New test script: tests/mdp with two test cases

Improve test defs: new execute_servald() function

Widen MDP port number format from 8 to ten chars to accommodate leading
"0x"
2014-04-29 15:31:50 +09:30
Andrew Bettison
684735b11c Fix config file timestamp bug
Ensure that timestamp always advances by at least one second every time
the config file is modified, add test case

Refactor test defs: detecting started HTTPD is not specific to Rhizome

Refactor struct file_meta and associated functions into "io.h" and io.c

Add various strbuf helper functions for formatting struct timespec
and struct file_meta to assist debug logging
2014-04-29 15:04:20 +09:30
Andrew Bettison
459c5a2303 Improve "mdp ping" command
Packet stats now exclude duplicate pong replies

With <count> arg, exits immediately once a pong has been received for
every ping, unless new --wait-for-duplicates option given

Routing tests for unreliable links now assert all pongs received and
very few duplicates
2014-04-17 16:09:34 +09:30
Andrew Bettison
8bc746af25 New "config sync" command 2014-04-16 19:28:16 +09:30
Andrew Bettison
2716228074 Add server watchdog and config reload test 2014-04-16 18:06:27 +09:30
Jeremy Lakeman
c6e461bcb0 Attempt to create folders for binding AF_UNIX sockets 2014-04-11 17:00:47 +09:30
Jeremy Lakeman
14be44d211 Enhance console command to allow multiple concurrent calls 2014-04-10 13:46:07 +09:30
Jeremy Lakeman
23cb4600e2 Improve VOMP testing 2014-04-10 11:14:46 +09:30
Andrew Bettison
6371203703 Finish removing SLIP code and test case
More simplification of "serval.h"
2014-04-07 16:22:31 +09:30
Andrew Bettison
53c1b1c04c Filesystem Heirarchy Standard (FHS) paths
If no instance directory specified, then use paths like
/etc/serval/serval.conf
/var/cache/serval
/var/log/serval
/var/run/serval
etc. for files, instead of all in a single directory.

Log all directory creation as INFO messages.

Interpretation of log.file.directory_path has changed slightly.

Updated servald configuration tech doc.
2014-03-26 15:35:43 +10:30
Andrew Bettison
53d5a8734b Fix Rhizome cleanup to remove temporary payloads
Fix test case to properly assert that all stale payloads are
deleted
2014-03-18 16:03:26 +10:30
Andrew Bettison
dc588e964d Fix "name=/file" bug in Rhizome add
Improved Rhizome test scripts and added new assert_manifest_fields()
function to Rhizome test defs
2014-03-12 15:48:00 +10:30
Jeremy Lakeman
a6d6175576 Add config set warning if attempting to drop packets on datagram sockets 2014-03-12 14:55:46 +10:30
Jeremy Lakeman
2e4835d7d6 Ensure tests with packet filtering are using the dummy network interface 2014-03-12 14:32:59 +10:30
Jeremy Lakeman
ee68856878 Reduce log spam from debug.rhizome 2014-03-12 14:32:59 +10:30
Andrew Bettison
0bef066949 Store Rhizome external blob files in "blob" subdirectory
This will make issue #50 a bit easier to tackle
2014-03-07 16:55:38 +10:30
Jeremy Lakeman
6b17cd580f Enable rhizome HTTP transfers over loopback while using local sockets 2014-03-05 09:55:23 -08:00
Andrew Bettison
fd1b36a9da Refactor new MissingPayload test case
Move more code into fixture, add TODO comment
2014-03-03 15:14:34 +10:30
Andrew Bettison
19aed8ecb5 Fix test defs: pop_instance did not restore instance
Add push_and_set_instance() function
Remove redundant set_instance calls in test cases
Cope if error() returns (can happen during teardown)
2014-03-03 15:14:34 +10:30
Andrew Bettison
08e02b18db Improve test framework: tfw_run() function 2014-03-03 15:14:34 +10:30
Jeremy Lakeman
974c87adf2 Remove orphan file rows if we notice the payload is missing 2014-03-01 05:45:32 +10:30
Jeremy Lakeman
1ade36f8df A missing payload should be re-fetched 2014-02-28 10:00:49 +10:30
Jeremy Lakeman
a6f408f013 Return mdp inet & http port numbers in servald start & status 2014-02-25 12:42:36 +10:30
Jeremy Lakeman
33cbc98300 Add proc files for http server & mdp udp ports 2014-02-24 17:15:28 +10:30
Jeremy Lakeman
dfda3f7a74 Move generic command line API to test package 2014-02-24 17:13:41 +10:30
Jeremy Lakeman
20494a90d5 Add Java MDP Client API
- support mdp clients over loopback UDP
  Note this is using an environment variable to specify the port number
  In future we expect to support environments where this port is already bound
- monitor mdp sockets in a single & separate thread
2014-02-21 16:39:47 +10:30
Jeremy Lakeman
693d1e9b60 Add clearer example java api usage 2014-02-20 15:35:31 +10:30
Jeremy Lakeman
7ed6be3690 Introduce basic vomp console test 2014-02-19 16:39:05 +10:30
Jeremy Lakeman
6f75d93460 Shift echo service to new internal api, ensure local services can respond to local clients 2014-02-19 16:34:47 +10:30
Jeremy Lakeman
487df0408d Refactor mdp packet processing to avoid using struct overlay_mdp_frame 2014-02-19 16:34:11 +10:30
Jeremy Lakeman
9e704e5b01 Capture servald log from test completion 2014-02-19 16:03:57 +10:30
Jeremy Lakeman
46ef40cf23 Migrate and refactor more interfaces from batphone 2014-02-19 15:35:08 +10:30
Andrew Bettison
4062b3adbf Fix typos in 'keyring' test suite 2014-02-17 17:13:58 +10:30
Jeremy Lakeman
348fc5f23b Fix meshms conversations parsing when looking for a single SID 2014-02-12 15:08:24 +10:30
Jeremy Lakeman
99fb8b6108 Move jni interface classes from batphone repo 2014-02-12 13:09:17 +10:30
Andrew Bettison
0769fa54e8 Improve MeshMS RESTful HTTP failure reporting
Add "meshms_status_code" element to simple JSON responses
2014-02-07 16:28:40 +10:30
Andrew Bettison
5dd9ea7a6b Improve MeshMS failure handling
Introduce "enum meshms_status" to replace int -1,0,1 for return values
from MeshMS internal functions

Add test cases to check that status 2 is returned when attempting to
send a message from an identity that is not known (unlocked)
2014-02-06 18:10:00 +10:30
Andrew Bettison
fd86a3d17f Implement HTTP POST /restful/meshms/<SID>/<SID>/sendmessage
Rename struct meshms_ply fields from "buffer" to "record" for
consistency with comment terminology
2014-02-05 14:28:15 +10:30
Andrew Bettison
015ed0b181 Refactor to rename "Rhizome HTTP server" to "HTTPD server"
Move HTTPD start/stop/dispatch logic from rhizome_httpd.c into new
httpd.c

Rename config.debug.httpd -> config.debug.http_server
Rename config.debug.rhizome_httpd -> config.debug.httpd
Rename config.debug.rhizome_nohttptx -> config.debug.nohttptx
2014-02-03 15:56:07 +10:30
Andrew Bettison
f4249707a4 Implement HTTP GET /restful/meshms/<SID>/<SID>/newsince/<token>/messagelist.json
Two new test cases
2014-01-31 14:58:30 +10:30
Andrew Bettison
ebe444ffe3 Refactor rhizomehttp MeshMS message list test
New function add_messages() to use in fixture

Improve existing test case by adding more ACKs and messages
2014-01-31 14:30:01 +10:30
Andrew Bettison
c73bc49cf0 Improve MeshMS RESTful HTTP requests
Include local (my) and remote (their) SIDs in JSON responses for
conversationlist.json and messagelist.json, so that the same JSON
structures can be used in future for non-SID-specific queries.

Refactor MeshMS message iterator, rename "sender" and "recipient" fields
to "my" and "their", for consistency with the rest of the MeshMS source
code, and because "sender" and "recipient" are properties that apply to
a single message or single ply, not to a message thread (conversation).
2014-01-31 14:29:57 +10:30
Andrew Bettison
7b5752a111 Implement HTTP GET /restful/meshms/<SID>/<SID>/messagelist.json 2014-01-28 15:09:51 +10:30
Andrew Bettison
8897563d09 Implement HTTP GET /restful/meshms/<SID>/conversationlist.json 2014-01-23 10:52:43 +10:30
Andrew Bettison
6a1c8bcf5a All HTTP RESTful responses are JSON, not HTTP
Even error responses (typically code 403)
2014-01-20 15:44:21 +10:30
Jeremy Lakeman
68421db114 Add msp debug flag 2014-01-09 15:06:35 +10:30
Jeremy Lakeman
48dce9a575 Add tcp forwarding on the listen end of an MSP connection 2014-01-09 15:06:35 +10:30
Jeremy Lakeman
99e08d1516 Add command to support forwarding TCP connections to an MSP service 2014-01-09 15:06:35 +10:30
Jeremy Lakeman
e09387b27d Refactor stdio handling 2014-01-09 15:06:34 +10:30
Jeremy Lakeman
6d942f8d4f Improve tests 2014-01-09 15:01:38 +10:30
Jeremy Lakeman
4bb3336b33 Generalise method for defining multiple interfaces 2014-01-09 15:01:38 +10:30
Jeremy Lakeman
506518ad06 Allow interfaces to use local unix addresses
- broadcast packets are sent to all sockets in the same folder
- switched most tests to use local sockets
2014-01-09 15:01:38 +10:30
Jeremy Lakeman
df6af96455 Test connection keep alive and timeouts 2014-01-09 15:01:37 +10:30
Jeremy Lakeman
741442c54c Pipe file contents across msp connection 2014-01-09 15:01:37 +10:30
Jeremy Lakeman
3a533187b7 Improve msp failure handling 2014-01-09 15:01:37 +10:30
Jeremy Lakeman
db7f68afd7 Initial msp library implementation with connection state tracking 2014-01-09 15:01:37 +10:30
Andrew Bettison
669080edc8 Add 'rhizomehttp' to 'all' tests
Disable unused 'rhizomehttp' tests
2013-12-30 18:10:39 +10:30
Andrew Bettison
ee9c96bb8c Implement HTTP POST /restful/rhizome/insert
Change HTTP request buffer pointers from (const char*) to (char*)
because some Rhizome operations can modify received data in-place, eg,
when decrypting it.
2013-12-30 18:09:56 +10:30
Andrew Bettison
34188fa489 Replace config 'rhizome.external_blobs' with 'rhizome.max_blob_size' 2013-12-30 16:30:43 +10:30
Andrew Bettison
72040517e1 New enum rhizome_payload_status
Refactor a lot of Rhizome bundle storage code to use the new "enum
rhizome_payload_status" instead of mysterious int values to represent
the outcome of the operation.
2013-12-30 16:30:35 +10:30
Andrew Bettison
9ebef81a49 Formalise "rhizome add file" exit status
Formalise add-bundle result in "enum rhizome_bundle_status"

Rewrite rhizome_manifest_finalise(), rhizome_find_duplicate() and
rhizome_add_manifest() to return enum rhizome_bundle_status

New function rhizome_manifest_check_stored() that compares a manifest
with its stored counterpart and returns enum rhizome_bundle_status

Remove redundant rhizome_manifest_check_sanity(), consolidating all
manifest validation rules in rhizome_manifest_validate(), which now
checks the 'id' field is present, and that 'sender' and 'recipient' are
both present for MeshMS

Correct manifest finalisation logic: set the 'finalised' flag in
rhizome_manifest_validate(), not in rhizome_manifest_verify() (which
sets 'selfSigned'), and consistently clear 'finalised' flag in all
attribute setter functions

Remove manifest 'ttl' field and all references thereof (leaving unused
space in Rhizome BAR)

Rename some payload functions for clarity
2013-12-21 14:37:18 +10:30
Andrew Bettison
39b2f3a6f5 Improve 'rhizomehttp' test descriptions 2013-12-19 11:29:28 +10:30
Andrew Bettison
6798e943ec Implement HTTP GET /restful/rhizome/<BID>/decrypted.bin 2013-12-16 15:09:46 +10:30
Andrew Bettison
8c9ac6c1a5 Improve test case for /restful/rhizome/<BID>/raw.bin
Include some encrypted payloads in the test
2013-12-13 17:41:26 +10:30
Andrew Bettison
b53cabaae5 Unify output manifest fields from CLI rhizome commands 2013-12-13 17:24:59 +10:30
Andrew Bettison
6361bfd757 Implement HTTP GET /restful/rhizome/<BID>/raw.bin 2013-12-13 16:36:37 +10:30
Andrew Bettison
183cb46000 Add Rhizome HTTP response headers to /restful/rhizome/<BID>.rhm 2013-12-13 14:49:55 +10:30
Andrew Bettison
26e0120898 Implement HTTP GET /restful/rhizome/<BID>.rhm 2013-12-05 17:25:58 +10:30
Andrew Bettison
6bff87f7e5 Increase 'rhizomeprotocol' test case timeout
The 50% MDP packet loss test case is now likely to take more than 60
seconds to transfer the bundles, probably because of stricter manifest
syntax
2013-12-01 05:49:50 +10:30
Andrew Bettison
8db5f9c14a Merge branch 'anyservice' into 'development'
Allows any valid "service" manifest field in the "rhizome add file"
command

Many improvements in Rhizome manifest parsing; stricter manifest syntax
rules (no comment or blank lines, field names must be alphanumeric
identifiers), faster preliminary manifest inspection when receiving
manifest advertisements or syncing manifests

The 'development' branch introduces "struct socket_address" which
coincidentally fixed the recently encountered Linux kernel 3.12
recvmsg(2) EINVAL problem, so that 'rhizomeprotocol' tests which fail on
the 'anyservice' branch will pass after this merge
2013-12-01 05:44:01 +10:30
Andrew Bettison
0dd1b302b5 Rewrite Rhizome manifest parsing
Move validation checks into new function rhizome_manifest_validate()

Remove rhizome_manifest 'errors' field

Replace rhizome_manifest 'warnings' with 'malformed'

Replace rhizome_manifest 'manifest_bytes' with 'manifest_body_bytes' and
refactor to use 'manifest_all_bytes' in all manifest i/o

Refactor rhizome_manifest_verify() and reverse sense of return value to
match rhizome_manifest_validate()

New function rhizome_manifest_inspect() -- lightweight manifest parser
used when receiving Rhizome advertisements

New 'rhizomeops' test case for invalid manifest "service" field values,
now passes
2013-12-01 05:13:34 +10:30
Andrew Bettison
5e5e097d7f Accept any well formed Rhizome service field
Do not limit service field to only recognised supported values 'file',
'MeshMS1' and 'MeshMS2'

New test case checks for malformed service field, fails
2013-12-01 05:13:33 +10:30
Jeremy Lakeman
1681abd620 Increase default mdp timeout & make it configurable 2013-11-27 13:10:11 +10:30
Jeremy Lakeman
1b34c2dc13 Improve test reliability 2013-11-27 13:10:09 +10:30
Jeremy Lakeman
ad33264834 Rename mavlink to radio_link
- split link state from slip structures and encapsulate it completely
- allocate interface specific buffers on demand
2013-11-27 13:09:26 +10:30
Jeremy Lakeman
5ea19ab538 Better approximation of TDMA timing 2013-11-27 13:00:02 +10:30
Andrew Bettison
851dbb4d64 Add 'debug.subscriber' config option
Add debug logging to find_subscriber() function
2013-11-26 15:33:54 +10:30
Andrew Bettison
59a6ded10c Update 'dnaprotocol' debug flags 2013-11-26 15:33:54 +10:30
Andrew Bettison
711364ad8d Merge branch 'meshextender' into 'development' 2013-11-21 17:05:03 +10:30
Andrew Bettison
513aa15968 Finish HTTP /restful/rhizome/newsince/... test case
Uses new test framework fork support
2013-11-20 15:06:58 +10:30
Andrew Bettison
a882b8ebc6 Fix Rhizome Direct stress test - longer command timeouts
Since a default timeout was added to the execute() primitive
(b1b5a79) the Rhizome Direct test fails.
2013-11-20 15:06:07 +10:30
Andrew Bettison
c5014baf62 Improve test framework: better fork functions
Replace forkKillAll with fork_terminate_all

Replace forkWaitAll with fork_wait_all

Add fork_terminate and fork_wait functions

Introduce fork labels '%<alphanumeric>' as arguments to all fork
functions

Treat a forked process exit status 143 (killed with SIGTERM) as a
failure or error -- it happens when a forked process is terminated
using fork_terminate or fork_terminate_all
2013-11-20 15:01:37 +10:30
Andrew Bettison
29fab6dcb7 Add 'rhizome.api.restful.newsince_poll_ms' config option 2013-11-20 14:50:57 +10:30
Andrew Bettison
833466411c Test case for HTTP /restful/rhizome/newsince/...
Not passing yet
2013-11-19 17:28:04 +10:30
Andrew Bettison
f8f7716544 Improve HTTP /restful/rhizome/bundlelist.json token
Now tokens are included in each row instead of a single token in the
enclosing JSON object.  Only puts tokens on the first row, and
thereafter all rows with a rowid higher than for the previous row that
had a token.
2013-11-18 15:10:28 +10:30
Andrew Bettison
973bb9c897 Merge branch 'naf4' into 'development' 2013-11-13 17:59:01 +10:30
Andrew Bettison
4380fdcccd Token in /restful/rhizome/bundlelist.json output 2013-11-13 16:58:28 +10:30
Jeremy Lakeman
0b404b0260 Implement id list command 2013-11-13 12:00:46 +10:30
Jeremy Lakeman
ff1f1ff093 Allow tag primitives to be used outside of keyring entries 2013-11-13 12:00:05 +10:30
Jeremy Lakeman
9680b24f23 Add support for adding and updating tags in the keyring 2013-11-13 11:59:21 +10:30
Jeremy Lakeman
534b01ba2a Add explicit set did test 2013-11-13 11:52:46 +10:30
Andrew Bettison
13634f8748 Add ROWID field to struct rhizome_manifest
New ".rowid" output field from rhizome add, import, extract, export
operations.  (Also added missing ".inserttime" and "date" fields to
some operations.)

Use new "rhizome add file" .rowid output field to check output of of
/restful/rhizome/bundlelist.json
2013-11-11 18:18:08 +10:30
Andrew Bettison
6b961c56ce Fix HTTP /restful/rhizome/bundlelist.json
Write test case assertions using jq(1) utility, increase from four
bundles to 100.

Fix bugs in HTTP server content generation logic.

Make payload content generator read payload 4KiB at a time, to
always read on filesystem block boundaries for performance.  Increase
size of payload in relevant test case.
2013-11-11 16:21:26 +10:30
Andrew Bettison
45442d3eb4 Rewrite bundle author authentication
Replaced 'int has_author' manifest element with new 'enum authorship'
element to record the result of author authentication, to avoid
repeating expensive crypto operations.

Separated the handling of bundle secret arguments from author lookup and
authentication.  The new rhizome_apply_bundle_secret(m,bsk) is now
called at the top level to set the manifest secret key (if it
validates), and thereafter there is no need to pass the 'bsk' argument
to any other functions, as they can simply check the 'haveSecret' field
of the manifest.

Removed rhizome_extract_privatekey() which combined author lookup and
bundle secret validation, and replaced it with functions that only deal
with the author: rhizome_lookup_author() and rhizome_authenticate_author().

Renamed other functions to make their purpose and effect clearer.

Formalised the semantics of only storing AUTHENTICATED author SIDs in
the 'author' column of the MANIFESTS table, which necessitated a change
to a 'rhizomeops' test case: when adding a file using a BK-less
manifest, the author column is set to null, so the Rhizome list output
does not show the bundle as ".fromhere" and does not give an author for
that bundle.
2013-11-06 23:58:17 +10:30
Andrew Bettison
016fbe0244 Merge branch 'development' into 'naf4' 2013-11-04 20:01:58 +10:30
Andrew Bettison
bacba19dc1 Refactor manifest: specific setter functions
Replace generic rhizome_manifest_set() and rhizome_manifest_set_ll()
with per-field setter functions, eg, rhizome_manifest_set_filesize().
Struct rhizome_manifest elements for all known fields, to replace the
use of rhizome_manifest_get() and rhizome_manifest_get_ll() everywhere:
sender, recipient, service, name, date, bundle_key.

Add boolean validity flags for binary blob types, to avoid having to compare
with many bytes of all-zero to detect presence, eg, has_sender, has_recipient,
has_author, has_bundle_key.  These maintained by the setter functions.

Rename existing manifest struct elements to be the same as their field
names: fileLength -> filesize, journalTail -> tail.

More use of unsigned int, size_t and uint64_t for payload sizes, offsets, byte
counts, etc. especially in rhizome_store.c and meshms.c.  More uniform use of
size_t to dimension memory buffers.  Fix some printf(3) style format strings
for 64-bit correctness on 32-bit systems.  Use new constant RHIZOME_SIZE_UNSET
instead of -1 to indicate unknown dimension, and explicitly assert its absence
before comparisons and arithmetic, for safety.

Replace some 'int' loop variables with 'unsigned' where appropriate.

Fix bugs discovered in MeshMS bundle private/public key generation and
bundle secret key handling for export/extract commands.

Instrument the first MeshMS test case to aid debugging.

New debug config flag: debug.manifest logs all modifications to all manifest
fields by setter functions.

Rename debug config flag: debug.rhizome_bind -> debug.rhizome_sql_bind.
2013-11-04 19:17:49 +10:30
Andrew Bettison
675e2021cc Add ".secret" output field to all rhizome export/extract commands 2013-11-04 17:42:22 +10:30
Andrew Bettison
2ac1bc3240 Get HTTP Basic authentication working
Second 'rhizomehttp' test now passes
2013-10-30 11:07:45 +10:30
Andrew Bettison
21fe12859f Implement HTTP basic authentication
Use it in /restful/rhizome/bundlelist.json -- first 'rhizomehttp' test
passes
2013-10-29 17:32:04 +10:30
Andrew Bettison
9d54c629b2 Merge branch 'naf4' into 'development'
Rewritten HTTP request parsing and response buffering
2013-10-28 18:20:40 +10:30
Andrew Bettison
6ee691b161 Fix bugs in new HTTP server MIME body parsing code
Fixes all remaining 'rhizomeprotocol' test failures.
2013-10-28 14:25:16 +10:30
Andrew Bettison
2a9329c0c8 Fix bugs in new HTTP server MIME body parsing code
Fixes 'rhizomeprotocol' test 24 HttpImport.  Five tests still fail.
2013-10-27 13:19:23 +10:30
Andrew Bettison
0397a47753 Fix some test failures in new HTTP server code 2013-10-26 20:28:47 +10:30
gardners
e9e0a1b4a4 allow setting of % of 252 byte RS protected packets that we want
to have delivered.  Adjust tests accordingly.  Create new test
for ~50% packet delivery on fakeradio in preparation for network
coding.
2013-10-23 19:45:50 -07:00
Andrew Bettison
640a61cbe5 Add 'debug.rhizome_httpd' config option 2013-10-16 11:23:45 +10:30
Jeremy Lakeman
d5d5737f1c Rename 'id revoke' to 'id relinquish' to match documentation 2013-10-15 14:52:07 +10:30
Andrew Bettison
017286b158 Merge branch 'development' into 'naf4' 2013-10-15 11:16:56 +10:30
Andrew Bettison
3d3e900e72 First HTTP RESTful interface tests 2013-10-14 17:46:09 +10:30
Andrew Bettison
70af3289f6 Refactor setup for 'rhizomeops' test script 2013-10-14 16:49:06 +10:30
Andrew Bettison
eb46cc99b3 Improve test defs: setup_curl()
Replace setup_curl_7() function in separate test scripts with a single,
general setup_curl() function in testdefs.sh which takes the minimum
version number as its argument.
2013-10-14 16:48:20 +10:30
Andrew Bettison
ef2cc1404c Improve test framework: tfw_cmp_version()
Also add unit tests for test framework itself
2013-10-14 16:45:56 +10:30
Jeremy Lakeman
b8ec568711 Send 3-way request to stop routing an identity 2013-10-14 14:40:19 +10:30
Jeremy Lakeman
0c1c767af0 Wait until sid from local keyring is unreachable before claiming ownership 2013-10-10 15:59:44 +10:30
Jeremy Lakeman
ef7351bddc Unload identities from a running daemon 2013-10-10 14:27:00 +10:30
Jeremy Lakeman
ae7e120ed5 Pass keyring entry pin to running daemon and unlock identities 2013-10-09 15:34:41 +10:30
Andrew Bettison
c9f9025581 Fix intermittent 'routing' test failures
Caused by tests running so slowly that nodes become unreachable due to
timeout before the assertion of reachability is made.  The timeout was
hard coded at five times the tick interval (2.5 seconds).

Introduced new config options: mdp.iftype.*.reachable_timeout_ms and
interface.*.mdp.reachable_timeout_ms to allow the timeout to be
configured.  Configure the timeout to 60 seconds in 'routing' tests.

Changed the config schema to move options interfaces.*.mdp_tick_ms and
interfaces.*.packet_interval into interfaces.*.mdp.tick_ms and
interfaces.*.mdp.packet_interval (introduced new sub-struct 'mdp').
2013-10-08 16:36:07 +10:30
Andrew Bettison
55edc74482 Fix intermittent 'rhizomeops' test failure 2013-10-07 04:47:14 +10:30
Andrew Bettison
c3a4ab1f25 Fix two Rhizome Direct test cases 2013-10-04 03:11:54 +09:30
Andrew Bettison
41c862ea30 Fix 'rhizomeops' test case
Was broken by 919455608f
2013-10-03 15:13:28 +09:30
Andrew Bettison
06d7620314 Add "rhizome add file" --force-new option
Overrides the default behaviour of de-duplicating added manifests

New 'rhizomeops' test case: AddForceDuplicate
2013-09-30 16:48:18 +09:30
Andrew Bettison
2868cf0adf Improve Rhizome test functions
Options to allow manifest name to be given explicitly to override the
conventional "<filename>.manifest"
2013-09-30 16:39:58 +09:30
Andrew Bettison
919455608f Rename 'rhizomeops' test case
AddDuplicate -> AddDeDuplicate
2013-09-30 16:34:17 +09:30
Andrew Bettison
5e164607aa Expand output of rhizome add and import
Rename 'secret:' field to '.secret:' (non-manifest-fields start with '.'
using the same convention as rhizome list)

Add '.author' and 'BK' fields to "rhizome add"

Add 'BK' field to "rhizome import"

Fix 'rhizomeops' tests to assert no 'author' and 'BK' output fields from
"rhizome add" with no author

Fiz testdefs.sh and testdefs_rhizome.sh to support new output fields
2013-09-27 22:34:22 +09:30
Jeremy Lakeman
dab07dd4dc Remove monitor.socket & mdp.socket from tests 2013-09-24 16:27:34 +09:30
Jeremy Lakeman
57577abca6 Merge branch 'mavlink' into development 2013-09-20 14:58:29 +09:30
Jeremy Lakeman
e3a5e8c353 Use heartbeat packets to control serial buffering
- Improve simulation of fakeradio
- Backtrack on partial RS errors in case of truncation / heartbeat insertion
2013-09-20 14:41:50 +09:30
Andrew Bettison
4851eaf536 Issue #20: remove unique socket names from tests
The recent work on abstract local socket names means that every instance has
its own unique abstract socket name anyway, so there is no need for test
scripts to use $TFWUNIQUE in MDP and Monitor socket names to avoid
intereference when run concurrently.
2013-09-20 14:24:55 +09:30
Jeremy Lakeman
2a5ba97b48 Add rhizome test over fakeradio 2013-09-13 14:02:35 +09:30
Jeremy Lakeman
3f123e1875 Send the next packet via stream interface ASAP 2013-09-12 16:14:01 +09:30
Jeremy Lakeman
6f26594447 Simplify mavlink link frame creation
- build one frame at a time directly from the prepared overlay buffer
2013-09-11 17:15:43 +09:30
Jeremy Lakeman
e29564bc6c Refactor fakeradio
- completely event driven and non-blocking
- modelling of tx & rx serial buffers that will truncate on overfilling
- each radio takes turns to send a packet, with approx TDMA overhead
2013-09-11 15:03:43 +09:30
gardners
0cbebedc77 Add tx throttling for packet radios
avoids missing data due to lack of flow control.
2013-09-10 11:53:38 +09:30
Andrew Bettison
7add45214c Add [keyring-pin] arg to keyring load command 2013-09-10 11:28:55 +09:30
Andrew Bettison
e74aec9354 Tests for starting daemon with keyring entry PINs 2013-09-09 14:47:02 +09:30
Andrew Bettison
d177512746 Add test for server start with --keyring-pin option 2013-09-09 14:47:02 +09:30
Andrew Bettison
18e2916cec Add more keyring load tests, fix duplicate identity bugs 2013-09-07 04:03:28 +09:30
Andrew Bettison
8a300c2520 Get new keyring dump command and test working 2013-09-05 16:34:01 +09:30
Andrew Bettison
4b0550afd1 Fix new keyring load test 2013-09-05 09:19:45 +09:30
Andrew Bettison
f95e41374b Implement keyring load command, not working yet
Write new keyring test, fails with ERROR because assertGrep -F option is
not supported.
2013-09-04 23:47:17 +09:30
Andrew Bettison
9d1c3e0cba Fix test case doc comment 2013-09-03 17:05:50 +09:30
Jeremy Lakeman
e3b616421f Remember outgoing links 2013-08-30 17:38:29 +09:30
Andrew Bettison
572d7c2b5d Add tests for new keyring environment vars 2013-08-30 11:20:19 +09:30
Jeremy Lakeman
44ab51e2ab Don't clean the database for every command line operation by default 2013-08-27 13:44:05 +09:30
gardners
9b09021605 add test for falling over to MDP when HTTP rhizome fetching fails. 2013-08-25 16:37:44 +09:30
Jeremy Lakeman
643b21d6a7 Fetch file payloads if they are missing, even if we already have the manifest 2013-08-22 15:14:21 +09:30
Jeremy Lakeman
f9b828c3dd Remove payload if hash doesn't match when reading back 2013-08-21 15:45:18 +09:30
Jeremy Lakeman
1468d2deb0 Pre-emptively send small payloads via MDP 2013-08-21 09:21:40 +09:30
Jeremy Lakeman
5c374e9e6e Basic routing test with simulated serial modem interface 2013-08-19 16:59:10 +09:30
Jeremy Lakeman
771cb4151b Fetch journal range via HTTP 2013-08-15 17:00:43 +09:30
Jeremy Lakeman
21132466f1 Serve a range of a file payload via HTTP 2013-08-15 16:31:26 +09:30
Jeremy Lakeman
e322114850 Only transfer new journal content via MDP 2013-08-15 14:36:56 +09:30
Jeremy Lakeman
e6ce14af45 Test rhizome over unicast links 2013-08-13 12:06:05 +09:30
Jeremy Lakeman
36b0fe51ae Fix ping via directory service test 2013-08-13 10:55:48 +09:30
Jeremy Lakeman
a516af616e Improve reliability
- don't drop packets that haven't been resolved yet on incoming acks
- statistically drop unicast packets during tests
2013-08-12 16:38:37 +09:30
Jeremy Lakeman
9a50d8a9ef Route mulit-hop via unicast links 2013-08-09 16:37:54 +09:30
Jeremy Lakeman
55657623aa Detect unicast links 2013-08-09 12:18:14 +09:30
Jeremy Lakeman
81afc42d8b Major refactor to packet queuing and delivery in preparation for unicast link tracking.
Unicast tests broken
2013-08-09 09:37:08 +09:30
Jeremy Lakeman
0f2d082a0d Add stress test that sends lots of messages while transferring 2013-08-05 11:46:30 +09:30
Jeremy Lakeman
91a600fba7 Add separate rows for delivered and read status
- add explicit transaction around blob writes so the commit can be retried
- remove status columns
- remove SID's from message log output to reduce size and complexity
2013-08-02 14:02:56 +09:30
Jeremy Lakeman
68bf04f69d Change manifest service to MeshMS2 2013-08-01 11:37:35 +09:30
Jeremy Lakeman
e4882f175b Output message read offsets in conversation list 2013-07-31 10:33:01 +09:30
Jeremy Lakeman
88c986ebb5 Use sid_t consistently, tweak command output 2013-07-30 14:05:05 +09:30
Jeremy Lakeman
94381bab49 Fix non-journal, non-block based payload encryption 2013-07-26 15:13:22 +09:30
Jeremy Lakeman
fa012347a8 Improve test reliability 2013-07-26 14:40:06 +09:30
Jeremy Lakeman
7634e1fcfb Add command to mark messages as read 2013-07-26 14:36:11 +09:30
Jeremy Lakeman
a1caa4bf68 ACK conversations when listed, track conversations in a bundle 2013-07-25 14:49:06 +09:30
Jeremy Lakeman
f1139d4c0e Reduce meshms entry encoding size 2013-07-23 16:21:07 +09:30
Jeremy Lakeman
ddfb7d9417 First cut of new meshms API, unfinished 2013-07-22 17:55:02 +09:30
Jeremy Lakeman
2c1a995275 Don't encrypt by default when the recipient is broadcast 2013-07-19 10:28:15 +09:30
Jeremy Lakeman
a4bf4edbfc Write out of order packets with MDP transport
- add random order writing to rhizome store API
- remove rhizome flush, force all callers to provide buffer
2013-07-17 16:57:59 +09:30
Jeremy Lakeman
49e0286b43 Reduce packet header size for point-to-point links
- new interface.x.point_to_point config
- disable local echo for point to point dummy interfaces
- add ME and YOU sid abbreviation codes after learning other sid
2013-07-16 15:38:00 +09:30
Jeremy Lakeman
98dcd973ec Add help / version output jni test 2013-07-09 16:24:47 +09:30
Jeremy Lakeman
d6c9dbc1a3 Improve reliability of multiple identity test 2013-07-08 17:12:56 +09:30
Jeremy Lakeman
88d88e685c Improve backward compatibility for broadcasts & packet retries 2013-07-08 16:01:58 +09:30
Jeremy Lakeman
62a4cd407d Identify a journal bundle by the presense of a tail field 2013-07-08 14:57:47 +09:30
Jeremy Lakeman
e758e0130f Create journal append command 2013-07-08 14:29:08 +09:30
Jeremy Lakeman
6254e0d81a Reduce IO strain from stress test while increasing bundle count 2013-06-20 11:10:02 +09:30
Jeremy Lakeman
3e57add9ca Fix, initialise read / write handles so we don't close fd=0 2013-06-19 12:05:56 +09:30
Jeremy Lakeman
40364be92f Fix and improve rhizome stress test
- reduce dependance on routing
- fail immediately if a servald instance crashes
2013-06-18 16:33:41 +09:30
Jeremy Lakeman
c6241c6634 Ensure routing table is updated when indirect nodes go offline 2013-06-11 10:18:28 +09:30
Jeremy Lakeman
b1f384cd63 Swap between broadcast & unicast links as they become available 2013-05-31 11:32:31 +09:30
Jeremy Lakeman
bc50d3e184 Establish a unicast link to nodes who are filtering broadcasts 2013-05-29 11:58:06 +09:30
Jeremy Lakeman
d6d3c1f413 Add ping backward compatibility test 2013-05-29 11:58:06 +09:30
Jeremy Lakeman
37ef97268b Disable rhizome when testing dnahelper to reduce IO load 2013-05-29 11:58:06 +09:30
Jeremy Lakeman
aa75e519f3 Rename 2-hop test 2013-05-29 11:58:06 +09:30
Jeremy Lakeman
8f575262ff Reduce packet storm to reduce I/O required for running tests 2013-05-29 11:58:06 +09:30
Jeremy Lakeman
8ab5073869 Adjust packet format for better duplicate detection 2013-05-29 11:58:05 +09:30
Jeremy Lakeman
02be4c2979 Add support for retransmitting broadcast mdp packets 2013-05-29 11:58:05 +09:30
Jeremy Lakeman
a213872f09 Don't ack if we aren't being used in a route 2013-05-29 11:58:05 +09:30
Jeremy Lakeman
5b4e4919cf Mark resent packets so they can be dropped 2013-05-29 11:58:05 +09:30
Jeremy Lakeman
e519633f7a Resend last ACK if it is explicitly NACKed 2013-05-29 11:58:05 +09:30