Commit Graph

731 Commits

Author SHA1 Message Date
Andrew Bettison
e34141a2a5 Upgrade to SQLite 3.14.2
Ignores a variety of warnings when compiling sqlite3.c.

Uses the new sqlite3_trace_v2() function for logging SQL statements, now
that sqlite3_trace() and sqlite3_profile() are deprecated.

Fixes a rhizomeops test failure caused because SQLite 3.12 increased the
default block size from 1024 to 4096, causing the fixed overhead of an
empty Rhizome db to exceed 32KB.  This caused the test to fail because
the first "file add" failed.  Now the test fixture adapts to the fixed
overhead.

Deletes the SQLite 3.10.2 source amalgamation.
2016-09-21 22:47:35 +09:30
Jeremy Lakeman
adf0a880bc Any identity could be the main one, check for any valid route 2016-09-21 17:27:39 +09:30
Andrew Bettison
298b83f97b Fix a failing keyringrestful test
The recent randomising of keyring slots broke an assumption
about the order of the 'keyring list' output.  Now the test's
assertions do not assume any order.
2016-09-21 12:16:50 +09:30
Jeremy Lakeman
a531c0e960 Use a random walk to find free keyring slots
Always round the file size up to the nearest 16 slots
2016-09-20 16:47:30 +09:30
Jeremy Lakeman
275c99bc4e Add cli option for changing an entry pin 2016-09-20 10:00:02 +09:30
Andrew Bettison
2f61c447a9 Add a new rhizomerestful test case: update a bundle to empty payload 2016-09-19 14:30:04 +09:30
Andrew Bettison
0657e27dea Fix a failing meshmsrestful test
On OS-X, "sed" is BSD sed(1); must use $SED instead to get GNU sed(1).
2016-09-15 17:17:34 +09:30
Andrew Bettison
21679e3865 Fix rhizomeprotocol test on OS-X 2016-09-12 15:09:46 +09:30
Andrew Bettison
43bd0bd33f Fix a failing keyring test
The in-memory identity creation test was failing intermittently because
the server only creates the in-memory identity on the first tick, and
under load the server sometimes does not tick for a while, leading to a
race with the "servald id self" command.

The test case now waits for the server to emit a tick before running the
"id self" command.  This necessitated a new DEBUG() statement and a new
debug flag 'overlaytick'.

Also disabled Rhizome for all the keyring tests, as unnecessary Rhizome
DB creation was slowing down the fixtures.
2016-09-06 15:00:21 +09:30
Andrew Bettison
2cfdafdb56 Add some MDP client DEBUG trace statements 2016-09-06 13:31:38 +09:30
Andrew Bettison
7f0fef2209 Fix an intermittent server test failure
The quit-on-monitor-client-disconnect test was non-deterministic
depending on load (eg, other concurrently running tests).  Under load,
it was likely that the server did not process the "monitor quit" command
before checking for disconnect, so the server did not terminate.

The fix was to make the monitor interface read and process all queued
input from the client before checking for HUP or ERR condition on the
socket.  With this fix, the "sleep 1" kludges before and after the echo
"monitor quit" to the console command are no longer needed.

In the process the monitor interface code was modernised: eg, now it
calls read_nonblock() instead of read(2).
2016-09-06 12:33:29 +09:30
Jeremy Lakeman
67b9aeeb4e Add simple meshmb find command 2016-08-29 11:06:53 +09:30
Jeremy Lakeman
e13b9c3c94 Reduce redundant reading of message ply's 2016-08-23 14:28:13 +09:30
Jeremy Lakeman
b2eb3a2199 Begin broadcast messaging service 2016-08-16 15:04:15 +09:30
Jeremy Lakeman
b6256ea89a Add identity_t type & rename signing key api to identity. 2016-08-16 12:22:13 +09:30
Jeremy Lakeman
c5d6579397 Add cli option to include text of last received message 2016-08-10 11:20:12 +09:30
Jeremy Lakeman
83b6ecb453 Roll an in memory identity when the keyring is empty 2016-08-02 11:01:32 +09:30
Jeremy Lakeman
dbded493c1 Refactor meshms ply handling into separate source file 2016-07-25 16:54:11 +09:30
Jeremy Lakeman
9f4f56b663 Add Java api to add identity with name and number 2016-06-07 12:30:42 +09:30
Jeremy Lakeman
ce2131870c Fix rhizome op test that used recipient field 2016-05-31 13:05:06 +09:30
Jeremy Lakeman
d9e89fd299 Add meshms restful stress test 2016-05-31 12:53:37 +09:30
Jeremy Lakeman
16a14269af Encrypt payloads without needing a sender
- reworked keyring identity handling to reduce memory searching
2016-05-31 12:50:32 +09:30
Jeremy Lakeman
89ab832c21 Link multiple versions of a journal payload to the same file 2016-05-24 14:47:49 +09:30
Jeremy Lakeman
dced3bbb91 Avoid 1 second delay in tests when using curl to post forms 2016-05-18 12:11:58 +09:30
Jeremy Lakeman
b55926b081 Add configuration for idle_tick_ms so tests can complete faster without wasting power 2016-05-10 11:19:49 +09:30
Jeremy Lakeman
6feaddf24d Rework route print command to use newer mdp api, show live updates & deprecate monitor messages 2016-05-10 10:28:14 +09:30
Jeremy Lakeman
6cedb2d0ac Cleanup sync state when complete, or peers move out of range 2016-05-09 11:10:50 +09:30
Jeremy Lakeman
c5957e9c85 Transfer bundles via msp, triggered by key sync 2016-05-09 11:10:50 +09:30
Jeremy Lakeman
6b7aa800e4 Start syncing rhizome based on manifest hashes 2016-05-09 11:10:50 +09:30
Jeremy Lakeman
7939a780fb Fix case where eof mime boundary overlaps buffer boundary 2016-04-11 13:10:37 +09:30
Jeremy Lakeman
2241a15e48 Use file locking to better detect if the server really is running 2016-04-06 14:57:30 +09:30
Jeremy Lakeman
5636edfe20 Append manifest into zip file comment 2016-03-30 16:25:43 +10:30
Jeremy Lakeman
9feffb89c0 Add test for missing file 2016-03-08 14:59:13 +10:30
Jeremy Lakeman
b5afcf9a8f Fix vomp stun test after route print change 2016-03-07 16:11:27 +10:30
Jeremy Lakeman
c1947cf774 Add prior hop SID to route print output 2016-03-02 10:51:16 +10:30
Andrew Bettison
fa4fd02fbd Fix MSP test case after switching to nc6 2016-02-15 18:41:54 +10:30
Andrew Bettison
3e4470bf4a Use nc6 in MSP tests instead of nc
On OS-X (BSD), nc6 appears to be the only available variant of
netcat that matches the Linux version.

Introduces a guard function to catch invocations of nc6 when
setup_netcat6 has not been called in the fixture.  Introduce
guard functions to prevent invocations of nc and netcat.
2016-02-15 18:41:54 +10:30
Jeremy Lakeman
3b88b50d01 Make sure the test is performed on the correct serval instance 2016-02-01 15:05:27 +10:30
Jeremy Lakeman
389f16d2bb Establish a unicast link to a configured host on demand 2016-01-27 16:27:47 +10:30
Jeremy Lakeman
d0da910b19 Avoid undefined behaviour as highlighted by clang 2016-01-27 16:27:47 +10:30
Andrew Bettison
0361b99ca7 Fix OSX bugs in test framework
Use GNU grep, sed and awk instead of BSD variants.  Developers will have
to install these using a package manager like homebrew.  Updated the
INSTALL.md and doc/Development.md tech docs with instructions.
2016-01-27 00:24:43 +10:30
Jeremy Lakeman
75523b7f3c Don't talk about SID's that don't match the whole abbreviation 2016-01-12 10:29:46 +10:30
Jeremy Lakeman
912806360c Tweak netlink setup code in an attempt to prevent gcc mis-optimization 2016-01-11 14:40:23 +10:30
Jeremy Lakeman
77162fbaec Improve config overflow message 2015-12-14 10:31:38 +10:30
Andrew Bettison
3f8f0f6fc7 Improve Rhizome HTTP API diagnostics
Add RHIZOME_BUNDLE_STATUS_MANIFEST_TOO_BIG enum option to indicate
that the manifest exceeded 8 KiB in size.

Refactor rhizome_add_manifest() and rhizome_manifest_finalise()
to return 'struct rhizome_bundle_result' instead of 'enum
rhizome_bundle_status', so that that their detailed failure messages
can reach the HTTP API layer instead of just being logged.

Fix HTTP response status codes produced Rhizome direct HTTP requests
to be consistent with the Rhizome RESTful API.
2015-12-07 22:35:42 +10:30
Andrew Bettison
39460947f1 Add 'rhizomeall' test script
Runs all Rhizome test cases.
2015-12-01 00:51:15 +10:30
Andrew Bettison
fd50a34fc8 Add minor safety check to a Rhizome test case
Ensure that two generated test files are different instead of just
assuming they are; makes the test logic more explicit.
2015-12-01 00:51:15 +10:30
Andrew Bettison
98ed0406c5 Refactor Rhizome result handling
- Introduce the new 'struct rhizome_bundle_result' that contains a
  rhizome_bundle_status enum value and an optional, nul-terminated
  string that provides an explanation of the cause of the status; add
  functions to construct, query, and free the struct

- Replace 'enum rhizome_add_result' with 'struct rhizome_bundle_result',
  removing an unnecessary level of enum interpretation

- Make rhizome_fill_manifest() return 'struct rhizome_bundle_result' and
  add logic to check that the supplied author SID is correct (previous
  behaviour was: if an incorrect author SID was supplied but the correct
  author could be found in the keyring, then the incorrect one was
  silently ignored)

- Simplify the response code in rhizome_restful.cc to take advantage of
  the new 'struct rhizome_bundle_result'; in particular, the mapping
  from 'enum rhizome_bundle_status' codes to HTTP status codes is now
  expressed in a single switch statement

- Fix some minor failures in test scripts revealed by the changes
2015-12-01 00:51:15 +10:30
Jeremy Lakeman
2ddbb86cb5 Don't ask for explanation of SIDs with dead routing links 2015-11-16 14:09:35 +10:30
Andrew Bettison
419364b5a9 Improve REST HTTP response status codes
List all the HTTP status codes in the REST API tech doc.

Only use 403 Forbidden for requests originating from a disallowed
origin (ie, not localhost).

- Return 400 for missing, unknown, duplicate and out-of-order form
  parts in POST requests.
- Return 415 Unsupported Media Type for unsupported form part
  Content-Disposition and Content-Type (including unsupported
  charset).
- Return 414 Request-URI Too Long for any buffer exhaustion while
  parsing request.
- Return 419 Authentication Timeout for missing crypto secret.
2015-11-02 12:26:40 +10:30
Andrew Bettison
4564e955e3 Improve HTTP Origin response headers
Re-introduce "Origin: null" handling, because actually it is in the
spec.
2015-11-02 11:58:24 +10:30
Andrew Bettison
404cc1476d Improve HTTP Origin header parsing
Parses the separate parts of the URI: scheme, hostname, port.
No longer supports "Origin: null", because that is not in the
spec.
2015-10-01 07:05:17 +09:30
Andrew Bettison
6123503c15 Improve Rhizome HTTP status codes
Also improve some reason phrases and test them
2015-09-21 18:43:24 +09:30
Andrew Bettison
7635e6b71b Add Keyring Java API (incomplete) with tests
keyringListIdentities()
keyringSetDidName()
2015-08-31 19:16:30 +09:30
Andrew Bettison
714ea63190 Change JSON response of GET HTTP /restful/keyring/...
Return "sid", "did" and "name" fields inside their own "identity"
object
2015-08-31 19:16:30 +09:30
Andrew Bettison
e956749727 Move utility functions from 'rhizomejava' test to testdefs_java.sh 2015-08-31 19:16:30 +09:30
Andrew Bettison
8ba612ffa1 HTTP GET /restful/keyring/set[?pin=PIN][&did=DID][&name=Name] 2015-08-24 22:54:45 +09:30
Andrew Bettison
8834a81575 Refactor HTTP GET /restful/keyring/add 2015-08-24 22:44:24 +09:30
Andrew Bettison
380a72113c pin= query param for HTTP GET /restful/keyring/identities.json 2015-08-24 22:41:53 +09:30
Andrew Bettison
b7ba297e31 Add TODO comments to randomise keyring slot allocation 2015-08-24 22:41:53 +09:30
Andrew Bettison
73ced0f93a pin= query param for HTTP GET /restful/keyring/add 2015-08-24 22:41:53 +09:30
Andrew Bettison
586c6b3060 Add HTTP GET /restful/keyring/add 2015-08-10 18:13:57 +09:30
Andrew Bettison
8ac156376d Fix MeshMS Java API when MeshMS disabled
If the MeshMS REST API is disabled, then the JSON response omits the
MeshMS status code, so the Java client API now handles this case.
2015-06-29 15:39:19 +09:30
Jeremy Lakeman
e97f7c4e29 Fix vomp stun tests
- Don't trust stun responses about 3rd parties
- Only ack neighbour links via 1hop destinations (not multi-hop routes)
- Don't override network destinations that were manually supplied
2015-06-22 14:50:28 +09:30
Jeremy Lakeman
568f431d91 Share private addresses during stun traversal 2015-06-22 12:15:49 +09:30
Jeremy Lakeman
73c2c9e6fe Relax unreliable path tests for now 2015-06-22 12:15:49 +09:30
Jeremy Lakeman
e9cdb67b10 MeshMS API should not fail in JSON parsing when rhizome is disabled 2015-06-15 16:13:00 +09:30
Andrew Bettison
6febcc350d Replace RESTful newsince polling with triggers
Remove the 'api.restful.newsince_poll_ms' config option, no longer
needed.
2015-06-02 02:23:06 +09:30
Andrew Bettison
ed1acca0ed Fix 'rhizomerestful' and 'rhizomejava' newsince-arrival tests
Use POST /restfule/rhizome/insert REST request to add bundles instead of
CLI 'rhizome file add', to avoid database locking storm and prepare for
replacing polling with trigger-based wakeup in newsince requests.
2015-06-02 02:23:06 +09:30
Andrew Bettison
858a7fc577 Refactor: define extract_http_header() in testdefs.sh
Also http_unquote_string()
2015-06-02 02:23:06 +09:30
Andrew Bettison
45c6b9ecfa Refactor: set $CR and $HT in testdefs.sh
Fix some bugs in test scripts that used $CR without setting it
2015-06-02 02:23:06 +09:30
Andrew Bettison
d5b96b9931 Fix 'meshms' and 'meshmsjava' newsince-arrival tests
Failed because of a database locking storm between the three waiting
newsince REST requests (which polled the database) and the concurrent
'meshms send' CLI commands.  Changed to use REST requests instead of CLI
for 'meshms send'.
2015-05-25 16:46:43 +09:30
Jeremy Lakeman
51ed6162dd Add support for running and stopping servald from a JVM thread
fdpoll will now run a callback when about to sleep / woke up.
A new Java interface to indicate server started / sleeping / waking up.
An android implementation may allow the CPU to sleep.
If there's a java exception the server will try to shutdown.
Calling servald stop is currently undefined.
2015-05-25 11:46:37 +09:30
Andrew Bettison
106f5d04d9 Fix 'msp' test failures: different netcat versions
The "traditional" variant of nc(1) ("netcat") (installed by default on
Debian and Ubuntu) behaves differently from the OpenBSD variant, causing
two 'msp' tests to fail.  Added -q options to nc(1) invocations to make
tests pass with either variant.
2015-05-18 17:20:01 +09:30
Jeremy Lakeman
d5e0835f21 Add explicit timeout parameter to mdp trace for slow links 2015-05-11 15:47:35 +09:30
Andrew Bettison
30f4a398ea Merge RESTful Rhizome journal append into development
Support for appending to Rhizome journals using the RESTful API
2015-04-13 16:59:06 +09:30
Andrew Bettison
979854ae0b Test that Rhizome RESTful append rejects non-journals 2015-04-13 16:24:05 +09:30
Andrew Bettison
0f8141af95 Test RESTful Rhizome journal append with shared payload
When an append produces a payload that is already in the store,
the append must succeed, and the returned status codes should
indicate that the payload already existed
2015-04-13 16:24:05 +09:30
Andrew Bettison
53e4e5be5f Improve 'rhizomerestful' journal append test case
Add asserts for the returned status codes
2015-04-13 14:43:52 +09:30
Andrew Bettison
7734e24006 /restful/rhizome/append on existing journal
Refactor Rhizome insert/append logic into functions used by both CLI and
RESTful API.  Improve RESTful diagnostic messages.
2015-03-30 18:24:12 +10:30
Andrew Bettison
214dad421b Rename 'rhizomerestful' journal test cases 2015-03-30 18:21:31 +10:30
Jeremy Lakeman
fea91f3f82 Use configured reachable timeout for network links, read dummy files even when not ticking 2015-03-30 17:38:40 +10:30
Jeremy Lakeman
943bca3bea Add transmit timeout config per network destination for low bandwidth links 2015-03-30 14:15:08 +10:30
Andrew Bettison
da22816784 Add 'rhizome add file --bundle=BID' option
A convenience to re-use an existing manifest, instead of having
to extract the manifest, erase some fields, then pass the result
back into the 'rhizome add file' command
2015-03-28 05:09:38 +10:30
Andrew Bettison
b17848a438 Improve 'rhizomeops' test fixtures
Use extract_stdout_manifestid() instead of extract_stdout_keyvalue()
2015-03-28 05:09:37 +10:30
Andrew Bettison
dbea301ce2 Rhizome test defs support 'tail' field 2015-03-28 05:09:37 +10:30
Andrew Bettison
987ec00183 Improve 'rhizomeops' test speed
Only create the identities needed, not five in every test
2015-03-25 00:55:37 +10:30
Andrew Bettison
16355df04b Fix failing 'rhizomejava' test
Handle missing payload-status headers in Java test client
2015-03-23 23:26:09 +10:30
Andrew Bettison
705dbeff09 Fix bug in rhizome_write_open_journal()
Was treating PAYLOAD_STATUS_NEW as an error
2015-03-23 17:40:08 +10:30
Andrew Bettison
015b4a0b07 Refactor and improve "journal append"
Distinguish between fatal and user-supplied-input errors, return
error descriptive text in a strbuf to allow dynamic content, apply
user-supplied field assgnments/deletions _after_ copying existing
manifest fields, use exit status 4 (invalid manifest) when applying
journal append to a non-journal or vice versa
2015-03-23 17:38:46 +10:30
Jeremy Lakeman
f7dbe06836 Rearrange interface config to support unicast settings 2015-03-23 13:35:22 +10:30
Andrew Bettison
655b94eb3c Fix failing 'rhizomeops' tests 2015-03-19 03:21:27 +10:30
Andrew Bettison
c9131f43a2 POST /restful/rhizome/append for journals 2015-03-02 18:52:12 +10:30
Andrew Bettison
4b234fbfbd Fix /restful/rhizome/insert HTTP status code
Return 403 "Forbidden" instead of 501 "Not implemented" when
given a journal bundle
2015-03-02 18:49:51 +10:30
Andrew Bettison
d44d18280a Rhizome test: cannot use "add file" to create a journal 2015-03-02 18:49:51 +10:30
Andrew Bettison
b2919a3a5e Rhizome test: cannot use "add file" to update a journal
Journal bundles can only be modified using "journal append"
2015-03-02 18:49:51 +10:30
Andrew Bettison
e0684567e8 Add Rhizome test case: import journal bundle
Also fix a bug thus revealed
2015-03-02 18:49:51 +10:30
Andrew Bettison
e2ef96a57d Improve some Rhizome test case comments 2015-03-02 18:49:50 +10:30
Jeremy Lakeman
3ae0e95e0b Treat null strings as empty 2015-03-02 15:44:34 +10:30
Andrew Bettison
e6bdeeb337 Increase timeouts on MeshMS newsince tests
So they don't fail during large parallel test runs
2015-02-23 15:57:24 +10:30