Commit Graph

3842 Commits

Author SHA1 Message Date
Andrew Bettison
a82babf9db Refactor test setup function 'create_single_identity'
Now produces exactly the same output variables as 'create_identities 1'
so that these two functions can be interchanged with minimal impact
on the test script.
2018-03-27 17:29:21 +10:30
Andrew Bettison
f8850535ad Improve the test framework
The executeOk() function should dump stderr to the log in event of
failure, but execute() need not.
2018-03-27 17:29:21 +10:30
Andrew Bettison
bc7f7aeaaf Refactor all REST API tests
Rename 'testdefs_json.sh' to 'testdefs_rest.sh'; add setup functions and
rest_request() function, and roll out through existing test scripts.
2018-03-27 17:29:21 +10:30
Andrew Bettison
9ef7d33d69 Log more debug in MDP_ROUTE_TABLE request 2018-03-27 09:06:41 +10:30
Jeremy Lakeman
3cbf60f2b7 Improve error messages when locating zip file comment 2018-03-26 15:03:28 +10:30
Jeremy Lakeman
240f69595b Remove dump_config from android specific code 2018-03-26 15:03:28 +10:30
Jeremy Lakeman
ad2f0364cf Add content type class for sending headers & matching responses
Note that android doesn't include javax so we can't use it.
2018-03-26 14:45:36 +10:30
Jeremy Lakeman
262df2804d By default clang only logs warnings for unsupported command line args 2018-03-26 14:45:36 +10:30
Andrew Bettison
3a56355111 Add 'Location' header to HTTP server 301 and 302 responses 2018-03-19 18:07:33 +10:30
Andrew Bettison
007a7af122 Update Java API to use new Keyring REST API
The ServalDCient class now requires a "verb" parameter as well as the
URL parameter.

Add Java API method to query a single keyring identity.
2018-03-19 18:07:33 +10:30
Andrew Bettison
98ec1c9608 Redesign the Keyring REST API (fixes #132)
The new API follows REST rules by using the proper request verbs:
POST, PUT, PATCH and DELETE, instead of just GET.

The legacy GET-only API is still supported for backward compatibility,
but not longer tested or documented.

Add a new query-single-identity operation.

Implement the lock-single-identity operation, which until now had been
documented but not yet implemented.  Whenever a single identity is
locked (released), any other unlocked identities with the same PIN are
flagged to indicate that the PIN is not "fully" unlocked, so that the
next time the PIN is entered, the slot decryption is re-tried for
non-loaded identities, and the locked identity will be unlocked again.

Update the 'keyring' and 'keyringrestful' test scripts:
- refactored to reduce curl command-line clutter in test cases
- now tests the redesigned request verbs and paths
- added a test for GET /restful/keyring/SID
- added a test for PUT /restful/keyring/SID/lock
2018-03-19 18:06:23 +10:30
Andrew Bettison
8242ca0a00 Fix a bug in MDP_IDENTITY request handling
The LOCK and UNLOCK by PIN requests were acting on the supplied PIN
and also the empty PIN, due to a bug in ob_get_str_ptr(), which returned
an empty string instead of NULL after reaching the end of the string
list.
2018-03-09 16:56:36 +10:30
Andrew Bettison
2333a116f3 Fix a bug in debug hexdumps
Introduced in the recent log system overhaul.
2018-03-09 16:56:36 +10:30
Andrew Bettison
fad187046e Add missing tests/alljava script 2018-03-06 18:31:26 +10:30
Andrew Bettison
c35af0617a Fix header files for Xcode iOS build
- make "debug.h" a private header so that the DEBUG() macro does not
  interfere with Xcode Debug builds, which set DEBUG=1
- move all #include "debug.h" from headers into .c files
- move 'struct idebug' into a new public header "idebug.h" so that
  "log.h" can be public
- move HTTPD port number defs from "httpd.h" to "constants.h", so that
  "httpd.h" can be private and "conf.h" can be public
- add missing nested includes so each header compiles stand-alone
  without error
- #include "sodium.h" instead of <sodium.h>
- #include "sqlite3.h" instead of <sqlite3.h>
- add header guard to "fifo.h"
- fix header guard in "sync_keys.h"
2018-03-06 15:29:29 +10:30
Andrew Bettison
ca8e846264 Add iOS Framework Bundle build
The new ios/configure script performs a separate ../configure for each of
the following iOS targets:

    iPhoneOS        on armv7
    iPhoneOS        on arm64
    iPhoneSimulator on i386
    iPhoneSimulator on x86_64

The script then creates ios/Makefile that builds a static iOS Framework
Bundle suitable for inclusion in an Xcode iOS project.

Add the --xcode-sdk=SDK option to configure.ac, to support cross
compiling using Apple Xcode.  It prefixes all compile/link toolchain
commands with the "xcrun --sdk SDK" command, ie, CC, AS, LD, AR, RANLIB,
etc.

Re-structure headerfiles.mk to separate "public" from "private" headers,
because the Framework module only exposes the public ones.  Moves the
SQLITE3_AMALGAMATION definition from Makefile.in into headerfiles.mk.

Update INSTALL.md and add a technical document for Apple iOS.

This enables development of Serval DNA within the context of an Xcode
iOS project using the standard edit-make-test cycle: after modifying a
Serval DNA source file, "cd ios; make" will recompile the changed file
for all the target architectures and update the Framework Bundle.
Rebuilding the Xcode project will then incorporate the changes, which
can be tested immediately.
2018-03-06 15:29:29 +10:30
Andrew Bettison
c418385f56 Update COPYRIGHT.txt 2018-03-06 15:29:29 +10:30
Andrew Bettison
a79156c4d0 Add Swift daemon API, use in 'servaldswift'
The daemon API is built as a Swift module called ServalDNA.

The new CliContext class allows easy implementation of CLI output using
Swift code.  The new CliContextFile subclass is the obvious first
implementation, equivalent to cli_stdio.c.  The 'servaldswift'
executable now uses CliContextFile to print its CLI output to standard
output.

The new delegated log output support constructs each log line in a
buffer and prints it by calling the delegate's 'print' function at
end-of-line.  The 'servaldswift' executable now provides a log output
delegate in Swift that simply prints to standard error, replacing
log_output_console.o, which is omitted from its link.
2018-03-06 15:29:17 +10:30
Andrew Bettison
3ffa4b10af Add Swift keyring client API with tests
Add a swift-client-api subdirectory containing a Swift source package
and a Makefile.in that compiles it into the "ServalClient" Swift module
using the Swift package manager.  The Swift API contains the following
classes:

- ServalKeyring provides the operations: add, remove, set, list
- AbstractId and its specialisation SubscriberId, already in near-final
  form, are data types for SID and the like
- ServalRestfulClient (internal) uses an HTTP client to access the
  Serval DNA RESTful interface

Improve the REST /keyring/set operation to only alter the DID or Name if
the corresponding query parameter is supplied.  Modify the internal
keyring_set_did() function to only assign the DID or Name if the
corresponding parameter is not a null pointer.

The configure script ensures that the Swift build target version is
10.10 or later when compiling for Mac OS-X, so that the package manager
will succeed.

Add autoconf macros for the Swift package manager.
2018-03-06 15:29:17 +10:30
Andrew Bettison
f2eb2bf9ab Add Swift 3/4 'servaldswift' executable
If the Swift 3 or Swift 4 compiler is present or passed to the configure
script in the SWIFTC variable, then the Makefile will compile a Swift
'servaldswift' executable to ensure that the module map links correctly
into a stand-alone Swift program.

Use the gold linker if supported, to avoid relocation errors on symbols
produced by Swift when linking dynamic libraries.
2018-03-06 15:29:17 +10:30
Andrew Bettison
e9d91d04ca Fix failing Rhizome Java API tests
Update the Java API to be consistent with recent changes to the Rhizome
REST API was that regularised the MIME types used in Content-Type
headers.

Add some missing CONTENT_TYPE_ constants to "httpd.h", and ensure they
are consistent with the Java getMimeType() methods in subclasses of
AbstractId.

Add the --stderr option to all executeJavaOk statements in the
'rhizomejava' test script, to reveal the exception message in the case
of failure.
2018-03-06 15:16:56 +10:30
Andrew Bettison
16d4a6d18e Fix the $JAVAC conditional in tests/all
Add a test script for all Java tests, called "alljava".

Include testconfig.sh in testdefs.sh, not testdefs_java.sh, so that the
tests/all script can successfully test the $JAVAC variable.
2018-03-06 15:16:56 +10:30
Andrew Bettison
92fa6c196a Rewrite logging system
Rename the logging primitive functions and utility functions, prefixing
all with 'serval_log', eg: logMessage() -> serval_logf() etc.

Add an XPRINTF xhexdump() function and use it to implement the
serval_log_hexdump() utility, renamed from dump().  Add macros
WHY_dump(), WARN_dump(), HINT_dump() and DEBUG_dump(), and use them
everywhere.

Remove the 'log.console.dump_config' and 'log.file.dump_config'
configuration options; configuration is now dumped in every log prolog.

The logging system now constructs the log prolog by invoking the new
'log_prolog' trigger, so that it no longer depends on the version string
and configuration system.  Any system that wants to present a message in
the log prolog can define its own trigger, which calls standard log
primitives to print the message.

Split the logging system into a front-end (log.c) that provides the
logging primitives and is independent of the configuration system, and a
set of back-end "outputters" (log_output_console.c, log_output_file.c,
log_output_android.c) that may depend on the configuration system and
are decoupled from the front-end using the 'logoutput' link section.

These log outputters are explicitly linked into executables by the
Makefile rules, but could also be linked in using USE_FEATURE().  The
USE_FEATURE() calls have _not_ been added to servald_features.c, so that
different daemon executables can be built with the same feature set but
different log outputs.
2018-03-06 15:16:56 +10:30
Andrew Bettison
71e654a615 Fix compile error in strbuf_local_buf() with CFLAGS=-g
The strbuf_local_buf() function attempted to detect if its argument was
not an array but a pointer, and fail at compile time.  This worked fine
when compiling with standard optimisation (-O2).

However, specifying the gcc -g option disables compiler optimisation, so
the compiler does not optimise away the call to the function with
__attribute__(error("...")), and so the compile fails.

The solution is to perform the check at run time using assert(), like
strbuf_local() already does.
2018-03-06 15:16:55 +10:30
Andrew Bettison
5fa8c1ffcf Add Vim modeline to configure.ac 2018-03-06 15:16:55 +10:30
Andrew Bettison
9b8a7c7384 Remove unnecessary autoconf macro invocation 2018-03-06 15:16:55 +10:30
Andrew Bettison
19607e7886 Improve MDP request debug logging 2018-03-06 15:16:55 +10:30
Andrew Bettison
c3cf86161f Improve validation of keyring DID and Name (fixes #131)
Correct the maximum DID length defined in "serval_types.h" from 32 to
31.  Add a definition of the maximum identity Name length and use it
instead of the bare constant 64, eg, in the MDP_DNALOOKUP request
handling code.

Introduce a dataformats.h function for validating an identity name, and
use it to validate the 'name' parameter in the CLI 'keyring set'
command.

Add 'did' and 'name' parameter validation to the GET /restful/keyring/add
and GET /restful/keyring/SID/set requests (#131).

Rename keyring_set_did() to keyring_set_did_name() and assert that DID
and Name lengths have been validated before storing in the keyring.

Update the Keyring REST API tech document.
2018-03-06 15:14:18 +10:30
Jeremy Lakeman
afa33484ba Always silently ignore incoming data once response has started 2018-01-08 12:24:42 +10:30
Jeremy Lakeman
60b9472472 Always include testconfig so java tests are run with all tests 2018-01-08 12:24:42 +10:30
Jeremy Lakeman
fbbbfdacc7 Move test for uninitialised mdp port to throw in more cases 2018-01-03 18:01:13 +10:30
Jeremy Lakeman
dc0c0f86d8 Fix java resful client after recent changes to mime types 2018-01-03 18:01:13 +10:30
Andrew Bettison
988250a576 Improve test framework
In the assert() function, if --message option is given, append the details
of the failed command to the message.
2017-12-06 16:36:42 +10:30
Andrew Bettison
bc6396ec28 Fix malformed test assert message in executeOk_servald()
The assertion message was omitting the details of the command executed.
2017-12-06 16:36:42 +10:30
Andrew Bettison
7edf1a912e Improve MDP technical documentation
The MDP technical document can now replace the Serval Developer wiki MDP
page.
2017-11-22 17:38:17 +10:30
Andrew Bettison
0650666acb Allow PREVIOUS overlay address to refer to SIGNKEY 2017-11-22 17:38:17 +10:30
Andrew Bettison
cc016f7bf7 Fix two broken rhizomerestful tests
On Debian testing, it appears that curl(1) no longer sends an Expect:
100-continue header by default, so the test script must explicitly
supply it.
2017-11-21 15:11:48 +10:30
Jeremy Lakeman
bb7d95ccd1 Log error reason if we fail to detect interface type from reading /sys/net 2017-11-20 15:05:36 +10:30
Jeremy Lakeman
577fbb34db Don't attempt to calculate the next alarm if the interface just went down 2017-11-20 10:39:32 +10:30
Jeremy Lakeman
3815900f29 Send route change when we detect a peer has a combined key 2017-11-16 11:08:43 +01:00
Andrew Bettison
c185b6c613 Fix syntax error in Makefile.in 2017-11-10 11:31:25 +10:30
Jeremy Lakeman
ce2c276570 Detect Wifi and cabled ethernet on linux systems, allow matching by type in config 2017-11-07 16:53:19 +10:30
Andrew Bettison
68deae8393 Improve out-of-tree build support
Use the conventional autoconf @srcdir@ substitution instead of requiring
the SOURCE_PREFIX variable to be passed to the Makefile.

Also support running test scripts in an out-of-tree build directory.
2017-10-23 17:51:42 +10:30
Andrew Bettison
cab9a430fb Improve comments and variable names in http_server.cc 2017-10-20 17:29:46 +10:30
Andrew Bettison
261fc43849 Log a clearer diagnostic on HTTP GET with chunked transfer 2017-10-20 17:29:46 +10:30
Andrew Bettison
58573a4345 Spruce up MeshMS REST API tech doc a bit 2017-10-20 17:22:38 +10:30
Andrew Bettison
f53be7c6a6 Improve tech doc of MDP and servald config 2017-10-20 17:22:34 +10:30
Andrew Bettison
34e2e8d4bc Improve and document HTTP REST Rhizome import
The REST Rhizome import request now requires the 'id' and 'version'
query parameters to either both be supplied or neither, and fails if
they do not match the manifest that is supplied in the request body.
Added a test case for this.

Added a test case to ensure that if the 'id' and 'version' query
parameters cause a hit (already in store) then the response is sent
immediately without reading the request body.

Improve the documentation for the REST Rhizome import request.
2017-10-20 15:50:18 +10:30
Andrew Bettison
5eb19f1a16 Improve REST API technical documentation (fixes #118)
Users and contributors have had difficulty working out how to use the
more complex requests in the REST interface, particularly POST
/restful/rhizome/insert.

Improve the Markdown documentation to provide concrete examples of POST
requests and provide much more extensive description of how to use the
"multipart/form-data" Content-Type.
2017-10-20 10:03:18 +10:30
Andrew Bettison
9e32c01e61 Set Content-Type in static file HTTP responses
The static file server now sets the Content-Type of the response
depending on the file name extension:

    .txt        =  text/plain; charset=utf-8
    .htm .html  =  text/html; charset=utf-8
    .json       =  application/json
    .rfm        =  rhizome/manifest; format=text+binarysig
2017-10-20 10:01:15 +10:30