Commit Graph

3831 Commits

Author SHA1 Message Date
Andrew Bettison
8242ca0a00 Fix a bug in MDP_IDENTITY request handling
The LOCK and UNLOCK by PIN requests were acting on the supplied PIN
and also the empty PIN, due to a bug in ob_get_str_ptr(), which returned
an empty string instead of NULL after reaching the end of the string
list.
2018-03-09 16:56:36 +10:30
Andrew Bettison
2333a116f3 Fix a bug in debug hexdumps
Introduced in the recent log system overhaul.
2018-03-09 16:56:36 +10:30
Andrew Bettison
fad187046e Add missing tests/alljava script 2018-03-06 18:31:26 +10:30
Andrew Bettison
c35af0617a Fix header files for Xcode iOS build
- make "debug.h" a private header so that the DEBUG() macro does not
  interfere with Xcode Debug builds, which set DEBUG=1
- move all #include "debug.h" from headers into .c files
- move 'struct idebug' into a new public header "idebug.h" so that
  "log.h" can be public
- move HTTPD port number defs from "httpd.h" to "constants.h", so that
  "httpd.h" can be private and "conf.h" can be public
- add missing nested includes so each header compiles stand-alone
  without error
- #include "sodium.h" instead of <sodium.h>
- #include "sqlite3.h" instead of <sqlite3.h>
- add header guard to "fifo.h"
- fix header guard in "sync_keys.h"
2018-03-06 15:29:29 +10:30
Andrew Bettison
ca8e846264 Add iOS Framework Bundle build
The new ios/configure script performs a separate ../configure for each of
the following iOS targets:

    iPhoneOS        on armv7
    iPhoneOS        on arm64
    iPhoneSimulator on i386
    iPhoneSimulator on x86_64

The script then creates ios/Makefile that builds a static iOS Framework
Bundle suitable for inclusion in an Xcode iOS project.

Add the --xcode-sdk=SDK option to configure.ac, to support cross
compiling using Apple Xcode.  It prefixes all compile/link toolchain
commands with the "xcrun --sdk SDK" command, ie, CC, AS, LD, AR, RANLIB,
etc.

Re-structure headerfiles.mk to separate "public" from "private" headers,
because the Framework module only exposes the public ones.  Moves the
SQLITE3_AMALGAMATION definition from Makefile.in into headerfiles.mk.

Update INSTALL.md and add a technical document for Apple iOS.

This enables development of Serval DNA within the context of an Xcode
iOS project using the standard edit-make-test cycle: after modifying a
Serval DNA source file, "cd ios; make" will recompile the changed file
for all the target architectures and update the Framework Bundle.
Rebuilding the Xcode project will then incorporate the changes, which
can be tested immediately.
2018-03-06 15:29:29 +10:30
Andrew Bettison
c418385f56 Update COPYRIGHT.txt 2018-03-06 15:29:29 +10:30
Andrew Bettison
a79156c4d0 Add Swift daemon API, use in 'servaldswift'
The daemon API is built as a Swift module called ServalDNA.

The new CliContext class allows easy implementation of CLI output using
Swift code.  The new CliContextFile subclass is the obvious first
implementation, equivalent to cli_stdio.c.  The 'servaldswift'
executable now uses CliContextFile to print its CLI output to standard
output.

The new delegated log output support constructs each log line in a
buffer and prints it by calling the delegate's 'print' function at
end-of-line.  The 'servaldswift' executable now provides a log output
delegate in Swift that simply prints to standard error, replacing
log_output_console.o, which is omitted from its link.
2018-03-06 15:29:17 +10:30
Andrew Bettison
3ffa4b10af Add Swift keyring client API with tests
Add a swift-client-api subdirectory containing a Swift source package
and a Makefile.in that compiles it into the "ServalClient" Swift module
using the Swift package manager.  The Swift API contains the following
classes:

- ServalKeyring provides the operations: add, remove, set, list
- AbstractId and its specialisation SubscriberId, already in near-final
  form, are data types for SID and the like
- ServalRestfulClient (internal) uses an HTTP client to access the
  Serval DNA RESTful interface

Improve the REST /keyring/set operation to only alter the DID or Name if
the corresponding query parameter is supplied.  Modify the internal
keyring_set_did() function to only assign the DID or Name if the
corresponding parameter is not a null pointer.

The configure script ensures that the Swift build target version is
10.10 or later when compiling for Mac OS-X, so that the package manager
will succeed.

Add autoconf macros for the Swift package manager.
2018-03-06 15:29:17 +10:30
Andrew Bettison
f2eb2bf9ab Add Swift 3/4 'servaldswift' executable
If the Swift 3 or Swift 4 compiler is present or passed to the configure
script in the SWIFTC variable, then the Makefile will compile a Swift
'servaldswift' executable to ensure that the module map links correctly
into a stand-alone Swift program.

Use the gold linker if supported, to avoid relocation errors on symbols
produced by Swift when linking dynamic libraries.
2018-03-06 15:29:17 +10:30
Andrew Bettison
e9d91d04ca Fix failing Rhizome Java API tests
Update the Java API to be consistent with recent changes to the Rhizome
REST API was that regularised the MIME types used in Content-Type
headers.

Add some missing CONTENT_TYPE_ constants to "httpd.h", and ensure they
are consistent with the Java getMimeType() methods in subclasses of
AbstractId.

Add the --stderr option to all executeJavaOk statements in the
'rhizomejava' test script, to reveal the exception message in the case
of failure.
2018-03-06 15:16:56 +10:30
Andrew Bettison
16d4a6d18e Fix the $JAVAC conditional in tests/all
Add a test script for all Java tests, called "alljava".

Include testconfig.sh in testdefs.sh, not testdefs_java.sh, so that the
tests/all script can successfully test the $JAVAC variable.
2018-03-06 15:16:56 +10:30
Andrew Bettison
92fa6c196a Rewrite logging system
Rename the logging primitive functions and utility functions, prefixing
all with 'serval_log', eg: logMessage() -> serval_logf() etc.

Add an XPRINTF xhexdump() function and use it to implement the
serval_log_hexdump() utility, renamed from dump().  Add macros
WHY_dump(), WARN_dump(), HINT_dump() and DEBUG_dump(), and use them
everywhere.

Remove the 'log.console.dump_config' and 'log.file.dump_config'
configuration options; configuration is now dumped in every log prolog.

The logging system now constructs the log prolog by invoking the new
'log_prolog' trigger, so that it no longer depends on the version string
and configuration system.  Any system that wants to present a message in
the log prolog can define its own trigger, which calls standard log
primitives to print the message.

Split the logging system into a front-end (log.c) that provides the
logging primitives and is independent of the configuration system, and a
set of back-end "outputters" (log_output_console.c, log_output_file.c,
log_output_android.c) that may depend on the configuration system and
are decoupled from the front-end using the 'logoutput' link section.

These log outputters are explicitly linked into executables by the
Makefile rules, but could also be linked in using USE_FEATURE().  The
USE_FEATURE() calls have _not_ been added to servald_features.c, so that
different daemon executables can be built with the same feature set but
different log outputs.
2018-03-06 15:16:56 +10:30
Andrew Bettison
71e654a615 Fix compile error in strbuf_local_buf() with CFLAGS=-g
The strbuf_local_buf() function attempted to detect if its argument was
not an array but a pointer, and fail at compile time.  This worked fine
when compiling with standard optimisation (-O2).

However, specifying the gcc -g option disables compiler optimisation, so
the compiler does not optimise away the call to the function with
__attribute__(error("...")), and so the compile fails.

The solution is to perform the check at run time using assert(), like
strbuf_local() already does.
2018-03-06 15:16:55 +10:30
Andrew Bettison
5fa8c1ffcf Add Vim modeline to configure.ac 2018-03-06 15:16:55 +10:30
Andrew Bettison
9b8a7c7384 Remove unnecessary autoconf macro invocation 2018-03-06 15:16:55 +10:30
Andrew Bettison
19607e7886 Improve MDP request debug logging 2018-03-06 15:16:55 +10:30
Andrew Bettison
c3cf86161f Improve validation of keyring DID and Name (fixes #131)
Correct the maximum DID length defined in "serval_types.h" from 32 to
31.  Add a definition of the maximum identity Name length and use it
instead of the bare constant 64, eg, in the MDP_DNALOOKUP request
handling code.

Introduce a dataformats.h function for validating an identity name, and
use it to validate the 'name' parameter in the CLI 'keyring set'
command.

Add 'did' and 'name' parameter validation to the GET /restful/keyring/add
and GET /restful/keyring/SID/set requests (#131).

Rename keyring_set_did() to keyring_set_did_name() and assert that DID
and Name lengths have been validated before storing in the keyring.

Update the Keyring REST API tech document.
2018-03-06 15:14:18 +10:30
Jeremy Lakeman
afa33484ba Always silently ignore incoming data once response has started 2018-01-08 12:24:42 +10:30
Jeremy Lakeman
60b9472472 Always include testconfig so java tests are run with all tests 2018-01-08 12:24:42 +10:30
Jeremy Lakeman
fbbbfdacc7 Move test for uninitialised mdp port to throw in more cases 2018-01-03 18:01:13 +10:30
Jeremy Lakeman
dc0c0f86d8 Fix java resful client after recent changes to mime types 2018-01-03 18:01:13 +10:30
Andrew Bettison
988250a576 Improve test framework
In the assert() function, if --message option is given, append the details
of the failed command to the message.
2017-12-06 16:36:42 +10:30
Andrew Bettison
bc6396ec28 Fix malformed test assert message in executeOk_servald()
The assertion message was omitting the details of the command executed.
2017-12-06 16:36:42 +10:30
Andrew Bettison
7edf1a912e Improve MDP technical documentation
The MDP technical document can now replace the Serval Developer wiki MDP
page.
2017-11-22 17:38:17 +10:30
Andrew Bettison
0650666acb Allow PREVIOUS overlay address to refer to SIGNKEY 2017-11-22 17:38:17 +10:30
Andrew Bettison
cc016f7bf7 Fix two broken rhizomerestful tests
On Debian testing, it appears that curl(1) no longer sends an Expect:
100-continue header by default, so the test script must explicitly
supply it.
2017-11-21 15:11:48 +10:30
Jeremy Lakeman
bb7d95ccd1 Log error reason if we fail to detect interface type from reading /sys/net 2017-11-20 15:05:36 +10:30
Jeremy Lakeman
577fbb34db Don't attempt to calculate the next alarm if the interface just went down 2017-11-20 10:39:32 +10:30
Jeremy Lakeman
3815900f29 Send route change when we detect a peer has a combined key 2017-11-16 11:08:43 +01:00
Andrew Bettison
c185b6c613 Fix syntax error in Makefile.in 2017-11-10 11:31:25 +10:30
Jeremy Lakeman
ce2c276570 Detect Wifi and cabled ethernet on linux systems, allow matching by type in config 2017-11-07 16:53:19 +10:30
Andrew Bettison
68deae8393 Improve out-of-tree build support
Use the conventional autoconf @srcdir@ substitution instead of requiring
the SOURCE_PREFIX variable to be passed to the Makefile.

Also support running test scripts in an out-of-tree build directory.
2017-10-23 17:51:42 +10:30
Andrew Bettison
cab9a430fb Improve comments and variable names in http_server.cc 2017-10-20 17:29:46 +10:30
Andrew Bettison
261fc43849 Log a clearer diagnostic on HTTP GET with chunked transfer 2017-10-20 17:29:46 +10:30
Andrew Bettison
58573a4345 Spruce up MeshMS REST API tech doc a bit 2017-10-20 17:22:38 +10:30
Andrew Bettison
f53be7c6a6 Improve tech doc of MDP and servald config 2017-10-20 17:22:34 +10:30
Andrew Bettison
34e2e8d4bc Improve and document HTTP REST Rhizome import
The REST Rhizome import request now requires the 'id' and 'version'
query parameters to either both be supplied or neither, and fails if
they do not match the manifest that is supplied in the request body.
Added a test case for this.

Added a test case to ensure that if the 'id' and 'version' query
parameters cause a hit (already in store) then the response is sent
immediately without reading the request body.

Improve the documentation for the REST Rhizome import request.
2017-10-20 15:50:18 +10:30
Andrew Bettison
5eb19f1a16 Improve REST API technical documentation (fixes #118)
Users and contributors have had difficulty working out how to use the
more complex requests in the REST interface, particularly POST
/restful/rhizome/insert.

Improve the Markdown documentation to provide concrete examples of POST
requests and provide much more extensive description of how to use the
"multipart/form-data" Content-Type.
2017-10-20 10:03:18 +10:30
Andrew Bettison
9e32c01e61 Set Content-Type in static file HTTP responses
The static file server now sets the Content-Type of the response
depending on the file name extension:

    .txt        =  text/plain; charset=utf-8
    .htm .html  =  text/html; charset=utf-8
    .json       =  application/json
    .rfm        =  rhizome/manifest; format=text+binarysig
2017-10-20 10:01:15 +10:30
Andrew Bettison
29ce8994f2 Improve HTTP Content-Type handling
Undo a change from late 2015 that allowed the format=text+binarysig
parameter of the rhizome/manifest MIME content-type to be optional -- it
is easy to provide that parameter via libcurl, and the lbard code has
just been updated to provide it.

Make Content-Type handling consistent throughout the Serval DNA code by
using 'struct mime_content_type' to represent both parsed (request) and
sent (response) Content-Type fields, replacing ad-hoc in-line string
literals.

Fix some TODOs by adding the "serval/sid", "rhizome/bid" and
"rhizome/bundlesecret" MIME types, which remain optional for the time
being to preserve REST API compatibility with clients that do not set
the Content-Type of their 'bundle-author', 'bundle-id' and
'bundle-secret' parameters.
2017-10-20 09:55:21 +10:30
Andrew Bettison
c64faadb85 Add str_endswith() and strcase_endswith() 2017-10-20 09:48:19 +10:30
Andrew Bettison
fc478d07ff Preserve curl trace files in all rhizomerestful test cases
These trace files make it easier to see the innards of REST
API requests and responses, which improves the value of tests
as API specification/documentation.
2017-10-20 09:47:34 +10:30
Jeremy Lakeman
a8dfc8bfc1 Allow null names in activity list 2017-10-16 10:32:18 +10:30
Jeremy Lakeman
bcb89d8351 Swap order of closing streams to avoid deadlocking issue on android 2017-10-11 15:32:56 +10:30
Jeremy Lakeman
ac43a629e6 Fix rename of UUID symbol specifically for building with the Android NDK 2017-10-10 15:05:28 +10:30
Jeremy Lakeman
38eee5ee2e Check query parameter format before opening meshmb state 2017-10-10 14:26:47 +10:30
Andrew Bettison
159c731e9e Add SERVAL_ prefix to UUID symbols to avoid collisions on OS-X 2017-10-09 11:41:44 +10:30
Andrew Bettison
33a17a01fe Improve HTTP authorization diagnostics 2017-10-09 09:19:39 +10:30
Andrew Bettison
4987c0d689 Fix 'tags' target in Makefile
Was not indexing the libsodium-dev/include directory correctly.
2017-10-09 09:19:39 +10:30
Andrew Bettison
2443811dad Fix ./configure Java version detection
Use detected POSIX-compliant $SED instead of system sed (which is BSD
sed on Mac OS-X).
2017-10-09 09:19:39 +10:30