Whenever the daemon re-loads its config, it keeps any socket-based
(AF_INET) SOCK_DGRAM interfaces open that are matched by the new config,
but it used to close and then re-open all local (AF_UNIX) interfaces and
file-based interfaces such as SOCK_FILE dummy files and SOCK_STREAM
device files. This made it very difficult to develop the new
'routejava' test suite, because the Java API test harness always causes
a config re-load when it sets the REST API user/password in the config,
which caused the local socket interfaces to bounce, which interfered
with the reachability of nodes.
Now, local socket and file-based interfaces remain up after a config
re-load, as long as they are still matched by the new configuration.
Added INFO messages for interfaces that remain up after a config
re-load, and tweak interface up/down INFO messages to be consistent.
This will require any android application to include their own shared library rules similar to the following;
include $(CLEAR_VARS)
LOCAL_STATIC_LIBRARIES := servaldstatic
LOCAL_C_INCLUDES += $(LOCAL_PATH)/serval-dna
LOCAL_SRC_FILES := $(LOCAL_PATH)/features.c
LOCAL_MODULE := servaldaemon
LOCAL_LDLIBS += -L$(SYSROOT)/usr/lib -llog
include $(BUILD_SHARED_LIBRARY)
Allowing just the set of required features to be linked into the final binary
Consists of a single, one-shot request, GET /restful/route/all.json, for
the time being. A "newsince" request can be added later.
Add a Markdown tech doc that specifies the new Route REST API and update
the MDP tech doc a little.
Add the 'routerestful' test script, which uses some test utility
fuctions that have been factored out of the 'routing' test script into
the new testdefs_routing.sh.
Add the new 'allrestful' test script.
This helps tests distinguish which of a daemon's keyring identities
is used as its primary identity, which is not otherwise obvious
without consulting the routing table, because slots are allocated
in random order.
The new struct tree_iterator and associated start/get/next/free
functions replace the recursive walk() function, removing the need for a
callback when iterating over all nodes in the tree, and allowing
iteration to be suspended while other pseudo-threads are run. This
allows an HTTP REST request to keep a tree_iterator in its state struct
and potentially simplifies other areas of the code.
The iterator free()s any empty internal tree nodes that it encounters,
as did the original tree_walk() function. To support the existence of
multiple iterators at once, a reference count has been added to the
tree_node struct, to prevent any iterator from free()ing a node while
any other iterators point to it; only the last iterator to pop out of an
empty node will free() it.
The tree_walk() and tree_walk_prefix() functions have been
re-implemented to use an iterator state object internally. This
resolves an outstanding TODO to perform tree-node freeing during a
prefix walk, and simplifies the code considerably.
Renamed some function parameters and struct members to make the
nibble-tree API a little more self-explanatory.
Added a nibble-tree test to the 'serval-tests' utility.
If runTests() terminated with a FATAL condition, the cause was not
always apparent, because the error message usually got overwritten by
the test progress output on the user's terminal. To fix this, the main
loop's standard error is now collected in a temporary file which is sent
to standard error just before exit.
The test framework was not always handling its internal FATAL error code
(255) correctly, so this has been fixed.
One cause of a FATAL termination was if a test's unique temporary
directory, which was based on its Process ID, already existed. Anything
that left a temporary directory behind increased the likelihood of a
FATAL in a subsequent test run. (For example, one keyring test case was
not killing its Serval DNA daemon, and the daemon was re-creating its
instance directory before eventually terminating itself. This test case
has been fixed.)
The test framework now allocates its temporary directory by re-trying
different random numbers until mkdir(1) succeeds, and only fataling
after 20 failures.
Now produces exactly the same output variables as 'create_identities 1'
so that these two functions can be interchanged with minimal impact
on the test script.
The new API follows REST rules by using the proper request verbs:
POST, PUT, PATCH and DELETE, instead of just GET.
The legacy GET-only API is still supported for backward compatibility,
but not longer tested or documented.
Add a new query-single-identity operation.
Implement the lock-single-identity operation, which until now had been
documented but not yet implemented. Whenever a single identity is
locked (released), any other unlocked identities with the same PIN are
flagged to indicate that the PIN is not "fully" unlocked, so that the
next time the PIN is entered, the slot decryption is re-tried for
non-loaded identities, and the locked identity will be unlocked again.
Update the 'keyring' and 'keyringrestful' test scripts:
- refactored to reduce curl command-line clutter in test cases
- now tests the redesigned request verbs and paths
- added a test for GET /restful/keyring/SID
- added a test for PUT /restful/keyring/SID/lock
The LOCK and UNLOCK by PIN requests were acting on the supplied PIN
and also the empty PIN, due to a bug in ob_get_str_ptr(), which returned
an empty string instead of NULL after reaching the end of the string
list.
- make "debug.h" a private header so that the DEBUG() macro does not
interfere with Xcode Debug builds, which set DEBUG=1
- move all #include "debug.h" from headers into .c files
- move 'struct idebug' into a new public header "idebug.h" so that
"log.h" can be public
- move HTTPD port number defs from "httpd.h" to "constants.h", so that
"httpd.h" can be private and "conf.h" can be public
- add missing nested includes so each header compiles stand-alone
without error
- #include "sodium.h" instead of <sodium.h>
- #include "sqlite3.h" instead of <sqlite3.h>
- add header guard to "fifo.h"
- fix header guard in "sync_keys.h"
The new ios/configure script performs a separate ../configure for each of
the following iOS targets:
iPhoneOS on armv7
iPhoneOS on arm64
iPhoneSimulator on i386
iPhoneSimulator on x86_64
The script then creates ios/Makefile that builds a static iOS Framework
Bundle suitable for inclusion in an Xcode iOS project.
Add the --xcode-sdk=SDK option to configure.ac, to support cross
compiling using Apple Xcode. It prefixes all compile/link toolchain
commands with the "xcrun --sdk SDK" command, ie, CC, AS, LD, AR, RANLIB,
etc.
Re-structure headerfiles.mk to separate "public" from "private" headers,
because the Framework module only exposes the public ones. Moves the
SQLITE3_AMALGAMATION definition from Makefile.in into headerfiles.mk.
Update INSTALL.md and add a technical document for Apple iOS.
This enables development of Serval DNA within the context of an Xcode
iOS project using the standard edit-make-test cycle: after modifying a
Serval DNA source file, "cd ios; make" will recompile the changed file
for all the target architectures and update the Framework Bundle.
Rebuilding the Xcode project will then incorporate the changes, which
can be tested immediately.
The daemon API is built as a Swift module called ServalDNA.
The new CliContext class allows easy implementation of CLI output using
Swift code. The new CliContextFile subclass is the obvious first
implementation, equivalent to cli_stdio.c. The 'servaldswift'
executable now uses CliContextFile to print its CLI output to standard
output.
The new delegated log output support constructs each log line in a
buffer and prints it by calling the delegate's 'print' function at
end-of-line. The 'servaldswift' executable now provides a log output
delegate in Swift that simply prints to standard error, replacing
log_output_console.o, which is omitted from its link.
Add a swift-client-api subdirectory containing a Swift source package
and a Makefile.in that compiles it into the "ServalClient" Swift module
using the Swift package manager. The Swift API contains the following
classes:
- ServalKeyring provides the operations: add, remove, set, list
- AbstractId and its specialisation SubscriberId, already in near-final
form, are data types for SID and the like
- ServalRestfulClient (internal) uses an HTTP client to access the
Serval DNA RESTful interface
Improve the REST /keyring/set operation to only alter the DID or Name if
the corresponding query parameter is supplied. Modify the internal
keyring_set_did() function to only assign the DID or Name if the
corresponding parameter is not a null pointer.
The configure script ensures that the Swift build target version is
10.10 or later when compiling for Mac OS-X, so that the package manager
will succeed.
Add autoconf macros for the Swift package manager.
If the Swift 3 or Swift 4 compiler is present or passed to the configure
script in the SWIFTC variable, then the Makefile will compile a Swift
'servaldswift' executable to ensure that the module map links correctly
into a stand-alone Swift program.
Use the gold linker if supported, to avoid relocation errors on symbols
produced by Swift when linking dynamic libraries.
Update the Java API to be consistent with recent changes to the Rhizome
REST API was that regularised the MIME types used in Content-Type
headers.
Add some missing CONTENT_TYPE_ constants to "httpd.h", and ensure they
are consistent with the Java getMimeType() methods in subclasses of
AbstractId.
Add the --stderr option to all executeJavaOk statements in the
'rhizomejava' test script, to reveal the exception message in the case
of failure.
Add a test script for all Java tests, called "alljava".
Include testconfig.sh in testdefs.sh, not testdefs_java.sh, so that the
tests/all script can successfully test the $JAVAC variable.
Rename the logging primitive functions and utility functions, prefixing
all with 'serval_log', eg: logMessage() -> serval_logf() etc.
Add an XPRINTF xhexdump() function and use it to implement the
serval_log_hexdump() utility, renamed from dump(). Add macros
WHY_dump(), WARN_dump(), HINT_dump() and DEBUG_dump(), and use them
everywhere.
Remove the 'log.console.dump_config' and 'log.file.dump_config'
configuration options; configuration is now dumped in every log prolog.
The logging system now constructs the log prolog by invoking the new
'log_prolog' trigger, so that it no longer depends on the version string
and configuration system. Any system that wants to present a message in
the log prolog can define its own trigger, which calls standard log
primitives to print the message.
Split the logging system into a front-end (log.c) that provides the
logging primitives and is independent of the configuration system, and a
set of back-end "outputters" (log_output_console.c, log_output_file.c,
log_output_android.c) that may depend on the configuration system and
are decoupled from the front-end using the 'logoutput' link section.
These log outputters are explicitly linked into executables by the
Makefile rules, but could also be linked in using USE_FEATURE(). The
USE_FEATURE() calls have _not_ been added to servald_features.c, so that
different daemon executables can be built with the same feature set but
different log outputs.
The strbuf_local_buf() function attempted to detect if its argument was
not an array but a pointer, and fail at compile time. This worked fine
when compiling with standard optimisation (-O2).
However, specifying the gcc -g option disables compiler optimisation, so
the compiler does not optimise away the call to the function with
__attribute__(error("...")), and so the compile fails.
The solution is to perform the check at run time using assert(), like
strbuf_local() already does.
Correct the maximum DID length defined in "serval_types.h" from 32 to
31. Add a definition of the maximum identity Name length and use it
instead of the bare constant 64, eg, in the MDP_DNALOOKUP request
handling code.
Introduce a dataformats.h function for validating an identity name, and
use it to validate the 'name' parameter in the CLI 'keyring set'
command.
Add 'did' and 'name' parameter validation to the GET /restful/keyring/add
and GET /restful/keyring/SID/set requests (#131).
Rename keyring_set_did() to keyring_set_did_name() and assert that DID
and Name lengths have been validated before storing in the keyring.
Update the Keyring REST API tech document.
