The new struct tree_iterator and associated start/get/next/free
functions replace the recursive walk() function, removing the need for a
callback when iterating over all nodes in the tree, and allowing
iteration to be suspended while other pseudo-threads are run. This
allows an HTTP REST request to keep a tree_iterator in its state struct
and potentially simplifies other areas of the code.
The iterator free()s any empty internal tree nodes that it encounters,
as did the original tree_walk() function. To support the existence of
multiple iterators at once, a reference count has been added to the
tree_node struct, to prevent any iterator from free()ing a node while
any other iterators point to it; only the last iterator to pop out of an
empty node will free() it.
The tree_walk() and tree_walk_prefix() functions have been
re-implemented to use an iterator state object internally. This
resolves an outstanding TODO to perform tree-node freeing during a
prefix walk, and simplifies the code considerably.
Renamed some function parameters and struct members to make the
nibble-tree API a little more self-explanatory.
Added a nibble-tree test to the 'serval-tests' utility.
The new ios/configure script performs a separate ../configure for each of
the following iOS targets:
iPhoneOS on armv7
iPhoneOS on arm64
iPhoneSimulator on i386
iPhoneSimulator on x86_64
The script then creates ios/Makefile that builds a static iOS Framework
Bundle suitable for inclusion in an Xcode iOS project.
Add the --xcode-sdk=SDK option to configure.ac, to support cross
compiling using Apple Xcode. It prefixes all compile/link toolchain
commands with the "xcrun --sdk SDK" command, ie, CC, AS, LD, AR, RANLIB,
etc.
Re-structure headerfiles.mk to separate "public" from "private" headers,
because the Framework module only exposes the public ones. Moves the
SQLITE3_AMALGAMATION definition from Makefile.in into headerfiles.mk.
Update INSTALL.md and add a technical document for Apple iOS.
This enables development of Serval DNA within the context of an Xcode
iOS project using the standard edit-make-test cycle: after modifying a
Serval DNA source file, "cd ios; make" will recompile the changed file
for all the target architectures and update the Framework Bundle.
Rebuilding the Xcode project will then incorporate the changes, which
can be tested immediately.
The daemon API is built as a Swift module called ServalDNA.
The new CliContext class allows easy implementation of CLI output using
Swift code. The new CliContextFile subclass is the obvious first
implementation, equivalent to cli_stdio.c. The 'servaldswift'
executable now uses CliContextFile to print its CLI output to standard
output.
The new delegated log output support constructs each log line in a
buffer and prints it by calling the delegate's 'print' function at
end-of-line. The 'servaldswift' executable now provides a log output
delegate in Swift that simply prints to standard error, replacing
log_output_console.o, which is omitted from its link.
Add a swift-client-api subdirectory containing a Swift source package
and a Makefile.in that compiles it into the "ServalClient" Swift module
using the Swift package manager. The Swift API contains the following
classes:
- ServalKeyring provides the operations: add, remove, set, list
- AbstractId and its specialisation SubscriberId, already in near-final
form, are data types for SID and the like
- ServalRestfulClient (internal) uses an HTTP client to access the
Serval DNA RESTful interface
Improve the REST /keyring/set operation to only alter the DID or Name if
the corresponding query parameter is supplied. Modify the internal
keyring_set_did() function to only assign the DID or Name if the
corresponding parameter is not a null pointer.
The configure script ensures that the Swift build target version is
10.10 or later when compiling for Mac OS-X, so that the package manager
will succeed.
Add autoconf macros for the Swift package manager.
If the Swift 3 or Swift 4 compiler is present or passed to the configure
script in the SWIFTC variable, then the Makefile will compile a Swift
'servaldswift' executable to ensure that the module map links correctly
into a stand-alone Swift program.
Use the gold linker if supported, to avoid relocation errors on symbols
produced by Swift when linking dynamic libraries.
Rename the logging primitive functions and utility functions, prefixing
all with 'serval_log', eg: logMessage() -> serval_logf() etc.
Add an XPRINTF xhexdump() function and use it to implement the
serval_log_hexdump() utility, renamed from dump(). Add macros
WHY_dump(), WARN_dump(), HINT_dump() and DEBUG_dump(), and use them
everywhere.
Remove the 'log.console.dump_config' and 'log.file.dump_config'
configuration options; configuration is now dumped in every log prolog.
The logging system now constructs the log prolog by invoking the new
'log_prolog' trigger, so that it no longer depends on the version string
and configuration system. Any system that wants to present a message in
the log prolog can define its own trigger, which calls standard log
primitives to print the message.
Split the logging system into a front-end (log.c) that provides the
logging primitives and is independent of the configuration system, and a
set of back-end "outputters" (log_output_console.c, log_output_file.c,
log_output_android.c) that may depend on the configuration system and
are decoupled from the front-end using the 'logoutput' link section.
These log outputters are explicitly linked into executables by the
Makefile rules, but could also be linked in using USE_FEATURE(). The
USE_FEATURE() calls have _not_ been added to servald_features.c, so that
different daemon executables can be built with the same feature set but
different log outputs.
Use the conventional autoconf @srcdir@ substitution instead of requiring
the SOURCE_PREFIX variable to be passed to the Makefile.
Also support running test scripts in an out-of-tree build directory.
The libsodium source code is now embedded in the Serval DNA sourcecode
as a git subtree, so configure.ac and Makefile.in now recurse into it as
part of the Serval DNA build.
This simplifies the build instructions and eliminates all external
dependencies on libsodium development files and operating system
packages, returning Serval DNA to a self-contained component whose only
external dependencies are standard system libraries.
Remove the build-libsodium.sh script, as it is no longer needed.
The 'make tags' indexes all the header and source files known to the
Makefile, and also all the non-standard library headers used for
compiling, such as libsodium, JNI and Android NDK.
This supports the sp-mktags utility in serval-tools, which tries "make
tags" before falling back to its own ctags command.
Re-organised all Java source code under the java-api subdirectory, which
also contains a Makefile.in that performs the Java compilation. This
makefile is only invoked if ./configure finds a working Java compiler.
Improves support for running make from a current working directory that
is not the repo root. Only the files that reside in the repo root are
prefixed with $(SOURCE_PREFIX); the files that are created by
./configure are in the current working directory.
Renamed libservald.so to libservaldaemon.so everywhere to avoid conflicts
with the "servald" module name in Android.mk, and to avoid the overly
generic name "libserval".
Include the JNI entry points in the Android build.
swiftc(1) requires every -D option to be prefixed with -Xcc, which
is not easy to achieve using autoconf.
Now instead of setting configuration macros like HAVE_LSEEK64 on the
command line using -D, they are defined in config.h, which is generated
by ./configure, ignored by Git, and included by all headers and source
files that use any configuration macro.
The dependency on Makefile has been replaced with a dependency on
Makefile.in and config.h, which helps avoid some redundant full
recompiles after running ./configure, because the configure script
does not overwrit config.h if its content does not change.
Recent changes such as the Makefile.in overhaul and the introduction of
feature-driven linking broke the Android build. This commit fixes the
breakage:
- detects the presence of gettid() in configure.ac and only defines
gettid() in serval.c if HAVE_GETTID is not defined
- builds libserval.so not libservald.so on Android, to avoid a conflict
on the module name "servald" in Android.mk
- renames cli_cleanup() to command_cleanup(), defines it in
commandline.h instead of cli.h, and supplies it in android.c
- supplies the 'keyring' global in android.c
- removes log_stderr.c from the Android build, since it conflicted with
log.c
This introduces a new way of linking Serval executables and dynamic
libraries from static libraries like libservald.a -- called
"feature-driven" linking.
The Makefile now links servald and serval-tests from libservald.a,
rather than from an explicit list of object (.o) files. Thanks to the
section-based method for registering functions such as HTTP handlers,
CLI commands and MDP handlers, these object files had become
"stand-alone" and hence were no longer included in the link because
there was no unresolved reference that required them to be linked in.
The new "feature.h" provides the DECLARE_FEATURE(name) macro that each
stand-alone source file uses to declare the named feature(s) it
provides. Each executable can call the USE_FEATURE(name) macro in any
of its explicitly-linked source files to cause the corresponding
object(s) to be included in the link, eg, servald_features.c.
The DEFINE_BINDING() macro has been extended so that every individual
MDP binding is given a feature name based on its port number macro, eg,
"mdp_binding_MDP_PORT_ECHO".
Some features have been factored into their own separate source files so
they can be omitted or included in a build independently of each other:
- the MDP bindings for MDP_PORT_DNALOOKUP, MDP_PORT_ECHO,
MDP_PORT_TRACE, MDP_PORT_KEYMAPREQUEST, MDP_PORT_RHIZOME_xxx,
MDP_PORT_PROBE, MDP_PORT_STUN, MDP_PORT_STUNREQ
- the CLI "log" and "echo" commands
- the CLI "rhizome direct" command
The JNI source files are only compiled if the <jni.h> header is present,
otherwise they are omitted from libservald.so.
The CLI and server main loop now have no conditional JNI code. All JNI
code has been moved into separate source files, which #include the new
"jni_common.h" instead of <jni.h>. The "cli.h" header no longer
includes <jni.h>, so the rest of the Serval source code is now
unaffected by JNI definitions.
The 'cf_limbo' global variable is now thread-local, so that each thread
has its own independent copy of the loaded configuration. The JNI
server entry point now calls cf_init() once. The new 'cf_initialised'
flag prevents clobbering the config state by redundant calls to
cf_init().
The CLI "stop" command now sends SIGHUP to the specific thread in which
the server is running. This is achieved by writing the PID and TID
(Linux Thread ID) into the pidfile, separated by a space, on systems
that support the Linux gettid() and tgkill() system calls. The server's
signal handler has been overhauled, and its logging improved.
Changed the built libraries and updated INSTALL.md accordingly:
- the 'servaldwrap' executable is now built
- libservald.[a,so] now contains the full daemon executable
- libservalclient.[a,so] contains the client library
- libserval.a is no longer built
Building the 'servaldwrap' executable is a step towards ensuring that
the libservald.so library can be linked and executed, without requiring
an Android build to reveal any failure.
Added the SOURCE_PREFIX variable that allows the make to be invoked from
within any current working directory, not just the serval-dna
repository's root directory.
Fixed the serval_version.o target to invoke the version_string.sh
script with the --repository=DIR option, so that the build will work
even if the current working directory is not within the serval-dna
repository.
Re-ordered and added some comments for clarity, and added some missing
.PHONY declarations.
Ignores a variety of warnings when compiling sqlite3.c.
Uses the new sqlite3_trace_v2() function for logging SQL statements, now
that sqlite3_trace() and sqlite3_profile() are deprecated.
Fixes a rhizomeops test failure caused because SQLite 3.12 increased the
default block size from 1024 to 4096, causing the fixed overhead of an
empty Rhizome db to exceed 32KB. This caused the test to fail because
the first "file add" failed. Now the test fixture adapts to the fixed
overhead.
Deletes the SQLite 3.10.2 source amalgamation.
Was not passing $(CFLAGS) $(DEFS) to compile servald_version.c, which
provoked a "cc1: note: someone does not honour COPTS correctly, passed 0
times" warning
Also was not doing "mkdir -p opts" before the compile, which produced an
"opts/version.o: No such file or directory: error
Make .o files in separate subdirectories for servald and library
Factor struct __sourceloc and __WHENCE__ from "log.h" into "whence.h"
Factor sid_t etc from "serval.h" into "serval_types.h"
Factor rhizome_bid_t etc from "rhizome.h" into "rhizome_types.h"
Do not include "serval.h" in library sources
Add log_stderr.c and logMessage.c for stand-alone executables
