diff --git a/.gitignore b/.gitignore index 4e1591ae..adf0a20d 100644 --- a/.gitignore +++ b/.gitignore @@ -19,9 +19,6 @@ /Makefile /testconfig.sh /config.h -/nacl/nacl-20110221/build -/nacl/naclinc.txt -/nacl/nacllib.txt /servald /servaldwrap /serval-tests diff --git a/INSTALL.md b/INSTALL.md index 4b8aaaec..29af81a8 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -1,13 +1,13 @@ Serval DNA Build and Test ========================= -[Serval Project][], October 2016 +[Serval Project][], September 2017 Supported Architectures ----------------------- These instructions will build [Serval DNA][] successfully for the following platforms: - * Debian Linux, ix86 and x86\_64, kernel versions 2.6 to 4.6, using [gcc + * Debian Linux, ix86 and x86\_64, kernel versions 2.6 to 4.12, using [gcc 4.4][] and later, [gcc 5][] and [gcc 6][] * Mac OS-X x86\_64, releases 10.7 “Lion” to 10.11 “El Capitan”, using [Xcode][] versions 3.2 to 8, and GNU tools available from [homebrew][] @@ -47,16 +47,10 @@ Mandatory dependencies: * network services library **libnsl** and headers * dynamic link library **libdl** and header `` * Native Posix Threads Library **libpthread** and header `` - * elliptic curve encryption library **libsodium** and header ``, - version 1.0.2 or greater * on Solaris, the realtime library **librt** (for the `nanosleep()` function) * **autoconf** 2.67-2.69 (2.70 may work but has not been tested) * **automake** 1.15 -The **libsodium** development files are available on Debian/Ubuntu systems in -the `libsodium-dev` package. On other systems, like Mac OS-X, it must be -compiled from source. The [Notes for Developers][] give more details. - Optional: * Java compiler and SDK @@ -103,28 +97,52 @@ A successful session should appear something like: config.status: creating Makefile config.status: creating testconfig.sh $ make - SERVALD CC conf.c - SERVALD CC cli.c + SERVALD CC servald_features.c + MAKE libsodium-dev + make[1]: Entering directory '/home/username/src/serval-dna/libsodium' + Making install in contrib + make[2]: Entering directory '/home/username/src/serval-dna/libsodium/contrib' ... - CC cli.c - CC commandline.c + make[4]: Leaving directory '/home/username/src/serval-dna/libsodium/src/libsodium/include' + make[4]: Entering directory '/home/username/src/serval-dna/libsodium/src/libsodium' + CC crypto_aead/chacha20poly1305/sodium/libsodium_la-aead_chacha20poly1305.lo + CC crypto_aead/xchacha20poly1305/sodium/libsodium_la-aead_xchacha20poly1305.lo + CC crypto_auth/libsodium_la-crypto_auth.lo + ... + ---------------------------------------------------------------------- + Libraries have been installed in: + /home/username/src/serval-dna/libsodium-dev/lib + ... + make[1]: Leaving directory '/home/username/src/serval-dna/libsodium' + SERVALD CC jni_common.c + SERVALD CC jni_commandline.c + SERVALD CC jni_instance.c + ... + CC version_servald.c + AR _servald.a + LINK libservaldaemon.so + LIB CC base64.c + LIB CC cli.c + LIB CC cli_stdio.c ... LINK simulator - SERVALD CC test_cli.c - SERVALD CC log_context.c - SERVALD CC log_stderr.c - SERVALD CC context1.c + SERVALD CC test_features.c LINK serval-tests + make[1]: Entering directory '/home/username/src/serval-dna/java-api' + JAVAC classes + JAVAC testclasses + Note: Some input files use or override a deprecated API. + Note: Recompile with -Xlint:deprecation for details. + make[1]: Leaving directory '/home/username/src/serval-dna/java-api' + rm _servalclient.a _monitorclient.a _servald.a $ -On Solaris, the system `make` command may not be GNU Make, and the system -`cc` command may not be GNU Gcc. The following may work: +On some systems, the system `make` command may not be GNU Make, and the system +`cc` command may not be GNU gcc. The following may work: $ cd $HOME/src/serval-dna $ autoreconf -f -i -I m4 - $ CC=gcc - $ export CC - $ ./configure + $ ./configure CC=gcc $ gmake $ @@ -140,14 +158,15 @@ Built artifacts The build process produces the following artifacts: * **servald** is the main Serval DNA daemon executable. All the Serval DNA - daemon code is statically linked into this executable, so it does not depend - on any built Serval libraries. However, it is dynamically linked with the - system libraries and with `libsodium.so`. + daemon, SQLite and libsodium code is statically linked into this executable, + so it does not need to load any Serval or libsodium shared libraries at + run-time. Its unstripped size is about 9.5 MB on a typical 64-bit system, of + which about 7 MB is SQLite. Its stripped size is about 3 MB. * **servaldwrap** is a Serval DNA executable identical to *servald*, but - dynamically linked with `libservald.so` instead of statically linked. This - executable mainly exists to ensure that the shared library is always - linkable. + it loads `libservaldaemon.so` at run-time using [dlopen(3)][] instead of + being statically linked, so it is only a dozen KB in size. This executable + mainly exists to test that the shared library is loadable. * **serval-tests** is an executable utility that performs various system tests such as memory speed, cryptographic speed, byte ordering, and configuration @@ -155,16 +174,16 @@ The build process produces the following artifacts: provided in a separate executable in order to keep the size of the *servald* executable to a minimum. -* **libservald.a** is a static library containing the complete executable code - of the Serval DNA daemon. An executable (such as *servald*) can be built - with any desired subset of Serval functions by linking in only the required - parts of this library using the *features* mechanism described in - [feature.h](./feature.h). +* **libservaldaemon.a** is a static library containing the complete executable + code of the Serval DNA daemon, including SQLite and libsodium cryptographic + functions. An executable (such as *servald*) can be built with any desired + subset of Serval functions by linking in only the required parts of this + library using the *features* mechanism described in [feature.h](./feature.h). -* **libservald.so** is a dynamic library containing the complete executable - code of the Serval DNA daemon, including [JNI][] entry points. The Serval - DNA Java API, which is used by [batphone][], and the *servaldwrap* executable - both use this dynamic library. +* **libservaldaemon.so** is a dynamic library containing the complete executable + code of the Serval DNA daemon, including [JNI][] entry points, SQLite and + libsodium cryptographic functions. The Serval DNA Java API, which is used by + [batphone][], and the *servaldwrap* executable both use this dynamic library. * **directory_service** is the executable for the [Serval Infrastructure][] daemon. @@ -242,7 +261,7 @@ and may need to be changed for your particular circumstances. ----- **Copyright 2013-2015 Serval Project Inc.** -**Copyright 2016 Flinders University** +**Copyright 2016-2017 Flinders University** ![CC-BY-4.0](./cc-by-4.0.png) This document is available under the [Creative Commons Attribution 4.0 International licence][CC BY 4.0]. diff --git a/Makefile.in b/Makefile.in index 58eb6df1..754f1294 100644 --- a/Makefile.in +++ b/Makefile.in @@ -12,6 +12,12 @@ SQLITE3_AMALGAMATION = sqlite-amalgamation-3140200 include $(SOURCE_PREFIX)sourcefiles.mk include $(SOURCE_PREFIX)headerfiles.mk +LIBSODIUM_SUBDIR = $(SOURCE_PREFIX)libsodium/ +LIBSODIUM_DEV = libsodium-dev +LIBSODIUM_HEADERS = $(LIBSODIUM_DEV)/include/sodium.h +LIBSODIUM_A = $(LIBSODIUM_DEV)/lib/libsodium.a +LIBSODIUM_SO = $(LIBSODIUM_DEV)/lib/libsodium.so + OBJSDIR_SERVALD = objs_servald OBJSDIR_LIB = objs_lib OBJSDIR_TOOLS = objs @@ -53,6 +59,7 @@ SIMULATOR_OBJS = \ $(addprefix $(OBJSDIR_TOOLS)/, $(SIMULATOR_SOURCES:.c=.o)) PREFIXED_HEADERS = $(addprefix $(SOURCE_PREFIX), $(HDRS)) +PREFIXED_SOURCES = $(addprefix $(SOURCE_PREFIX), $(ALL_SOURCES)) CC= @CC@ @@ -87,6 +94,14 @@ DEFS= @DEFS@ CONFIG_H = @CONFIG_H@ +# Usage: $(call MERGE_STATIC_LIBS, libdest.a, libsrcfoo.a libsrcbar.a) +MERGE_STATIC_LIBS = echo MERGE $(1) = $(2); \ + tmp="$$(mktemp -d)" && \ + ( cd "$$tmp" $(foreach LIB, $(2), && $(AR) -x $(abspath $(LIB))) ) && \ + $(RM) $@ && \ + $(AR) -cr $(1) "$$tmp"/* && \ + $(RM) -r "$$tmp" + .PHONY: all libs test install uninstall clean all: libs servald servaldwrap test @@ -104,13 +119,15 @@ uninstall: $(RM) $(DESTDIR)$(sbindir)/servald clean: - @$(RM) -r $(OBJSDIRS:%=%/*) \ - servald \ - libservaldaemon.so libservaldaemon.a \ - libservalclient.so libservalclient.a \ - libmonitorclient.so libmonitorclient.a \ - tfw_createfile directory_service fakeradio simulator serval-tests \ - tags + cd $(LIBSODIUM_SUBDIR) && $(MAKE) clean + $(RM) -r $(LIBSODIUM_DEV) \ + $(OBJSDIRS:%=%/*) \ + servald \ + libservaldaemon.so libservaldaemon.a \ + libservalclient.so libservalclient.a \ + libmonitorclient.so libmonitorclient.a \ + tfw_createfile directory_service fakeradio simulator serval-tests \ + tags # Only provide Java targets if the Java compiler is available. ifneq ($(JAVAC),) @@ -136,6 +153,32 @@ java-api-clean: endif # $(JAVAC) +# Build the Sodium elliptic curve encryption library within its 'libsodium' +# subtree, and install its development files (headers and libraries) into our +# $(LIBSODIUM_DEV) subdirectory. Then use the contents of this subdirectory to +# provide system headers, eg , and libraries, eg, libsodium.a, to +# compile and link Serval DNA. This completely avoids any dependency on the +# system's installed libsodium run-time and development packages. + +CFLAGS+= -isystem $(LIBSODIUM_DEV)/include + +$(LIBSODIUM_HEADERS) $(LIBSODIUM_A): $(LIBSODIUM_DEV)/.installed + +# The libsodium package uses libtool, so by default its static libraries get +# compiled without the -fPIC option. This prevents the libsodium static +# library from being linked into a shared library. However, this Makefile +# needs to do exactly that, to make libservaldaemon.so, so passes the +# -prefer-pic libtool option, which causes libtool to use -fPIC even when +# compiling for static libraries. + +$(LIBSODIUM_DEV)/.installed: + @echo MAKE $(LIBSODIUM_DEV) + @mkdir -p $(LIBSODIUM_DEV) + @$(RM) $@ + @touch $@.in-progress + @cd $(LIBSODIUM_SUBDIR) && $(MAKE) CFLAGS+=-prefer-pic prefix=$(abspath $(LIBSODIUM_DEV)) install + @mv -f $@.in-progress $@ + # Source code test coverage support -- see doc/Testing.md .PHONY: covzero covinit covhtml is_built_with_coverage has_coverage_data @@ -193,7 +236,7 @@ $(SOURCE_PREFIX)configure: $(SOURCE_PREFIX)configure.ac # Embed Serval DNA's version into libraries and executables. -$(OBJSDIR_TOOLS)/version.o: $(addprefix $(SOURCE_PREFIX), $(ALL_SOURCES)) \ +$(OBJSDIR_TOOLS)/version.o: $(PREFIXED_SOURCES) \ $(PREFIXED_HEADERS) \ $(SOURCE_PREFIX)version_servald.c \ $(SOURCE_PREFIX)version_string.sh \ @@ -243,28 +286,32 @@ $(OBJSDIR_LIB)/%.o: $(SOURCE_PREFIX)%.c # broad so it sometimes results in unnecessary re-compilation, but that is # better than too narrow, which can result in missed re-compilation. -$(SERVAL_DAEMON_OBJS): Makefile $(CONFIG_H) $(PREFIXED_HEADERS) -$(SERVALD_OBJS): Makefile -$(LIB_SERVAL_OBJS): Makefile $(CONFIG_H) $(PREFIXED_HEADERS) +$(SERVAL_DAEMON_OBJS): Makefile $(CONFIG_H) $(PREFIXED_HEADERS) $(LIBSODIUM_HEADERS) +$(SERVALD_OBJS): Makefile $(LIBSODIUM_HEADERS) +$(LIB_SERVAL_OBJS): Makefile $(CONFIG_H) $(PREFIXED_HEADERS) $(LIBSODIUM_HEADERS) $(OBJSDIR_TOOLS)/tfw_createfile.o: Makefile str.h -$(OBJSDIR_TOOLS)/directory_service.o: Makefile $(CONFIG_H) $(PREFIXED_HEADERS) -$(MONITOR_CLIENT_OBJS): Makefile $(CONFIG_H) $(PREFIXED_HEADERS) -$(SIMULATOR_OBJS): Makefile $(CONFIG_H) $(PREFIXED_HEADERS) +$(OBJSDIR_TOOLS)/directory_service.o: Makefile $(CONFIG_H) $(PREFIXED_HEADERS) $(LIBSODIUM_HEADERS) +$(MONITOR_CLIENT_OBJS): Makefile $(CONFIG_H) $(PREFIXED_HEADERS) $(LIBSODIUM_HEADERS) +$(SIMULATOR_OBJS): Makefile $(CONFIG_H) $(PREFIXED_HEADERS) $(LIBSODIUM_HEADERS) # Rules for main targets. -libservaldaemon.a: \ - $(SERVALD_OBJS) \ - $(OBJSDIR_TOOLS)/version.o +.INTERMEDIATE: _servald.a +_servald.a: $(SERVALD_OBJS) $(OBJSDIR_TOOLS)/version.o @echo AR $@ + @$(RM) $@ @$(AR) -cr $@ $^ +libservaldaemon.a: _servald.a $(LIBSODIUM_A) + @$(call MERGE_STATIC_LIBS, $@, $^) + libservaldaemon.so: \ $(OBJSDIR_SERVALD)/servald_features.o \ $(SERVAL_DAEMON_JNI_OBJS) \ - libservaldaemon.a + _servald.a \ + $(LIBSODIUM_A) @echo LINK $@ - @$(CC) -Wall -shared -o $@ $^ $(LDFLAGS) + @$(CC) -Wall -shared -o $@ $(LDFLAGS) $(filter %.o, $^) $(filter %.a, $^) servald: $(OBJSDIR_SERVALD)/servald_features.o libservaldaemon.a @echo LINK $@ @@ -294,27 +341,38 @@ simulator: $(SIMULATOR_OBJS) libservalclient.a @echo LINK $@ @$(CC) -Wall -o $@ $^ $(LDFLAGS) -libservalclient.a: $(LIB_SERVAL_OBJS) $(OBJSDIR_TOOLS)/version.o +.INTERMEDIATE: _servalclient.a +_servalclient.a: $(LIB_SERVAL_OBJS) $(OBJSDIR_TOOLS)/version.o @echo AR $@ + @$(RM) $@ @$(AR) -cr $@ $^ -libservalclient.so: $(LIB_SERVAL_OBJS) $(OBJSDIR_TOOLS)/version.o - @echo LINK $@ - @$(CC) -Wall -shared -o $@ $^ $(LDFLAGS) +libservalclient.a: _servalclient.a $(LIBSODIUM_A) + @$(call MERGE_STATIC_LIBS, $@, $^) -libmonitorclient.a: $(MONITOR_CLIENT_OBJS) $(OBJSDIR_TOOLS)/version.o +libservalclient.so: $(LIB_SERVAL_OBJS) $(OBJSDIR_TOOLS)/version.o $(LIBSODIUM_A) + @echo LINK $@ + @$(CC) -Wall -shared -o $@ $(LDFLAGS) $(filter %.o, $^) $(filter %.a, $^) + +.INTERMEDIATE: _monitorclient.a +_monitorclient.a: $(MONITOR_CLIENT_OBJS) $(OBJSDIR_TOOLS)/version.o @echo AR $@ + @$(RM) $@ @$(AR) -cr $@ $^ -libmonitorclient.so: $(MONITOR_CLIENT_OBJS) $(OBJSDIR_TOOLS)/version.o +libmonitorclient.a: _monitorclient.a $(LIBSODIUM_A) + @$(call MERGE_STATIC_LIBS, $@, $^) + +libmonitorclient.so: $(MONITOR_CLIENT_OBJS) $(OBJSDIR_TOOLS)/version.o $(LIBSODIUM_A) @echo LINK $@ - @$(CC) -Wall -shared -o $@ $^ $(LDFLAGS) + @$(CC) -Wall -shared -o $@ $(LDFLAGS) $(filter %.o, $^) $(filter %.a, $^) # The tags will always index all the Serval DNA headers and source files. If # serval-tools is installed, then it will also index all the libsodium, JNI # and Android NDK header files. -tags: Makefile $(HDRS) $(ALL_SOURCES) - { for file in $(HDRS) $(ALL_SOURCES); do echo "$$file"; done; \ +$(SOURCE_PREFIX)tags: Makefile $(PREFIXED_HEADERS) $(PREFIXED_SOURCES) + { for file in $(PREFIXED_HEADERS) $(PREFIXED_SOURCES); do echo "$$file"; done; \ + find $(LIBSODIUM_DEV)/install -type f ; \ sp-find-gcc-headers $(CFLAGS) 2>/dev/null; \ ndk_prefix=$$(sp-ndk-prefix . 2>/dev/null) && find "$$ndk_prefix/arch-arm/usr/include" -type f; \ } | ctags -L- -f $@ --tag-relative=yes --c-kinds=defgmpstuv diff --git a/README.md b/README.md index 6bc559c7..bd953174 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ Serval DNA ========== -[Serval Project][], September 2016 +[Serval Project][], September 2017 [Serval DNA][] is the core component of the [Serval Mesh][] app for Android and the [Serval Mesh Extender][] long-range mesh networking device. It is a daemon @@ -53,10 +53,15 @@ Bugs can be reported and inspected using the [GitHub issue tracker][Serval DNA i What is in this repository? --------------------------- -This repository contains the [GNU C][] source code for the **servald** -executable, a [test framework](./testframework.sh) and [test scripts](./tests/) -written in [Bash][], some [technical documentation](./doc/), and various -support files for installation and configuration on various platforms. +This repository contains: + +* the [GNU C][] source code for the **servald** executable, +* the [Java][] source code for the Serval API, +* some [technical documentation](./doc/), +* a [test framework](./testframework.sh) and [test scripts](./tests/) written + in [Bash][], +* various support files for installation and configuration on various + platforms. The **servald** executable is a multi-purpose program that can be invoked directly from the command line, run as a daemon process, or invoked via [JNI][] @@ -158,7 +163,7 @@ Entity][entity]. ----- **Copyright 2015 Serval Project Inc.** -**Copyright 2016 Flinders University** +**Copyright 2016-2017 Flinders University** ![CC-BY-4.0](./cc-by-4.0.png) This document is available under the [Creative Commons Attribution 4.0 International licence][CC BY 4.0]. @@ -176,6 +181,7 @@ This document is available under the [Creative Commons Attribution 4.0 Internati [Serval DNA issues]: https://github.com/servalproject/serval-dna/issues [issue #28]: https://github.com/servalproject/serval-dna/issues/28 [GNU C]: http://gcc.gnu.org/ +[Java]: https://en.wikipedia.org/wiki/Java_(programming_language) [daemon]: http://en.wikipedia.org/wiki/Daemon_(computing) [free software]: http://www.gnu.org/philosophy/free-sw.html [contributors]: /servalproject/serval-dna/blob/development/CONTRIBUTORS.md diff --git a/build-libsodium.sh b/build-libsodium.sh deleted file mode 100755 index f65d99f6..00000000 --- a/build-libsodium.sh +++ /dev/null @@ -1,266 +0,0 @@ -#!/bin/bash -# -# Copyright 2016 Flinders University -# -# This script downloads the libsodium source code from GitHub, then compiles it -# and sets up the Serval DNA source code to build against the compiled -# libsodium. -# -# This script is useful for building Serval DNA on platforms such as Max OS X -# that do not provide the libsodium development package. Debian and Ubuntu -# provide the libsodium-dev package which can be installed instead of using this -# script. -# -# By default, the script creates a 'libsodium' directory in the same directory -# that contains the 'serval-dna' directory, ie, '../libsodium' relative to the -# script. This location can be overridden by giving an alternative directory -# path on the command line, which the script will create if it does not exist. - -usage() { - echo "Usage: ${0##*/} [options]" - echo "Options: --src PATH download and build in PATH [$LIBSODIUM_BUILD_DIR]" - echo " --branch BRANCH checkout BRANCH [$LIBSODIUM_BRANCH]" - echo " --prefix PATH install into PATH [$INSTALL_PREFIX-$DIST]" - echo " --no-update do not update if already downloaded" - echo " --no-clean do not clean if already built" - echo " --ssh download from GitHub using SSH instead of HTTPS" - echo " --dist DIST build using libsodium's 'dist-build/DIST.sh' script" - echo " --make-arg ARG pass ARG to the 'make' command" - echo " --help display this message on standard output" -} - -# Work out the path of the directory that contains this script. -case "$0" in -*/*) SCRIPT_DIR="${0%/*}";; -*) SCRIPT_DIR=".";; -esac -case "$SCRIPT_DIR" in -*/*) SCRIPT_PARENT_DIR="${SCRIPT_DIR%/*}";; -.) SCRIPT_PARENT_DIR="..";; -*) SCRIPT_PARENT_DIR="$SCRIPT_DIR/..";; -esac - -# Download location. -LIBSODIUM_URL_PATH="jedisct1/libsodium.git" -LIBSODIUM_GIT_URL_HTTPS="https://github.com/$LIBSODIUM_URL_PATH" -LIBSODIUM_GIT_URL_SSH="git@github.com:$LIBSODIUM_URL_PATH" -LIBSODIUM_GIT_URL="$LIBSODIUM_GIT_URL_HTTPS" - -# The branch to check out. -LIBSODIUM_BRANCH=stable - -# The distribution to build. -DIST=native - -# The directory in which to install the built libsodium. -INSTALL_PREFIX="$SCRIPT_DIR/libsodium" - -# The directory in which to download and build libsodium. -LIBSODIUM_BUILD_DIR="$SCRIPT_PARENT_DIR/libsodium" - -# Exit on error -set -e - -usage_error() { - echo "${0##*/}: $*" >&2 - usage >&2 - exit 1 -} - -fatal() { - echo "${0##*/}: $*" >&2 - exit 1 -} - -# Parse the command-line, preserving all the arguments for later reference. -PRESERVED_ARGS=() -OPT_UPDATE=true -OPT_CLEAN=true -MAKE_ARGS=() -while [ $# -ne 0 ]; do - opt="$1" - shift - case "$opt" in - -h | --help | '-?' ) - usage - exit 0 - ;; - --src=*) - PRESERVED_ARGS+=("$opt") - LIBSODIUM_BUILD_DIR="${opt#*=}" - ;; - --src) - [ $# -ge 1 ] || usage_error "missing argument after $opt" - PRESERVED_ARGS+=("$opt" "$1") - LIBSODIUM_BUILD_DIR="$1" - shift - ;; - --branch=*) - PRESERVED_ARGS+=("$opt") - LIBSODIUM_BRANCH="${opt#*=}" - ;; - --branch) - [ $# -ge 1 ] || usage_error "missing argument after $opt" - PRESERVED_ARGS+=("$opt" "$1") - LIBSODIUM_BRANCH="$1" - shift - ;; - --prefix=*) - PRESERVED_ARGS+=("$opt") - INSTALL_PREFIX="${opt#*=}" - ;; - --prefix) - [ $# -ge 1 ] || usage_error "missing argument after $opt" - PRESERVED_ARGS+=("$opt" "$1") - INSTALL_PREFIX="$1" - shift - ;; - --ssh) - PRESERVED_ARGS+=("$opt") - LIBSODIUM_GIT_URL="$LIBSODIUM_GIT_URL_SSH" - ;; - --no-update) - OPT_UPDATE=false - ;; - --no-clean) - OPT_CLEAN=false - ;; - --dist=*) - PRESERVED_ARGS+=("$opt") - DIST="${opt#*=}" - ;; - --dist) - [ $# -ge 1 ] || usage_error "missing argument after $opt" - PRESERVED_ARGS+=("$opt" "$1") - DIST="$1" - shift - ;; - --make-arg=*) - PRESERVED_ARGS+=("$opt") - MAKE_ARGS+=("${opt#*=}") - ;; - --make-arg) - [ $# -ge 1 ] || usage_error "missing argument after $opt" - PRESERVED_ARGS+=("$opt" "$1") - MAKE_ARGS+=("$1") - shift - ;; - -*) - usage_error "unrecognised option: $1" - ;; - *) - usage_error "spurious argument: $1" - ;; - esac -done - -abspath() { - case "$1" in - /*) echo "$1";; - .) echo "$PWD";; - *) echo "$PWD/${1#./}";; - esac -} - -if [ ! -d "$LIBSODIUM_BUILD_DIR" ]; then - echo "Create $LIBSODIUM_BUILD_DIR" - mkdir -p "$LIBSODIUM_BUILD_DIR" -fi - -LIBSODIUM_INSTALL_ABSDIR="$(abspath "$INSTALL_PREFIX")-$DIST" - -is_libsodium_downloaded() { - [ -r "$1/src/libsodium/include/sodium.h" -a \ - -r "$1/libsodium-uninstalled.pc.in" \ - ] -} - -if ! is_libsodium_downloaded "$LIBSODIUM_BUILD_DIR"; then - echo "Download libsodium from $LIBSODIUM_GIT_URL..." - git clone --branch "$LIBSODIUM_BRANCH" "$LIBSODIUM_GIT_URL" "$LIBSODIUM_BUILD_DIR" - cd "$LIBSODIUM_BUILD_DIR" >/dev/null - is_libsodium_downloaded . || fatal "Download did not produce expected source files" -else - echo "Libsodium appears to already be downloaded" - cd "$LIBSODIUM_BUILD_DIR" >/dev/null - git checkout "$LIBSODIUM_BRANCH" - if $OPT_UPDATE; then - echo "Update from" $(git remote get-url origin) - git pull --ff-only origin "$LIBSODIUM_BRANCH" - fi -fi - -if [ -d "$LIBSODIUM_INSTALL_ABSDIR" ]; then - echo "Delete the previous installation" - rm -rf "$LIBSODIUM_INSTALL_ABSDIR" -fi - -if $OPT_CLEAN; then - echo "Clean the previous build" - git clean -f -d -x -fi - -if [ ! -x configure ]; then - echo "No configure script present; run autogen.sh..." - ./autogen.sh -fi - -if [ "$DIST" = native ]; then - echo "Native build..." - [ -r Makefile ] || ./configure --prefix="$LIBSODIUM_INSTALL_ABSDIR" - make -j3 "${MAKE_ARGS[@]}" check - make -j3 "${MAKE_ARGS[@]}" install -elif [ -x "dist-build/$DIST.sh" ]; then - installed="libsodium-$DIST" - case "$DIST" in - arm) installed="libsodium-armv6";; - esac - [ -e "$installed" ] && fatal "previous build remains in $installed" - echo "Build using 'dist-build/$DIST.sh'..." - "dist-build/$DIST.sh" - [ -d "$installed" ] || fatal "build did not produce $installed" - echo "Copy built installation into $LIBSODIUM_INSTALL_ABSDIR" - cp -R -p "$installed" "$LIBSODIUM_INSTALL_ABSDIR" -else - fatal "script "dist-build/$DIST_BUILD.sh" does not exist." -fi - -# Create a shell script that will set up the environment to use the built and -# installed libsodium. -cat >"$LIBSODIUM_INSTALL_ABSDIR/settings.sh" <"$LIBSODIUM_INSTALL_ABSDIR/README.txt" < header file. */ #undef HAVE_LINUX_IF_H @@ -144,9 +141,6 @@ /* Define to 1 if the sinf() function is available. */ #undef HAVE_SINF -/* Define to 1 if you have the header file. */ -#undef HAVE_SODIUM_H - /* Define to 1 if you have the header file. */ #undef HAVE_STDINT_H diff --git a/configure.ac b/configure.ac index 16e46c9f..e7b44c20 100644 --- a/configure.ac +++ b/configure.ac @@ -153,23 +153,10 @@ AC_CHECK_HEADERS( AC_SUBST([HAVE_JNI_H], [$ac_cv_header_jni_h]) -dnl libsodium +dnl The entire libsodium source is in a subdirectory, and has its own configure +dnl script. -m4_define([LIBSODIUM_MESSAGE], [ -*** If your platform provides a libsodium development package, then -*** install it and try again, eg, on Debian/Ubuntu: -*** sudo apt-get install libsodium-dev -*** If using a libsodium built locally from source, ensure that the -*** $1 environment variable is set, eg, with the command: -*** . libsodium/settings.sh -*** See INSTALL.md and doc/Development.md for more information. -]) - -AC_CHECK_HEADERS( - sodium.h,, [AC_MSG_ERROR([missing sodium.h LIBSODIUM_MESSAGE([C_INCLUDE_PATH])])] -) - -AC_CHECK_LIB(sodium, sodium_init,, [AC_MSG_ERROR([missing libsodium LIBSODIUM_MESSAGE([LIBRARY_PATH])])]) +AC_CONFIG_SUBDIRS([libsodium]) dnl Check if the Linux gettid() and tgkill() system calls are supported. AC_CHECK_FUNCS([gettid tgkill]) diff --git a/doc/Development.md b/doc/Development.md index abdb17c7..d9e45d3f 100644 --- a/doc/Development.md +++ b/doc/Development.md @@ -1,6 +1,6 @@ Notes for Serval DNA Developers =============================== -[Serval Project][], September 2016 +[Serval Project][], September 2017 Introduction ------------ @@ -8,8 +8,8 @@ Introduction This document is intended for all developers of [Serval DNA][], and also for non-developers who are experiencing errors in the [build][] process. -Autoconf --------- +Autotools +--------- The [configure.ac](../configure.ac) file is an [autoconf][] script that contains instructions for adapting the build of Serval DNA to different @@ -30,13 +30,12 @@ directory. In turn, it then includes this `aclocal.m4` file when invoking [GNU M4][] to convert the [configure.ac](../configure.ac) file into the `./configure` script. -Internally, [autoconf][] generates the `aclocal.m4` file by invoking the -[aclocal][] utility. Used without arguments, [aclocal][] may emit some warning -messages that look like this: +When invoked without arguments, all [autoreconf][] versions up to 2.69 will +emit warning messages like this: $ cd serval-dna $ aclocal - configure.ac:18: warning: Unsupported attribute section, the test may fail + configure.ac:19: warning: Unsupported attribute section, the test may fail ../../lib/autoconf/lang.m4:224: AC_LANG_SOURCE is expanded from... ../../lib/autoconf/lang.m4:241: AC_LANG_PROGRAM is expanded from... ../../lib/autoconf/lang.m4:193: AC_LANG_CONFTEST is expanded from... @@ -46,21 +45,30 @@ messages that look like this: ../../lib/autoconf/general.m4:2042: AC_CACHE_VAL is expanded from... ../../lib/autoconf/general.m4:2063: AC_CACHE_CHECK is expanded from... /usr/share/aclocal/ax_gcc_var_attribute.m4:57: AX_GCC_VAR_ATTRIBUTE is expanded from... - configure.ac:18: the top level + configure.ac:19: the top level $ -These messages are harmless; the correct `aclocal.m4` is still generated. To -suppress most of these messages from the output of [aclocal][] and -[autoreconf][], give the `-I m4` option: +These messages mean that the generated `aclocal.m4` file does not contain the +M4 macros provided by Serval DNA, but instead contains the default ones that +from the [autoconf macro archive][]. As a consequence, the `./configure` +script may not test for certain features correctly, leading to compilation +failure or subtle bugs. + +The correct way to invoke [autoreconf][] versions 2.69 or earlier is to give +the `-I m4` option, which will eliminate the warnings: $ cd serval-dna $ autoreconf -f -i -I m4 $ +The `-I m4` option should be unnecessary in [autoreconf][] versions 2.70 and +later, because they will deduce it from the `AC_CONFIG_MACRO_DIR([m4])` +directive near the top of `configure.ac`. + libsodium --------- -**Libsodium** is the cryptographic library that [Serval DNA][] uses for all its +[Serval DNA][] uses the [libsodium][] cryptographic library for elliptic curve encryption, authentication and secure hashing. [libsodium][] is a portable, cross-compilable fork of [NaCl][], with a @@ -72,64 +80,37 @@ not describe the specific origins of certain constants. Despite the emphasis on higher security, NaCl-libsodium primitives are faster across-the-board than most implementations of the NIST standards. - * In order to [build][] Serval DNA, the libsodium development files must be - available, such as the source header file `` and the static - library `libsodium.a`. +A copy of the libsodium source code is embedded within the Serval DNA source +code under the `libsodium` subdirectory, using [git subtree][]. Developers do +not need to take any special steps to compile or install this libsodium source +code, because Serval DNA does it automatically: - * In order to **run** Serval DNA, the libsodium run-time files must be - available, such as the dynamic library `libsodium.so`. +* the [autoreconf][] command automatically recurses into the libsodium + subdirectory; +* [Autotools](#autotools) `./configure` script automatically runs the + `libsodium/configure` script; +* the `make` command automatically recurses into the libsodium directory *the + first time that it is run*. -### libsodium-dev +The Serval DNA build system has not been set up to facilitate development of +the libsodium source code itself. The Serval DNA `make` command will only +recurse into the libsodium directory the first time it runs. If a developer +subsequently alters a libsodium source file, he/she must run `cd libsodium; +make` manually to compile it, then run the Serval DNA `make`, which will +recompile the entire Serval DNA source code. -On systems that provide a libsodium development package, simply install that -package. For example, on [Debian][] and [Ubuntu][]: +Upgrading libsodium +------------------- - $ sudo apt-get install libsodium-dev - Reading package lists... Done - Building dependency tree - Reading state information... Done - The following NEW packages will be installed: - libsodium-dev - 0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded. - Selecting previously unselected package libsodium-dev:amd64. - (Reading database ... 223406 files and directories currently installed.) - Preparing to unpack .../libsodium-dev_1.0.11-1_amd64.deb ... - Unpacking libsodium-dev:amd64 (1.0.11-1) ... - Setting up libsodium-dev:amd64 (1.0.11-1) ... +To upgrade the embedded [libsodium](#libsodium) source code to a later version, +for example to its (hypothetical) tag `1.0.77`: + + $ cd serval-dna + $ git subtree pull --prefix libsodium git@github.com:jedisct1/libsodium.git \ + --squash 1.0.77 --message 'Merge libsodium 1.0.77' $ -### build-libsodium.sh - -On systems that do not provide a libsodium development package (even though -they may provide a run-time package), such as [Mac OS X](#apple-mac-os-x), the -libsodium development files can be locally installed by downloading the -[libsodium source code][] and building it. The -[build-libsodium.sh](../build-libsodium.sh) script will do this: - - $ ./build-libsodium.sh - ... - The libsodium run-time and development files have been installed in: - /absolute/path/name/serval-dna/libsodium - - To use this installation of libsodium, set up the environment using the - shell's "dot" command to source its settings.sh script, for example: - - . libsodium/settings.sh ; ./configure - - $ - -In the event of failure, check that: - -* the [github.com][libsodium source code] web site can be reached -* there is at least 60 MB of available disk space -* there is no other libsodium development package already installed - -For more information, refer to the [libsodium installation documentation][] -and the script's help message: - - $ ./build-libsodium.sh --help - ... - $ +Beware: Git does not support rebasing of subtree merge commits. Debian/Ubuntu ------------- @@ -137,20 +118,19 @@ Debian/Ubuntu A single [apt-get][] command will install all mandatory and testing dependencies before building on [Debian][] and [Ubuntu][] systems: - $ sudo apt-get --yes install libc6-dev libsodium-dev jq curl + $ sudo apt-get --yes install libc6-dev jq curl Reading package lists... Done Building dependency tree Reading state information... Done libc6-dev is already the newest version (2.23-5). The following NEW packages will be installed: - curl jq libsodium-dev - 0 upgraded, 3 newly installed, 0 to remove and 0 not upgraded. + curl jq + 0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded. Need to get 544 kB of archives. After this operation, 1,683 kB of additional disk space will be used. Get:1 http://ftp.us.debian.org/debian testing/main amd64 curl amd64 7.50.1-1 [218 kB] Get:2 http://ftp.us.debian.org/debian testing/main amd64 jq amd64 1.5+dfsg-1 [156 kB] - Get:3 http://ftp.us.debian.org/debian testing/main amd64 libsodium-dev amd64 1.0.11-1 [170 kB] - Fetched 544 kB in 1s (304 kB/s) + Fetched 374 kB in 1s (304 kB/s) Selecting previously unselected package curl. (Reading database ... 205089 files and directories currently installed.) Preparing to unpack .../0-curl_7.50.1-1_amd64.deb ... @@ -158,28 +138,14 @@ dependencies before building on [Debian][] and [Ubuntu][] systems: Selecting previously unselected package jq. Preparing to unpack .../1-jq_1.5+dfsg-1_amd64.deb ... Unpacking jq (1.5+dfsg-1) ... - Selecting previously unselected package libsodium-dev:amd64. - Preparing to unpack .../2-libsodium-dev_1.0.11-1_amd64.deb ... - Unpacking libsodium-dev:amd64 (1.0.11-1) ... Setting up jq (1.5+dfsg-1) ... Setting up curl (7.50.1-1) ... - Setting up libsodium-dev:amd64 (1.0.11-1) ... Processing triggers for man-db (2.7.5-1) ... $ Apple Mac OS X -------------- -### libsodium for OS X - -Although the [libsodium][] package is available for Mac OS X using the -[homebrew][] package manager, that package only provides the run-time library, -not the development files; for example, it does not provide the `#include -` header. - -The libsodium development files can be installed locally on Mac OS X using the -[build-libsodium.sh](#build-libsodiumsh) script. - ### Test utilities The [OS X grep(1)][] , [OS X sed(1)][] and [OS X awk(1)][] tools provided by @@ -266,7 +232,7 @@ and may need to be changed for your particular circumstances. ----- **Copyright 2015 Serval Project Inc.** -**Copyright 2016 Flinders University** +**Copyright 2016-2017 Flinders University** ![CC-BY-4.0](./cc-by-4.0.png) Available under the [Creative Commons Attribution 4.0 International licence][CC BY 4.0]. @@ -280,17 +246,16 @@ Available under the [Creative Commons Attribution 4.0 International licence][CC [GNU M4]: http://www.gnu.org/software/m4/m4.html [GCC]: https://gcc.gnu.org/ [Clang]: http://clang.llvm.org/ +[libsodium]: https://libsodium.org/ [NaCl]: https://nacl.cr.yp.to/ [NIST]: https://en.wikipedia.org/wiki/National_Institute_of_Standards_and_Technology [Curve25519]: https://en.wikipedia.org/wiki/Curve25519 [build]: ../INSTALL.md [aclocal]: https://www.gnu.org/software/automake/manual/html_node/aclocal-Invocation.html [autoreconf]: https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/autoconf.html#autoreconf-Invocation +[git subtree]: http://git-memo.readthedocs.io/en/latest/subtree.html [Debian]: http://www.debian.org/ [Ubuntu]: http://www.ubuntu.com/ -[libsodium]: https://download.libsodium.org/doc/ -[libsodium source code]: https://github.com/jedisct1/libsodium -[libsodium installation documentation]: https://download.libsodium.org/libsodium/content/installation/ [apt-get]: https://www.debian.org/doc/manuals/apt-guide/ch2.en.html [OS X grep(1)]: https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/grep.1.html [OS X sed(1)]: https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/sed.1.html