Commit Graph

3876 Commits

Author SHA1 Message Date
Paul Gardner-Stephen
ffeb25aa67 remove C99-ism 2018-04-18 16:45:51 +09:30
Paul Gardner-Stephen
504302aea1 pass exec_prefix to libsodium recursed make 2018-04-18 16:45:51 +09:30
Andrew Bettison
faa0f392c5 Improve Rhizome DB path configuration
Change the default 'rhizome.datastore_path' to "rhizome", ie, a sub-
directory of the instance directory.

Add a new erename() operation to "os.h".

Clean up the code around the fix 39f582cca (two commits ago) for a bug
introduced in 2014: the 'rhizome.db' file was always being created in
the instance directory, regardless of the setting of the
'rhizome.datastore_path' config option.

Whenever the Rhizome database is opened (by the server or by a CLI
invocation), if the database file does not exist but there is a
'rhizome.db' file in the legacy location, then move that file (and any
accompanying "blob" and "hash" sub-directories) to the configured
location.

Update copyright notices.
2018-04-17 12:01:02 +09:30
Jeremy Lakeman
9ba27b28a0 Possible race condition in command cleanup due to linking order, (Fixes #136) 2018-04-16 09:36:17 +09:30
Jeremy Lakeman
39f582ccab Auto move rhizome store when config is first set 2018-04-10 16:50:09 +09:30
Jeremy Lakeman
5e0e59e8b8 Fix tests/rhizomeall 2018-04-10 10:52:44 +09:30
Jeremy Lakeman
35fefb1680 Fix compilation and real interfaces on android 2018-04-10 10:52:44 +09:30
Jeremy Lakeman
59b6a70374 Use config to set and remember rhizome database location 2018-04-10 10:52:44 +09:30
Jeremy Lakeman
57767e444d Keep rhizome's external blobs in subfolders, migrating each file on first use 2018-04-10 10:52:44 +09:30
Andrew Bettison
a691a39870 Add Route Swift API
Uses the Route REST API internally.

Create the 'allswift' test script that aggregates the Keyring and Route
Swift API tests.
2018-04-09 14:43:37 +09:30
Andrew Bettison
dce8b378d2 Refactor routing and Swift test set-up
Add some servald config set-up functions to 'testdefs_routing.sh' and
'testdefs_swift.sh' that can be re-used in the Swift Routing API tests.

Improve some commenting on existing functions, for consistency.
2018-04-09 09:22:36 +09:30
Andrew Bettison
12bc773d4d Refactor Keyring Swift API
Factor JSON table parsing into a separate method of ServalRestfulClient
so it can be re-used by other APIs.
2018-04-09 07:50:42 +09:30
Andrew Bettison
bc637e1a5c Update Swift API to use new Keyring REST API
Now uses the correct, documented REST requests (not the legacy ones) and
includes a single-identity query.
2018-04-06 08:51:31 +09:30
Andrew Bettison
e349c8fc9c Remove -Wimplicit-fallthrough #pragma from sqlite3.c (fixes #128)
Replace it with the -Wno-implicit-fallthrough CFLAGS_SQLITE option in
configure.ac, so it no longer causes compile errors with older compilers
that don't support that warning.
2018-04-05 21:21:26 +09:30
Andrew Bettison
189485a513 Add Route Java API 2018-04-05 18:12:59 +09:30
Andrew Bettison
cf9e0b4730 Keep AF_UNIX and file-based interfaces up whenever config re-loads
Whenever the daemon re-loads its config, it keeps any socket-based
(AF_INET) SOCK_DGRAM interfaces open that are matched by the new config,
but it used to close and then re-open all local (AF_UNIX) interfaces and
file-based interfaces such as SOCK_FILE dummy files and SOCK_STREAM
device files.  This made it very difficult to develop the new
'routejava' test suite, because the Java API test harness always causes
a config re-load when it sets the REST API user/password in the config,
which caused the local socket interfaces to bounce, which interfered
with the reachability of nodes.

Now, local socket and file-based interfaces remain up after a config
re-load, as long as they are still matched by the new configuration.

Added INFO messages for interfaces that remain up after a config
re-load, and tweak interface up/down INFO messages to be consistent.
2018-04-05 18:12:23 +09:30
Andrew Bettison
1930c51da4 Minor clean-ups 2018-04-04 12:22:26 +09:30
Andrew Bettison
d1e113d4f3 Fix 'servaldswift' link error 2018-04-04 12:22:07 +09:30
Jeremy Lakeman
5c2f553ed2 Change Android.mk to only compile a single .a file
This will require any android application to include their own shared library rules similar to the following;

include $(CLEAR_VARS)
LOCAL_STATIC_LIBRARIES := servaldstatic
LOCAL_C_INCLUDES += $(LOCAL_PATH)/serval-dna
LOCAL_SRC_FILES := $(LOCAL_PATH)/features.c
LOCAL_MODULE := servaldaemon
LOCAL_LDLIBS += -L$(SYSROOT)/usr/lib -llog
include $(BUILD_SHARED_LIBRARY)

Allowing just the set of required features to be linked into the final binary
2018-04-03 16:41:10 +09:30
Jeremy Lakeman
4809d7d585 Link java api into shared library by specifying features 2018-04-03 16:41:10 +09:30
Jeremy Lakeman
d844b0f6bd Reduce coupling with console, dna_helper, monitor & directory_client features 2018-04-03 14:32:06 +09:30
Jeremy Lakeman
331e0ae576 Add missing include for bzero 2018-04-03 09:44:28 +09:30
Andrew Bettison
8345d896a6 Add Route REST API (fixes #96)
Consists of a single, one-shot request, GET /restful/route/all.json, for
the time being.  A "newsince" request can be added later.

Add a Markdown tech doc that specifies the new Route REST API and update
the MDP tech doc a little.

Add the 'routerestful' test script, which uses some test utility
fuctions that have been factored out of the 'routing' test script into
the new testdefs_routing.sh.

Add the new 'allrestful' test script.
2018-03-29 15:31:03 +10:30
Andrew Bettison
1a091aa8a1 Expose daemon's primary SID in 'proc/primary_sid'
This helps tests distinguish which of a daemon's keyring identities
is used as its primary identity, which is not otherwise obvious
without consulting the routing table, because slots are allocated
in random order.
2018-03-29 15:19:54 +10:30
Andrew Bettison
798e34cc5b Add strbuf_json_integer() 2018-03-29 15:19:54 +10:30
Andrew Bettison
c7a2fb4573 Add nibble-tree iterator
The new struct tree_iterator and associated start/get/next/free
functions replace the recursive walk() function, removing the need for a
callback when iterating over all nodes in the tree, and allowing
iteration to be suspended while other pseudo-threads are run.  This
allows an HTTP REST request to keep a tree_iterator in its state struct
and potentially simplifies other areas of the code.

The iterator free()s any empty internal tree nodes that it encounters,
as did the original tree_walk() function.  To support the existence of
multiple iterators at once, a reference count has been added to the
tree_node struct, to prevent any iterator from free()ing a node while
any other iterators point to it; only the last iterator to pop out of an
empty node will free() it.

The tree_walk() and tree_walk_prefix() functions have been
re-implemented to use an iterator state object internally.  This
resolves an outstanding TODO to perform tree-node freeing during a
prefix walk, and simplifies the code considerably.

Renamed some function parameters and struct members to make the
nibble-tree API a little more self-explanatory.

Added a nibble-tree test to the 'serval-tests' utility.
2018-03-29 15:19:54 +10:30
Andrew Bettison
2ef315b692 Fix tfw_createfile utility
Was not omitting characters given by the --omit option.
2018-03-29 15:19:54 +10:30
Andrew Bettison
2e0cb4e6dc Improve test framework error handling
If runTests() terminated with a FATAL condition, the cause was not
always apparent, because the error message usually got overwritten by
the test progress output on the user's terminal.  To fix this, the main
loop's standard error is now collected in a temporary file which is sent
to standard error just before exit.

The test framework was not always handling its internal FATAL error code
(255) correctly, so this has been fixed.

One cause of a FATAL termination was if a test's unique temporary
directory, which was based on its Process ID, already existed.  Anything
that left a temporary directory behind increased the likelihood of a
FATAL in a subsequent test run.  (For example, one keyring test case was
not killing its Serval DNA daemon, and the daemon was re-creating its
instance directory before eventually terminating itself.  This test case
has been fixed.)

The test framework now allocates its temporary directory by re-trying
different random numbers until mkdir(1) succeeds, and only fataling
after 20 failures.
2018-03-29 15:09:45 +10:30
Andrew Bettison
288fae4f2b Use enum for keyring key type codes 2018-03-29 12:20:56 +10:30
Jeremy Lakeman
3388170468 Don't allow a delegate log to be opened if it isn't used 2018-03-28 15:42:18 +10:30
Jeremy Lakeman
65d0e3a47c Remove implementation specific Java hack and allow POST verb for keyring set command 2018-03-28 15:42:18 +10:30
Andrew Bettison
4f5567e2c1 Use rigorous quoting throughout test scripts
This makes it much easier to refactor test fixtures and utilities.
2018-03-27 17:29:21 +10:30
Andrew Bettison
1936015ac7 Fix Rhizome protocol "ManyFiles" test
Increase the timeout from 30 to 240 seconds.  Reduce debug logging.  Fix
bundle authors.
2018-03-27 17:29:21 +10:30
Andrew Bettison
b822378c37 Add a precautionary assert to test function 'transform_list_json' 2018-03-27 17:29:21 +10:30
Andrew Bettison
a82babf9db Refactor test setup function 'create_single_identity'
Now produces exactly the same output variables as 'create_identities 1'
so that these two functions can be interchanged with minimal impact
on the test script.
2018-03-27 17:29:21 +10:30
Andrew Bettison
f8850535ad Improve the test framework
The executeOk() function should dump stderr to the log in event of
failure, but execute() need not.
2018-03-27 17:29:21 +10:30
Andrew Bettison
bc7f7aeaaf Refactor all REST API tests
Rename 'testdefs_json.sh' to 'testdefs_rest.sh'; add setup functions and
rest_request() function, and roll out through existing test scripts.
2018-03-27 17:29:21 +10:30
Andrew Bettison
9ef7d33d69 Log more debug in MDP_ROUTE_TABLE request 2018-03-27 09:06:41 +10:30
Jeremy Lakeman
3cbf60f2b7 Improve error messages when locating zip file comment 2018-03-26 15:03:28 +10:30
Jeremy Lakeman
240f69595b Remove dump_config from android specific code 2018-03-26 15:03:28 +10:30
Jeremy Lakeman
ad2f0364cf Add content type class for sending headers & matching responses
Note that android doesn't include javax so we can't use it.
2018-03-26 14:45:36 +10:30
Jeremy Lakeman
262df2804d By default clang only logs warnings for unsupported command line args 2018-03-26 14:45:36 +10:30
Andrew Bettison
3a56355111 Add 'Location' header to HTTP server 301 and 302 responses 2018-03-19 18:07:33 +10:30
Andrew Bettison
007a7af122 Update Java API to use new Keyring REST API
The ServalDCient class now requires a "verb" parameter as well as the
URL parameter.

Add Java API method to query a single keyring identity.
2018-03-19 18:07:33 +10:30
Andrew Bettison
98ec1c9608 Redesign the Keyring REST API (fixes #132)
The new API follows REST rules by using the proper request verbs:
POST, PUT, PATCH and DELETE, instead of just GET.

The legacy GET-only API is still supported for backward compatibility,
but not longer tested or documented.

Add a new query-single-identity operation.

Implement the lock-single-identity operation, which until now had been
documented but not yet implemented.  Whenever a single identity is
locked (released), any other unlocked identities with the same PIN are
flagged to indicate that the PIN is not "fully" unlocked, so that the
next time the PIN is entered, the slot decryption is re-tried for
non-loaded identities, and the locked identity will be unlocked again.

Update the 'keyring' and 'keyringrestful' test scripts:
- refactored to reduce curl command-line clutter in test cases
- now tests the redesigned request verbs and paths
- added a test for GET /restful/keyring/SID
- added a test for PUT /restful/keyring/SID/lock
2018-03-19 18:06:23 +10:30
Andrew Bettison
8242ca0a00 Fix a bug in MDP_IDENTITY request handling
The LOCK and UNLOCK by PIN requests were acting on the supplied PIN
and also the empty PIN, due to a bug in ob_get_str_ptr(), which returned
an empty string instead of NULL after reaching the end of the string
list.
2018-03-09 16:56:36 +10:30
Andrew Bettison
2333a116f3 Fix a bug in debug hexdumps
Introduced in the recent log system overhaul.
2018-03-09 16:56:36 +10:30
Andrew Bettison
fad187046e Add missing tests/alljava script 2018-03-06 18:31:26 +10:30
Andrew Bettison
c35af0617a Fix header files for Xcode iOS build
- make "debug.h" a private header so that the DEBUG() macro does not
  interfere with Xcode Debug builds, which set DEBUG=1
- move all #include "debug.h" from headers into .c files
- move 'struct idebug' into a new public header "idebug.h" so that
  "log.h" can be public
- move HTTPD port number defs from "httpd.h" to "constants.h", so that
  "httpd.h" can be private and "conf.h" can be public
- add missing nested includes so each header compiles stand-alone
  without error
- #include "sodium.h" instead of <sodium.h>
- #include "sqlite3.h" instead of <sqlite3.h>
- add header guard to "fifo.h"
- fix header guard in "sync_keys.h"
2018-03-06 15:29:29 +10:30
Andrew Bettison
ca8e846264 Add iOS Framework Bundle build
The new ios/configure script performs a separate ../configure for each of
the following iOS targets:

    iPhoneOS        on armv7
    iPhoneOS        on arm64
    iPhoneSimulator on i386
    iPhoneSimulator on x86_64

The script then creates ios/Makefile that builds a static iOS Framework
Bundle suitable for inclusion in an Xcode iOS project.

Add the --xcode-sdk=SDK option to configure.ac, to support cross
compiling using Apple Xcode.  It prefixes all compile/link toolchain
commands with the "xcrun --sdk SDK" command, ie, CC, AS, LD, AR, RANLIB,
etc.

Re-structure headerfiles.mk to separate "public" from "private" headers,
because the Framework module only exposes the public ones.  Moves the
SQLITE3_AMALGAMATION definition from Makefile.in into headerfiles.mk.

Update INSTALL.md and add a technical document for Apple iOS.

This enables development of Serval DNA within the context of an Xcode
iOS project using the standard edit-make-test cycle: after modifying a
Serval DNA source file, "cd ios; make" will recompile the changed file
for all the target architectures and update the Framework Bundle.
Rebuilding the Xcode project will then incorporate the changes, which
can be tested immediately.
2018-03-06 15:29:29 +10:30