Commit Graph

304 Commits

Author SHA1 Message Date
Jeremy Lakeman
4aeb6545ad Refactor storage space measurement to expose new status API
Change semantics of fake space limit to exclude used space, otherwise the result will keep changing on each call
2018-06-26 16:41:27 +09:30
Andrew Bettison
f91033820b Use explicit serverMode enum values everywhere 2018-06-01 17:40:09 +09:30
Andrew Bettison
b8ddad6ff2 Move Rhizome function declaration from server.h to rhizome.h and rename 2018-06-01 10:13:01 +09:30
Andrew Bettison
faa0f392c5 Improve Rhizome DB path configuration
Change the default 'rhizome.datastore_path' to "rhizome", ie, a sub-
directory of the instance directory.

Add a new erename() operation to "os.h".

Clean up the code around the fix 39f582cca (two commits ago) for a bug
introduced in 2014: the 'rhizome.db' file was always being created in
the instance directory, regardless of the setting of the
'rhizome.datastore_path' config option.

Whenever the Rhizome database is opened (by the server or by a CLI
invocation), if the database file does not exist but there is a
'rhizome.db' file in the legacy location, then move that file (and any
accompanying "blob" and "hash" sub-directories) to the configured
location.

Update copyright notices.
2018-04-17 12:01:02 +09:30
Jeremy Lakeman
39f582ccab Auto move rhizome store when config is first set 2018-04-10 16:50:09 +09:30
Jeremy Lakeman
59b6a70374 Use config to set and remember rhizome database location 2018-04-10 10:52:44 +09:30
Jeremy Lakeman
57767e444d Keep rhizome's external blobs in subfolders, migrating each file on first use 2018-04-10 10:52:44 +09:30
Andrew Bettison
c35af0617a Fix header files for Xcode iOS build
- make "debug.h" a private header so that the DEBUG() macro does not
  interfere with Xcode Debug builds, which set DEBUG=1
- move all #include "debug.h" from headers into .c files
- move 'struct idebug' into a new public header "idebug.h" so that
  "log.h" can be public
- move HTTPD port number defs from "httpd.h" to "constants.h", so that
  "httpd.h" can be private and "conf.h" can be public
- add missing nested includes so each header compiles stand-alone
  without error
- #include "sodium.h" instead of <sodium.h>
- #include "sqlite3.h" instead of <sqlite3.h>
- add header guard to "fifo.h"
- fix header guard in "sync_keys.h"
2018-03-06 15:29:29 +10:30
Andrew Bettison
92fa6c196a Rewrite logging system
Rename the logging primitive functions and utility functions, prefixing
all with 'serval_log', eg: logMessage() -> serval_logf() etc.

Add an XPRINTF xhexdump() function and use it to implement the
serval_log_hexdump() utility, renamed from dump().  Add macros
WHY_dump(), WARN_dump(), HINT_dump() and DEBUG_dump(), and use them
everywhere.

Remove the 'log.console.dump_config' and 'log.file.dump_config'
configuration options; configuration is now dumped in every log prolog.

The logging system now constructs the log prolog by invoking the new
'log_prolog' trigger, so that it no longer depends on the version string
and configuration system.  Any system that wants to present a message in
the log prolog can define its own trigger, which calls standard log
primitives to print the message.

Split the logging system into a front-end (log.c) that provides the
logging primitives and is independent of the configuration system, and a
set of back-end "outputters" (log_output_console.c, log_output_file.c,
log_output_android.c) that may depend on the configuration system and
are decoupled from the front-end using the 'logoutput' link section.

These log outputters are explicitly linked into executables by the
Makefile rules, but could also be linked in using USE_FEATURE().  The
USE_FEATURE() calls have _not_ been added to servald_features.c, so that
different daemon executables can be built with the same feature set but
different log outputs.
2018-03-06 15:16:56 +10:30
Andrew Bettison
34e2e8d4bc Improve and document HTTP REST Rhizome import
The REST Rhizome import request now requires the 'id' and 'version'
query parameters to either both be supplied or neither, and fails if
they do not match the manifest that is supplied in the request body.
Added a test case for this.

Added a test case to ensure that if the 'id' and 'version' query
parameters cause a hit (already in store) then the response is sent
immediately without reading the request body.

Improve the documentation for the REST Rhizome import request.
2017-10-20 15:50:18 +10:30
Andrew Bettison
159c731e9e Add SERVAL_ prefix to UUID symbols to avoid collisions on OS-X 2017-10-09 11:41:44 +10:30
Jeremy Lakeman
e158a38137 Clean up a number of memory leaks, as revealed by clang's sanitiser 2017-09-11 14:09:27 +09:30
Andrew Bettison
093bdfdcd3 Add FALLTHROUGH macro to silence -Wimplicit-fallthrough in GCC7 2017-09-11 12:43:38 +09:30
Jeremy Lakeman
8cdd849c3e Return STATUS_OLD from is_interesting 2017-05-17 14:24:28 +09:30
Jeremy Lakeman
a473304c06 Add restful import API 2017-05-17 10:16:46 +09:30
Jeremy Lakeman
4b98def664 Add newsince request for incoming activity 2017-04-19 14:46:37 +09:30
Jeremy Lakeman
2f60b8417f Test and fix rhizome clean verify 2017-04-19 14:46:36 +09:30
Jeremy Lakeman
585e573ecf Refactor rhizome storage methods to return database busy status, and pause rhizome sync 2017-03-06 16:12:55 +10:30
Jeremy Lakeman
2540c9e633 Cause rhizome add to poke the daemon and trigger all bundle add notifications 2017-03-06 16:12:55 +10:30
Jeremy Lakeman
1e70bcaac6 Clean up some more warnings 2016-11-21 10:34:12 +10:30
Andrew Bettison
6d2eb268ed Add 'cmd_cleanup' trigger
The new 'cmd_cleanup' trigger replaces the old command_cleanup()
function, which was causing linking problems on OS X and inverted strict
dependency.  The keyring cmd_cleanup calls keyring_free(global_keyring),
instead of merely asserting keyring == NULL, so the error exit cases of
many CLI functions have been simplified.
2016-11-14 11:28:03 +10:30
Jeremy Lakeman
b927af79a8 Introduce new typedef for signing keys to expose their internal structure 2016-09-27 10:28:46 +09:30
Andrew Bettison
e34141a2a5 Upgrade to SQLite 3.14.2
Ignores a variety of warnings when compiling sqlite3.c.

Uses the new sqlite3_trace_v2() function for logging SQL statements, now
that sqlite3_trace() and sqlite3_profile() are deprecated.

Fixes a rhizomeops test failure caused because SQLite 3.12 increased the
default block size from 1024 to 4096, causing the fixed overhead of an
empty Rhizome db to exceed 32KB.  This caused the test to fail because
the first "file add" failed.  Now the test fixture adapts to the fixed
overhead.

Deletes the SQLite 3.10.2 source amalgamation.
2016-09-21 22:47:35 +09:30
Jeremy Lakeman
a854105598 Support rhizome newsince request on an empty store 2016-07-25 16:54:11 +09:30
Jeremy Lakeman
89ab832c21 Link multiple versions of a journal payload to the same file 2016-05-24 14:47:49 +09:30
Jeremy Lakeman
177f695671 Rework handling of race conditions when storing the same bundle (#109) 2016-05-23 15:38:40 +09:30
Jeremy Lakeman
c5957e9c85 Transfer bundles via msp, triggered by key sync 2016-05-09 11:10:50 +09:30
Jeremy Lakeman
6b7aa800e4 Start syncing rhizome based on manifest hashes 2016-05-09 11:10:50 +09:30
Jeremy Lakeman
e15e332072 Swap from included nacl to host libsodium 2016-05-09 10:32:04 +09:30
Jeremy Lakeman
2efcfa24d5 Only trigger bundle add hooks within the daemon thread / process 2016-03-30 17:18:40 +10:30
Jeremy Lakeman
edc1658cfd Rework detection of rhizome add from cli to trigger bundle_add 2016-03-08 13:33:23 +10:30
Jeremy Lakeman
3cacf63eaa Add sqlite column for the hash of the manifest (before any signatures) 2016-03-07 14:34:53 +10:30
Jeremy Lakeman
2da3f12cfa Expose more rhizome add command line arguments to the Java API 2016-02-15 15:34:29 +10:30
Jeremy Lakeman
ef208c9932 Workaround stack alignment bugs when compiling with clang 2016-01-27 16:27:47 +10:30
Andrew Bettison
bbc7f8fbce Remove terminating semicolons from within macros
The ALARM_STRUCT() and DECLARE_TRIGGER() macros included terminating
semicolons, but their invocation should always be followed by a
semicolon, otherwise etags(1) gets confused, so their included semicolon
was redundant.
2015-11-02 10:11:41 +10:30
Andrew Bettison
a060642fdb Add strbuf_local_buf() macro 2015-11-02 10:11:41 +10:30
Andrew Bettison
6123503c15 Improve Rhizome HTTP status codes
Also improve some reason phrases and test them
2015-09-21 18:43:24 +09:30
gardners
dab244d92f shorten section names (and precursors) to keep in the 16 char limit
of the O-MACH assembler on OSX.
2015-08-24 10:17:04 +09:30
Andrew Bettison
7d9a5faa4e Move if(config.debug.xxx) tests into DEBUGF()
Original DEBUG() and DEBUGF() macros renamed to _DEBUG() and _DEBUGF()
New DEBUG() and DEBUGF() macros, first argument is flagname
New DEBUGF2(foo, bar, ...) macro does if(config.debug.foo||config.debug.bar) test
Replace almost all config.debug.xxx references to IF_DEBUG(xxx)
2015-07-13 16:00:05 +09:30
Andrew Bettison
008f296026 Generic trigger functions "trigger.h" 2015-06-02 02:23:06 +09:30
Jeremy Lakeman
51ed6162dd Add support for running and stopping servald from a JVM thread
fdpoll will now run a callback when about to sleep / woke up.
A new Java interface to indicate server started / sleeping / waking up.
An android implementation may allow the CPU to sleep.
If there's a java exception the server will try to shutdown.
Calling servald stop is currently undefined.
2015-05-25 11:46:37 +09:30
Andrew Bettison
a23c56626d Update some copyright notices 2014-11-20 00:01:12 +10:30
Andrew Bettison
6d0c56b62a Fix MeshMS failure caused by recent changes
Commit 65d6005 changed the code that creates a new manifest from
a bundle secret, and broke MeshMS
2014-11-19 22:59:59 +10:30
Andrew Bettison
3c4703feb4 Reduce DB locking issues in RESTful tests
Add environment variable SERVALD_RHIZOME_DB_RETRY_LIMIT to extend the
Rhizome DB retry period, set it to 60 seconds in RESTful tests to
override the default 5 seconds
2014-11-12 23:08:19 +10:30
Jeremy Lakeman
2c72875508 Ensure sql lock failures are returned. 2014-09-12 15:24:23 +09:30
Andrew Bettison
7e1fbe8c76 Refactor to fix some OS-X compile warnings
Fixes two minor bugs that escaped notice until now
2014-08-14 17:39:44 +09:30
Jeremy Lakeman
eaf069e1c3 Dont treat database lock errors the same as missing records 2014-07-14 15:26:50 +09:30
Andrew Bettison
606f087dd5 Merge latest Rhizome Java API into development 2014-07-11 12:29:24 +09:30
Andrew Bettison
cf43635789 Improve Rhizome HTTP RESTful interface
Add RHIZOME_BUNDLE_STATUS_READONLY enum value

Tighten up switch statements on bundle and payload status enums (no
default labels)

Rename some recently added enum entries

Return bundle status and payload status in HTTP responses

Add test for failing to decrypt a foreign encrypted bundle payload,
fix bug that caused an assertion failure

Add tests for fetching a non-existent manifest and fetching bundles
whose payload blob is not in the store
2014-07-10 06:29:51 +09:30
Jeremy Lakeman
63226143ac Don't log an error when MDP fetch fails due to missing bundle 2014-07-07 11:31:09 +09:30