Commit Graph

342 Commits

Author SHA1 Message Date
Andrew Bettison
faa0f392c5 Improve Rhizome DB path configuration
Change the default 'rhizome.datastore_path' to "rhizome", ie, a sub-
directory of the instance directory.

Add a new erename() operation to "os.h".

Clean up the code around the fix 39f582cca (two commits ago) for a bug
introduced in 2014: the 'rhizome.db' file was always being created in
the instance directory, regardless of the setting of the
'rhizome.datastore_path' config option.

Whenever the Rhizome database is opened (by the server or by a CLI
invocation), if the database file does not exist but there is a
'rhizome.db' file in the legacy location, then move that file (and any
accompanying "blob" and "hash" sub-directories) to the configured
location.

Update copyright notices.
2018-04-17 12:01:02 +09:30
Jeremy Lakeman
59b6a70374 Use config to set and remember rhizome database location 2018-04-10 10:52:44 +09:30
Jeremy Lakeman
57767e444d Keep rhizome's external blobs in subfolders, migrating each file on first use 2018-04-10 10:52:44 +09:30
Andrew Bettison
c35af0617a Fix header files for Xcode iOS build
- make "debug.h" a private header so that the DEBUG() macro does not
  interfere with Xcode Debug builds, which set DEBUG=1
- move all #include "debug.h" from headers into .c files
- move 'struct idebug' into a new public header "idebug.h" so that
  "log.h" can be public
- move HTTPD port number defs from "httpd.h" to "constants.h", so that
  "httpd.h" can be private and "conf.h" can be public
- add missing nested includes so each header compiles stand-alone
  without error
- #include "sodium.h" instead of <sodium.h>
- #include "sqlite3.h" instead of <sqlite3.h>
- add header guard to "fifo.h"
- fix header guard in "sync_keys.h"
2018-03-06 15:29:29 +10:30
Andrew Bettison
34e2e8d4bc Improve and document HTTP REST Rhizome import
The REST Rhizome import request now requires the 'id' and 'version'
query parameters to either both be supplied or neither, and fails if
they do not match the manifest that is supplied in the request body.
Added a test case for this.

Added a test case to ensure that if the 'id' and 'version' query
parameters cause a hit (already in store) then the response is sent
immediately without reading the request body.

Improve the documentation for the REST Rhizome import request.
2017-10-20 15:50:18 +10:30
Andrew Bettison
159c731e9e Add SERVAL_ prefix to UUID symbols to avoid collisions on OS-X 2017-10-09 11:41:44 +10:30
Jeremy Lakeman
8cdd849c3e Return STATUS_OLD from is_interesting 2017-05-17 14:24:28 +09:30
Jeremy Lakeman
940b8c2d4b Ack incoming feeds for threading 2017-04-19 14:46:36 +09:30
Jeremy Lakeman
a8c29bbb15 Add MeshMB cli commands to follow and ignore feeds 2017-04-19 14:46:36 +09:30
Jeremy Lakeman
650db68a69 Set service of private bundles 2017-03-21 13:09:47 +10:30
Jeremy Lakeman
585e573ecf Refactor rhizome storage methods to return database busy status, and pause rhizome sync 2017-03-06 16:12:55 +10:30
Jeremy Lakeman
4c538a7686 Return payload busy if write fails to flush, treat as an error in most cases 2017-02-28 13:13:09 +10:30
Jeremy Lakeman
c05775a320 Generalise deterministic key generation 2016-11-21 11:09:27 +10:30
Jeremy Lakeman
1e70bcaac6 Clean up some more warnings 2016-11-21 10:34:12 +10:30
Jeremy Lakeman
159188d08c Don't overflow the stack if logging config causes more logging due to using memory diagnostics 2016-10-31 15:23:06 +10:30
Jeremy Lakeman
30aa1c16b8 Set authorship to remote if the sender sid can be generated from the id 2016-10-20 08:51:46 -04:00
Andrew Bettison
71cbe86566 Switch to feature-driven linking
This introduces a new way of linking Serval executables and dynamic
libraries from static libraries like libservald.a -- called
"feature-driven" linking.

The Makefile now links servald and serval-tests from libservald.a,
rather than from an explicit list of object (.o) files.  Thanks to the
section-based method for registering functions such as HTTP handlers,
CLI commands and MDP handlers, these object files had become
"stand-alone" and hence were no longer included in the link because
there was no unresolved reference that required them to be linked in.

The new "feature.h" provides the DECLARE_FEATURE(name) macro that each
stand-alone source file uses to declare the named feature(s) it
provides.  Each executable can call the USE_FEATURE(name) macro in any
of its explicitly-linked source files to cause the corresponding
object(s) to be included in the link, eg, servald_features.c.

The DEFINE_BINDING() macro has been extended so that every individual
MDP binding is given a feature name based on its port number macro, eg,
"mdp_binding_MDP_PORT_ECHO".

Some features have been factored into their own separate source files so
they can be omitted or included in a build independently of each other:
- the MDP bindings for MDP_PORT_DNALOOKUP, MDP_PORT_ECHO,
  MDP_PORT_TRACE, MDP_PORT_KEYMAPREQUEST, MDP_PORT_RHIZOME_xxx,
  MDP_PORT_PROBE, MDP_PORT_STUN, MDP_PORT_STUNREQ
- the CLI "log" and "echo" commands
- the CLI "rhizome direct" command

The JNI source files are only compiled if the <jni.h> header is present,
otherwise they are omitted from libservald.so.
2016-10-19 09:33:01 +10:30
Jeremy Lakeman
ced74d0b6b Simplify BK handling using new signing key typedefs 2016-09-27 15:20:12 +09:30
Jeremy Lakeman
b927af79a8 Introduce new typedef for signing keys to expose their internal structure 2016-09-27 10:28:46 +09:30
Andrew Bettison
a10207f471 Remove unnecessary #includes from "rhizome.h" 2016-09-21 18:47:49 +09:30
Jeremy Lakeman
6a0b7a1da6 Allow for arbitrary extra manifest fields on broadcast messages 2016-08-24 16:33:39 +09:30
Jeremy Lakeman
b2eb3a2199 Begin broadcast messaging service 2016-08-16 15:04:15 +09:30
Jeremy Lakeman
dc7226b6a5 Remove file hash command 2016-08-16 12:22:13 +09:30
Jeremy Lakeman
a854105598 Support rhizome newsince request on an empty store 2016-07-25 16:54:11 +09:30
Jeremy Lakeman
16a14269af Encrypt payloads without needing a sender
- reworked keyring identity handling to reduce memory searching
2016-05-31 12:50:32 +09:30
Jeremy Lakeman
89ab832c21 Link multiple versions of a journal payload to the same file 2016-05-24 14:47:49 +09:30
Jeremy Lakeman
177f695671 Rework handling of race conditions when storing the same bundle () 2016-05-23 15:38:40 +09:30
Jeremy Lakeman
c5957e9c85 Transfer bundles via msp, triggered by key sync 2016-05-09 11:10:50 +09:30
Jeremy Lakeman
6b7aa800e4 Start syncing rhizome based on manifest hashes 2016-05-09 11:10:50 +09:30
Jeremy Lakeman
c625a598d9 Use malloc / free for manifest structs 2016-05-09 11:10:50 +09:30
Jeremy Lakeman
e15e332072 Swap from included nacl to host libsodium 2016-05-09 10:32:04 +09:30
Jeremy Lakeman
5636edfe20 Append manifest into zip file comment 2016-03-30 16:25:43 +10:30
Jeremy Lakeman
edc1658cfd Rework detection of rhizome add from cli to trigger bundle_add 2016-03-08 13:33:23 +10:30
Jeremy Lakeman
231ab257e4 Define internal port bindings with section linking tricks 2016-03-07 15:50:34 +10:30
Jeremy Lakeman
3cacf63eaa Add sqlite column for the hash of the manifest (before any signatures) 2016-03-07 14:34:53 +10:30
Jeremy Lakeman
f2e2b4acb5 Replace nacl & sha methods with versions from libsodium to avoid undefined behaviour 2016-01-27 16:27:04 +10:30
Andrew Bettison
3f8f0f6fc7 Improve Rhizome HTTP API diagnostics
Add RHIZOME_BUNDLE_STATUS_MANIFEST_TOO_BIG enum option to indicate
that the manifest exceeded 8 KiB in size.

Refactor rhizome_add_manifest() and rhizome_manifest_finalise()
to return 'struct rhizome_bundle_result' instead of 'enum
rhizome_bundle_status', so that that their detailed failure messages
can reach the HTTP API layer instead of just being logged.

Fix HTTP response status codes produced Rhizome direct HTTP requests
to be consistent with the Rhizome RESTful API.
2015-12-07 22:35:42 +10:30
Andrew Bettison
98ed0406c5 Refactor Rhizome result handling
- Introduce the new 'struct rhizome_bundle_result' that contains a
  rhizome_bundle_status enum value and an optional, nul-terminated
  string that provides an explanation of the cause of the status; add
  functions to construct, query, and free the struct

- Replace 'enum rhizome_add_result' with 'struct rhizome_bundle_result',
  removing an unnecessary level of enum interpretation

- Make rhizome_fill_manifest() return 'struct rhizome_bundle_result' and
  add logic to check that the supplied author SID is correct (previous
  behaviour was: if an incorrect author SID was supplied but the correct
  author could be found in the keyring, then the incorrect one was
  silently ignored)

- Simplify the response code in rhizome_restful.cc to take advantage of
  the new 'struct rhizome_bundle_result'; in particular, the mapping
  from 'enum rhizome_bundle_status' codes to HTTP status codes is now
  expressed in a single switch statement

- Fix some minor failures in test scripts revealed by the changes
2015-12-01 00:51:15 +10:30
Andrew Bettison
bdb3b99143 Add some comments to clarify Rhizome enums 2015-11-17 00:42:06 +10:30
Andrew Bettison
bbc7f8fbce Remove terminating semicolons from within macros
The ALARM_STRUCT() and DECLARE_TRIGGER() macros included terminating
semicolons, but their invocation should always be followed by a
semicolon, otherwise etags(1) gets confused, so their included semicolon
was redundant.
2015-11-02 10:11:41 +10:30
gardners
dab244d92f shorten section names (and precursors) to keep in the 16 char limit
of the O-MACH assembler on OSX.
2015-08-24 10:17:04 +09:30
Andrew Bettison
008f296026 Generic trigger functions "trigger.h" 2015-06-02 02:23:06 +09:30
Jeremy Lakeman
51ed6162dd Add support for running and stopping servald from a JVM thread
fdpoll will now run a callback when about to sleep / woke up.
A new Java interface to indicate server started / sleeping / waking up.
An android implementation may allow the CPU to sleep.
If there's a java exception the server will try to shutdown.
Calling servald stop is currently undefined.
2015-05-25 11:46:37 +09:30
Andrew Bettison
7734e24006 /restful/rhizome/append on existing journal
Refactor Rhizome insert/append logic into functions used by both CLI and
RESTful API.  Improve RESTful diagnostic messages.
2015-03-30 18:24:12 +10:30
Andrew Bettison
86c3c7312c Better support for Rhizome logging status enums
Provide functions that return non-NULL even on invalid enum
values
2015-03-30 18:19:26 +10:30
Andrew Bettison
da22816784 Add 'rhizome add file --bundle=BID' option
A convenience to re-use an existing manifest, instead of having
to extract the manifest, erase some fields, then pass the result
back into the 'rhizome add file' command
2015-03-28 05:09:38 +10:30
Andrew Bettison
015b4a0b07 Refactor and improve "journal append"
Distinguish between fatal and user-supplied-input errors, return
error descriptive text in a strbuf to allow dynamic content, apply
user-supplied field assgnments/deletions _after_ copying existing
manifest fields, use exit status 4 (invalid manifest) when applying
journal append to a non-journal or vice versa
2015-03-23 17:38:46 +10:30
Andrew Bettison
5b7cfb8afb crypto_sign_compute_public_key() return void not int
It can never fail, so no need to check return value for -1
2015-03-23 17:31:06 +10:30
Andrew Bettison
655b94eb3c Fix failing 'rhizomeops' tests 2015-03-19 03:21:27 +10:30
Andrew Bettison
19119e759c Refactor "rhizome add file" and "rhizome journal append"
Put manifest creation logic into new rhizome_bundle_add_file()
function, in preparation for implementing new HTTP POST
/restful/rhizome/append request

Several 'rhizomeops' tests fail
2015-03-16 22:44:15 +10:30