Commit Graph

68 Commits

Author SHA1 Message Date
Jeremy Lakeman
18bc162361 Add Java API for restful storage information request 2018-06-26 16:41:27 +09:30
Jeremy Lakeman
4aeb6545ad Refactor storage space measurement to expose new status API
Change semantics of fake space limit to exclude used space, otherwise the result will keep changing on each call
2018-06-26 16:41:27 +09:30
Jeremy Lakeman
59b6a70374 Use config to set and remember rhizome database location 2018-04-10 10:52:44 +09:30
Andrew Bettison
c35af0617a Fix header files for Xcode iOS build
- make "debug.h" a private header so that the DEBUG() macro does not
  interfere with Xcode Debug builds, which set DEBUG=1
- move all #include "debug.h" from headers into .c files
- move 'struct idebug' into a new public header "idebug.h" so that
  "log.h" can be public
- move HTTPD port number defs from "httpd.h" to "constants.h", so that
  "httpd.h" can be private and "conf.h" can be public
- add missing nested includes so each header compiles stand-alone
  without error
- #include "sodium.h" instead of <sodium.h>
- #include "sqlite3.h" instead of <sqlite3.h>
- add header guard to "fifo.h"
- fix header guard in "sync_keys.h"
2018-03-06 15:29:29 +10:30
Andrew Bettison
e9d91d04ca Fix failing Rhizome Java API tests
Update the Java API to be consistent with recent changes to the Rhizome
REST API was that regularised the MIME types used in Content-Type
headers.

Add some missing CONTENT_TYPE_ constants to "httpd.h", and ensure they
are consistent with the Java getMimeType() methods in subclasses of
AbstractId.

Add the --stderr option to all executeJavaOk statements in the
'rhizomejava' test script, to reveal the exception message in the case
of failure.
2018-03-06 15:16:56 +10:30
Andrew Bettison
34e2e8d4bc Improve and document HTTP REST Rhizome import
The REST Rhizome import request now requires the 'id' and 'version'
query parameters to either both be supplied or neither, and fails if
they do not match the manifest that is supplied in the request body.
Added a test case for this.

Added a test case to ensure that if the 'id' and 'version' query
parameters cause a hit (already in store) then the response is sent
immediately without reading the request body.

Improve the documentation for the REST Rhizome import request.
2017-10-20 15:50:18 +10:30
Andrew Bettison
29ce8994f2 Improve HTTP Content-Type handling
Undo a change from late 2015 that allowed the format=text+binarysig
parameter of the rhizome/manifest MIME content-type to be optional -- it
is easy to provide that parameter via libcurl, and the lbard code has
just been updated to provide it.

Make Content-Type handling consistent throughout the Serval DNA code by
using 'struct mime_content_type' to represent both parsed (request) and
sent (response) Content-Type fields, replacing ad-hoc in-line string
literals.

Fix some TODOs by adding the "serval/sid", "rhizome/bid" and
"rhizome/bundlesecret" MIME types, which remain optional for the time
being to preserve REST API compatibility with clients that do not set
the Content-Type of their 'bundle-author', 'bundle-id' and
'bundle-secret' parameters.
2017-10-20 09:55:21 +10:30
Andrew Bettison
159c731e9e Add SERVAL_ prefix to UUID symbols to avoid collisions on OS-X 2017-10-09 11:41:44 +10:30
Jeremy Lakeman
87d171cdf7 Reject rhizome import before sending manifest & payload 2017-05-17 14:55:56 +09:30
Jeremy Lakeman
a473304c06 Add restful import API 2017-05-17 10:16:46 +09:30
Jeremy Lakeman
6c4e7de574 Add sender & recipient filters to rhizome restful api 2017-05-08 11:38:20 +09:30
Jeremy Lakeman
4c538a7686 Return payload busy if write fails to flush, treat as an error in most cases 2017-02-28 13:13:09 +10:30
Jeremy Lakeman
8e9877b3ce Add filter by service / name to restful rhizome list 2016-10-26 13:09:32 +10:30
Jeremy Lakeman
30aa1c16b8 Set authorship to remote if the sender sid can be generated from the id 2016-10-20 08:51:46 -04:00
Andrew Bettison
71cbe86566 Switch to feature-driven linking
This introduces a new way of linking Serval executables and dynamic
libraries from static libraries like libservald.a -- called
"feature-driven" linking.

The Makefile now links servald and serval-tests from libservald.a,
rather than from an explicit list of object (.o) files.  Thanks to the
section-based method for registering functions such as HTTP handlers,
CLI commands and MDP handlers, these object files had become
"stand-alone" and hence were no longer included in the link because
there was no unresolved reference that required them to be linked in.

The new "feature.h" provides the DECLARE_FEATURE(name) macro that each
stand-alone source file uses to declare the named feature(s) it
provides.  Each executable can call the USE_FEATURE(name) macro in any
of its explicitly-linked source files to cause the corresponding
object(s) to be included in the link, eg, servald_features.c.

The DEFINE_BINDING() macro has been extended so that every individual
MDP binding is given a feature name based on its port number macro, eg,
"mdp_binding_MDP_PORT_ECHO".

Some features have been factored into their own separate source files so
they can be omitted or included in a build independently of each other:
- the MDP bindings for MDP_PORT_DNALOOKUP, MDP_PORT_ECHO,
  MDP_PORT_TRACE, MDP_PORT_KEYMAPREQUEST, MDP_PORT_RHIZOME_xxx,
  MDP_PORT_PROBE, MDP_PORT_STUN, MDP_PORT_STUNREQ
- the CLI "log" and "echo" commands
- the CLI "rhizome direct" command

The JNI source files are only compiled if the <jni.h> header is present,
otherwise they are omitted from libservald.so.
2016-10-19 09:33:01 +10:30
Jeremy Lakeman
b927af79a8 Introduce new typedef for signing keys to expose their internal structure 2016-09-27 10:28:46 +09:30
Andrew Bettison
6e65a2399f Add missing #include "str.h" 2016-09-21 18:47:49 +09:30
Andrew Bettison
5b3d997896 Move Base64 conversions from "str.h" to "base64.h" 2016-09-21 18:47:49 +09:30
Jeremy Lakeman
a854105598 Support rhizome newsince request on an empty store 2016-07-25 16:54:11 +09:30
Andrew Bettison
518740bcf2 Report missing Content-Disposition header 2016-02-22 18:14:53 +10:30
Andrew Bettison
bd45186a6a Document REST POST /restful/rhizome/insert
Adds definitions for Rhizome concepts such as Bundle ID, Bundle Secret,
payload, etc. and a detailed step-by-step definition of the insertion
logic.
2015-12-07 22:39:19 +10:30
Andrew Bettison
3f8f0f6fc7 Improve Rhizome HTTP API diagnostics
Add RHIZOME_BUNDLE_STATUS_MANIFEST_TOO_BIG enum option to indicate
that the manifest exceeded 8 KiB in size.

Refactor rhizome_add_manifest() and rhizome_manifest_finalise()
to return 'struct rhizome_bundle_result' instead of 'enum
rhizome_bundle_status', so that that their detailed failure messages
can reach the HTTP API layer instead of just being logged.

Fix HTTP response status codes produced Rhizome direct HTTP requests
to be consistent with the Rhizome RESTful API.
2015-12-07 22:35:42 +10:30
Andrew Bettison
cc26dd99e5 Cosmetic: rename variable 'result' to 'http_status'
For more consistency with standard terminology.
2015-12-07 11:25:40 +10:30
Andrew Bettison
98ed0406c5 Refactor Rhizome result handling
- Introduce the new 'struct rhizome_bundle_result' that contains a
  rhizome_bundle_status enum value and an optional, nul-terminated
  string that provides an explanation of the cause of the status; add
  functions to construct, query, and free the struct

- Replace 'enum rhizome_add_result' with 'struct rhizome_bundle_result',
  removing an unnecessary level of enum interpretation

- Make rhizome_fill_manifest() return 'struct rhizome_bundle_result' and
  add logic to check that the supplied author SID is correct (previous
  behaviour was: if an incorrect author SID was supplied but the correct
  author could be found in the keyring, then the incorrect one was
  silently ignored)

- Simplify the response code in rhizome_restful.cc to take advantage of
  the new 'struct rhizome_bundle_result'; in particular, the mapping
  from 'enum rhizome_bundle_status' codes to HTTP status codes is now
  expressed in a single switch statement

- Fix some minor failures in test scripts revealed by the changes
2015-12-01 00:51:15 +10:30
Andrew Bettison
3c993f0273 Add fromhere=2 to Rhizome list output
Need a way for the client to distinguish between authenticated (certain)
and unauthenticated (likely) author SIDs in the context of a bundle list,
since the bundle list does not verify manifest signatures for performance
and battery life reasons.
2015-11-02 12:26:40 +10:30
Andrew Bettison
419364b5a9 Improve REST HTTP response status codes
List all the HTTP status codes in the REST API tech doc.

Only use 403 Forbidden for requests originating from a disallowed
origin (ie, not localhost).

- Return 400 for missing, unknown, duplicate and out-of-order form
  parts in POST requests.
- Return 415 Unsupported Media Type for unsupported form part
  Content-Disposition and Content-Type (including unsupported
  charset).
- Return 414 Request-URI Too Long for any buffer exhaustion while
  parsing request.
- Return 419 Authentication Timeout for missing crypto secret.
2015-11-02 12:26:40 +10:30
Andrew Bettison
6123503c15 Improve Rhizome HTTP status codes
Also improve some reason phrases and test them
2015-09-21 18:43:24 +09:30
Jeremy Lakeman
84caf21969 Use section linking to define httpd pages 2015-08-24 12:49:09 +09:30
Andrew Bettison
7d9a5faa4e Move if(config.debug.xxx) tests into DEBUGF()
Original DEBUG() and DEBUGF() macros renamed to _DEBUG() and _DEBUGF()
New DEBUG() and DEBUGF() macros, first argument is flagname
New DEBUGF2(foo, bar, ...) macro does if(config.debug.foo||config.debug.bar) test
Replace almost all config.debug.xxx references to IF_DEBUG(xxx)
2015-07-13 16:00:05 +09:30
Andrew Bettison
6febcc350d Replace RESTful newsince polling with triggers
Remove the 'api.restful.newsince_poll_ms' config option, no longer
needed.
2015-06-02 02:23:06 +09:30
Andrew Bettison
f602674ea5 RESTful Rhizome add/insert always return HTTP 201
The 200 status is reserved to indicate a well-formed and
legal request but the bundle was not added because the
store rejected it.
2015-04-13 14:39:51 +09:30
Andrew Bettison
7734e24006 /restful/rhizome/append on existing journal
Refactor Rhizome insert/append logic into functions used by both CLI and
RESTful API.  Improve RESTful diagnostic messages.
2015-03-30 18:24:12 +10:30
Andrew Bettison
99ff49d502 /restful/rhizome/insert reject out-of-order parameters
Must have the 'bundle-secret', 'bundle-author' and 'bundle-id'
parameters before the 'manifest' part
2015-03-28 05:09:38 +10:30
Andrew Bettison
ccf4824605 Add 'manifest-id' parameter to /restful/rhizome/insert 2015-03-28 05:09:38 +10:30
Andrew Bettison
da22816784 Add 'rhizome add file --bundle=BID' option
A convenience to re-use an existing manifest, instead of having
to extract the manifest, erase some fields, then pass the result
back into the 'rhizome add file' command
2015-03-28 05:09:38 +10:30
Andrew Bettison
7f564e4dcf Make strn_to_xxx() functions consistent
All strn_to_xxx() functions should take a 'size_t len' parameter

All functions that take a 'const char **endp' parameter should
be named parse_xxx(), and should also have a 'ssize_t len' parameter
which can be -1 to indicate a null-terminated string
2015-03-28 04:53:09 +10:30
Andrew Bettison
015b4a0b07 Refactor and improve "journal append"
Distinguish between fatal and user-supplied-input errors, return
error descriptive text in a strbuf to allow dynamic content, apply
user-supplied field assgnments/deletions _after_ copying existing
manifest fields, use exit status 4 (invalid manifest) when applying
journal append to a non-journal or vice versa
2015-03-23 17:38:46 +10:30
Andrew Bettison
217fc7cde9 Improve diagnostic logging and HTTP responses 2015-03-16 22:40:46 +10:30
Andrew Bettison
4d5cded047 Change return type of rhizome_fill_manifest()
To help improve diagnostic feedback through APIs
2015-03-16 22:40:08 +10:30
Andrew Bettison
c9131f43a2 POST /restful/rhizome/append for journals 2015-03-02 18:52:12 +10:30
Andrew Bettison
4b234fbfbd Fix /restful/rhizome/insert HTTP status code
Return 403 "Forbidden" instead of 501 "Not implemented" when
given a journal bundle
2015-03-02 18:49:51 +10:30
gardners
2802f75fa2 allow blank format field in post, so that libcurl can be used to
insert rhizome bundles.
2015-02-15 17:48:47 +10:30
Jeremy Lakeman
ce21c64809 Allow serving of static http content from [etc]/static/ 2014-12-01 10:21:35 +10:30
Andrew Bettison
081f7ab8c8 Add recent bundle secret improvements to RESTful API
The "#passphrase" notation was already supported, and now the RESTful
API can be used to create and update anonymous and authored bundles
using only the bundle secret, just like the CLI.
2014-11-14 17:31:08 +10:30
Andrew Bettison
d8f26a76ee Accept '#passphrase' for bundle secret arguments
As well as a hex string.  Applies to CLI and also REST API.
Maximum pass phrase length is 80 ASCII chars.
2014-11-14 13:11:44 +10:30
Jeremy Lakeman
851144ea0a Refactor keyring identity iteration 2014-10-31 15:19:59 +10:30
Jeremy Lakeman
2c72875508 Ensure sql lock failures are returned. 2014-09-12 15:24:23 +09:30
Jeremy Lakeman
31cf3a67b5 Allow lcal http servers to perform cross site scripting of restful APIs 2014-07-28 12:54:57 +09:30
Andrew Bettison
d436705e64 Fix conflict on UNUSED() macro from OpenJDK 7
The OpenJDK 7 recently introduced the UNUSED() macro in their jni_md.h
header file, which is included from <jni.h>.  This causes a
compile-error if "constants.h", which defines our own UNUSED() macro, is
included as well as <jni.h>.

The OpenJDK UNUSED() macro is unsuitable for our own use, because it
prefixes the unused identifier with "UNUSED_" whereas we depend on the
parameter name remaining unchanged.

I have reported this as a Request for Enhancement with Oracle Java,
asking them to remove the UNUSED() macro, since it is not used by any
JNI or Java extension header files.  Review ID: JI-9013689.

In the meantime, constants.h now undefines UNUSED before defining it, so
including <jni.h> before "constants.h" will avoid a compile error.
2014-07-23 11:55:55 +09:30
Andrew Bettison
8842f32b19 Rhizome Java API: insert journal is not implemented
Change the HTTP response code for unimplemented operations from 403 to
501
2014-07-10 22:23:11 +09:30