Commit Graph

289 Commits

Author SHA1 Message Date
Andrew Bettison
a23c56626d Update some copyright notices 2014-11-20 00:01:12 +10:30
Andrew Bettison
c95807b002 Refactor manifest parsing
Consolidate tests for valid field names and values into
one place
2014-11-19 13:17:40 +10:30
Andrew Bettison
2f1843858d Support unrecognised manifest fields
Refactor Rhizome manifest parsing in preparation for command-line
field assignment
2014-11-19 10:47:31 +10:30
Andrew Bettison
e743e4d7f9 Create authorless bundle from bundle secret
Until now, the bundle secret was never used to create a new bundle,
only to update a bundle whose secret was known from its creation.
2014-11-14 13:11:44 +10:30
Jeremy Lakeman
2c72875508 Ensure sql lock failures are returned. 2014-09-12 15:24:23 +09:30
Jeremy Lakeman
4aac9e768e Fix clang error due to copy paste 2014-07-14 16:01:10 +09:30
Andrew Bettison
606f087dd5 Merge latest Rhizome Java API into development 2014-07-11 12:29:24 +09:30
Andrew Bettison
cf43635789 Improve Rhizome HTTP RESTful interface
Add RHIZOME_BUNDLE_STATUS_READONLY enum value

Tighten up switch statements on bundle and payload status enums (no
default labels)

Rename some recently added enum entries

Return bundle status and payload status in HTTP responses

Add test for failing to decrypt a foreign encrypted bundle payload,
fix bug that caused an assertion failure

Add tests for fetching a non-existent manifest and fetching bundles
whose payload blob is not in the store
2014-07-10 06:29:51 +09:30
Andrew Bettison
61023287b1 Refactor Rhizome manifest validation
New function returns text describing the failed validation
2014-07-10 06:29:09 +09:30
Jeremy Lakeman
096c9670ca Remember when we last verified each payload 2014-07-04 14:25:28 +09:30
Jeremy Lakeman
31dbfa4e58 Improve debug info for rhizome sync 2014-06-28 20:27:29 +09:30
Jeremy Lakeman
58ea5a1c8b Simplify fetch decisions when manifest arrives 2014-06-28 17:41:05 +09:30
Jeremy Lakeman
588b95ab20 Ensure temporary id is unique 2014-06-28 17:36:06 +09:30
Jeremy Lakeman
47f051917d Rhizome cleanup will now expire payloads and vacuum the db file. 2014-06-19 14:23:43 +09:30
Jeremy Lakeman
7ff89afcf4 Reinstate rhizome database storage limit
- old / large payloads should be evicted to fit more payloads
- if there isn't enough space, new payloads will not be added
2014-06-19 11:09:09 +09:30
Jeremy Lakeman
80e139c2d9 Ensure orphan payloads or manifests are cleaned up 2014-06-16 15:56:31 +09:30
Jeremy Lakeman
f63e7fee19 Introduce typedef for handling BAR values 2014-06-16 14:38:14 +09:30
Jeremy Lakeman
2a572d19bf Rework rhizome store to write content more lazily and cleanup more carefully 2014-06-04 17:13:08 +09:30
Jeremy Lakeman
b6de3fcd34 Only send rhizome sync packets when we have neighbours 2014-05-30 15:03:22 +09:30
Andrew Bettison
21f51965c1 Overhaul Makefile and header files
Make .o files in separate subdirectories for servald and library
Factor struct __sourceloc and __WHENCE__ from "log.h" into "whence.h"
Factor sid_t etc from "serval.h" into "serval_types.h"
Factor rhizome_bid_t etc from "rhizome.h" into "rhizome_types.h"
Do not include "serval.h" in library sources
Add log_stderr.c and logMessage.c for stand-alone executables
2014-05-26 15:36:26 +09:30
Andrew Bettison
53c1b1c04c Filesystem Heirarchy Standard (FHS) paths
If no instance directory specified, then use paths like
/etc/serval/serval.conf
/var/cache/serval
/var/log/serval
/var/run/serval
etc. for files, instead of all in a single directory.

Log all directory creation as INFO messages.

Interpretation of log.file.directory_path has changed slightly.

Updated servald configuration tech doc.
2014-03-26 15:35:43 +10:30
Andrew Bettison
0bef066949 Store Rhizome external blob files in "blob" subdirectory
This will make issue #50 a bit easier to tackle
2014-03-07 16:55:38 +10:30
Jeremy Lakeman
b56f4c27d3 Replace internal usage of overlay_mdp_dispatch 2014-02-19 16:39:42 +10:30
Jeremy Lakeman
487df0408d Refactor mdp packet processing to avoid using struct overlay_mdp_frame 2014-02-19 16:34:11 +10:30
Andrew Bettison
a77b0702b1 Rename uuid_t --> serval_uuid_t
To avoid compile errors on OSX
2014-02-17 13:26:03 +10:30
Andrew Bettison
015ed0b181 Refactor to rename "Rhizome HTTP server" to "HTTPD server"
Move HTTPD start/stop/dispatch logic from rhizome_httpd.c into new
httpd.c

Rename config.debug.httpd -> config.debug.http_server
Rename config.debug.rhizome_httpd -> config.debug.httpd
Rename config.debug.rhizome_nohttptx -> config.debug.nohttptx
2014-02-03 15:56:07 +10:30
Andrew Bettison
94274ba5fa Refactor sqlite_retry_state into rhizome_list_cursor 2014-01-22 17:20:06 +10:30
Andrew Bettison
e3e3e1e046 rhizome_http.h 2014-01-22 15:51:59 +10:30
Jeremy Lakeman
5f2c6e364a Refactor interface addresses to use new socket_address struct 2014-01-09 15:01:38 +10:30
Jeremy Lakeman
64fe96ebfa Only drop BAR's from the sync if they are no longer interesting 2014-01-09 15:01:37 +10:30
Andrew Bettison
ee9c96bb8c Implement HTTP POST /restful/rhizome/insert
Change HTTP request buffer pointers from (const char*) to (char*)
because some Rhizome operations can modify received data in-place, eg,
when decrypting it.
2013-12-30 18:09:56 +10:30
Andrew Bettison
dd5048bd43 rhizome_manifest_set_name_from_path()
Refactor rhizome_fill_manifest() to use it
2013-12-30 16:30:43 +10:30
Andrew Bettison
7cecdf7a3a Add Rhizome db blob primitive functions
sqlite_blob_open_retry()
sqlite_blob_write_retry()
sqlite_blob_close()
2013-12-30 16:30:43 +10:30
Andrew Bettison
72040517e1 New enum rhizome_payload_status
Refactor a lot of Rhizome bundle storage code to use the new "enum
rhizome_payload_status" instead of mysterious int values to represent
the outcome of the operation.
2013-12-30 16:30:35 +10:30
Andrew Bettison
9ebef81a49 Formalise "rhizome add file" exit status
Formalise add-bundle result in "enum rhizome_bundle_status"

Rewrite rhizome_manifest_finalise(), rhizome_find_duplicate() and
rhizome_add_manifest() to return enum rhizome_bundle_status

New function rhizome_manifest_check_stored() that compares a manifest
with its stored counterpart and returns enum rhizome_bundle_status

Remove redundant rhizome_manifest_check_sanity(), consolidating all
manifest validation rules in rhizome_manifest_validate(), which now
checks the 'id' field is present, and that 'sender' and 'recipient' are
both present for MeshMS

Correct manifest finalisation logic: set the 'finalised' flag in
rhizome_manifest_validate(), not in rhizome_manifest_verify() (which
sets 'selfSigned'), and consistently clear 'finalised' flag in all
attribute setter functions

Remove manifest 'ttl' field and all references thereof (leaving unused
space in Rhizome BAR)

Rename some payload functions for clarity
2013-12-21 14:37:18 +10:30
Andrew Bettison
273c5f2689 Make rhizome_manifest_pack_variables() static 2013-12-20 11:29:26 +10:30
Andrew Bettison
d52ba4c871 Remove rhizome_read_manifest_file()
Replace with memcpy() followed by rhizome_manifest_parse()
for memory buffers

Replace with rhizome_read_manifest_from_file() for files
2013-12-20 11:29:17 +10:30
Andrew Bettison
250309f4d8 Fix strn_to_sid_t(), add strn_to_rhizome_bk_t()
Improve strn_to_rhizome_bid_t() and strn_to_rhizome_filehash_t() to
accept a NULL first arg -- useful for data validation
2013-12-18 17:44:16 +10:30
Andrew Bettison
6361bfd757 Implement HTTP GET /restful/rhizome/<BID>/raw.bin 2013-12-13 16:36:37 +10:30
Andrew Bettison
183cb46000 Add Rhizome HTTP response headers to /restful/rhizome/<BID>.rhm 2013-12-13 14:49:55 +10:30
Andrew Bettison
075f9c7c27 Use uint64_t for Rhizome manifest version
Instead of int64_t.  Fixes some -Wsign-compare warnings.

Replace sqlite_exec_int64() with sqlite_exec_uint64().

Also store rowid as uint64_t, and use 0 not -1 to indicate
unset.
2013-12-11 11:11:34 +10:30
Andrew Bettison
b7528412df Use UINT64_MAX instead of -1 for Rhizome HTTP response content_length
Fixes -Wsign-compare warnings
2013-12-10 17:06:45 +10:30
Andrew Bettison
4af6cf9d6a Fix -Wsign-compare warnings: use size_t or unsigned for byte counts 2013-12-10 16:52:53 +10:30
Andrew Bettison
30b2059608 Fix -Wunused-parameter warnings
New UNUSED(param) macro suppresses the warning for a specific parameter

In some cases, removed the unused parameters.  In others, used the
parameter in a DEBUGF() or assert() statement to document the intent.
2013-12-09 18:22:18 +10:30
Andrew Bettison
5226e47ef6 Uniform #ifdef __SERVAL_DNA__... for headers 2013-12-04 17:15:36 +10:30
Andrew Bettison
42ab9aec4c Merge branch 'development' into 'naf4'
Remove two redundant calls to rhizome_retrieve_manifest() in meshms.c,
revealed by an assert() in the stricter manifest parsing code

Fix header files included by socket.h
2013-12-02 17:17:47 +10:30
Andrew Bettison
8db5f9c14a Merge branch 'anyservice' into 'development'
Allows any valid "service" manifest field in the "rhizome add file"
command

Many improvements in Rhizome manifest parsing; stricter manifest syntax
rules (no comment or blank lines, field names must be alphanumeric
identifiers), faster preliminary manifest inspection when receiving
manifest advertisements or syncing manifests

The 'development' branch introduces "struct socket_address" which
coincidentally fixed the recently encountered Linux kernel 3.12
recvmsg(2) EINVAL problem, so that 'rhizomeprotocol' tests which fail on
the 'anyservice' branch will pass after this merge
2013-12-01 05:44:01 +10:30
Andrew Bettison
0dd1b302b5 Rewrite Rhizome manifest parsing
Move validation checks into new function rhizome_manifest_validate()

Remove rhizome_manifest 'errors' field

Replace rhizome_manifest 'warnings' with 'malformed'

Replace rhizome_manifest 'manifest_bytes' with 'manifest_body_bytes' and
refactor to use 'manifest_all_bytes' in all manifest i/o

Refactor rhizome_manifest_verify() and reverse sense of return value to
match rhizome_manifest_validate()

New function rhizome_manifest_inspect() -- lightweight manifest parser
used when receiving Rhizome advertisements

New 'rhizomeops' test case for invalid manifest "service" field values,
now passes
2013-12-01 05:13:34 +10:30
Jeremy Lakeman
99d5d9fa1f Refactor dataformats into separate header 2013-11-27 13:11:42 +10:30
Jeremy Lakeman
28fc1da2d2 Encapsulate rhizome fetch logging 2013-11-27 13:10:55 +10:30