Commit Graph

466 Commits

Author SHA1 Message Date
Jeremy Lakeman
99d5d9fa1f Refactor dataformats into separate header 2013-11-27 13:11:42 +10:30
Jeremy Lakeman
0e6a3cd7cc Use interval parameter correctly even for unlimited pings 2013-11-27 13:11:37 +10:30
Jeremy Lakeman
0d5d878521 Refactor socket name and address length handling 2013-11-27 13:08:37 +10:30
Andrew Bettison
a9ccd38adc Improve overlay buffer ob_xxx() primitives
All ob_append_xxx(b,...) functions return void

ob_makespace() returns 1 if successful, 0 if not

Add ob_overrun(b) predicate to check for overrun after any number of
appends
2013-11-26 15:33:54 +10:30
Andrew Bettison
9ddcfb87aa Fix output delimiter in "reverse lookup" command 2013-11-22 11:55:09 +10:30
Jeremy Lakeman
a13189e554 Fix printf formats 2013-11-21 18:11:47 +10:30
Andrew Bettison
67a724b29d Fix Android compiler warnings ("%zd" and ssize_t)
Also squash some unconditional DEBUG statements
2013-11-21 17:32:58 +10:30
Andrew Bettison
711364ad8d Merge branch 'meshextender' into 'development' 2013-11-21 17:05:03 +10:30
Andrew Bettison
8c06881fa3 Fix compiler warning 2013-11-21 16:49:44 +10:30
Andrew Bettison
973bb9c897 Merge branch 'naf4' into 'development' 2013-11-13 17:59:01 +10:30
Jeremy Lakeman
0b404b0260 Implement id list command 2013-11-13 12:00:46 +10:30
Jeremy Lakeman
ff1f1ff093 Allow tag primitives to be used outside of keyring entries 2013-11-13 12:00:05 +10:30
Jeremy Lakeman
c618c724e1 Ensure that invalid subscriber id arguments fail validation 2013-11-13 12:00:02 +10:30
Jeremy Lakeman
9680b24f23 Add support for adding and updating tags in the keyring 2013-11-13 11:59:21 +10:30
Jeremy Lakeman
8c03f18e5e Refactor keyring searching 2013-11-13 11:59:13 +10:30
Jeremy Lakeman
4434b1b65d Create keyring.h and move definitions there 2013-11-13 11:55:12 +10:30
Andrew Bettison
13634f8748 Add ROWID field to struct rhizome_manifest
New ".rowid" output field from rhizome add, import, extract, export
operations.  (Also added missing ".inserttime" and "date" fields to
some operations.)

Use new "rhizome add file" .rowid output field to check output of of
/restful/rhizome/bundlelist.json
2013-11-11 18:18:08 +10:30
Andrew Bettison
ca66a387dd Add "date" output field to Rhizome commands
To wit: "add file" and "import bundle"
2013-11-11 16:16:04 +10:30
Andrew Bettison
051eca4775 Improve Rhizome list cursor
Remove 'rowcount' element

Order by descending ROWID, which is functionally the same as descending
inserttime but more reliable

Replace '_offset' cursor element with rowid first-last range to record
the expanding window of rows already returned, which allows release and
re-open of cursor mid-listing without missing rows or producing duplicates
2013-11-07 17:22:06 +10:30
Andrew Bettison
9606b4b1b8 Refactor "rhizome list" command
Move output formatting into commandline.c, leave database query in
rhizome_database.c
2013-11-07 10:21:11 +10:30
Andrew Bettison
e00c945fd1 Merge branch 'naf4' into 'development'
Big refactor of Rhizome author and bundle secret handling.
2013-11-07 00:42:42 +10:30
Andrew Bettison
45442d3eb4 Rewrite bundle author authentication
Replaced 'int has_author' manifest element with new 'enum authorship'
element to record the result of author authentication, to avoid
repeating expensive crypto operations.

Separated the handling of bundle secret arguments from author lookup and
authentication.  The new rhizome_apply_bundle_secret(m,bsk) is now
called at the top level to set the manifest secret key (if it
validates), and thereafter there is no need to pass the 'bsk' argument
to any other functions, as they can simply check the 'haveSecret' field
of the manifest.

Removed rhizome_extract_privatekey() which combined author lookup and
bundle secret validation, and replaced it with functions that only deal
with the author: rhizome_lookup_author() and rhizome_authenticate_author().

Renamed other functions to make their purpose and effect clearer.

Formalised the semantics of only storing AUTHENTICATED author SIDs in
the 'author' column of the MANIFESTS table, which necessitated a change
to a 'rhizomeops' test case: when adding a file using a BK-less
manifest, the author column is set to null, so the Rhizome list output
does not show the bundle as ".fromhere" and does not give an author for
that bundle.
2013-11-06 23:58:17 +10:30
Jeremy Lakeman
7629f24d3f Free subscriber entries after all commands 2013-11-06 15:36:21 +10:30
Jeremy Lakeman
b84925f4ed Free the keyring once it's no longer needed 2013-11-06 15:35:44 +10:30
Andrew Bettison
bacba19dc1 Refactor manifest: specific setter functions
Replace generic rhizome_manifest_set() and rhizome_manifest_set_ll()
with per-field setter functions, eg, rhizome_manifest_set_filesize().
Struct rhizome_manifest elements for all known fields, to replace the
use of rhizome_manifest_get() and rhizome_manifest_get_ll() everywhere:
sender, recipient, service, name, date, bundle_key.

Add boolean validity flags for binary blob types, to avoid having to compare
with many bytes of all-zero to detect presence, eg, has_sender, has_recipient,
has_author, has_bundle_key.  These maintained by the setter functions.

Rename existing manifest struct elements to be the same as their field
names: fileLength -> filesize, journalTail -> tail.

More use of unsigned int, size_t and uint64_t for payload sizes, offsets, byte
counts, etc. especially in rhizome_store.c and meshms.c.  More uniform use of
size_t to dimension memory buffers.  Fix some printf(3) style format strings
for 64-bit correctness on 32-bit systems.  Use new constant RHIZOME_SIZE_UNSET
instead of -1 to indicate unknown dimension, and explicitly assert its absence
before comparisons and arithmetic, for safety.

Replace some 'int' loop variables with 'unsigned' where appropriate.

Fix bugs discovered in MeshMS bundle private/public key generation and
bundle secret key handling for export/extract commands.

Instrument the first MeshMS test case to aid debugging.

New debug config flag: debug.manifest logs all modifications to all manifest
fields by setter functions.

Rename debug config flag: debug.rhizome_bind -> debug.rhizome_sql_bind.
2013-11-04 19:17:49 +10:30
Andrew Bettison
675e2021cc Add ".secret" output field to all rhizome export/extract commands 2013-11-04 17:42:22 +10:30
Petter Reinholdtsen
93c43da89b Merge current development branch. 2013-10-30 18:10:08 +01:00
Andrew Bettison
416b82b6c0 Refactor manifest: change haveSecret to enum 2013-10-30 13:14:26 +10:30
Andrew Bettison
9d54c629b2 Merge branch 'naf4' into 'development'
Rewritten HTTP request parsing and response buffering
2013-10-28 18:20:40 +10:30
Andrew Bettison
05d4215752 Add 'strlen' argument to strn_fromprint() 2013-10-25 00:19:58 +10:30
gardners
31b314d2da add context switching speed test to "test memory" command. 2013-10-22 13:12:10 -07:00
gardners
6ff2ae4871 add memory write speed test. 2013-10-22 12:53:34 -07:00
gardners
17d2dda6ae add simple memory performance test 2013-10-22 12:37:10 -07:00
Petter Reinholdtsen
8a19411c26 Clean up after merge with upstream. 2013-10-21 21:29:35 +02:00
Jeremy Lakeman
d5d5737f1c Rename 'id revoke' to 'id relinquish' to match documentation 2013-10-15 14:52:07 +10:30
Petter Reinholdtsen
6564449f49 Initialize some struct members reported by the compiler. 2013-10-13 09:01:53 +02:00
Andrew Bettison
97cbebc91e Issue #11: improve types in file i/o functions
Use open(2)/read(2)/write(2) instead of fopen(3)/fread(3)/fwrite(3) in
several places to avoid unnecessary buffering

Fix a bug in Rhizome HTTP add's handling of unconfigured manifest
template file

Improve some debug and error logging for file i/o
2013-10-11 16:24:51 +10:30
Andrew Bettison
49729cc768 Remove default service=file for new manifests 2013-10-11 15:39:10 +10:30
Andrew Bettison
974c7a56a0 Issue #11: Use rhizome_filehash_t everywhere 2013-10-11 15:35:27 +10:30
Jeremy Lakeman
ef7351bddc Unload identities from a running daemon 2013-10-10 14:27:00 +10:30
Andrew Bettison
221fc4a4fc mdp_port_t 2013-10-09 19:22:51 +10:30
Andrew Bettison
a95ef79139 Use sid_t everywhere
Remove stowSid() at last

Change API of tohex() and strbuf_tohex(), to pass string length instead of
binary byte count.  This allows odd numbers of hex digits to be produced.

Remove alloca_tohex_sid(); replace with alloca_tohex_sid_t()

New alloca_tohex_sid_t_trunc() macro
2013-10-09 18:54:21 +10:30
Jeremy Lakeman
ae7e120ed5 Pass keyring entry pin to running daemon and unlock identities 2013-10-09 15:34:41 +10:30
Jeremy Lakeman
60e3f5a3fc Fix printf formats for 64bit compilation 2013-10-09 10:31:52 +10:30
Andrew Bettison
ffafae86b1 Issue #11: replace long long with int64_t 2013-10-07 05:54:46 +10:30
Andrew Bettison
ab31420faf rhizome_bid_t
Consistent type for internal binary representation of Rhizome Bundle ID
(aka Manifest ID)
2013-10-03 23:16:45 +09:30
Andrew Bettison
06d7620314 Add "rhizome add file" --force-new option
Overrides the default behaviour of de-duplicating added manifests

New 'rhizomeops' test case: AddForceDuplicate
2013-09-30 16:48:18 +09:30
Andrew Bettison
ae5ae2d394 Cosmetic changes to whitespace
Make it easier to edit in Vim
2013-09-30 16:06:23 +09:30
Andrew Bettison
5e164607aa Expand output of rhizome add and import
Rename 'secret:' field to '.secret:' (non-manifest-fields start with '.'
using the same convention as rhizome list)

Add '.author' and 'BK' fields to "rhizome add"

Add 'BK' field to "rhizome import"

Fix 'rhizomeops' tests to assert no 'author' and 'BK' output fields from
"rhizome add" with no author

Fiz testdefs.sh and testdefs_rhizome.sh to support new output fields
2013-09-27 22:34:22 +09:30
Romain Vimont (®om)
954a8a01a4 Use one MDP socket per binding instead of a single global socket. 2013-09-24 16:09:44 +09:30
Andrew Bettison
7add45214c Add [keyring-pin] arg to keyring load command 2013-09-10 11:28:55 +09:30
Andrew Bettison
befb658958 All keyring commands now close (free) the keyring 2013-09-05 16:34:52 +09:30
Andrew Bettison
8a300c2520 Get new keyring dump command and test working 2013-09-05 16:34:01 +09:30
Andrew Bettison
08c02e0e20 More progress towards keyring load command 2013-09-03 17:31:10 +09:30
Andrew Bettison
425aa1005c Start work on "keyring load" command 2013-09-02 17:33:52 +09:30
Andrew Bettison
080ccd957a Replace str_fromprint() with strn_fromprint() 2013-09-02 17:31:13 +09:30
Andrew Bettison
7d30b3cce7 Pass command-line args to server() function
So that the daemon can open the keyring file with --keyring-pin and --entry-pin
options.

Also remove some unused code.
2013-08-30 11:20:19 +09:30
Jeremy Lakeman
8fea1523b1 More consistent use of sleep_ms 2013-08-27 15:14:17 +09:30
Jeremy Lakeman
f2e7391672 Fix format warnings on 64bit machines 2013-08-26 11:54:46 +09:30
gardners
76f90f6004 add simple test to make sure byte order manipulation routines work. 2013-08-25 22:30:06 +09:30
Jeremy Lakeman
81afc42d8b Major refactor to packet queuing and delivery in preparation for unicast link tracking.
Unicast tests broken
2013-08-09 09:37:08 +09:30
Jeremy Lakeman
8a1c0a39e3 Merge branch 'meshmsv2' into development 2013-08-05 13:05:44 +09:30
Jeremy Lakeman
6c85999917 Log crashes and unexpected signals as FATAL 2013-07-31 14:53:52 +09:30
Jeremy Lakeman
7634e1fcfb Add command to mark messages as read 2013-07-26 14:36:11 +09:30
Jeremy Lakeman
ddfb7d9417 First cut of new meshms API, unfinished 2013-07-22 17:55:02 +09:30
Jeremy Lakeman
4dd2c8e2a2 Revert sid output format change for now 2013-07-15 13:22:24 +09:30
Jeremy Lakeman
3855ff35c4 Clean up string formatting for 64bit compilation 2013-07-15 09:59:24 +09:30
Jeremy Lakeman
8ac359e550 Use consistent integer types for 64bit compilation 2013-07-13 14:47:06 +09:30
Jeremy Lakeman
62a4cd407d Identify a journal bundle by the presense of a tail field 2013-07-08 14:57:47 +09:30
Jeremy Lakeman
e758e0130f Create journal append command 2013-07-08 14:29:08 +09:30
Jeremy Lakeman
2bc07a8eaa Fix stupid syntax error 2013-07-04 16:02:19 +09:30
Jeremy Lakeman
53f3920b90 Move cli global state into a structure and pass it around 2013-07-03 16:51:27 +09:30
Jeremy Lakeman
b67516558e Add clean command to verify all bundles 2013-06-14 11:58:19 +09:30
Jeremy Lakeman
e2c34b62fd Allow mdp ping to flood packets rapidly 2013-05-27 14:25:59 +09:30
Jeremy Lakeman
118dcca40b Improve quality and readability of some routing tests 2013-05-10 12:05:28 +09:30
gardners
dc49671583 take sleep() out of urandomdev(), since it just causes problems, and
read() should block if there aren't bytes available yet.
2013-05-07 14:42:47 +09:30
Jeremy Lakeman
b8a2026618 Use appropriate cli interface functions for console output 2013-05-03 16:21:29 +09:30
Jeremy Lakeman
9ee09e36a0 Replace old routing engine with new link state engine 2013-05-02 14:29:28 +09:30
Andrew Bettison
4e48725c93 Merge branch 'new-keyring' into 'development' 2013-04-30 17:57:59 +09:30
Andrew Bettison
1f10f05302 Add "keyring dump" command 2013-04-30 17:29:06 +09:30
Jeremy Lakeman
85b885dde2 Add basic link state routing 2013-04-26 16:54:01 +09:30
Andrew Bettison
121ae2d609 Improve timeout control in routing tests 2013-04-22 16:01:38 +09:30
Andrew Bettison
81af4460f3 Fix broken "ping" command arg parsing 2013-04-12 17:18:25 +09:30
Andrew Bettison
a8b88a462f Fix clang compiler warnings (revealed some bugs) 2013-04-12 17:16:50 +09:30
Andrew Bettison
a9a1ffecde Simplify "start" commands
Remove all "in <instancepath>" variants

Make "foreground" and "exec <path>" variants mutually exclusive, using
new CLI alternatives syntax

Fix some log levels to assist newbies
2013-04-11 17:54:18 +09:30
Andrew Bettison
58ff69c1d3 Fix #59: "-h" and "--help" synonyms for "help" 2013-04-09 17:26:38 +09:30
Andrew Bettison
5bbdef5587 Improve command-line parsing (issue #59)
Emit HINT log message about using "help" command

Support alternatives syntax "word|word|..." in CLI schema

Better return values from cli_parse()
2013-04-09 17:23:20 +09:30
Andrew Bettison
2c379fadb1 Merge branch 'naf2-logging' into 'development'
Conflicts:
	commandline.c
	tests/rhizomeprotocol
2013-04-05 17:39:23 +10:30
Andrew Bettison
4be43b76d5 Remove CLIFLAG_NONOVERLAY and CLIFLAG_STANDALONE
Not used for anything
2013-04-02 17:49:56 +10:30
Andrew Bettison
21991726ae Add logging tests, add "log" command
Refactor log.c ready for log file rotation
2013-04-02 17:01:48 +10:30
Andrew Bettison
0f83b9a5fc Fix #60 - "rhizome direct" command line URL
Add test case for "rhizome push <url>" command
2013-04-02 14:54:46 +10:30
Andrew Bettison
2d09ec545e Improve configuration dump to log
Refactor much of the new code in log.c

Add new functions: cf_load_strict() and cf_reload_strict() for use by the
server, which does not want to overwrite its in-memory config at all if there
is any defect in the config file.  However, commands always want to overwrite
the default in-memory config, even with a defective config file, so that
logging has a chance to succeed.

Added `log_format.dump_config` boolean config option
2013-03-29 00:22:08 +10:30
Andrew Bettison
95b0c028f2 Log to file, Android and stderr simultaneously
Indepent configuration of show_pid, show_time and log level for each
destination

Update test scripts for new config options

Include xprintf.c in MDP client source files (now used by log.c)
2013-03-27 16:17:38 +10:30
Andrew Bettison
a88234ce9d Add optional filter args to 'help' command 2013-03-25 12:05:26 +10:30
Andrew Bettison
300ec986f1 Fix warnings from gcc 4.6.3 2013-03-18 16:48:15 +10:30
Andrew Bettison
b17f5a6f63 Servald version string from git describe 2013-03-12 14:03:33 +10:30
Andrew Bettison
1d1f7a37a4 Fix Solaris SIGBUS bug in MDP ping
Also fixed non-portable (endian dependent) code
2013-03-07 14:54:40 +10:30
Andrew Bettison
8771d6181b Merge branch 'naf2-logging' into 'development' 2013-03-01 17:32:57 +10:30
Andrew Bettison
ae787339ec Improve "config dump" command
All STRUCT cf_fmt_config_xxx() functions omit invalid values, silently
if the invalid value is the default, otherwise with a non-CFOK return
value.

The "config dump --full" option includes all options with valid values,
otherwise omits options having default values.
2013-02-28 15:18:48 +10:30
Andrew Bettison
46eeacb823 Improve new "config dump" command
No more SEGV.  Omits invalid (default) values.  Logs unconditional DEBUG
output, to be removed before merging into development.  Still missing
cf_cmp_ functions to prune out default values.

Improved config Object Model iterator logic to barf on internal NULL
nodes.
2013-02-27 16:26:07 +10:30