Add "keyring remove" command

Adds a CLI and RESTful API operation for "keyring remove", with simple
test cases.  Added the corresponding Java API operation.  Updated the
API documentation.

API change: for consistency with RESTful API design, the GET
/restful/keyring/add operation now returns "201 Created" not "200 OK" if
successful.
This commit is contained in:
Andrew Bettison 2016-11-04 17:00:40 +10:30
parent e47d0ce1c7
commit a8e394d299
13 changed files with 329 additions and 37 deletions

View File

@ -1,6 +1,6 @@
Keyring REST API
================
[Serval Project][], February 2016
[Serval Project][], November 2016
Introduction
------------
@ -10,24 +10,55 @@ easily be created by any node at any time. Each [Serval DNA][] daemon that
runs on a node in the network stores its own identities in the [Keyring][], an
encrypted store protected by passwords, and gives applications access to the
Keyring via the **Keyring REST API** described in this document. Using this
API, client applications can query, unlock, lock, create, and modify identities
in the keyring.
API, client applications can add, remove, unlock, lock, query, and modify
identities in the keyring.
### Basic concepts
Basic concepts
--------------
#### Serval ID
### Serval ID
Every identity in the [Serval mesh network][] is represented by its **Serval
ID**, (usually abbreviated to [SID][], and formerly known as “Subscriber ID”),
which is a unique 256-bit public key in the [Curve25519][] key space that is
generated from the random *Serval ID secret* when the identity is created. The
SID is used:
which is a unique 256-bit public key in the [Curve25519][] *crypto-box* key
space that is generated from the random *Serval ID secret* when the identity is
created. The SID is used:
* as the network address in the [Serval Mesh network][]
* to encrypt [MDP][] messages
* to identify the senders, recipients and authors of [Rhizome bundles][]
* to identify the parties in a [MeshMS conversation][]
#### Rhizome Secret
### Serval Signing ID
Every identity in the [Serval mesh network][] has a **Serval Signing ID**,
which is a unique 256-bit public key in the [Curve25519][] *crypto-sign* key
space that is generated at the same time as the [Serval ID](#serval-id) when
the identity is created. The Signing ID is used:
* to prevent forgery of [Serval Mesh network][] routing messages
* to authenticate non-encrypted [MDP][] messages
### DID
The **DID** ([Dialled Identity][]) is a telephone number, represented as a
string of five or more digits from the set `123456789#0*`. It is used by the
[DNA][] protocol to allow [Serval mesh network][] users to discover each other
by telephone number; the first step in establishing a mesh voice call.
### Name
The **Name** is a short, non-blank, non-empty, unstructured string assigned by
a human user to an identity. It is used to represent the identity to human
users, as it is more recognisable than a hexadecimal [SID](#serval-id) or a
[DID](#did) (telephone number).
The name is encoded using [UTF-8][]. Since it is intended for human
consumption, it may be constrained to contain only printable characters and no
carriage-motion characters (eg, TAB U+0009 or LF U+0010), and to not start or
end with white space.
### Rhizome Secret
The *Rhizome Secret* is a secret key, separate from the [SID](#serval-id)
secret, that is generated randomly for each new identity, and stored in the
@ -36,7 +67,7 @@ the [Bundle Secret][] of a bundle into its [manifest][], in the form of the
[Bundle Key][], thus relieving [Rhizome][] applications of the burden of having
to store and protect Bundle Secrets themselves.
#### PIN
### PIN
When an identity is created, it can optionally be given a PIN (passphrase). If
the PIN is *empty* then the identity is permanently unlocked (visible).
@ -50,13 +81,44 @@ identities.
If a PIN is lost and forgotten, then the identity (identities) it unlocks will
remain locked and unusable forever. There is no “master PIN” or back-door.
#### Identity unlocking
### Identity unlocking
All Keyring API requests can supply a passphrase using the optional **pin**
All Keyring requests can supply a passphrase using the optional **pin**
parameter, which unlocks all keyring identities protected by that password,
prior to performing the request. Serval DNA caches every password it receives
until the password is revoked using the *lock* request, so once an identity is
unlocked, it remains visible until explicitly locked.
prior to performing the request. Serval DNA caches every PIN it receives until
the PIN is revoked using the [lock request](#get-restful-keyring-lock), so once
an identity is unlocked, it remains visible until explicitly locked.
Keyring REST API common features
--------------------------------
### Keyring JSON result
All Keyring requests relating to a single identity that do not produce a
special response content for the outcome, return the following augmented [JSON
result][] object as the HTTP response content:
{
"http_status_code": ...,
"http_status_message": "...",
"identity": {
"sid": "<hex64>",
"identity": "<hex64>",
"did": "...",
"name": "..."
}
}
* the `sid` field is the [SID](#serval-id); a string containing 64 uppercase
hexadecimal digits
* the `identity` field is the [Signing Id](#serval-signing-id); a string
containing 64 uppercase hexadecimal digits
* the `did` field is the string [DID](#did); omitted if the identity has no DID
* the `name` field is the string [Name](#name); omitted if the identity has no
name
Keyring REST API operations
---------------------------
### GET /restful/keyring/identities.json
@ -65,10 +127,12 @@ The table columns are:
* **sid**: the [SID](#serval-id) of the identity, a string of 64 uppercase
hex digits
* **did**: the optional [DID][] (telephone number) of the identity, either
*null* or a string of five or more digits from the set `123456789#0*`
* **name**: the optional name of the identity, either *null* or a non-empty
string of [UTF-8] characters
* **identity**: the [Signing ID](#serval-signing-id) of the identity, a
string of 64 uppercase hex digits
* **did**: the optional [DID](#did) (telephone number) of the identity;
`null` if none is assigned
* **name**: the optional string [Name](#name) of the identity; `null` if none
is assigned
### GET /restful/keyring/add
@ -77,9 +141,20 @@ parameter is supplied, then the new identity will be protected by that
password, and the password will be cached by Serval DNA so that the new
identity is unlocked.
Returns [201 Created][201] if an identity is created; the [JSON
result](#keyring-json-result) describes the identity that was created.
### GET /restful/keyring/SID/remove
Removes an existing identity with a given [SID](#serval-id).
If there is no unlocked identity with the given SID, this request returns [404
Not Found][404]. Otherwise it returns [200 OK][200] and the [JSON
result](#keyring-json-result) describes the identity that was removed.
### GET /restful/keyring/SID/set
Sets the [DID][] and/or name of the unlocked identity that has the given
Sets the [DID](#did) and/or name of the unlocked identity that has the given
[SID](#serval-id). The following parameters are recognised:
* **did**: sets the DID (phone number); must be a string of five or more
@ -89,9 +164,18 @@ Sets the [DID][] and/or name of the unlocked identity that has the given
If there is no unlocked identity with the given SID, this request returns [404
Not Found][404].
### GET /restful/keyring/SID/lock
Locks an existing identity with a given [SID](#serval-id).
If there is no unlocked identity with the given SID, this request returns [404
Not Found][404]. Otherwise it returns [200 OK][200] and the [JSON
result](#keyring-json-result) describes the identity that was locked.
-----
**Copyright 2015 Serval Project Inc.**
**Copyright 2016 Flinders University**
![CC-BY-4.0](./cc-by-4.0.png)
Available under the [Creative Commons Attribution 4.0 International licence][CC BY 4.0].
@ -105,14 +189,18 @@ Available under the [Creative Commons Attribution 4.0 International licence][CC
[cryptographic identities]: http://developer.servalproject.org/dokuwiki/doku.php?id=content:tech:security_framework
[Curve25519]: https://en.wikipedia.org/wiki/Curve25519
[SID]: http://developer.servalproject.org/dokuwiki/doku.php?id=content:tech:sid
[DID]: http://developer.servalproject.org/dokuwiki/doku.php?id=content:tech:did
[Dialled Identity]: http://developer.servalproject.org/dokuwiki/doku.php?id=content:tech:did
[DNA]: http://developer.servalproject.org/dokuwiki/doku.php?id=content:tech:dna
[MDP]: ./Mesh-Datagram-Protocol.md
[Rhizome]: ./REST-API-Rhizome.md
[Rhizome bundles]: ./REST-API-Rhizome.md#bundle
[manifest]: ./REST-API-Rhizome.md#manifest
[Bundle Secret]: ./REST-API-Rhizome.md#bundle-secret
[Bundle Key]: ./REST-API-Rhizome.md#bundle-key
[MeshMS conversation]: ./REST-API-MeshMS.md#conversation
[JSON result]: ./REST-API.md#json-result
[JSON table]: ./REST-API.md#json-table
[UTF-8]: https://en.wikipedia.org/wiki/UTF-8
[200]: ./REST-API.md#200-ok
[201]: ./REST-API.md#201-created
[202]: ./REST-API.md#202-accepted

View File

@ -399,15 +399,15 @@ remove them. They reveal internal details of the storage of the bundle:
All Rhizome requests to fetch or insert a single bundle that do not produce a
special response content for the outcome, return the following augmented [JSON
result](#json-result) object as the HTTP response content:
result][] object as the HTTP response content:
{
"http_status_code": ...,
"http_status_message": "...",
"rhizome_bundle_status_code": ...,
"rhizome_bundle_status_message": "...",
"rhizome_payload_status_code": ...,
"rhizome_payload_status_message": "..."
"http_status_code": ...,
"http_status_message": "...",
"rhizome_bundle_status_code": ...,
"rhizome_bundle_status_message": "...",
"rhizome_payload_status_code": ...,
"rhizome_payload_status_message": "..."
}
* the `rhizome_bundle_status_code` field is the integer [bundle status code](#bundle-status-code)
@ -946,6 +946,7 @@ Available under the [Creative Commons Attribution 4.0 International licence][CC
[Serval Mesh network]: http://developer.servalproject.org/dokuwiki/doku.php?id=content:tech:mesh_network
[Serval DNA]: ../README.md
[REST-API]: ./REST-API.md
[JSON result]: ./REST-API.md#json-result
[store and forward]: https://en.wikipedia.org/wiki/Store_and_forward
[SID]: ./REST-API-Keyring.md#serval-id
[Keyring]: ./REST-API-Keyring.md

View File

@ -442,7 +442,7 @@ for example, some *404* responses from Rhizome have phrases like, “Bundle not
found”, “Payload not found”, etc.
Some responses augment the *JSON result* object with extra fields; for example,
[Rhizome JSON result](#rhizome-json-result).
[Rhizome JSON result][] and [Keyring JSON result][].
### JSON table
@ -521,6 +521,8 @@ Available under the [Creative Commons Attribution 4.0 International licence][CC
[Internet Media Type]: https://www.iana.org/assignments/media-types/media-types.xhtml
[Rhizome bundle]: ./REST-API-Rhizome.md#bundle
[Rhizome manifest]: ./REST-API-Rhizome.md#manifest
[Rhizome JSON result]: ./REST-API-Rhizome.md#rhizome-json-result
[Keyring JSON result]: ./REST-API-Keyring.md#keyring-json-result
[bundle secret]: ./REST-API-Rhizome.md#bundle-secret
[text+binarysig format]: ./REST-API-Rhizome.md#textbinarysig-manifest-format
[JSON]: https://en.wikipedia.org/wiki/JSON

View File

@ -91,6 +91,11 @@ public class ServalDClient implements ServalDHttpConnectionFactory {
return KeyringCommon.addIdentity(this, did, name, pin);
}
public KeyringIdentity keyringRemove(SubscriberId sid, String pin) throws ServalDInterfaceException, IOException
{
return KeyringCommon.removeIdentity(this, sid, pin);
}
public RhizomeBundleList rhizomeListBundles() throws ServalDInterfaceException, IOException
{
RhizomeBundleList list = new RhizomeBundleList(this);

View File

@ -238,13 +238,34 @@ public class KeyringCommon
query_params.add(new ServalDHttpConnectionFactory.QueryParam("pin", pin));
HttpURLConnection conn = connector.newServalDHttpConnection("/restful/keyring/add", query_params);
conn.connect();
Status status = receiveRestfulResponse(conn, HttpURLConnection.HTTP_CREATED);
try {
decodeRestfulStatus(status);
dumpStatus(status, System.err);
if (status.identity == null)
throw new ServalDInterfaceException("invalid JSON response; missing identity");
return status.identity;
}
finally {
if (status.input_stream != null)
status.input_stream.close();
}
}
public static KeyringIdentity removeIdentity(ServalDHttpConnectionFactory connector, SubscriberId sid, String pin)
throws IOException, ServalDInterfaceException
{
Vector<ServalDHttpConnectionFactory.QueryParam> query_params = new Vector<ServalDHttpConnectionFactory.QueryParam>();
if (pin != null)
query_params.add(new ServalDHttpConnectionFactory.QueryParam("pin", pin));
HttpURLConnection conn = connector.newServalDHttpConnection("/restful/keyring/" + sid.toHex() + "/remove", query_params);
conn.connect();
Status status = receiveRestfulResponse(conn, HttpURLConnection.HTTP_OK);
try {
decodeRestfulStatus(status);
dumpStatus(status, System.err);
if (status.identity == null)
throw new ServalDInterfaceException("invalid JSON response; missing identity");
return status.identity;
}
finally {

View File

@ -81,6 +81,17 @@ public class Keyring {
System.exit(0);
}
static void remove(SubscriberId sid, String pin) throws ServalDInterfaceException, IOException, InterruptedException
{
ServalDClient client = new ServerControl().getRestfulClient();
KeyringIdentity id = client.keyringRemove(sid, pin);
System.out.println("sid=" + id.sid +
", did=" + id.did +
", name=" + id.name
);
System.exit(0);
}
public static void main(String... args)
{
if (args.length < 1)
@ -93,6 +104,8 @@ public class Keyring {
set(new SubscriberId(args[1]), args[2], args[3], args.length >= 5 ? args[4] : null);
else if (methodName.equals("add"))
add(args[1], args[2], args.length >= 4 ? args[3] : null);
else if (methodName.equals("remove"))
remove(new SubscriberId(args[1]), args.length >= 3 ? args[2] : null);
} catch (Exception e) {
e.printStackTrace();
System.exit(1);

View File

@ -368,7 +368,6 @@ void keyring_free_identity(keyring_identity *id)
link_stop_routing(id->subscriber);
bzero(id,sizeof(keyring_identity));
free(id);
return;
}
/*
@ -1423,7 +1422,7 @@ keyring_identity *keyring_create_identity(keyring_file *k, const char *pin)
{
DEBUGF(keyring, "k=%p", k);
/* Check obvious abort conditions early */
if (!k->bam) { WHY("keyring lacks BAM (not to be confused with KAPOW)"); return NULL; }
if (!k->bam) { WHY("keyring lacks BAM"); return NULL; }
if (!pin) pin="";
@ -1479,6 +1478,35 @@ static int write_random_slot(keyring_file *k, unsigned slot)
return 0;
}
/* Remove the given identity from the keyring by overwriting it's slot in the keyring file with
* random data, and unlinking it from the in-memory cache list. Does NOT call
* keyring_free_identity(id), so the identity's contents remain intact; the caller must free the
* identity if desired.
*/
void keyring_destroy_identity(keyring_file *k, keyring_identity *id)
{
DEBUGF(keyring, "k=%p, id=%p", k, id);
if (!k->bam) {
WHY("keyring lacks BAM");
return;
}
assert(id->slot != 0);
DEBUGF(keyring, "Destroy identity in slot %u", id->slot);
// Mark the slot as unused in the BAM.
set_slot(k, id->slot, 0);
// Fill the slot in the file with random bytes.
write_random_slot(k, id->slot);
// Unlink the identity from the in-memory cache.
keyring_identity **i = &k->identities;
while (*i && *i != id)
i = &(*i)->next;
if (*i == id)
*i = id->next;
}
int keyring_commit(keyring_file *k)
{
DEBUGF(keyring, "k=%p", k);

View File

@ -132,6 +132,7 @@ int keyring_commit(keyring_file *k);
keyring_identity *keyring_inmemory_identity();
void keyring_free_identity(keyring_identity *id);
keyring_identity *keyring_create_identity(keyring_file *k, const char *pin);
void keyring_destroy_identity(keyring_file *k, keyring_identity *id);
void keyring_identity_extract(const keyring_identity *id, const char **didp, const char **namep);
int keyring_load_from_dump(keyring_file *k, unsigned entry_pinc, const char **entry_pinv, FILE *input);
int keyring_dump(keyring_file *k, XPRINTF xpf, int include_secret);

View File

@ -255,6 +255,39 @@ static int app_keyring_add(const struct cli_parsed *parsed, struct cli_context *
return 0;
}
DEFINE_CMD(app_keyring_remove, 0,
"Remove an identity from the keyring",
"keyring","remove" KEYRING_PIN_OPTIONS,"<sid>");
static int app_keyring_remove(const struct cli_parsed *parsed, struct cli_context *context)
{
DEBUG_cli_parsed(verbose, parsed);
const char *sidhex;
if (cli_arg(parsed, "sid", &sidhex, str_is_subscriber_id, "") == -1)
return -1;
sid_t sid;
if (str_to_sid_t(&sid, sidhex) == -1){
keyring_free(keyring);
keyring = NULL;
return WHY("str_to_sid_t() failed");
}
if (!(keyring = keyring_open_instance_cli(parsed)))
return -1;
keyring_identity *id = keyring_find_identity_sid(keyring, &sid);
int r=0;
if (!id)
r=WHY("No matching SID");
keyring_destroy_identity(keyring, id);
if (keyring_commit(keyring) == -1) {
keyring_free(keyring);
return WHY("Could not destroy identity");
}
cli_output_identity(context, id);
keyring_free_identity(id);
keyring_free(keyring);
keyring = NULL;
return r;
}
DEFINE_CMD(app_keyring_set_did, 0,
"Set the DID for the specified SID (must supply PIN to unlock the SID record in the keyring)",
"keyring", "set","did" KEYRING_PIN_OPTIONS,"<sid>","<did>","<name>", "[<new_pin>]");
@ -305,6 +338,7 @@ DEFINE_CMD(app_keyring_set_tag, 0,
"keyring", "set","tag" KEYRING_PIN_OPTIONS,"<sid>","<tag>","<value>");
static int app_keyring_set_tag(const struct cli_parsed *parsed, struct cli_context *context)
{
DEBUG_cli_parsed(verbose, parsed);
const char *sidhex, *tag, *value;
if (cli_arg(parsed, "sid", &sidhex, str_is_subscriber_id, "") == -1 ||
cli_arg(parsed, "tag", &tag, NULL, "") == -1 ||

View File

@ -33,6 +33,7 @@ DECLARE_HANDLER("/restful/keyring/", restful_keyring_);
static HTTP_HANDLER restful_keyring_identitylist_json;
static HTTP_HANDLER restful_keyring_add;
static HTTP_HANDLER restful_keyring_remove;
static HTTP_HANDLER restful_keyring_set;
static int restful_keyring_(httpd_request *r, const char *remainder)
@ -57,7 +58,11 @@ static int restful_keyring_(httpd_request *r, const char *remainder)
}
else if (parse_sid_t(&r->sid1, remainder, -1, &end) != -1) {
remainder = end;
if (strcmp(remainder, "/set") == 0) {
if (strcmp(remainder, "/remove") == 0) {
handler = restful_keyring_remove;
remainder = "";
}
else if (strcmp(remainder, "/set") == 0) {
handler = restful_keyring_set;
remainder = "";
}
@ -228,7 +233,25 @@ static int restful_keyring_add(httpd_request *r, const char *remainder)
}
if (keyring_commit(keyring) == -1)
return http_request_keyring_response(r, 500, "Could not store new identity");
return http_request_keyring_response_identity(r, 200, id);
return http_request_keyring_response_identity(r, 201, id);
}
static int restful_keyring_remove(httpd_request *r, const char *remainder)
{
if (*remainder)
return 404;
const char *pin = http_request_get_query_param(&r->http, "pin");
if (pin)
keyring_enter_pin(keyring, pin);
keyring_identity *id = keyring_find_identity_sid(keyring, &r->sid1);
if (!id)
return http_request_keyring_response(r, 404, "Identity not found");
keyring_destroy_identity(keyring, id);
if (keyring_commit(keyring) == -1)
return http_request_keyring_response(r, 500, "Could not erase removed identity");
int ret = http_request_keyring_response_identity(r, 200, id);
keyring_free_identity(id);
return ret;
}
static int restful_keyring_set(httpd_request *r, const char *remainder)

View File

@ -2,7 +2,8 @@
# Tests for Serval keyring
#
# Copyright 2012-2013 Serval Project, Inc.
# Copyright 2012-2015 Serval Project, Inc.
# Copyright 2016 Flinders University
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
@ -47,6 +48,40 @@ setup_instances() {
done
}
doc_KeyringAdd="Add new identity to keyring"
test_KeyringAdd() {
executeOk_servald keyring add ''
tfw_cat --stderr
executeOk_servald keyring list
assert_keyring_list 1
executeOk_servald keyring add ''
tfw_cat --stderr
executeOk_servald keyring list
assert_keyring_list 2
}
doc_KeyringRemove="Remove identity from keyring"
setup_KeyringRemove() {
setup
executeOk_servald keyring add ''
extract_stdout_keyvalue SID1 sid "$rexp_sid"
executeOk_servald keyring add ''
extract_stdout_keyvalue SID2 sid "$rexp_sid"
executeOk_servald keyring list
assert_keyring_list 2
assert [ "$SID1" != "$SID2" ]
}
test_KeyringRemove() {
executeOk_servald keyring remove "$SID1"
executeOk_servald keyring list
assert_keyring_list 1
assertStdoutGrep --matches=0 "^$SID1:"
assertStdoutGrep --matches=1 "^$SID2:"
executeOk_servald keyring remove "$SID2"
executeOk_servald keyring list
assert_keyring_list 0
}
doc_KeyringCreate="Create keyring destroys existing keys"
test_KeyringCreate() {
for i in {1..20}

View File

@ -116,6 +116,25 @@ test_keyringAddDidName() {
assertStdoutGrep --stderr --matches=1 "^${rexp_sid}:${rexp_id}:987654321:Joe Bloggs\$"
}
doc_keyringRemove="Java API remove existing identity"
setup_keyringRemove() {
IDENTITY_COUNT=2
setup
}
test_keyringRemove() {
executeJavaOk org.servalproject.test.Keyring remove "$SIDA1"
tfw_cat --stdout --stderr
assertStdoutGrep --matches=1 "sid=$SIDA1, did=${DIDA1:-null}, name=${NAMEA1:-null}$"
executeOk_servald keyring list
assert_keyring_list 1
assertStdoutGrep --stderr --matches=0 "$SIDA1"
assertStdoutGrep --stderr --matches=1 "^$SIDA2:${rexp_id}:$DIDA2:$NAMEA2\$"
executeJavaOk org.servalproject.test.Keyring remove "$SIDA2"
tfw_cat --stdout --stderr
executeOk_servald keyring list
assert_keyring_list 0
}
doc_keyringSetDidName="Java API set DID and name"
setup_keyringSetDidName() {
IDENTITY_COUNT=2

View File

@ -140,7 +140,7 @@ test_keyringAdd() {
"http://$addr_localhost:$PORTA/restful/keyring/add"
tfw_cat http.headers add.json
tfw_preserve add.json
assertStdoutIs '200'
assertStdoutIs '201'
SID="$(jq -r '.identity.sid' add.json)"
ID="$(jq -r '.identity.identity' add.json)"
assert matches_rexp "^${rexp_sid}$" "$SID"
@ -164,7 +164,7 @@ test_keyringAddPin() {
"http://$addr_localhost:$PORTA/restful/keyring/add?pin=1234"
tfw_cat http.headers add.json
tfw_preserve add.json
assertStdoutIs '200'
assertStdoutIs '201'
SID="$(jq -r '.identity.sid' add.json)"
ID="$(jq -r '.identity.identity' add.json)"
executeOk_servald keyring list
@ -187,6 +187,28 @@ test_keyringAddPin() {
assertJq ids.json 'contains([{"sid": "'$SIDA1'"}])'
}
doc_keyringRemove="HTTP RESTful remove keyring identity"
setup_keyringRemove() {
IDENTITY_COUNT=2
setup
}
test_keyringRemove() {
executeOk curl \
--silent --show-error --write-out '%{http_code}' \
--output add.json \
--dump-header http.headers \
--basic --user harry:potter \
"http://$addr_localhost:$PORTA/restful/keyring/$SIDA1/remove"
tfw_cat http.headers add.json
tfw_preserve add.json
assertStdoutIs '200'
SID="$(jq -r '.identity.sid' add.json)"
assert [ "$SID" = "$SIDA1" ]
executeOk_servald keyring list
assert_keyring_list 1
assertStdoutGrep --stderr --matches=0 "^$SID:"
}
doc_keyringSetDidName="HTTP RESTful set DID and name"
setup_keyringSetDidName() {
IDENTITY_COUNT=2