serval-dna/doc/Cooee.md

Cooee Service Discovery Protocol

Serval Project, May 2014

Cooee is a protocol for discovering services offered by reachable nodes in a Serval mesh network. It is named after the Australian bush shout.

Cooee was funded by a grant from the New America Foundation's Open Technology Institute.

What is a Service?

A service is a program (called a server) running on a single network node, that accepts spontaneous requests from other programs (called clients) via the network, performs a useful function, and replies to the client via the network.

A server may decline to respond to some clients based on their SID (typically a blacklist or whitelist) or some other criterion such as load or some aspect of the request. In general, however, a service is not selective about which clients it will serve, and does not have prior knowledge of who will contact it, or when. In order to avoid service delays or absences, a server must be able to handle many requests concurrently.

Service protocols and state

Each service has its own particular protocol, and clients wishing to use the service must use that protocol.

A one-shot service protocol contains the entire request in a single MDP packet, and typically the response is a single packet sent back to the client's originating MDP port number. A one-shot service with very rapid reply generation may not need to be concurrent; it need only queue all incoming requests and deal with them in turn. In a one-shot service, it is possible for the server to be fully stateless, because it need not remember any prior requests or responses. Service congestion can be dealt with by simply dropping new requests when the queue is full. The absence of a response causes the client to re-send the request after some time-out.

More elaborate service protocols involve establishing a session between the client and server, which is discarded once the service has been performed. The Mesh Stream Protocol is commonly used for this kind of service, to provide lossless transmission of messages between client and server. Session-based servers must retain some state per session.

Server MDP port number

A server operates by listening for MDP packets sent to a single MDP port on its node. The port number may be reserved in advance for that service, which means that the service can have at most one server running at a time per node. Alternatively, the port number may be allocated whenever the server starts, so is not known in advance, but each node may support many servers offering the same service.

In general, clients that wish to use a service do not know the MDP port number or SID of the node that hosts the server. They must use Cooee to discover the SID and port number.

What is Cooee?

Cooee is a one-shot MDP service available on MDP port 11 of every node, that resolves service names to port numbers and other qualifying information.

In Cooee, every service is described by a stanza of name=value pairs, using the same UTF-8 text format as the Serval DNA configuration file.

A Cooee request packet contains one or more patterns that are each matched against the names in each service's stanza. The request packet is typically broadcast to all nodes in the reachable (local) Mesh network, and every service with a name that matches any pattern sends a reply packet containing the lines from its stanza that match.

How does a client use Cooee?

Client programs use Cooee by sending a broadcast packet to port 11 and collecting all the reply packets that it provokes. Any packet whose content does not strictly conform to the stanza grammar should be ignored.

For example, a client searching for a SOCKS5-over-MSP Internet access point may send a broadcast packet containing the following pattern:

socks5.msp.*

It may receive the following response from one node:

socks5.msp.port=34
socks5.msp.name.en=Filtered Internet service
socks5.msp.rx_bps=174000
socks5.msp.tx_bps=36000

... and the following response from another node:

socks5.msp.port=116
socks5.msp.name.en=Telco mobile data plan
socks5.msp.name.es=Móvil internet de Telco

The exact meanings of the lines in the replies is defined below.

Clients may simply ignore any lines that they do not recognize, or may present them to the human user to assist in choosing between services. For example, even though the .rx_bps and .tx_bps lines shown above may not be recognised by all clients, their presence or absence will not cause any client to disregard the service itself.

The remote (sending) SID of the Cooee reply packet gives the node that hosts the service, so a node cannot reply for services available on other nodes.

How does a server advertise itself using Cooee?

Every server that wishes to make itself discoverable via Cooee must know its own stanza, and must respond to Cooee requests on port 11 (as well as responding to the port on which it provides its own service). The MDP port sharing feature of Serval DNA allows many servers to listen on port 11 at the same time, and all of them will receive a copy of every packet received on port 11.

Whenever a server receives a Cooee request packet, it must match the requested pattern against its own stanza. If any lines match, it must construct and send a reply packet containing the matching lines. If a server provides many services by listening on many ports, then it will know one stanza per service, and if more than one of those stanzas match a Cooee request, then it must send one reply packet per matching stanza -- it cannot combine all the matches into a single packet.

This means that a single Cooee request may provoke more than one response packet from one node, if more than one server running on that node provides the service being sought.

Service description stanzas

A Cooee service description stanza is a block of UTF-8 text which conforms to the following grammar:

STANZA := LINE {1..}

LINE := NAME "=" VALUE "\n"

NAME := WORD ( "." WORD ){0..}
WORD := WCHAR {1..}
WCHAR := DIGIT | UPPERCASE | LOWERCASE | "_"
DIGIT := "0".."9"      (ASCII digits)
UPPERCASE := "A".."Z"  (ASCII uppercase)
LOWERCASE := "a".."z"  (ASCII lowercase)

VALUE := VCHAR {0..}
VCHAR := any UTF-8 character except "\n" and NUL

In other words, NAME is a sequence of one or more alphanumeric words separated by period characters, and VALUE is any sequence of characters not containing NUL (zero) or newline "\n".

Note that the grammar does not permit blank lines or white space before, in or after a NAME.

Standard service names

In general, every service description NAME must have the form:

servicename.protocol.attribute

Standard servicename values:

• socks5 -- a SOCKS5 forward proxy that provides access to the public Internet

• http_proxy -- a Web forward proxy that provides access to the public World Wide Web

Standard protocol values:

• mdp -- Mesh Datagram Protocol, generally only of use for one-shot services

• msp -- Mesh Stream Protocol, generally used by session-based services

Standard attribute values:

• port -- the MDP port number on which the service may be found, the value is either decimal (eg, 31) or hexadecimal with a 0x prefix (eg, 0x1f)

• name.en -- a textual description of the service that can be presented to an English-speaking human user (this is just a case of the name.XX attribute below)

• name.XX -- a textual description of the service in a human language identified by the XX two-letter ISO 639-1 code, eg, name.es for Spanish, name.ja for Japanese, etc.

Non-standard service names

Services may put any lines they wish into their Cooee stanza. However, names not defined in this standard run the risk of being incompatible with a future expansion of the standard.

If a service wishes to publish extra-standard information about itself, it must use NAME components that start with the underscore _ character. The Cooee standard will never use these names, so the only risk is collision with other services that have used the same name independently.

Service name patterns

A Cooee request pattern uses a glob-like syntax. For example:

[A-Z_]*.(ms|tc)p.(port|name.es)

will match all service names that start with an uppercase letter or underscore, and whose protocol is either msp or tcp, and only matches the port and name.es attributes.

• text -- matches exactly text, which may contain periods

• * -- matches zero or more of any character excluding period

• ** -- matches zero or more of any character including periods

• [SET] -- matches any single character in SET, where SET is a concatenation of:

  • C -- a single character C
  • A-B -- a range of characters in ASCII code order between A and B inclusive

• [!SET] -- matches any single character not in SET

• (one|two|...) matches either exactly one or two or any other alternatives separated by bars | and enclosed in parentheses

Some more examples:

• Find all MSP services: *.msp.**

• Find only the port numbers of all MDP services: *.mdp.port

• Find all lines of all stanzas of all services: **

• Find all non-standard lines of all stanzas of all services: (_**|**._**)

---

Copyright 2014 Serval Project Inc.
Available under the Creative Commons Attribution 4.0 International licence.