diff --git a/doc/Mesh-Streaming-Protocol.md b/doc/Mesh-Streaming-Protocol.md index 76616511..5cd1a14a 100644 --- a/doc/Mesh-Streaming-Protocol.md +++ b/doc/Mesh-Streaming-Protocol.md @@ -9,9 +9,7 @@ an HTTP session, or carry quasi-real-time streaming data. MSP uses the unreliable [MDP][] protocol to send datagram packets between the two end points. MSP uses sequence numbering, acknowledgement messages and a -sliding window to achieve eventual reliable delivery of all packets. MSP also -uses [network coding][] to transmit redundant copies of packets in advance -without the inefficiencies of simple packet repetition. +sliding window to achieve eventual reliable delivery of all packets. MSP was funded by a [grant][] from the [New America Foundation][NAF]'s [Open Technology Institute][OTI]. @@ -19,12 +17,184 @@ Technology Institute][OTI]. Protocol description -------------------- -TBC +Mesh Streaming Protocol (MSP) is a TCP-like protocol for providing reliable, +in-order delivery of messages via the Mesh Datagram Protocol (MDP). + +Being based on MDP all data will be encrypted end-to-end, and authentication +is implicit. Since encryption doesn't depend on negotiating a session token, +the first few packets of data can be sent before knowing if the connection +attempt is successful. + +The MSP protocol requires processing timed events for handling retransmission or +connection failures. It is necessary for the programmer to manually schedule +calls to handle these events. It would be possible to have this happen +automatically from a helper-thread, however the existing manual mode will +likely be retained in the long-term as it allows for greater flexibility. + +MSP is a work in progress, and has not been subjected to rigorous testing, so +expect speed humps and sub-optimal performance depending on your operating +conditions. Please report any issues that you may encounter so that MSP can +be improved for all users. Client API ---------- -TBC +The MSP API builds on the underlying MDP API. + +See `msp_client.h` for full function prototypes and pre-processor definitions. + +The provisional API is as follows. Expect this to evolve as MSP development +continues. + +#### Create an MSP socket +``` +int mdp_fd = mdp_socket(); +struct msp_sock *sock = msp_socket(mdp_fd); +``` + +#### Receive any queued inbound events on an MSP socket + +For example, if the file descriptor is identified as having waiting input using `poll()` or `select()` +``` +msp_recv(mdp_fd); +``` + +After calling `msp_recv()` you should then call `msp_processing()` as soon as possible, +so that it can schedule any necessary alarms for asynchronous processing in a timely manner. + +#### Handle any asynchronous processing alarms that are due +``` +time_ms_t next_time; +msp_processing(&next_time); +``` + +This will set `next_time` to the next time at which `msp_processing()` should be called. +You should attempt to call `msp_processing()` again at or before the time indicated. +Typically, this might be done with a `poll()` or `select()` with an appropriate time out, +so that if no input events occur before `next_time`, it can be called. + + +#### To listen for incoming MSP connections + +The socket can be bound to listen on all local SIDs (`BIND_ALL`), +only the primary SID of the node (`BIND_PRIMARY`), +or to any particular SID passed in via `addr.sid`. + +Any number of incoming connections can be handled by a single mdp socket. +An MSP socket will be allocated for each incoming connection with the local +and remote addresses of the connection. Each socket will initially be given +the same call-back handler as the original listen socket. + +``` +struct mdp_sockaddr addr; +addr.port = ; +addr.sid = ; +msp_set_local(sock, addr); +msp_listen(sock); +``` + +#### To connect to a remote socket; + +Only one outgoing connection is supported on a single mdp socket. You may call +`msp_set_local()` before `msp_set_remote()` to manually bind to a specific source address. +If you do not call set_local, the connection will be bound to the Primary +address on the next available port. + + +``` +struct mdp_sockaddr addr; +addr.port = ; +addr.sid = ; +msp_set_remote(sock, addr); +``` + +You may start sending data immediately. A small number of data packets can +be sent before we know that the connection has been accepted. +You must call `msp_processing()` to establish the connection. + +#### Queue a message for transmission; + +Use `msp_send()` to send messages. + +``` +int msp_send(struct msp_sock *sock, const uint8_t *payload, size_t len); +``` + +Data is not actually sent until the next call to `msp_processing()`, which +should be triggered as soon as possible. +Message boundaries are preserved from end to end. Zero length messages are not currently supported. + +In the event that there is insufficient buffer space to send the message, `msp_send()` +will return -1. If this occurs, you should wait until the next time your handler +function is called with the `MSP_STATE_DATAOUT` flag set in the `state` argument. + +#### Handling asynchronous events; + +The handler is specified by calling `msp_set_handler()` similar to the following: +``` +msp_set_handler(sock, handler_function, context); +``` + +The handler function should process incoming data and deal with other state changes related to the connection, as indicated in the example below. + +``` +int handler_function(struct msp_sock *sock, msp_state_t state, const uint8_t *payload, size_t len, void *context){ + + if (state & MSP_STATE_ERROR){ + // something went wrong, eg connection timeout or an unspecified error was returned from serval's daemon. + } + + if (payload && len){ + // process incoming message bytes and return 0 + // all incoming messages will be delivered in the same order they were sent + + // if you can't process the whole message now, return 1 + // later when you believe you will be able to process this message, call msp_processing again + } + + if (state & MSP_STATE_DATA_OUT){ + // a call to msp_send() should now succeed. + } + + if (state & MSP_STATE_SHUTDOWN_REMOTE){ + // the remote party has stopped sending data and would like to close. + // you may keep your end of the connection open and continue to send data. + // When both ends have shutdown the connection will be closed. + } + + if (state & MSP_STATE_CLOSED){ + // No matter how the socket it closed, this function should always be called exactly once with a closed state before the struct msp_sock* is free'd + // Any other resources associated with the socket should now be released. + } + + // if you return -1, the socket will be flagged a closed +} +``` + +#### Graceful shutdown of MSP handling on a socket + +Simply call `msp_shutdown(sock);`, replacing `sock` with the MSP socket in question. + +After calling shutdown, you can no longer queue any messages to send. Once both parties have +shutdown the socket and all remaining data has been transferred, the socket will be closed. + + +#### Closing an MSP connection/socket. + +Call `msp_close()` on the socket in question. + +Note that calling `msp_close()` immediately destroys all socket state on the local side, without negotiating with or notifying the remote side. + +This can be called at anytime, because it only marks the socket for cleanup, which will not occur until the next time `msp_processing()` is called. + +#### Free all sockets immediately; + +``` +msp_close_all(mdp_fd); +``` + +This immediately frees all MSP sockets. It cannot be called from within a handler function. + [Serval Project]: http://www.servalproject.org/