tahoe-lafs/docs/proposed/GridID.txt

= Grid Identifiers =

What makes up a Tahoe "grid"? The rough answer is a fairly-stable set of
Storage Servers.

The read- and write- caps that point to files and directories are scoped to a
particular set of servers. The Tahoe peer-selection and erasure-coding
algorithms provide high availability as long as there is significant overlap
between the servers that were used for upload and the servers that are
available for subsequent download. When new peers are added, the shares will
get spread out in the search space, so clients must work harder to download
their files. When peers are removed, shares are lost, and file health is
threatened. Repair bandwidth must be used to generate new shares, so cost
increases with the rate of server departure. If servers leave the grid too
quickly, repair may not be able to keep up, and files will be lost.

So to get long-term stability, we need that peer set to remain fairly stable.
A peer which joins the grid needs to stick around for a while.

== Multiple Grids ==

The current Tahoe read-cap format doesn't admit the existence of multiple
grids. In fact, the "URI:" prefix implies that these cap strings are
universal: it suggests that this string (plus some protocol definition) is
completely sufficient to recover the file.

However, there are a variety of reasons why we may want to have more than one
Tahoe grid in the world:

 * scaling: there are a variety of problems that are likely to be encountered
   as we attempt to grow a Tahoe grid from a few dozen servers to a few
   thousand, some of which are easier to deal with than others. Maintaining
   connections to servers and keeping up-to-date on the locations of servers
   is one issue. There are design improvements that can work around these,
   but they will take time, and we may not want to wait for that work to be
   done. Begin able to deploy multiple grids may be the best way to get a
   large number of clients using tahoe at once.

 * managing quality of storage, storage allocation: the members of a
   friendnet may want to restrict access to storage space to just each other,
   and may want to run their grid without involving any external coordination

 * commercial goals: a company using Tahoe may want to restrict access to
   storage space to just their customers

 * protocol upgrades, development: new and experimental versions of the tahoe
   software may need to be deployed and analyzed in isolation from the grid
   that clients are using for active storage

So if we define a grid to be a set of storage servers, then two distinct
grids will have two distinct sets of storage servers. Clients are free to use
whichever grid they like (and have permission to use), however each time they
upload a file, they must choose a specific grid to put it in. Clients can
upload the same file to multiple grids in two separate upload operations.

== Grid IDs in URIs ==

Each URI needs to be scoped to a specific grid, to avoid confusion ("I looked
for URI123 and it said File Not Found.. oh, which grid did you upload that
into?"). To accomplish this, the URI will contain a "grid identifier" that
references a specific Tahoe grid. The grid ID is shorthand for a relatively
stable set of storage servers.

To make the URIs actually Universal, there must be a way to get from the grid
ID to the actual grid. This document defines a protocol by which a client
that wants to download a file from a previously-unknown grid will be able to
locate and connect to that grid.

== Grid ID specification ==

The Grid ID is a string, using a fairly limited character set, alphanumerics
plus possibly a few others. It can be very short: a gridid of just "0" can be
used. The gridID will be copied into the cap string for every file that is
uploaded to that grid, so there is pressure to keep them short.

The cap format needs to be able to distinguish the gridID from the rest of
the cap. This could be expressed in DNS-style dot notation, for example the
directory write-cap with a write-key of "0ZrD.." that lives on gridID "foo"
could be expressed as "D0ZrDNAHuxs0XhYJNmkdicBUFxsgiHzMdm.foo" .

 * design goals: non-word-breaking, double-click-pasteable, maybe
   human-readable (do humans need to know which grid is being used? probably
   not).
 * does not need to be Secure (i.e. long and unguessable), but we must
   analyze the sorts of DoS attack that can result if it is not (and even
   if it is)
 * does not need to be human-memorable, although that may assist debugging
   and discussion ("my file is on grid 4, where is yours?)
 * *does* need to be unique, but the total number of grids is fairly small
   (counted in the hundreds or thousands rather than millions or billions)
   and we can afford to coordinate the use of short names. Folks who don't
   like coordination can pick a largeish random string.

Each announcement that a Storage Server publishes (to introducers) will
include its grid id. If a server participates in multiple grids, it will make
multiple announcements, each with a single grid id. Clients will be able to
ask an introducer for information about all storage servers that participate
in a specific grid.

Clients are likely to have a default grid id, to which they upload files. If
a client is adding a file to a directory that lives in a different grid, they
may upload the file to that other grid instead of their default.

== Getting from a Grid ID to a grid ==

When a client decides to download a file, it starts by unpacking the cap and
extracting the grid ID.

Then it attempts to connect to at least one introducer for that grid, by
leveraging DNS:

 hash $GRIDID id (with some tag) to get a long base32-encoded string: $HASH

 GET http://tahoe-$HASH.com/introducer/gridid/$GRIDID

 the results should be a JSON-encoded list of introducer FURLs

 for extra redundancy, if that query fails, perform the following additional
 queries:

  GET http://tahoe-$HASH.net/introducer/gridid/$GRIDID
  GET http://tahoe-$HASH.org/introducer/gridid/$GRIDID
  GET http://tahoe-$HASH.tv/introducer/gridid/$GRIDID
  GET http://tahoe-$HASH.info/introducer/gridid/$GRIDID
   etc
  GET http://tahoe-grids.allmydata.com/introducer/gridid/$GRIDID

 The first few introducers should be able to announce other introducers, via
 the distributed gossip-based introduction scheme of #68.

Properties:

 * claiming a grid ID is cheap: a single domain name registration (in an
   uncontested namespace), and a simple web server. allmydata.com can publish
   introducer FURLs for grids that don't want to register their own domain.

 * lookup is at least as robust as DNS. By using benevolent public services
   like tahoe-grids.allmydata.com, reliability can be increased further. The
   HTTP fetch can return a list of every known server node, all of which can
   act as introducers.

 * not secure: anyone who can interfere with DNS lookups (or claims
   tahoe-$HASH.com before you do) can cause clients to connect to their
   servers instead of yours. This admits a moderate DoS attack against
   download availability. Performing multiple queries (to .net, .org, etc)
   and merging the results may mitigate this (you'll get their servers *and*
   your servers; the download search will be slower but is still likely to
   succeed). It may admit an upload DoS attack as well, or an upload
   file-reliability attack (trick you into uploading to unreliable servers)
   depending upon how the "server selection policy" (see below) is
   implemented.

Once the client is connected to an introducer, it will see if there is a
Helper who is willing to assist with the upload or download. (For download,
this might reduce the number of connections that the grid's storage servers
must deal with). If not, ask the introducers for storage servers, and connect
to them directly.

== Controlling Access ==

The introducers are not used to enforce access control. Instead, a system of
public keys are used.

There are a few kinds of access control that we might want to implement:

 * protect storage space: only let authorized clients upload/consume storage
 * protect download bandwidth: only give shares to authorized clients
 * protect share reliability: only upload shares to "good" servers

The first two are implemented by the server, to protect their resources. The
last is implemented by the client, to avoid uploading shares to unreliable
servers (specifically, to maximize the utility of the client's limited upload
bandwidth: there's no problem with putting shares on unreliable peers per se,
but it is a problem if doing so means the client won't put a share on a more
reliable peer).

The first limitation (protect storage space) will be implemented by public
keys and signed "storage authority" certificates. The client will present
some credentials to the storage server to convince it that the client
deserves the space. When storage servers are in this mode, they will have a
certificate that names a public key, and any credentials that can demonstrate
a path from that key will be accepted. This scheme is described in
docs/accounts-pubkey.txt .

The second limitation is unexplored. The read-cap does not currently contain
any notion of who must pay for the bandwidth incurred.

The third limitation (only upload to "good" servers), when enabled, is
implemented by a "server selection policy" on the client side, which defines
which server credentials will be accepted. This is just like the first
limitation in reverse. Before clients consider including a server in their
peer selection algorithm, they check the credentials, and ignore any that do
not meet them.

This means that a client may not wish to upload anything to "foreign grids",
because they have no promise of reliability. The reasons that a client might
want to upload to a foreign grid need to be examined: reliability may not be
important, or it might be good enough to upload the file to the client's
"home grid" instead.

The server selection policy is intended to be fairly open-ended: we can
imagine a policy that says "upload to any server that has a good reputation
among group X", or more complicated schemes that require less and less
centralized management. One important and simple scheme is to simply have a
list of acceptable keys: a friendnet with 5 members would include 5 such keys
in each policy, enabling every member to use the services of the others,
without having a single central manager with unilateral control over the
definition of the group.

== Closed Grids ==

To implement these access controls, each client needs to be configured with
three things:

 * home grid ID (used to find introducers, helpers, storage servers)
 * storage authority (certificate to enable uploads)
 * server selection policy (identify good/reliable servers)

If the server selection policy indicates centralized control (i.e. there is
some single key X which is used to sign the credentials for all "good"
servers), then this could be built in to the grid ID. By using the base32
hash of the pubkey as the grid ID, clients would only need to be configured
with two things: the grid ID, and their storage authority. In this case, the
introducer would provide the pubkey, and the client would compare the hashes
to make sure they match. This is analogous to how a TubID is used in a FURL.

Such grids would have significantly larger grid IDs, 24 characters or more.