CODE OVERVIEW

A brief map to where the code lives in this distribution:

 src/zfec: the erasure-coding library, turns data into shares and back again.
           When installed, this provides the 'zfec' package.

 src/Crypto: a modified version of PyCrypto, which includes a patch to
             greatly improve the speed of CTR mode, which unfortunately makes
             it incompatible with the normal version of PyCrypto. When
             installed, this provides the 'allmydata.Crypto' package.

 src/allmydata: the bulk of the code for this project. When installed, this
                provides the 'allmydata' package

Within src/allmydata/ :

 interfaces.py: declaration of zope.interface-style Interfaces for most
                components, also defines Foolscap RemoteInterfaces for
                all remotely-accessible components

 node.py: the base Node, which handles connection establishment and
          application startup

 client.py, introducer_and_vdrive.py:
   these are two specialized subclasses of Node, for users and the central
   introducer/vdrive handler, respectively. Each works by assembling a
   collection of services underneath a top-level Node instance.

 introducer.py: node introduction handlers, client is used by client.py,
                server is used by introducer_and_vdrive.py

 storageserver.py: provides storage services to other nodes

 codec.py: low-level erasure coding, wraps zfec

 encode.py: handles turning data into shares and blocks, computes hash trees
 upload.py: upload-side peer selection, reading data from upload sources

 download.py: download-side peer selection, share retrieval, decoding

 dirnode.py: implements the directory nodes. One part runs on the
             global vdrive server, the other runs inside a client
             (starting with vdrive.py)

 vdrive.py: provides a client-side service that accesses the global
            shared virtual drive and the per-user private drive.

 webish.py, web/*.xhtml: provides the web frontend, using a Nevow server

 uri.py: URI packing/parsing routines

 hashtree.py: Merkle hash tree classes

 debugshell.py, manhole.py: SSH-connected python shell, for debug purposes

 util/*.py: misc utility classes

 test/*.py: unit tests


Both the client and the central introducer-and-vdrive node runs as a tree of
(twisted.application.service) Service instances. The Foolscap "Tub" is one of
these. Client nodes have an Uploader service and a Downloader service that
turn data into URIs and back again. They also have a VirtualDrive service
which provides access to the single global shared filesystem.

The Uploader is given an "upload source" (which could be an open filehandle,
a filename on local disk, or even a string), and returns a Deferred that
fires with the URI once the upload is complete. The Downloader is given a URI
and a "download target" (an open filehandle, filename, or instructions to
provide a string), and fires a Deferred with a target-specific value when the
download is complete. The source/target API is intended to make it easy to
stream the incoming data to a media player or HTTP connection without having
to consume a lot of memory for buffering.