mirror of
https://github.com/tahoe-lafs/tahoe-lafs.git
synced 2024-12-22 14:22:25 +00:00
d19d1058e0
This is a potentially disruptive and potentially ugly change to the code base, because I renamed the object that serves in both roles from "Queen" to "IntroducerAndVdrive", which is a bit of an ugly name. However, I think that clarity is important enough in this release to make this change. All unit tests pass. I'm now darcs recording this patch in order to pull it to other machines for more testing.
75 lines
3.0 KiB
Plaintext
75 lines
3.0 KiB
Plaintext
|
|
CODE OVERVIEW
|
|
|
|
A brief map to where the code lives in this distribution:
|
|
|
|
src/zfec: the erasure-coding library, turns data into shares and back again.
|
|
When installed, this provides the 'zfec' package.
|
|
|
|
src/Crypto: a modified version of PyCrypto, which includes a patch to
|
|
greatly improve the speed of CTR mode, which unfortunately makes
|
|
it incompatible with the normal version of PyCrypto. When
|
|
installed, this provides the 'allmydata.Crypto' package.
|
|
|
|
src/allmydata: the bulk of the code for this project. When installed, this
|
|
provides the 'allmydata' package
|
|
|
|
Within src/allmydata/ :
|
|
|
|
interfaces.py: declaration of zope.interface-style Interfaces for most
|
|
components, also defines Foolscap RemoteInterfaces for
|
|
all remotely-accessible components
|
|
|
|
node.py: the base Node, which handles connection establishment and
|
|
application startup
|
|
|
|
client.py, introducer_and_vdrive.py: two specialized subclasses of Node, for users
|
|
and the central introducer/vdrive handler, respectively
|
|
|
|
introducer.py: node introduction handlers, client is used by client.py,
|
|
server is used by introducer_and_vdrive.py
|
|
|
|
storageserver.py: provides storage services to other nodes
|
|
|
|
codec.py: low-level erasure coding, wraps zfec
|
|
|
|
encode.py: handles turning data into shares and blocks, computes hash trees
|
|
upload.py: upload-side peer selection, reading data from upload sources
|
|
|
|
download.py: download-side peer selection, share retrieval, decoding
|
|
|
|
filetable.py, vdrive.py: implements the current one-global-vdrive layer,
|
|
part runs on client nodes, part runs on the
|
|
central vdrive handler
|
|
|
|
webish.py, web/*.xhtml: provides the web frontend, using a Nevow server
|
|
|
|
workqueue.py, filetree/*.py: building blocks for the future filetree work
|
|
|
|
hashtree.py: Merkle hash tree classes
|
|
|
|
debugshell.py, manhole.py: SSH-connected python shell, for debug purposes
|
|
|
|
uri.py: URI packing/parsing routines
|
|
|
|
util/*.py: misc utility classes
|
|
|
|
test/*.py: unit tests
|
|
|
|
|
|
Both the client and the central introducer-and-vdrive node runs as a tree of
|
|
(twisted.application.service) Service instances. The Foolscap "Tub" is one of
|
|
these. Client nodes have an Uploader service and a Downloader service that turn
|
|
data into URIs and back again. They also have a VDrive service which provides
|
|
access to the single global shared filesystem.
|
|
|
|
The Uploader is given an "upload source" (which could be an open filehandle,
|
|
a filename on local disk, or even a string), and returns a Deferred that
|
|
fires with the URI once the upload is complete. The Downloader is given a URI
|
|
and a "download target" (an open filehandle, filename, or instructions to
|
|
provide a string), and fires a Deferred with a target-specific value when the
|
|
download is complete. The source/target API is intended to make it easy to
|
|
stream the incoming data to a media player or HTTP connection without having
|
|
to consume a lot of memory for buffering.
|
|
|