2007-04-19 20:55:13 +00:00
|
|
|
Allmydata, Inc. [1], provider of the "Allmydata" consumer backup product, is
|
2007-04-27 05:19:20 +00:00
|
|
|
pleased announce the first public release of "Tahoe", a secure, distributed
|
|
|
|
storage mesh with a free-software licence.
|
2007-04-19 20:55:13 +00:00
|
|
|
|
|
|
|
The source code that we are releasing is the current working prototype for
|
2007-04-30 04:24:55 +00:00
|
|
|
Allmydata's next-generation product. This release is targeted at hackers who
|
|
|
|
are willing to build from source and use a minimal, text-oriented web user
|
|
|
|
interface.
|
|
|
|
|
|
|
|
This prototype is not recommended for storage of confidential data nor for
|
2007-04-30 15:57:00 +00:00
|
|
|
data which is not otherwise backed up, but it already implements a functional
|
|
|
|
distributed storage mesh and is useful for experimentation, prototyping, and
|
|
|
|
extension.
|
2007-04-19 20:55:13 +00:00
|
|
|
|
|
|
|
|
2007-04-20 00:30:06 +00:00
|
|
|
INSTALLATION
|
2007-04-19 20:55:13 +00:00
|
|
|
|
2007-04-27 23:37:52 +00:00
|
|
|
This release of Tahoe works and passes all unit tests on Linux/x86,
|
|
|
|
Linux/amd64, Mac/Intel, Mac/PPC, Windows-native, and Cygwin.
|
2007-04-19 20:55:13 +00:00
|
|
|
|
2007-04-19 21:09:15 +00:00
|
|
|
To install, download the tarball [2], untar it, cd into the resulting
|
|
|
|
directory, and follow the directions in the README [3].
|
2007-04-19 20:55:13 +00:00
|
|
|
|
|
|
|
|
2007-04-20 00:30:06 +00:00
|
|
|
USAGE
|
|
|
|
|
2007-04-20 03:53:19 +00:00
|
|
|
Once installed, create a "client node" as described in the README. Instruct
|
|
|
|
this client node to connect to a specific "introducer node" by means of
|
|
|
|
config files in the client node's working directory. To join a public mesh,
|
|
|
|
copy in the .furl files for that mesh. To create a private mesh, run your
|
2007-04-27 23:37:52 +00:00
|
|
|
own introducer, and copy its .furl files.
|
2007-04-20 00:30:06 +00:00
|
|
|
|
|
|
|
Each client node runs a local webserver (enabled by writing the desired port
|
2007-04-20 03:53:19 +00:00
|
|
|
number into a file called 'webport'). The front page of this webserver shows
|
2007-04-20 00:30:06 +00:00
|
|
|
the node's status, including which introducer is being used and which other
|
2007-04-20 03:53:19 +00:00
|
|
|
nodes are connected. Links from the status page lead to others that give
|
2007-04-20 00:30:06 +00:00
|
|
|
access to a shared virtual filesystem, in which each directory is represented
|
2007-04-20 03:53:19 +00:00
|
|
|
by a separate page. Each directory page shows a list of the files available
|
2007-04-20 00:30:06 +00:00
|
|
|
there, with download links, and forms to upload new files.
|
2007-04-19 20:55:13 +00:00
|
|
|
|
2007-04-20 00:30:06 +00:00
|
|
|
Other ways to access the filesystem are planned, as well as other structures
|
|
|
|
than the single globally-shared namespace implemented by this release: please
|
|
|
|
see the roadmap.txt for some rough details.
|
2007-04-19 20:55:13 +00:00
|
|
|
|
|
|
|
|
2007-04-20 00:30:06 +00:00
|
|
|
HACKING AND COMMUNITY
|
2007-04-19 20:55:13 +00:00
|
|
|
|
2007-04-20 00:30:06 +00:00
|
|
|
Please join the mailing list [4] to discuss the ideas behind Tahoe and
|
2007-04-19 20:55:13 +00:00
|
|
|
extensions of and uses of Tahoe. Patches that extend and improve Tahoe are
|
|
|
|
gratefully accepted -- roadmap.txt [5] shows the next improvements that we
|
|
|
|
plan to make. You can browse the revision control history, source code, and
|
2007-04-19 21:11:00 +00:00
|
|
|
issue tracking at the Trac instance [6]. Please note the buildbot results
|
2007-04-19 20:55:13 +00:00
|
|
|
page [7], which show how Tahoe builds and passes unit tests on each checkin,
|
|
|
|
and the code coverage results [8] and percentage-covered graph [9], which
|
|
|
|
show how much of the Tahoe source code is currently exercised the test suite.
|
|
|
|
|
|
|
|
|
2007-04-20 00:30:06 +00:00
|
|
|
LICENCE
|
2007-04-19 20:55:13 +00:00
|
|
|
|
2007-04-27 20:47:15 +00:00
|
|
|
Tahoe is offered under the GNU General Public License (v2 or later), with the
|
|
|
|
added permission that, if you become obligated to release a derived work
|
|
|
|
under this licence (as per section 2.b), you may delay the fulfillment of
|
|
|
|
this obligation for up to 12 months.
|
2007-04-19 20:55:13 +00:00
|
|
|
|
|
|
|
|
2007-04-20 00:30:06 +00:00
|
|
|
NETWORK ARCHITECTURE
|
2007-04-19 20:55:13 +00:00
|
|
|
|
|
|
|
Each peer maintains a connection to each other peer. A single distinct
|
|
|
|
server called an "introducer" is used to discover other peers with which to
|
|
|
|
connect.
|
|
|
|
|
|
|
|
To store a file, the file is encrypted and erasure coded, and each resulting
|
|
|
|
share is uploaded to a different peer. The secure hash of the encrypted file
|
|
|
|
and the encryption key are packed into a URI, knowledge of which is necessary
|
|
|
|
and sufficient to recover the file.
|
|
|
|
|
|
|
|
To fetch a file, starting with the URI, a subset of shares is downloaded from
|
|
|
|
peers, the file is reconstructed from the shares, and then decrypted.
|
|
|
|
|
2007-04-20 00:30:06 +00:00
|
|
|
A single distinct server called a "vdrive server" maintains a global share
|
|
|
|
mapping from pathnames/filenames to URIs.
|
2007-04-19 20:55:13 +00:00
|
|
|
|
|
|
|
We are well aware of the limitations of decentralization and scalability
|
|
|
|
inherent in this prototype. In particular, the completely-connected property
|
|
|
|
of the mesh and the requirement of a single distinct introducer and vdrive
|
|
|
|
server limits the possible size of the mesh. We have plans to loosen these
|
|
|
|
limitations (see roadmap.txt [5]). Currently it should be noted that the
|
|
|
|
mesh already depends as little as possible on the accessibility and
|
2007-04-19 21:16:48 +00:00
|
|
|
correctness of the introduction server and the vdrive server. Also note that
|
2007-04-27 20:47:15 +00:00
|
|
|
the choice of which servers to use is easily configured -- you should be able
|
|
|
|
to set up a private mesh for you and your friends almost as easily as to
|
|
|
|
connect to our public test mesh.
|
2007-04-19 20:55:13 +00:00
|
|
|
|
|
|
|
|
2007-04-20 00:30:06 +00:00
|
|
|
SOFTWARE ARCHITECTURE
|
2007-04-19 20:55:13 +00:00
|
|
|
|
|
|
|
Tahoe is a "from the ground-up" rewrite, inspired by Allmydata's existing
|
2007-04-27 20:47:15 +00:00
|
|
|
consumer backup service. It is primarily written in Python.
|
2007-04-19 20:55:13 +00:00
|
|
|
|
|
|
|
It uses the Foolscap library [10] which provides a remote object protocol
|
|
|
|
inspired by the capability-secure "E" programming language [11]. Foolscap
|
2007-04-19 21:11:00 +00:00
|
|
|
allows us to express the intended behavior of the distributed mesh directly
|
2007-04-19 20:55:13 +00:00
|
|
|
in object-oriented terms while relying on a well-engineered, secure transport
|
|
|
|
layer.
|
|
|
|
|
|
|
|
The underlying networking is provided by the Twisted library [12].
|
|
|
|
Computationally intensive operations are performed in native compiled code,
|
|
|
|
such as the "zfec" library for fast erasure coding (also available
|
|
|
|
separately: [13]).
|
|
|
|
|
|
|
|
|
|
|
|
[1] http://allmydata.com
|
2007-04-30 06:05:39 +00:00
|
|
|
[2] http://allmydata.org/source/tahoe/tahoe-0.1.2-0-UNSTABLE.tar.gz
|
2007-04-27 20:47:15 +00:00
|
|
|
[3] http://allmydata.org/trac/tahoe/browser/README
|
|
|
|
[4] http://allmydata.org/cgi-bin/mailman/listinfo/tahoe-dev
|
|
|
|
[5] http://allmydata.org/trac/tahoe/browser/roadmap.txt
|
2007-04-27 23:12:44 +00:00
|
|
|
[6] http://allmydata.org/trac/tahoe
|
2007-04-20 03:11:21 +00:00
|
|
|
[7] http://allmydata.org/buildbot
|
2007-04-27 20:47:15 +00:00
|
|
|
[8] http://allmydata.org/tahoe-figleaf/figleaf/
|
|
|
|
[9] http://allmydata.org/tahoe-figleaf-graph/hanford.allmydata.com-tahoe_figleaf.html
|
2007-04-19 20:55:13 +00:00
|
|
|
[10] http://twistedmatrix.com/trac/wiki/FoolsCap
|
|
|
|
[11] http://erights.org/
|
|
|
|
[12] http://twistedmatrix.com/
|
2007-04-27 23:12:44 +00:00
|
|
|
[13] http://allmydata.org/trac/tahoe/browser/src/zfec
|