Welcome to the AllMyData "tahoe" project. This project implements a scalable distributed fault-tolerant filestore. The basic idea is that the data in this filesystem is spread over all participating nodes, using an algorithm that can recover the data even if a majority of the nodes are no longer available. The main application code is in the 'allmydata' package, under src/allmydata/ . There is also a patched version of PyCrypto (adding a faster CTR-mode) in src/Crypto/ which gets installed to the 'allmydata.Crypto' package (since the API is different than the normal Crypto package). It also includes Zooko's PyFEC library, a fast python wrapper around the Rizzo 'fec' C library, installed to the 'pyfec' package and located in src/pyfec/ . DEPENDENCIES: * Python 2.4 or newer (tested against both 2.4 and 2.5) http://python.org/ * Twisted (tested against both 2.4 and 2.5) http://twistedmatrix.com/ The default subpackages should all be included by default, but in case they aren't they are the following: * core (the standard Twisted package) * web, trial, conch Note that Twisted requires zope.interface, and that the standard batteries-included Twisted distribution includes a copy. * Foolscap (0.1.2 or newer) - note: since the Foolscap wire protocol is not yet compatible from one release to the next, make sure all of your nodes are using the same version of Foolscap http://twistedmatrix.com/trac/wiki/FoolsCap * Nevow (probably 0.9.0 or later) http://divmod.org/trac/wiki/DivmodNevow * PyOpenSSL (0.6 or later) http://pyopenssl.sourceforge.net * a C compiler * GNU make BUILDING: Just type 'make'. This works on Windows too, provided that you have the dependencies mentioned above (either a normal cygwin build or a mingw-style native build is supported by the makefile -- the cygwin build is the default). If the desired version of 'python' is not already on your PATH, then type 'make PYTHON=/path/to/your/preferred/python'. 'make test' runs the unit test suite. INSTALLING: If you're running on a debian system, use 'make deb-dapper' or 'make deb-sid' to construct a debian package named 'allmydata-tahoe', which you can then install. If not, you'll need to run three separate install steps, one for each of the three subpackages (allmydata, allmydata.Crypto, and pyfec). You may wish to use a different version of 'python' for these steps, or provide a --prefix or --root argument for the install. cd src/pyfec && python setup.py install && cd ../.. cd src/Crypto && python setup.py install && cd ../.. # the allmydata subpackage's setup.py script is in the root directory python setup.py install To test that all the modules got installed properly, start a python interpreter and import modules as follows: % python Python 2.4.4 (#2, Jan 13 2007, 17:50:26) [GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import allmydata.Crypto >>> import allmydata.interfaces >>> import fec >>> To run from a source tree (without installing first), type 'make', which will put all the necessary libraries into a local directory named instdir/lib/pythonN.N/site-packages/ , which you can then add to your PYTHONPATH . RUNNING: If you installed one of the debian packages constructed by "make deb-*" then it creates an 'allmydata-tahoe' executable, usually in /usr/bin . If you didn't install a package you can find allmydata-tahoe in instdir/scripts. This tool is used to create, start, and stop nodes. Each node lives in a separate base directory, inside of which you can add files to configure and control the node. Nodes also read and write files within that directory. A mesh consists of a single central 'queen' node and a large number of 'client' nodes. If you are joining an existing mesh, the queen node will already be running, and you'll just need to create a client node. If you're creating a brand new mesh, you'll need to create both a queen and a client (and then invite other people to create their own client nodes and join your mesh). The queen node is constructed by running 'allmydata-tahoe create-queen --basedir WHERE'. Once constructed, you can start the queen by running 'allmydata-tahoe start --basedir WHERE'. Inside that base directory, there will be a pair of files 'introducer.furl' and 'vdrive.furl'. Make a copy of these, as they'll be needed on the client nodes. To construct a client node, pick a new working directory for it, then run 'allmydata-tahoe create-client --basedir $HERE'. Copy the two .furl files from the queen into this new directory, then run 'allmydata-tahoe start --basedir $HERE'. After that, the client node should be off and running. The first thing it will do is connect to the queen and introduce itself to all other nodes on the mesh. You can follow its progress by looking at the $HERE/twistd.log file. To actually use the client, enable the web interface by writing a port number (like "8080") into a file named $HERE/webport and then restarting the node with 'allmydata-tahoe restart --basedir $HERE'. This will prompt the client node to run a webserver on the desired port, through which you can view, upload, download, and delete files.