mirror of
https://github.com/tahoe-lafs/tahoe-lafs.git
synced 2024-12-30 09:48:56 +00:00
fix warnings from rst2html
Apparently the in-line link syntax with "<>" in them causes these warnings. I don't know why. I changed them all to a slightly more verbose syntax. Thanks to Mike Kazantzsev's review comment (https://github.com/tahoe-lafs/tahoe-lafs/pull/67#commitcomment-4561370), I moved the links to the end of each section.
This commit is contained in:
Zooko Wilcox-O'Hearn
parent
82579cec96
commit
0bebbe3290
@ -20,7 +20,7 @@ Tahoe-LAFS Architecture
|
||||
Overview
|
||||
========
|
||||
|
||||
(See the `docs/specifications directory <specifications>`_ for more details.)
|
||||
(See the `docs/specifications directory`_ for more details.)
|
||||
|
||||
There are three layers: the key-value store, the filesystem, and the
|
||||
application.
|
||||
@ -45,10 +45,10 @@ Allmydata.com used it for a backup service: the application periodically
|
||||
copies files from the local disk onto the decentralized filesystem. We later
|
||||
provide read-only access to those files, allowing users to recover them.
|
||||
There are several other applications built on top of the Tahoe-LAFS
|
||||
filesystem (see the `RelatedProjects
|
||||
<https://tahoe-lafs.org/trac/tahoe-lafs/wiki/RelatedProjects>`_ page of the
|
||||
wiki for a list).
|
||||
filesystem (see the RelatedProjects_ page of the wiki for a list).
|
||||
|
||||
.. _docs/specifications directory: specifications
|
||||
.. _RelatedProjects: https://tahoe-lafs.org/trac/tahoe-lafs/wiki/RelatedProjects
|
||||
|
||||
The Key-Value Store
|
||||
===================
|
||||
@ -316,7 +316,9 @@ commercially-run grid for which all of the storage servers are in a colo
|
||||
facility with high interconnect bandwidth. In this case, the helper is placed
|
||||
in the same facility, so the helper-to-storage-server bandwidth is huge.
|
||||
|
||||
See `<helper.rst>`_ for details about the upload helper.
|
||||
See helper.rst_ for details about the upload helper.
|
||||
|
||||
.. _helper.rst: helper.rst
|
||||
|
||||
|
||||
The Filesystem Layer
|
||||
@ -368,8 +370,10 @@ clients are responsible for renewing their leases on a periodic basis at
|
||||
least frequently enough to prevent any of the leases from expiring before the
|
||||
next renewal pass.
|
||||
|
||||
See `<garbage-collection.rst>`_ for further information, and for how to
|
||||
configure garbage collection.
|
||||
See garbage-collection.rst_ for further information, and for how to configure
|
||||
garbage collection.
|
||||
|
||||
.. _garbage-collection.rst: garbage-collection.rst
|
||||
|
||||
|
||||
File Repairer
|
||||
|
||||
@ -45,16 +45,17 @@ The item descriptions below use the following types:
|
||||
|
||||
a Twisted listening-port specification string, like "``tcp:80``" or
|
||||
"``tcp:3456:interface=127.0.0.1``". For a full description of the format,
|
||||
see `the Twisted strports documentation
|
||||
<https://twistedmatrix.com/documents/current/api/twisted.application.strports.html>`_.
|
||||
Please note, if interface= is not specified, Tahoe-LAFS will attempt to
|
||||
bind the port specified on all interfaces.
|
||||
see `the Twisted strports documentation`_. Please note, if interface= is
|
||||
not specified, Tahoe-LAFS will attempt to bind the port specified on all
|
||||
interfaces.
|
||||
|
||||
``FURL string``
|
||||
|
||||
a Foolscap endpoint identifier, like
|
||||
``pb://soklj4y7eok5c3xkmjeqpw@192.168.69.247:44801/eqpwqtzm``
|
||||
|
||||
.. _the Twisted strports documentation: https://twistedmatrix.com/documents/current/api/twisted.application.strports.html
|
||||
|
||||
|
||||
Node Types
|
||||
==========
|
||||
@ -103,7 +104,7 @@ set the ``tub.location`` option described below.
|
||||
|
||||
This controls where the node's web server should listen, providing node
|
||||
status and, if the node is a client/server, providing web-API service as
|
||||
defined in `webapi.rst <frontends/webapi.rst>_`.
|
||||
defined in webapi.rst_.
|
||||
|
||||
This file contains a Twisted "strports" specification such as "``3456``"
|
||||
or "``tcp:3456:interface=127.0.0.1``". The "``tahoe create-node``" or
|
||||
@ -286,6 +287,8 @@ set the ``tub.location`` option described below.
|
||||
used for files that usually (on a Unix system) go into ``/tmp``. The
|
||||
string will be interpreted relative to the node's base directory.
|
||||
|
||||
.. _webapi.rst: frontends/webapi.rst
|
||||
|
||||
|
||||
Client Configuration
|
||||
====================
|
||||
@ -303,7 +306,7 @@ Client Configuration
|
||||
``helper.furl = (FURL string, optional)``
|
||||
|
||||
If provided, the node will attempt to connect to and use the given helper
|
||||
for uploads. See `<helper.rst>`_ for details.
|
||||
for uploads. See helper.rst_ for details.
|
||||
|
||||
``key_generator.furl = (FURL string, optional)``
|
||||
|
||||
@ -339,7 +342,7 @@ Client Configuration
|
||||
ratios are more reliable, and small ``N``/``k`` ratios use less disk
|
||||
space. ``N`` cannot be larger than 256, because of the 8-bit
|
||||
erasure-coding algorithm that Tahoe-LAFS uses. ``k`` can not be greater
|
||||
than ``N``. See `<performance.rst>`_ for more details.
|
||||
than ``N``. See performance.rst_ for more details.
|
||||
|
||||
``shares.happy`` allows you control over how well to "spread out" the
|
||||
shares of an immutable file. For a successful upload, shares are
|
||||
@ -377,8 +380,11 @@ Client Configuration
|
||||
controlled by this parameter and will always use SDMF. We may revisit
|
||||
this decision in future versions of Tahoe-LAFS.
|
||||
|
||||
See `<specifications/mutable.rst>`_ for details about mutable file
|
||||
formats.
|
||||
See mutable.rst_ for details about mutable file formats.
|
||||
|
||||
.. _helper.rst: helper.rst
|
||||
.. _performance.rst: performance.rst
|
||||
.. _mutable.rst: specifications/mutable.rst
|
||||
|
||||
Frontend Configuration
|
||||
======================
|
||||
@ -395,30 +401,33 @@ HTTP
|
||||
directories and files, as well as a number of pages to check on the
|
||||
status of your Tahoe node. It also provides a machine-oriented "WAPI",
|
||||
with a REST-ful HTTP interface that can be used by other programs
|
||||
(including the CLI tools). Please see `<frontends/webapi.rst>`_ for full
|
||||
details, and the ``web.port`` and ``web.static`` config variables above.
|
||||
The `<frontends/download-status.rst>`_ document also describes a few WUI
|
||||
status pages.
|
||||
(including the CLI tools). Please see webapi.rst_ for full details, and
|
||||
the ``web.port`` and ``web.static`` config variables above. The
|
||||
`download-status.rst`_ document also describes a few WUI status pages.
|
||||
|
||||
CLI
|
||||
|
||||
The main "bin/tahoe" executable includes subcommands for manipulating the
|
||||
filesystem, uploading/downloading files, and creating/running Tahoe
|
||||
nodes. See `<frontends/CLI.rst>`_ for details.
|
||||
nodes. See CLI.rst_ for details.
|
||||
|
||||
SFTP, FTP
|
||||
|
||||
Tahoe can also run both SFTP and FTP servers, and map a username/password
|
||||
pair to a top-level Tahoe directory. See `<frontends/FTP-and-SFTP.rst>`_
|
||||
for instructions on configuring these services, and the ``[sftpd]`` and
|
||||
pair to a top-level Tahoe directory. See FTP-and-SFTP.rst_ for
|
||||
instructions on configuring these services, and the ``[sftpd]`` and
|
||||
``[ftpd]`` sections of ``tahoe.cfg``.
|
||||
|
||||
Drop-Upload
|
||||
|
||||
As of Tahoe-LAFS v1.9.0, a node running on Linux can be configured to
|
||||
automatically upload files that are created or changed in a specified
|
||||
local directory. See `<frontends/drop-upload.rst>`_ for details.
|
||||
local directory. See drop-upload.rst_ for details.
|
||||
|
||||
.. _download-status.rst: frontends/download-status.rst
|
||||
.. _CLI.rst: frontends/CLI.rst
|
||||
.. _FTP-and-SFTP.rst: frontends/FTP-and-SFTP.rst
|
||||
.. _drop-upload.rst: frontends/drop-upload.rst
|
||||
|
||||
|
||||
Storage Server Configuration
|
||||
@ -441,9 +450,8 @@ Storage Server Configuration
|
||||
that are being decommissioned: the ``storage/`` directory could be
|
||||
mounted read-only, while shares are moved to other servers. Note that
|
||||
this currently only affects immutable shares. Mutable shares (used for
|
||||
directories) will be written and modified anyway. See ticket `#390
|
||||
<https://tahoe-lafs.org/trac/tahoe-lafs/ticket/390>`_ for the current
|
||||
status of this bug. The default value is ``False``.
|
||||
directories) will be written and modified anyway. See ticket `#390`_ for
|
||||
the current status of this bug. The default value is ``False``.
|
||||
|
||||
``reserved_space = (str, optional)``
|
||||
|
||||
@ -479,7 +487,10 @@ Storage Server Configuration
|
||||
|
||||
These settings control garbage collection, in which the server will
|
||||
delete shares that no longer have an up-to-date lease on them. Please see
|
||||
`<garbage-collection.rst>`_ for full details.
|
||||
garbage-collection.rst_ for full details.
|
||||
|
||||
.. _#390: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/390
|
||||
.. _garbage-collection.rst: garbage-collection.rst
|
||||
|
||||
|
||||
Running A Helper
|
||||
@ -492,12 +503,12 @@ service.
|
||||
|
||||
``enabled = (boolean, optional)``
|
||||
|
||||
If ``True``, the node will run a helper (see `<helper.rst>`_ for
|
||||
details). The helper's contact FURL will be placed in
|
||||
``private/helper.furl``, from which it can be copied to any clients that
|
||||
wish to use it. Clearly nodes should not both run a helper and attempt to
|
||||
use one: do not create ``helper.furl`` and also define
|
||||
``[helper]enabled`` in the same node. The default is ``False``.
|
||||
If ``True``, the node will run a helper (see helper.rst_ for details).
|
||||
The helper's contact FURL will be placed in ``private/helper.furl``, from
|
||||
which it can be copied to any clients that wish to use it. Clearly nodes
|
||||
should not both run a helper and attempt to use one: do not create
|
||||
``helper.furl`` and also define ``[helper]enabled`` in the same node.
|
||||
The default is ``False``.
|
||||
|
||||
|
||||
Running An Introducer
|
||||
@ -588,7 +599,7 @@ This section describes these other files.
|
||||
``private/helper.furl``
|
||||
|
||||
If the node is running a helper (for use by other clients), its contact
|
||||
FURL will be placed here. See `<helper.rst>`_ for more details.
|
||||
FURL will be placed here. See helper.rst_ for more details.
|
||||
|
||||
``private/root_dir.cap`` (optional)
|
||||
|
||||
@ -650,7 +661,7 @@ Other files
|
||||
files. The web-API has a facility to block access to filecaps by their
|
||||
storage index, returning a 403 "Forbidden" error instead of the original
|
||||
file. For more details, see the "Access Blacklist" section of
|
||||
`<frontends/webapi.rst>`_.
|
||||
webapi.rst_.
|
||||
|
||||
|
||||
Example
|
||||
@ -693,4 +704,6 @@ Old Configuration Files
|
||||
Tahoe-LAFS releases before v1.3.0 had no ``tahoe.cfg`` file, and used
|
||||
distinct files for each item. This is no longer supported and if you have
|
||||
configuration in the old format you must manually convert it to the new
|
||||
format for Tahoe-LAFS to detect it. See `<historical/configuration.rst>`_.
|
||||
format for Tahoe-LAFS to detect it. See `historical/configuration.rst`_.
|
||||
|
||||
.. _historical/configuration.rst: historical/configuration.rst
|
||||
|
||||
@ -23,4 +23,6 @@ automatically, but older filesystems may not have it enabled::
|
||||
|
||||
If "dir_index" is present in the "features:" line, then you're all set. If
|
||||
not, you'll need to use tune2fs and e2fsck to enable and build the index. See
|
||||
`<http://wiki.dovecot.org/MailboxFormat/Maildir>`_ for some hints.
|
||||
`http://wiki.dovecot.org/MailboxFormat/Maildir`_ for some hints.
|
||||
|
||||
.. _`http://wiki.dovecot.org/MailboxFormat/Maildir`: http://wiki.dovecot.org/MailboxFormat/Maildir
|
||||
|
||||
@ -133,10 +133,9 @@ of the same name in the upload directory, it will be unlinked and replaced
|
||||
with an immutable file. (`#1712`_)
|
||||
|
||||
If a file in the upload directory is changed (actually relinked to a new
|
||||
file), then the old file is still present on the grid, and any other caps
|
||||
to it will remain valid. See `docs/garbage-collection.rst
|
||||
<../garbage-collection.rst>`_ for how to reclaim the space used by files
|
||||
that are no longer needed.
|
||||
file), then the old file is still present on the grid, and any other caps to
|
||||
it will remain valid. See `docs/garbage-collection.rst`_ for how to reclaim
|
||||
the space used by files that are no longer needed.
|
||||
|
||||
Unicode names are supported, but the local name of a file must be encoded
|
||||
correctly in order for it to be uploaded. The expected encoding is that
|
||||
@ -154,3 +153,6 @@ printed by ``python -c "import sys; print sys.getfilesystemencoding()"``.
|
||||
.. _`#1710`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1710
|
||||
.. _`#1711`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1711
|
||||
.. _`#1712`: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/1712
|
||||
|
||||
.. _docs/garbage-collection.rst: ../garbage-collection.rst
|
||||
|
||||
|
||||
@ -54,8 +54,7 @@ section of $NODEDIR/tahoe.cfg will cause the node to run a webserver on port
|
||||
This string is actually a Twisted "strports" specification, meaning you can
|
||||
get more control over the interface to which the server binds by supplying
|
||||
additional arguments. For more details, see the documentation on
|
||||
`twisted.application.strports
|
||||
<https://twistedmatrix.com/documents/current/api/twisted.application.strports.html>`_.
|
||||
`twisted.application.strports`_.
|
||||
|
||||
Writing "tcp:3456:interface=127.0.0.1" into the web.port line does the same
|
||||
but binds to the loopback interface, ensuring that only the programs on the
|
||||
@ -66,18 +65,19 @@ This webport can be set when the node is created by passing a --webport
|
||||
option to the 'tahoe create-node' command. By default, the node listens on
|
||||
port 3456, on the loopback (127.0.0.1) interface.
|
||||
|
||||
.. _twisted.application.strports: https://twistedmatrix.com/documents/current/api/twisted.application.strports.html
|
||||
|
||||
|
||||
Basic Concepts: GET, PUT, DELETE, POST
|
||||
======================================
|
||||
|
||||
As described in `docs/architecture.rst <../architecture.rst>`_, each file
|
||||
and directory in a Tahoe virtual filesystem is referenced by an identifier
|
||||
that combines the designation of the object with the authority to do something
|
||||
with it (such as read or modify the contents). This identifier is called a
|
||||
"read-cap" or "write-cap", depending upon whether it enables read-only or
|
||||
read-write access. These "caps" are also referred to as URIs (which may be
|
||||
confusing because they are not currently `RFC3986
|
||||
<https://tools.ietf.org/html/rfc3986>`_-compliant URIs).
|
||||
As described in `docs/architecture.rst`_, each file and directory in a Tahoe
|
||||
virtual filesystem is referenced by an identifier that combines the
|
||||
designation of the object with the authority to do something with it (such as
|
||||
read or modify the contents). This identifier is called a "read-cap" or
|
||||
"write-cap", depending upon whether it enables read-only or read-write
|
||||
access. These "caps" are also referred to as URIs (which may be confusing
|
||||
because they are not currently RFC3986_-compliant URIs).
|
||||
|
||||
The Tahoe web-based API is "REST-ful", meaning it implements the concepts of
|
||||
"REpresentational State Transfer": the original scheme by which the World
|
||||
@ -127,6 +127,9 @@ a plain text stack trace instead. If the Accept header contains ``*/*``, or
|
||||
``text/*``, or text/html (or if there is no Accept header), HTML tracebacks will
|
||||
be generated.
|
||||
|
||||
.. _RFC3986: https://tools.ietf.org/html/rfc3986
|
||||
.. _docs/architecture.rst: ../architecture.rst
|
||||
|
||||
|
||||
URLs
|
||||
====
|
||||
@ -501,9 +504,9 @@ Creating a New Directory
|
||||
|
||||
The metadata may have a "no-write" field. If this is set to true in the
|
||||
metadata of a link, it will not be possible to open that link for writing
|
||||
via the SFTP frontend; see `<FTP-and-SFTP.rst>`_ for details.
|
||||
Also, if the "no-write" field is set to true in the metadata of a link to
|
||||
a mutable child, it will cause the link to be diminished to read-only.
|
||||
via the SFTP frontend; see FTP-and-SFTP.rst_ for details. Also, if the
|
||||
"no-write" field is set to true in the metadata of a link to a mutable
|
||||
child, it will cause the link to be diminished to read-only.
|
||||
|
||||
Note that the web-API-using client application must not provide the
|
||||
"Content-Type: multipart/form-data" header that usually accompanies HTML
|
||||
@ -660,6 +663,8 @@ Creating a New Directory
|
||||
This operation will return an error if the parent directory is immutable,
|
||||
or already has a child named NAME.
|
||||
|
||||
.. _FTP-and-SFTP.rst: FTP-and-SFTP.rst
|
||||
|
||||
|
||||
Getting Information About a File Or Directory (as JSON)
|
||||
-------------------------------------------------------
|
||||
@ -2070,7 +2075,9 @@ Tahoe-1.1; back with Tahoe-1.0 the web client was responsible for serializing
|
||||
web requests themselves).
|
||||
|
||||
For more details, please see the "Consistency vs Availability" and "The Prime
|
||||
Coordination Directive" sections of `mutable.rst <../specifications/mutable.rst>`_.
|
||||
Coordination Directive" sections of mutable.rst_.
|
||||
|
||||
.. _mutable.rst: ../specifications/mutable.rst
|
||||
|
||||
|
||||
Access Blacklist
|
||||
@ -2121,19 +2128,18 @@ the ``logs/twistd.log`` file.
|
||||
.. [1] URLs and HTTP and UTF-8, Oh My
|
||||
|
||||
HTTP does not provide a mechanism to specify the character set used to
|
||||
encode non-ASCII names in URLs
|
||||
(`RFC3986#2.1 <https://tools.ietf.org/html/rfc3986#section-2.1>`_).
|
||||
We prefer the convention that the ``filename=`` argument shall be a
|
||||
URL-escaped UTF-8 encoded Unicode string.
|
||||
For example, suppose we want to provoke the server into using a filename of
|
||||
"f i a n c e-acute e" (i.e. f i a n c U+00E9 e). The UTF-8 encoding of this
|
||||
is 0x66 0x69 0x61 0x6e 0x63 0xc3 0xa9 0x65 (or "fianc\\xC3\\xA9e", as python's
|
||||
``repr()`` function would show). To encode this into a URL, the non-printable
|
||||
characters must be escaped with the urlencode ``%XX`` mechanism, giving
|
||||
us "fianc%C3%A9e". Thus, the first line of the HTTP request will be
|
||||
"``GET /uri/CAP...?save=true&filename=fianc%C3%A9e HTTP/1.1``". Not all
|
||||
browsers provide this: IE7 by default uses the Latin-1 encoding, which is
|
||||
"fianc%E9e" (although it has a configuration option to send URLs as UTF-8).
|
||||
encode non-ASCII names in URLs (`RFC3986#2.1`_). We prefer the convention
|
||||
that the ``filename=`` argument shall be a URL-escaped UTF-8 encoded Unicode
|
||||
string. For example, suppose we want to provoke the server into using a
|
||||
filename of "f i a n c e-acute e" (i.e. f i a n c U+00E9 e). The UTF-8
|
||||
encoding of this is 0x66 0x69 0x61 0x6e 0x63 0xc3 0xa9 0x65 (or
|
||||
"fianc\\xC3\\xA9e", as python's ``repr()`` function would show). To encode
|
||||
this into a URL, the non-printable characters must be escaped with the
|
||||
urlencode ``%XX`` mechanism, giving us "fianc%C3%A9e". Thus, the first line
|
||||
of the HTTP request will be "``GET
|
||||
/uri/CAP...?save=true&filename=fianc%C3%A9e HTTP/1.1``". Not all browsers
|
||||
provide this: IE7 by default uses the Latin-1 encoding, which is "fianc%E9e"
|
||||
(although it has a configuration option to send URLs as UTF-8).
|
||||
|
||||
The response header will need to indicate a non-ASCII filename. The actual
|
||||
mechanism to do this is not clear. For ASCII filenames, the response header
|
||||
@ -2150,17 +2156,15 @@ the ``logs/twistd.log`` file.
|
||||
(note, the last four bytes of that line, not including the newline, are
|
||||
0xC3 0xA9 0x65 0x22)
|
||||
|
||||
`RFC2231#4 <https://tools.ietf.org/html/rfc2231#section-4>`_
|
||||
(dated 1997): suggests that the following might work, and
|
||||
`some developers have reported <http://markmail.org/message/dsjyokgl7hv64ig3>`_
|
||||
that it is supported by Firefox (but not IE7)::
|
||||
`RFC2231#4`_ (dated 1997): suggests that the following might work, and `some
|
||||
developers have reported`_ that it is supported by Firefox (but not IE7)::
|
||||
|
||||
#2: Content-Disposition: attachment; filename*=utf-8''fianc%C3%A9e
|
||||
|
||||
My reading of `RFC2616#19.5.1 <https://tools.ietf.org/html/rfc2616#section-19.5.1>`_
|
||||
(which defines Content-Disposition) says that the filename= parameter is
|
||||
defined to be wrapped in quotes (presumably to allow spaces without breaking
|
||||
the parsing of subsequent parameters), which would give us::
|
||||
My reading of `RFC2616#19.5.1`_ (which defines Content-Disposition) says
|
||||
that the filename= parameter is defined to be wrapped in quotes (presumably
|
||||
to allow spaces without breaking the parsing of subsequent parameters),
|
||||
which would give us::
|
||||
|
||||
#3: Content-Disposition: attachment; filename*=utf-8''"fianc%C3%A9e"
|
||||
|
||||
@ -2175,3 +2179,9 @@ the ``logs/twistd.log`` file.
|
||||
into the response header, rather than enforcing the UTF-8 convention. This
|
||||
means it does not try to decode the filename from the URL argument, nor does
|
||||
it encode the filename into the response header.
|
||||
|
||||
.. _RFC3986#2.1: https://tools.ietf.org/html/rfc3986#section-2.1
|
||||
.. _RFC2231#4: https://tools.ietf.org/html/rfc2231#section-4
|
||||
.. _some developers have reported: http://markmail.org/message/dsjyokgl7hv64ig3
|
||||
.. _RFC2616#19.5.1: https://tools.ietf.org/html/rfc2616#section-19.5.1
|
||||
|
||||
|
||||
@ -30,7 +30,7 @@ next renewal pass.
|
||||
|
||||
There are several tradeoffs to be considered when choosing the renewal timer
|
||||
and the lease duration, and there is no single optimal pair of values. See
|
||||
the `<lease-tradeoffs.svg>`_ diagram to get an idea for the tradeoffs involved.
|
||||
the lease-tradeoffs.svg_ diagram to get an idea for the tradeoffs involved.
|
||||
If lease renewal occurs quickly and with 100% reliability, than any renewal
|
||||
time that is shorter than the lease duration will suffice, but a larger ratio
|
||||
of duration-over-renewal-time will be more robust in the face of occasional
|
||||
@ -46,6 +46,9 @@ processed) to something other than 31 days.
|
||||
Renewing leases can be expected to take about one second per file/directory,
|
||||
depending upon the number of servers and the network speeds involved.
|
||||
|
||||
.. _lease-tradeoffs.svg: lease-tradeoffs.svg
|
||||
|
||||
|
||||
Client-side Renewal
|
||||
===================
|
||||
|
||||
@ -69,12 +72,14 @@ lease too: the ``--add-lease`` process is only needed to ensure that all
|
||||
older objects have up-to-date leases on them.
|
||||
|
||||
A separate "rebalancing manager/service" is also planned -- see ticket
|
||||
`#543 <http://tahoe-lafs.org/trac/tahoe-lafs/ticket/543>`_. The exact
|
||||
details of what this service will do are not settled, but it is likely to
|
||||
work by acquiring manifests from rootcaps on a periodic basis, keeping track
|
||||
of checker results, managing lease-addition, and prioritizing repair and
|
||||
rebalancing of shares. Eventually it may use multiple worker nodes to perform
|
||||
these jobs in parallel.
|
||||
`#543`_. The exact details of what this service will do are not settled, but
|
||||
it is likely to work by acquiring manifests from rootcaps on a periodic
|
||||
basis, keeping track of checker results, managing lease-addition, and
|
||||
prioritizing repair and rebalancing of shares. Eventually it may use multiple
|
||||
worker nodes to perform these jobs in parallel.
|
||||
|
||||
.. _#543: http://tahoe-lafs.org/trac/tahoe-lafs/ticket/543
|
||||
|
||||
|
||||
Server Side Expiration
|
||||
======================
|
||||
|
||||
113
docs/running.rst
113
docs/running.rst
@ -9,7 +9,7 @@ Intro
|
||||
|
||||
This is how to run a Tahoe-LAFS client or a complete Tahoe-LAFS grid.
|
||||
First you have to install the Tahoe-LAFS software, as documented in
|
||||
`quickstart.rst <quickstart.rst>`_.
|
||||
quickstart.rst_.
|
||||
|
||||
The ``tahoe`` program in the ``bin`` directory is used to create,
|
||||
start, and stop nodes. Each node lives in a separate base directory, in
|
||||
@ -20,30 +20,26 @@ A grid consists of a set of *storage nodes* and *client nodes* running
|
||||
the Tahoe-LAFS code. There is also an *introducer node* that is
|
||||
responsible for getting the other nodes talking to each other.
|
||||
|
||||
If you're getting started we recommend you try connecting to
|
||||
the `public test grid
|
||||
<https://tahoe-lafs.org/trac/tahoe-lafs/wiki/TestGrid>`_ as you only
|
||||
need to create a client node. When you want to create your own grid
|
||||
you'll need to create the introducer and several initial storage nodes
|
||||
(see the note about small grids below).
|
||||
If you're getting started we recommend you try connecting to the `public test
|
||||
grid`_ as you only need to create a client node. When you want to create your
|
||||
own grid you'll need to create the introducer and several initial storage
|
||||
nodes (see the note about small grids below).
|
||||
|
||||
If the Tahoe-LAFS ``bin`` directory is not on your PATH, then in all
|
||||
the command lines below, specify the full path to ``bin/tahoe``.
|
||||
|
||||
To construct a client node, run "``tahoe create-client``", which will
|
||||
create ``~/.tahoe`` to be the node's base directory. Acquire the
|
||||
``introducer.furl`` (see below if you are running your own introducer,
|
||||
or use the one from the `TestGrid page
|
||||
<https://tahoe-lafs.org/trac/tahoe-lafs/wiki/TestGrid>`_), and paste it
|
||||
after ``introducer.furl =`` in the ``[client]`` section of
|
||||
``~/.tahoe/tahoe.cfg``. Then use "``tahoe run ~/.tahoe``". After that,
|
||||
the node should be off and running. The first thing it will do is
|
||||
connect to the introducer and get itself connected to all other nodes
|
||||
on the grid.
|
||||
To construct a client node, run "``tahoe create-client``", which will create
|
||||
``~/.tahoe`` to be the node's base directory. Acquire the ``introducer.furl``
|
||||
(see below if you are running your own introducer, or use the one from the
|
||||
`TestGrid page`_), and paste it after ``introducer.furl =`` in the
|
||||
``[client]`` section of ``~/.tahoe/tahoe.cfg``. Then use "``tahoe run
|
||||
~/.tahoe``". After that, the node should be off and running. The first thing
|
||||
it will do is connect to the introducer and get itself connected to all other
|
||||
nodes on the grid.
|
||||
|
||||
By default, "``tahoe create-client``" creates a client-only node, that
|
||||
does not offer its disk space to other nodes. To configure other behavior,
|
||||
use "``tahoe create-node``" or see `configuration.rst <configuration.rst>`_.
|
||||
use "``tahoe create-node``" or see configuration.rst_.
|
||||
|
||||
To construct an introducer, create a new base directory for it (the
|
||||
name of the directory is up to you), ``cd`` into it, and run
|
||||
@ -52,28 +48,34 @@ name of the directory is up to you), ``cd`` into it, and run
|
||||
``introducer.furl`` into the ``private/`` subdirectory of that base
|
||||
directory. This file contains the URL the other nodes must use in order
|
||||
to connect to this introducer. (Note that "``tahoe run .``" doesn't
|
||||
work for introducers, this is a known issue: `#937
|
||||
<http://allmydata.org/trac/tahoe-lafs/ticket/937>`_.)
|
||||
work for introducers, this is a known issue: `#937`_.)
|
||||
|
||||
The "``tahoe run``" command above will run the node in the foreground.
|
||||
On Unix, you can run it in the background instead by using the
|
||||
"``tahoe start``" command. To stop a node started in this way, use
|
||||
"``tahoe stop``". ``tahoe --help`` gives a summary of all commands.
|
||||
|
||||
See `configuration.rst <configuration.rst>`_ for more details about how
|
||||
to configure Tahoe-LAFS, including how to get other clients to connect
|
||||
to your node if it is behind a firewall or NAT device.
|
||||
See configuration.rst_ for more details about how to configure Tahoe-LAFS,
|
||||
including how to get other clients to connect to your node if it is behind a
|
||||
firewall or NAT device.
|
||||
|
||||
.. _quickstart.rst: quickstart.rst
|
||||
.. _public test grid: https://tahoe-lafs.org/trac/tahoe-lafs/wiki/TestGrid
|
||||
.. _TestGrid page: https://tahoe-lafs.org/trac/tahoe-lafs/wiki/TestGrid
|
||||
.. _configuration.rst: configuration.rst
|
||||
.. _#937: https://tahoe-lafs.org/trac/tahoe-lafs/ticket/937
|
||||
|
||||
|
||||
A note about small grids
|
||||
------------------------
|
||||
|
||||
By default, Tahoe-LAFS ships with the configuration parameter
|
||||
``shares.happy`` set to 7. If you are using Tahoe-LAFS on a
|
||||
grid with fewer than 7 storage nodes, this won't work well for
|
||||
you -- none of your uploads will succeed. To fix this, see
|
||||
`configuration.rst <configuration.rst>`_ to learn how to set
|
||||
``shares.happy`` set to 7. If you are using Tahoe-LAFS on a grid with fewer
|
||||
than 7 storage nodes, this won't work well for you -- none of your uploads
|
||||
will succeed. To fix this, see configuration.rst_ to learn how to set
|
||||
``shares.happy`` to a more suitable value for your grid.
|
||||
|
||||
|
||||
Do Stuff With It
|
||||
================
|
||||
|
||||
@ -82,9 +84,8 @@ This is how to use your Tahoe-LAFS node.
|
||||
The WUI
|
||||
-------
|
||||
|
||||
Point your web browser to `http://127.0.0.1:3456
|
||||
<http://127.0.0.1:3456>`_ -- which is the URL of the gateway running on
|
||||
your own local computer -- to use your newly created node.
|
||||
Point your web browser to `http://127.0.0.1:3456`_ -- which is the URL of the
|
||||
gateway running on your own local computer -- to use your newly created node.
|
||||
|
||||
Create a new directory (with the button labelled "create a directory").
|
||||
Your web browser will load the new directory. Now if you want to be
|
||||
@ -95,48 +96,60 @@ then you can never again come back to this directory.
|
||||
You can do more or less everything you want to do with a decentralized
|
||||
filesystem through the WUI.
|
||||
|
||||
.. _http://127.0.0.1:3456: http://127.0.0.1:3456
|
||||
|
||||
|
||||
The CLI
|
||||
-------
|
||||
|
||||
Prefer the command-line? Run "``tahoe --help``" (the same command-line
|
||||
tool that is used to start and stop nodes serves to navigate and use
|
||||
the decentralized filesystem). To get started, create a new directory
|
||||
and mark it as the 'tahoe:' alias by running
|
||||
"``tahoe create-alias tahoe``". Once you've done that, you can do
|
||||
"``tahoe ls tahoe:``" and "``tahoe cp LOCALFILE tahoe:foo.txt``" to
|
||||
work with your filesystem. The Tahoe-LAFS CLI uses similar syntax to
|
||||
the well-known scp and rsync tools. See `CLI.rst <frontends/CLI.rst>`_
|
||||
for more details.
|
||||
Prefer the command-line? Run "``tahoe --help``" (the same command-line tool
|
||||
that is used to start and stop nodes serves to navigate and use the
|
||||
decentralized filesystem). To get started, create a new directory and mark it
|
||||
as the 'tahoe:' alias by running "``tahoe create-alias tahoe``". Once you've
|
||||
done that, you can do "``tahoe ls tahoe:``" and "``tahoe cp LOCALFILE
|
||||
tahoe:foo.txt``" to work with your filesystem. The Tahoe-LAFS CLI uses
|
||||
similar syntax to the well-known scp and rsync tools. See CLI.rst_ for more
|
||||
details.
|
||||
|
||||
As with the WUI (and with all current interfaces to Tahoe-LAFS), you
|
||||
are responsible for remembering directory capabilities yourself. If you
|
||||
create a new directory and lose the capability to it, then you cannot
|
||||
access that directory ever again.
|
||||
|
||||
.. _CLI.rst: frontends/CLI.rst
|
||||
|
||||
|
||||
The SFTP and FTP frontends
|
||||
--------------------------
|
||||
|
||||
You can access your Tahoe-LAFS grid via any `SFTP
|
||||
<http://en.wikipedia.org/wiki/SSH_file_transfer_protocol>`_ or `FTP
|
||||
<http://en.wikipedia.org/wiki/File_Transfer_Protocol>`_ client.
|
||||
See `FTP-and-SFTP.rst <frontends/FTP-and-SFTP.rst>`_ for how to set
|
||||
You can access your Tahoe-LAFS grid via any SFTP_ or FTP_ client.
|
||||
See `FTP-and-SFTP.rst`_ for how to set
|
||||
this up. On most Unix platforms, you can also use SFTP to plug
|
||||
Tahoe-LAFS into your computer's local filesystem via ``sshfs``.
|
||||
|
||||
The `SftpFrontend
|
||||
<https://tahoe-lafs.org/trac/tahoe-lafs/wiki/SftpFrontend>`_ page on the
|
||||
wiki has more information about using SFTP with Tahoe-LAFS.
|
||||
The SftpFrontend_ page on the wiki has more information about using SFTP with
|
||||
Tahoe-LAFS.
|
||||
|
||||
.. _SFTP: https://en.wikipedia.org/wiki/SSH_file_transfer_protocol
|
||||
.. _FTP: http://en.wikipedia.org/wiki/File_Transfer_Protocol
|
||||
.. _FTP-and-SFTP.rst: frontends/FTP-and-SFTP.rst
|
||||
.. _SftpFrontend: https://tahoe-lafs.org/trac/tahoe-lafs/wiki/SftpFrontend
|
||||
|
||||
|
||||
The WAPI
|
||||
--------
|
||||
|
||||
Want to program your Tahoe-LAFS node to do your bidding? Easy! See
|
||||
`webapi.rst <frontends/webapi.rst>`_.
|
||||
webapi.rst_.
|
||||
|
||||
.. _webapi.rst: frontends/webapi.rst
|
||||
|
||||
|
||||
Socialize
|
||||
=========
|
||||
|
||||
You can chat with other users of and hackers of this software on the
|
||||
#tahoe-lafs IRC channel at ``irc.freenode.net``, or on the `tahoe-dev
|
||||
mailing list
|
||||
<https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev>`_.
|
||||
#tahoe-lafs IRC channel at ``irc.freenode.net``, or on the `tahoe-dev mailing
|
||||
list`_.
|
||||
|
||||
.. _tahoe-dev mailing list: https://tahoe-lafs.org/cgi-bin/mailman/listinfo/tahoe-dev
|
||||
|
||||
@ -102,7 +102,7 @@ just a special way of interpreting the contents of a specific mutable file.
|
||||
Earlier releases used a "vdrive server": this server was abolished in the
|
||||
v0.7.0 release.
|
||||
|
||||
For details of how mutable files work, please see `<mutable.rst>`_ in this
|
||||
For details of how mutable files work, please see mutable.rst_ in this
|
||||
directory.
|
||||
|
||||
For releases since v0.7.0, we achieve most of our desired properties. The
|
||||
@ -124,13 +124,15 @@ multiple versions of each mutable file, and you might have some shares of
|
||||
version 1 and other shares of version 2). In extreme cases of simultaneous
|
||||
update, mutable files might suffer from non-monotonicity.
|
||||
|
||||
.. _mutable.rst: mutable.rst
|
||||
|
||||
|
||||
Dirnode secret values
|
||||
=====================
|
||||
|
||||
As mentioned before, dirnodes are simply a special way to interpret the
|
||||
contents of a mutable file, so the secret keys and capability strings
|
||||
described in `<mutable.rst>`_ are all the same. Each dirnode contains an RSA
|
||||
described in mutable.rst_ are all the same. Each dirnode contains an RSA
|
||||
public/private keypair, and the holder of the "write capability" will be able
|
||||
to retrieve the private key (as well as the AES encryption key used for the
|
||||
data itself). The holder of the "read capability" will be able to obtain the
|
||||
|
||||
@ -129,6 +129,7 @@ delivered to the user.
|
||||
|
||||
.. image:: file-encoding6.svg
|
||||
|
||||
|
||||
Hashes
|
||||
======
|
||||
|
||||
@ -145,8 +146,9 @@ Using SHA-256d (instead of plain SHA-256) guards against length-extension
|
||||
attacks. Using the tag protects our Merkle trees against attacks in which the
|
||||
hash of a leaf is confused with a hash of two children (allowing an attacker
|
||||
to generate corrupted data that nevertheless appears to be valid), and is
|
||||
simply good "cryptograhic hygiene". The `"Chosen Protocol Attack" by Kelsey,
|
||||
Schneier, and Wagner <http://www.schneier.com/paper-chosen-protocol.html>`_ is
|
||||
relevant. Putting the tag in a netstring guards against attacks that seek to
|
||||
confuse the end of the tag with the beginning of the subsequent value.
|
||||
simply good "cryptograhic hygiene". The `“Chosen Protocol Attack” by Kelsey,
|
||||
Schneier, and Wagner`_ is relevant. Putting the tag in a netstring guards
|
||||
against attacks that seek to confuse the end of the tag with the beginning of
|
||||
the subsequent value.
|
||||
|
||||
.. _“Chosen Protocol Attack” by Kelsey, Schneier, and Wagner: http://www.schneier.com/paper-chosen-protocol.html
|
||||
|
||||
@ -85,7 +85,7 @@ representation of the size of the data represented by this URI. All base32
|
||||
encodings are expressed in lower-case, with the trailing '=' signs removed.
|
||||
|
||||
For example, the following is a CHK URI, generated from a previous version of
|
||||
the contents of `<../architecture.rst>`_::
|
||||
the contents of architecture.rst_::
|
||||
|
||||
URI:CHK:ihrbeov7lbvoduupd4qblysj7a:bg5agsdt62jb34hxvxmdsbza6do64f4fg5anxxod2buttbo6udzq:3:10:28733
|
||||
|
||||
@ -100,6 +100,9 @@ about the file's contents (except filesize), which improves privacy. The
|
||||
URI:CHK: prefix really indicates that an immutable file is in use, without
|
||||
saying anything about how the key was derived.
|
||||
|
||||
.. _architecture.rst: ../architecture.rst
|
||||
|
||||
|
||||
LIT URIs
|
||||
--------
|
||||
|
||||
@ -143,7 +146,7 @@ The format of the write-cap for mutable files is::
|
||||
Where (writekey) is the base32 encoding of the 16-byte AES encryption key
|
||||
that is used to encrypt the RSA private key, and (fingerprint) is the base32
|
||||
encoded 32-byte SHA-256 hash of the RSA public key. For more details about
|
||||
the way these keys are used, please see `<mutable.rst>`_.
|
||||
the way these keys are used, please see mutable.rst_.
|
||||
|
||||
The format for mutable read-caps is::
|
||||
|
||||
@ -159,13 +162,16 @@ Historical note: the "SSK" prefix is a perhaps-inaccurate reference to
|
||||
"Sub-Space Keys" from the Freenet project, which uses a vaguely similar
|
||||
structure to provide mutable file access.
|
||||
|
||||
.. _mutable.rst: mutable.rst
|
||||
|
||||
|
||||
Directory URIs
|
||||
==============
|
||||
|
||||
The grid layer provides a mapping from URI to data. To turn this into a graph
|
||||
of directories and files, the "vdrive" layer (which sits on top of the grid
|
||||
layer) needs to keep track of "directory nodes", or "dirnodes" for short.
|
||||
`<dirnodes.rst>`_ describes how these work.
|
||||
dirnodes.rst_ describes how these work.
|
||||
|
||||
Dirnodes are contained inside mutable files, and are thus simply a particular
|
||||
way to interpret the contents of these files. As a result, a directory
|
||||
@ -181,6 +187,9 @@ directory) look much like mutable-file read-caps::
|
||||
Historical note: the "DIR2" prefix is used because the non-distributed
|
||||
dirnodes in earlier Tahoe releases had already claimed the "DIR" prefix.
|
||||
|
||||
.. _dirnodes.rst: dirnodes.rst
|
||||
|
||||
|
||||
Internal Usage of URIs
|
||||
======================
|
||||
|
||||
|
||||
@ -307,7 +307,9 @@ keep its FURL consistent). To explicitly control which port it uses, write
|
||||
the desired portnumber into a file named "portnum" (i.e. $BASEDIR/portnum),
|
||||
and the next time the gatherer is started, it will start listening on the
|
||||
given port. The portnum file is actually a "strports specification string",
|
||||
as described in `docs/configuration.rst <configuration.rst>`_.
|
||||
as described in configuration.rst_.
|
||||
|
||||
.. _configuration.rst: configuration.rst
|
||||
|
||||
Once running, the stats gatherer will create a standard python "pickle" file
|
||||
in $BASEDIR/stats.pickle . Once a minute, the gatherer will pull stats
|
||||
|
||||
Loading…
Reference in New Issue
Block a user