rolling back:
Thu Sep 25 14:42:23 BST 2008 robk-tahoe@allmydata.com
* fuse/impl_c: add --auto-fsid option
this was inspired by reading the fuse docs and discovering the 'fsid' option
to fuse_main, and was _intended_ to support a sort of 'stability' to the
filesystem (specifically derived from the root-uri mounted, whether directly
or via an alias) to support mac aliases across unmount/remount etc.
some experimentation shows that that doesn't actually work, and that, at
least for mac aliases in my testing, they're tied to path-to-mountpoint and
not to the fsid - which seems to have no bearing. perhaps the 'local' flag
is causing weirdness therein.
at any rate, I'm recording it simply for posterity, in case it turns out to
be useful after all somewhere down the road.
M ./contrib/fuse/impl_c/blackmatch.py +13
this was inspired by reading the fuse docs and discovering the 'fsid' option
to fuse_main, and was _intended_ to support a sort of 'stability' to the
filesystem (specifically derived from the root-uri mounted, whether directly
or via an alias) to support mac aliases across unmount/remount etc.
some experimentation shows that that doesn't actually work, and that, at
least for mac aliases in my testing, they're tied to path-to-mountpoint and
not to the fsid - which seems to have no bearing. perhaps the 'local' flag
is causing weirdness therein.
at any rate, I'm recording it simply for posterity, in case it turns out to
be useful after all somewhere down the road.
For a variety of reasons, high amongst them the fact that many people
interested in fuse support for tahoe seem to have missed its existence,
the existing fuse implementation for tahoe, previously 'mac/tahoefuse.py'
has been renamed and moved.
It was suggested that, even though the mac build depends upon it, that
the mac/tahoefuse implementation be moved into contrib/fuse along with
the other fuse implementations. The fact that it's not as extensively
covered by unit tests as mainline tahoe was given as corroboration.
In a bid to try and stem the confusion inherent in having tahoe_fuse,
tfuse and tahoefuse jumbled together (not necessarily helped by
referring to them as impl_a, b and c respectively) I'm hereby renaming
tahoefuse as 'blackmatch' (black match is, per wikipedia "a type of
crude fuse" hey, I'm a punny guy) Maybe one day it'll be promoted to
be 'quickmatch' instead...
Anyway, this patch moves mac/tahoefuse.py out to contrib/fuse/impl_c/
as blackmatch.py, and makes appropriate changes to the mac build process
to transclude blackmatch therein. this leaves the extant fuse.py and
fuseparts business in mac/ as-is and doesn't attempt to address such
issues in contrib/fuse/impl_c.
it is left as an exercise to the reader (or the reader of a message
to follow) as to how to deal with the 'fuse' python module on the mac.
as of this time, blackmatch should work on both mac and linux, and
passes the four extant tests in runtests. (fwiw neither impl_a nor
impl_b have I managed to get working on the mac yet)
since blackmatch supports a read-write and caching fuse interface to
tahoe, some write tests obviously need to be added to runtests.
This patch makes a significant number of changes to the fuse 'runtests' script
which stem from my efforts to integrate the third fuse implementation into this
framework. Perhaps not all were necessary to that end, and I beg nejucomo's
forebearance if I got too carried away.
- cleaned up the blank lines; imho blank lines should be empty
- made the unmount command switch based on platform, since macfuse just uses
'umount' not the 'fusermount' command (which doesn't exist)
- made the expected working dir for runtests the contrib/fuse dir, not the
top-level tahoe source tree - see also discussion of --path-to-tahoe below
- significantly reworked the ImplProcManager class. rather than subclassing
for each fuse implementation to be tested, the new version is based on
instantiating objects and providing relevant config info to the constructor.
this was motivated by a desire to eliminate the duplication of similar but
subtly different code between instances, framed by consideration of increasing
the number of platforms and implementations involved. each implementation to
test is thus reduced to the pertinent import and an entry in the
'implementations' table defining how to handle that implementation. this also
provides a way to specify which sets of tests to run for each implementation,
more on that below.
- significantly reworked the command line options parsing, using twisted.usage;
what used to be a single optional argument is now represented by the
--test-type option which allows one to choose between running unittests, the
system tests, or both.
the --implementations option allows for a specific (comma-separated) list of
implemenations to be tested, or the default 'all'
the --tests option allows for a specific (comma-separated) list of tests sets
to be run, or the default 'all'. note that only the intersection of tests
requested on the command line and tests relevant to each implementation will
be run. see below for more on tests sets.
the --path-to-tahoe open allows for the path to the 'tahoe' executable to be
specified. it defaults to '../../bin/tahoe' which is the location of the tahoe
script in the source tree relative to the contrib/fuse dir by default.
the --tmp-dir option controls where temporary directories (and hence
mountpoints) are created during the test. this defaults to /tmp - a change
from the previous behaviour of using the system default dir for calls to
tempfile.mkdtemp(), a behaviour which can be obtained by providing an empty
value, e.g. "--tmp-dir="
the --debug-wait flag causes the test runner to pause waiting upon user
input at various stages through the testing, which facilitates debugging e.g.
by allowing the user to open a browser and explore or modify the contents of
the ephemeral grid after it has been instantiated but before tests are run,
or make environmental adjustments before actually triggering fuse mounts etc.
note that the webapi url for the first client node is printed out upon its
startup to facilitate this sort of debugging also.
- the default tmp dir was changed, and made configurable. previously the
default behaviour of tempfile.mkdtemp() was used. it turns out that, at least
on the mac, that led to temporary directories to be created in a location
which ultimately led to mountpoint paths longer than could be handled by
macfuse - specifically mounted filesystems could not be unmounted and would
'leak'. by changing the default location to be rooted at /tmp this leads to
mountpoint paths short enough to be supported without problems.
- tests are now grouped into 'sets' by method name prefix. all the existing
tests have been moved into the 'read' set, i.e. with method names starting
'test_read_'. this is intended to facilitate the fact that some implementations
are read-only, and some support write, so the applicability of tests will vary
by implementation. the 'implementations' table, which governs the configuration
of the ImplProcManager responsible for a given implementation, provides a list
of 'test' (i.e test set names) which are applicable to that implementation.
note no 'write' tests yet exist, this is merely laying the groundwork.
- the 'expected output' of the tahoe command, which is checked for 'surprising'
output by regex match, can be confused by spurious output from libraries.
specfically, testing on the mac produced a warning message about zope interface
resolution various multiple eggs. the 'check_tahoe_output()' function now has
a list of 'ignorable_lines' (each a regex) which will be discarded before the
remainder of the output of the tahoe script is matched against expectation.
- cleaned up a typo, and a few spurious imports caught by pyflakes
specifically change the expectation of the code to be such that the node-url
(self.url) always includes the trailing slash to be a correctly formed url
moreover read the node-url from the 'node.url' file found in the node 'basedir'
and only if that doesn't exist, then fall back to reading the 'webport' file
from therein and assuming localhost. This then supports the general tahoe
pattern that tools needing only a webapi server can be pointed at a directory
containing the node.url file, which can optionally point to another server,
rather than requiring a complete node dir and locally running node instance.
from testing on linux (specifically ubuntu hardy) the libfuse dll has a
different name, specifically libfuse.so.2. this patch tries libfuse.so
and then falls back to trying .2 if the former fails.
it also changes the unmount behaviour, to simply return from the handler's
loop_forever() loop upon being unmounted, rather than raising an EOFError,
since none of the client code I looked at actually handled that exception,
but did seem to expect to fall off of main() when loop_forever() returned.
Additionally, from my testing unmount typically led to an OSError from the
fuse fd read, rather than an empty read, as the code seemed to expect.
also removed a spurious import pyflakes quibbled about.
This prevents the third client from failing to start due to a port
collision with the second client. The first client, which is used for
testing has a random high port written to webport, and thus does not
interfere.
The flow control has been de-obfuscated a bit.
Some output changes.
The test framework has quite a few race conditions, but it does a reasonable job of setting up and cleaning up.
This is a little convoluted because of the "layer" design, but it appears
to function correctly and do properly ordered cleanup.
Before system test setup is complete, tahoe_fuse.py needs to be modified
to allow arbitrary client base directories.
