On linux, write tests are failing because data written to fuse isn't showing
up in tahoe by the time it's checked. it's not clear where this is originating,
since the fuse implementation [should be] waiting for completion of tahoe
operations before returning from its calls. This adds an option to control the
duration of a pause between the fuse write and the check of tahoe, which is by
default set to 2s on linux, which - somewhat inexplicably - seems to 'fix' the
problem, in as far as it allows tests to complete.
previously, upon opening a file for reading, the open() call would block
while the entire file was retrieved from tahoe into the cache directory.
This change adds a DownloaderWithReadQueue class, and associated plumbing,
such that an open() will return promptly with the download initiated 'in
the background'. Subsequent read() operations will block until enough
data has been downloaded to satisfy that request. This provides a behaviour
similar to streaming, i.e. the client application will be able to read
data from the fuse interface while the remainder of the file is still being
downloaded.
this test uploads a test file to tahoe, and then reads the file from fuse,
but reads the blocks of the file in a random order; this is designed to
exercise the asynchronous download feature of blackmatch - where the file
is downloaded from tahoe asynchronously, and rather than blocking open()
for the entirety of the download, instead individual read() calls are
blocked until enough of the file has been downloaded to satisfy them
the code had a 'fullcleanup' flag internally which controlled whether
working directories were cleaned up. this promotes that to a command
line option (negated) '--no-cleanup' defaulting to False, i.e. do cleanup
this avoids dumping the repr of 1Mb of random data to stdout in the event
of a test failure, but rather just dumps the start/end of the errant strings
if the amount of data is > 200 chars repr'd
the repr_flags debug/logging function had a list of fields from the os
module that might be passed into an open() call, but it included at
least one which was available on the mac but not on linux. symmetrically
linux has numerous flags which are not present on the mac. the repr_flags
function is now tolerant of flags not being present, and has an expanded
list of flags
since the current tests assume that the implementation responds to changes made
to tahoe after mount, and impl_b prefetches and cached directory data, impl_b
fails the current 'read' test suite.
rather than reflect that problem in the overall failure of the runtests exit
code, this adds a 'todo' flag to the implementations table, and sets the todo
flag for impl_b. Thus errors will therein be reported in output, but not cause
a failing exit code.
previously the runtests suite removed the webport file created by
tahoe create-client in all but the first node. now that the node config
is in tahoe.cfg by default this file might not exist.
simplejson v2 returns strings as either unicode or str, depending upon its
mood. thus the interpretation of the node's json repr of a directory, and
the serialisation of strings in the json based rpc both exploded when built
against simplejson v2. this makes both of these places liberal in their
acceptance of either str or unicode.
humphf. my build runs the fuse stuff fine, but the build from the buildslave
doesn't seem to start up properly. hopefully this will elicit some useful info
adds a couple of functions to unpack 'mode' and 'flags' for open() calls, to
facilitate debugging.
adds a fix to ensure that all tmp files created for writing are opened with
permissions 0600 - one problem I had with testing with the Finder was that
files were being opened write only (0200) and were then failing to upload
to tahoe due to internal permission denied errors.
there remain a variety of problems with finder access which I'm unable to
comprehend at this time. sometimes copies to tahoe will work fine, sometimes
they yield "the finder cannot complete the operation because some data ...
could not be read or written. (Error code -36)" sometimes "You may need to
enter the name and password for an administrator on this computer to change
the item" sometimes "The operation cannot be completed because an item with
the name ... already exists." and sometimes "The operation cannot be completed
because the item ... is locked." What seems to be absent is rhyme or reason.
unix operations (cp, mv) work fine, rsync works fine.
the daemonize() function imported from twisted was causing problems when
run from a frozen (py2app) build. I simply copied the daemonize function
into this file, and that fixes the problem.
also removed a couple of lines of debugging spam that slipped through.
though it seemed to work before the 'fstype' passed to fuse of 'allmydata' was
today throwing errors that len(fstype) must be at most 7.
fixed a typo in changes to 'mount_filesystem()' args
bumped the delay between mounting a filesystem and 'open'ing it in Finder to
4s, as it seems to take a little longer to mount now the client and server
fuse processes need to coordinate.
