macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
#!/usr/bin/env python
|
|
|
|
|
|
|
|
#-----------------------------------------------------------------------------------------------
|
|
|
|
from allmydata.uri import CHKFileURI, NewDirectoryURI, LiteralFileURI
|
|
|
|
from allmydata.scripts.common_http import do_http as do_http_req
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
from allmydata.util.hashutil import tagged_hash
|
2008-03-07 00:43:25 +00:00
|
|
|
from allmydata.util.assertutil import precondition
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
from allmydata.util import base32
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
import base64
|
|
|
|
import sha
|
2008-02-20 00:16:08 +00:00
|
|
|
import sys
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
import os
|
|
|
|
import errno
|
|
|
|
import stat
|
|
|
|
# pull in some spaghetti to make this stuff work without fuse-py being installed
|
|
|
|
try:
|
|
|
|
import _find_fuse_parts
|
|
|
|
junk = _find_fuse_parts
|
|
|
|
del junk
|
|
|
|
except ImportError:
|
|
|
|
pass
|
|
|
|
import fuse
|
|
|
|
|
|
|
|
import time
|
|
|
|
import traceback
|
|
|
|
import simplejson
|
|
|
|
import urllib
|
|
|
|
|
2008-02-20 00:16:08 +00:00
|
|
|
USAGE = 'usage: tahoe fuse [dir_cap_name] [fuse_options] mountpoint'
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
DEFAULT_DIRECTORY_VALIDITY=26
|
2008-02-20 00:16:08 +00:00
|
|
|
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
if not hasattr(fuse, '__version__'):
|
|
|
|
raise RuntimeError, \
|
|
|
|
"your fuse-py doesn't know of fuse.__version__, probably it's too old."
|
|
|
|
|
|
|
|
fuse.fuse_python_api = (0, 2)
|
|
|
|
fuse.feature_assert('stateful_files', 'has_init')
|
|
|
|
|
|
|
|
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
logfile = file('tfuse.log', 'ab')
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
def log(msg):
|
|
|
|
logfile.write("%s: %s\n" % (time.asctime(), msg))
|
|
|
|
#time.sleep(0.1)
|
|
|
|
logfile.flush()
|
|
|
|
|
|
|
|
fuse.flog = log
|
|
|
|
|
2008-03-07 00:43:25 +00:00
|
|
|
def unicode_to_utf8(u):
|
|
|
|
precondition(isinstance(u, unicode), repr(u))
|
|
|
|
return u.encode('utf-8')
|
|
|
|
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
def do_http(method, url, body=''):
|
|
|
|
resp = do_http_req(method, url, body)
|
|
|
|
log('do_http(%s, %s) -> %s, %s' % (method, url, resp.status, resp.reason))
|
|
|
|
if resp.status not in (200, 201):
|
|
|
|
raise RuntimeError('http response (%s, %s)' % (resp.status, resp.reason))
|
|
|
|
else:
|
|
|
|
return resp.read()
|
|
|
|
|
|
|
|
def flag2mode(flags):
|
|
|
|
log('flag2mode(%r)' % (flags,))
|
|
|
|
#md = {os.O_RDONLY: 'r', os.O_WRONLY: 'w', os.O_RDWR: 'w+'}
|
|
|
|
md = {os.O_RDONLY: 'rb', os.O_WRONLY: 'wb', os.O_RDWR: 'w+b'}
|
|
|
|
m = md[flags & (os.O_RDONLY | os.O_WRONLY | os.O_RDWR)]
|
|
|
|
|
|
|
|
if flags | os.O_APPEND:
|
|
|
|
m = m.replace('w', 'a', 1)
|
|
|
|
|
|
|
|
return m
|
|
|
|
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
class TFSIOError(IOError):
|
|
|
|
pass
|
|
|
|
|
|
|
|
class ENOENT(TFSIOError):
|
|
|
|
def __init__(self, msg):
|
|
|
|
TFSIOError.__init__(self, errno.ENOENT, msg)
|
|
|
|
|
|
|
|
class EINVAL(TFSIOError):
|
|
|
|
def __init__(self, msg):
|
|
|
|
TFSIOError.__init__(self, errno.EINVAL, msg)
|
|
|
|
|
|
|
|
class EACCESS(TFSIOError):
|
|
|
|
def __init__(self, msg):
|
|
|
|
TFSIOError.__init__(self, errno.EACCESS, msg)
|
|
|
|
|
|
|
|
class EEXIST(TFSIOError):
|
|
|
|
def __init__(self, msg):
|
|
|
|
TFSIOError.__init__(self, errno.EEXIST, msg)
|
|
|
|
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
def logargsretexc(meth):
|
|
|
|
def inner(self, *args, **kwargs):
|
|
|
|
log("%s(%r, %r)" % (meth, args, kwargs))
|
|
|
|
try:
|
|
|
|
ret = meth(self, *args, **kwargs)
|
|
|
|
except:
|
|
|
|
log('exception:\n%s' % (traceback.format_exc(),))
|
|
|
|
raise
|
|
|
|
log("ret: %r" % (ret, ))
|
|
|
|
return ret
|
|
|
|
inner.__name__ = '<logwrap(%s)>' % (meth,)
|
|
|
|
return inner
|
|
|
|
|
|
|
|
def logexc(meth):
|
|
|
|
def inner(self, *args, **kwargs):
|
|
|
|
try:
|
|
|
|
ret = meth(self, *args, **kwargs)
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
except TFSIOError, tie:
|
|
|
|
log('error: %s' % (tie,))
|
|
|
|
raise
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
except:
|
|
|
|
log('exception:\n%s' % (traceback.format_exc(),))
|
|
|
|
raise
|
|
|
|
return ret
|
|
|
|
inner.__name__ = '<logwrap(%s)>' % (meth,)
|
|
|
|
return inner
|
|
|
|
|
|
|
|
def log_exc():
|
|
|
|
log('exception:\n%s' % (traceback.format_exc(),))
|
|
|
|
|
|
|
|
class TahoeFuseFile(object):
|
|
|
|
|
|
|
|
def __init__(self, path, flags, *mode):
|
|
|
|
log("TFF: __init__(%r, %r, %r)" % (path, flags, mode))
|
|
|
|
|
|
|
|
self._path = path # for tahoe put
|
|
|
|
try:
|
|
|
|
self.parent, self.name, self.fnode = self.tfs.get_parent_name_and_child(path)
|
|
|
|
m = flag2mode(flags)
|
|
|
|
log('TFF: flags2(mode) -> %s' % (m,))
|
|
|
|
if m[0] in 'wa':
|
|
|
|
# write
|
|
|
|
self.fname = self.tfs.cache.tmp_file(os.urandom(20))
|
|
|
|
if self.fnode is None:
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
log('TFF: [%s] open() for write: no file node, creating new File %s' % (self.name, self.fname, ))
|
|
|
|
self.fnode = File(0, 'URI:LIT:')
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
self.fnode.tmp_fname = self.fname # XXX kill this
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
self.parent.add_child(self.name, self.fnode, {})
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
elif hasattr(self.fnode, 'tmp_fname'):
|
|
|
|
self.fname = self.fnode.tmp_fname
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
log('TFF: [%s] open() for write: existing file node lists %s' % (self.name, self.fname, ))
|
|
|
|
else:
|
|
|
|
log('TFF: [%s] open() for write: existing file node lists no tmp_file, using new %s' % (self.name, self.fname, ))
|
|
|
|
self.file = os.fdopen(os.open(self.fname, flags|os.O_CREAT, *mode), m)
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
self.fd = self.file.fileno()
|
|
|
|
log('TFF: opened(%s) for write' % self.fname)
|
|
|
|
self.open_for_write = True
|
|
|
|
else:
|
|
|
|
# read
|
|
|
|
assert self.fnode is not None
|
|
|
|
uri = self.fnode.get_uri()
|
|
|
|
|
|
|
|
# XXX make this go away
|
|
|
|
if hasattr(self.fnode, 'tmp_fname'):
|
|
|
|
self.fname = self.fnode.tmp_fname
|
|
|
|
log('TFF: reopening(%s) for reading' % self.fname)
|
|
|
|
else:
|
|
|
|
log('TFF: fetching file from cache for reading')
|
|
|
|
self.fname = self.tfs.cache.get_file(uri)
|
|
|
|
|
|
|
|
self.file = os.fdopen(os.open(self.fname, flags, *mode), m)
|
|
|
|
self.fd = self.file.fileno()
|
|
|
|
self.open_for_write = False
|
|
|
|
log('TFF: opened(%s) for read' % self.fname)
|
|
|
|
except:
|
|
|
|
log_exc()
|
|
|
|
raise
|
|
|
|
|
|
|
|
def log(self, msg):
|
|
|
|
log("<TFF(%s:%s)> %s" % (os.path.basename(self.fname), self.name, msg))
|
|
|
|
|
|
|
|
@logexc
|
|
|
|
def read(self, size, offset):
|
|
|
|
self.log('read(%r, %r)' % (size, offset, ))
|
|
|
|
self.file.seek(offset)
|
|
|
|
return self.file.read(size)
|
|
|
|
|
|
|
|
@logexc
|
|
|
|
def write(self, buf, offset):
|
|
|
|
self.log("write(-%s-, %r)" % (len(buf), offset))
|
|
|
|
if not self.open_for_write:
|
|
|
|
return -errno.EACCES
|
|
|
|
self.file.seek(offset)
|
|
|
|
self.file.write(buf)
|
|
|
|
return len(buf)
|
|
|
|
|
|
|
|
@logexc
|
|
|
|
def release(self, flags):
|
|
|
|
self.log("release(%r)" % (flags,))
|
|
|
|
self.file.close()
|
|
|
|
if self.open_for_write:
|
|
|
|
size = os.path.getsize(self.fname)
|
|
|
|
self.fnode.size = size
|
|
|
|
file_cap = self.tfs.upload(self.fname)
|
|
|
|
self.fnode.ro_uri = file_cap
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
# XXX [ ] TODO: set metadata
|
|
|
|
# write new uri into parent dir entry
|
|
|
|
self.parent.add_child(self.name, self.fnode, {})
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
self.log("uploaded: %s" % (file_cap,))
|
|
|
|
|
|
|
|
# dbg
|
|
|
|
print_tree()
|
|
|
|
|
|
|
|
def _fflush(self):
|
|
|
|
if 'w' in self.file.mode or 'a' in self.file.mode:
|
|
|
|
self.file.flush()
|
|
|
|
|
|
|
|
@logexc
|
|
|
|
def fsync(self, isfsyncfile):
|
|
|
|
self.log("fsync(%r)" % (isfsyncfile,))
|
|
|
|
self._fflush()
|
|
|
|
if isfsyncfile and hasattr(os, 'fdatasync'):
|
|
|
|
os.fdatasync(self.fd)
|
|
|
|
else:
|
|
|
|
os.fsync(self.fd)
|
|
|
|
|
|
|
|
@logexc
|
|
|
|
def flush(self):
|
|
|
|
self.log("flush()")
|
|
|
|
self._fflush()
|
|
|
|
# cf. xmp_flush() in fusexmp_fh.c
|
|
|
|
os.close(os.dup(self.fd))
|
|
|
|
|
|
|
|
@logexc
|
|
|
|
def fgetattr(self):
|
|
|
|
self.log("fgetattr()")
|
|
|
|
s = os.fstat(self.fd)
|
|
|
|
self.log("fgetattr() -> %r" % (s,))
|
|
|
|
return s
|
|
|
|
|
|
|
|
@logexc
|
|
|
|
def ftruncate(self, len):
|
|
|
|
self.log("ftruncate(%r)" % (len,))
|
|
|
|
self.file.truncate(len)
|
|
|
|
|
|
|
|
class TahoeFuse(fuse.Fuse):
|
|
|
|
|
|
|
|
def __init__(self, tfs, *args, **kw):
|
|
|
|
log("TF: __init__(%r, %r)" % (args, kw))
|
|
|
|
|
|
|
|
self.tfs = tfs
|
2008-02-20 00:16:08 +00:00
|
|
|
_tfs_ = tfs
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
class MyFuseFile(TahoeFuseFile):
|
2008-02-20 00:16:08 +00:00
|
|
|
tfs = _tfs_
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
self.file_class = MyFuseFile
|
|
|
|
log("TF: file_class: %r" % (self.file_class,))
|
|
|
|
|
|
|
|
fuse.Fuse.__init__(self, *args, **kw)
|
|
|
|
|
|
|
|
#import thread
|
|
|
|
#thread.start_new_thread(self.launch_reactor, ())
|
|
|
|
|
|
|
|
def log(self, msg):
|
|
|
|
log("<TF> %s" % (msg, ))
|
|
|
|
|
|
|
|
@logexc
|
|
|
|
def readlink(self, path):
|
|
|
|
self.log("readlink(%r)" % (path,))
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
node = self.tfs.get_path(path)
|
|
|
|
if node:
|
|
|
|
raise EINVAL('Not a symlink') # nothing in tahoe is a symlink
|
|
|
|
else:
|
|
|
|
raise ENOENT('Invalid argument')
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
@logexc
|
|
|
|
def unlink(self, path):
|
|
|
|
self.log("unlink(%r)" % (path,))
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
self.tfs.unlink(path)
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
@logexc
|
|
|
|
def rmdir(self, path):
|
|
|
|
self.log("rmdir(%r)" % (path,))
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
self.tfs.unlink(path)
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
@logexc
|
|
|
|
def symlink(self, path, path1):
|
|
|
|
self.log("symlink(%r, %r)" % (path, path1))
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
self.tfs.link(path, path1)
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
@logexc
|
|
|
|
def rename(self, path, path1):
|
|
|
|
self.log("rename(%r, %r)" % (path, path1))
|
|
|
|
self.tfs.rename(path, path1)
|
|
|
|
|
|
|
|
@logexc
|
|
|
|
def link(self, path, path1):
|
|
|
|
self.log("link(%r, %r)" % (path, path1))
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
self.tfs.link(path, path1)
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
@logexc
|
|
|
|
def chmod(self, path, mode):
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
self.log("XX chmod(%r, %r)" % (path, mode))
|
|
|
|
#return -errno.EOPNOTSUPP
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
@logexc
|
|
|
|
def chown(self, path, user, group):
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
self.log("XX chown(%r, %r, %r)" % (path, user, group))
|
|
|
|
#return -errno.EOPNOTSUPP
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
@logexc
|
|
|
|
def truncate(self, path, len):
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
self.log("XX truncate(%r, %r)" % (path, len))
|
|
|
|
#return -errno.EOPNOTSUPP
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
@logexc
|
|
|
|
def utime(self, path, times):
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
self.log("XX utime(%r, %r)" % (path, times))
|
|
|
|
#return -errno.EOPNOTSUPP
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
@logexc
|
|
|
|
def statfs(self):
|
|
|
|
self.log("statfs()")
|
|
|
|
"""
|
|
|
|
Should return an object with statvfs attributes (f_bsize, f_frsize...).
|
|
|
|
Eg., the return value of os.statvfs() is such a thing (since py 2.2).
|
|
|
|
If you are not reusing an existing statvfs object, start with
|
|
|
|
fuse.StatVFS(), and define the attributes.
|
|
|
|
|
|
|
|
To provide usable information (ie., you want sensible df(1)
|
|
|
|
output, you are suggested to specify the following attributes:
|
|
|
|
|
|
|
|
- f_bsize - preferred size of file blocks, in bytes
|
|
|
|
- f_frsize - fundamental size of file blcoks, in bytes
|
|
|
|
[if you have no idea, use the same as blocksize]
|
|
|
|
- f_blocks - total number of blocks in the filesystem
|
|
|
|
- f_bfree - number of free blocks
|
|
|
|
- f_files - total number of file inodes
|
|
|
|
- f_ffree - nunber of free file inodes
|
|
|
|
"""
|
|
|
|
|
|
|
|
return os.statvfs(".")
|
|
|
|
|
|
|
|
def fsinit(self):
|
|
|
|
self.log("fsinit()")
|
|
|
|
|
|
|
|
def main(self, *a, **kw):
|
|
|
|
self.log("main(%r, %r)" % (a, kw))
|
|
|
|
|
|
|
|
return fuse.Fuse.main(self, *a, **kw)
|
|
|
|
|
|
|
|
##################################################################
|
|
|
|
|
|
|
|
@logexc
|
|
|
|
def readdir(self, path, offset):
|
|
|
|
log('readdir(%r, %r)' % (path, offset))
|
|
|
|
node = self.tfs.get_path(path)
|
|
|
|
if node is None:
|
|
|
|
return -errno.ENOENT
|
|
|
|
dirlist = ['.', '..'] + node.children.keys()
|
|
|
|
log('dirlist = %r' % (dirlist,))
|
|
|
|
return [fuse.Direntry(d) for d in dirlist]
|
|
|
|
|
|
|
|
@logexc
|
|
|
|
def getattr(self, path):
|
|
|
|
log('getattr(%r)' % (path,))
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
|
|
|
|
if path == '/':
|
|
|
|
# we don't have any metadata for the root (no edge leading to it)
|
|
|
|
mode = (stat.S_IFDIR | 755)
|
|
|
|
mtime = self.tfs.root.mtime
|
|
|
|
s = TStat({}, st_mode=mode, st_nlink=1, st_mtime=mtime)
|
|
|
|
log('getattr(%r) -> %r' % (path, s))
|
|
|
|
return s
|
|
|
|
|
|
|
|
parent, name, child = self.tfs.get_parent_name_and_child(path)
|
|
|
|
if not child: # implicitly 'or not parent'
|
|
|
|
raise ENOENT('No such file or directory')
|
|
|
|
return parent.get_stat(name)
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
@logexc
|
|
|
|
def access(self, path, mode):
|
|
|
|
self.log("access(%r, %r)" % (path, mode))
|
|
|
|
node = self.tfs.get_path(path)
|
|
|
|
if not node:
|
|
|
|
return -errno.ENOENT
|
|
|
|
accmode = os.O_RDONLY | os.O_WRONLY | os.O_RDWR
|
|
|
|
if (mode & 0222):
|
|
|
|
if not node.writable():
|
|
|
|
log('write access denied for %s (req:%o)' % (path, mode, ))
|
|
|
|
return -errno.EACCES
|
|
|
|
#else:
|
|
|
|
#log('access granted for %s' % (path, ))
|
|
|
|
|
|
|
|
@logexc
|
|
|
|
def mkdir(self, path, mode):
|
|
|
|
self.log("mkdir(%r, %r)" % (path, mode))
|
|
|
|
self.tfs.mkdir(path)
|
|
|
|
|
2008-02-20 00:16:08 +00:00
|
|
|
def launch_tahoe_fuse(tfs, argv):
|
|
|
|
sys.argv = ['tahoe fuse'] + list(argv)
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
server = TahoeFuse(tfs, version="%prog " + fuse.__version__,
|
2008-02-20 00:16:08 +00:00
|
|
|
usage=USAGE,
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
dash_s_do='setsingle')
|
|
|
|
server.parse(errex=1)
|
|
|
|
server.main()
|
|
|
|
|
|
|
|
|
2008-02-20 00:16:08 +00:00
|
|
|
def getbasedir(cap_name='root_dir'):
|
|
|
|
fname = os.path.expanduser("~/.tahoe/private/%s.cap" % (cap_name,))
|
|
|
|
if os.path.exists(fname):
|
|
|
|
f = file(fname, 'rb')
|
|
|
|
bd = f.read().strip()
|
|
|
|
f.close()
|
|
|
|
return bd
|
|
|
|
else:
|
|
|
|
return None
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
def getnodeurl():
|
|
|
|
f = file(os.path.expanduser("~/.tahoe/node.url"), 'rb')
|
|
|
|
nu = f.read().strip()
|
|
|
|
f.close()
|
|
|
|
if nu[-1] != "/":
|
|
|
|
nu += "/"
|
|
|
|
return nu
|
|
|
|
|
|
|
|
def fingerprint(uri):
|
|
|
|
if uri is None:
|
|
|
|
return None
|
|
|
|
return base64.b32encode(sha.new(uri).digest()).lower()[:6]
|
|
|
|
|
|
|
|
class TStat(fuse.Stat):
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
fields = [ 'st_mode', 'st_ino', 'st_dev', 'st_nlink', 'st_uid', 'st_gid', 'st_size',
|
|
|
|
'st_atime', 'st_mtime', 'st_ctime', ]
|
|
|
|
def __init__(self, metadata, **kwargs):
|
|
|
|
# first load any stat fields present in 'metadata'
|
|
|
|
for st in [ 'mtime', 'ctime' ]:
|
|
|
|
if st in metadata:
|
|
|
|
setattr(self, "st_%s" % st, metadata[st])
|
|
|
|
for st in self.fields:
|
|
|
|
if st in metadata:
|
|
|
|
setattr(self, st, metadata[st])
|
|
|
|
|
|
|
|
# then set any values passed in as kwargs
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
fuse.Stat.__init__(self, **kwargs)
|
|
|
|
|
|
|
|
def __repr__(self):
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
return "<Stat%r>" % {
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
'st_mode': self.st_mode,
|
|
|
|
'st_ino': self.st_ino,
|
|
|
|
'st_dev': self.st_dev,
|
|
|
|
'st_nlink': self.st_nlink,
|
|
|
|
'st_uid': self.st_uid,
|
|
|
|
'st_gid': self.st_gid,
|
|
|
|
'st_size': self.st_size,
|
|
|
|
'st_atime': self.st_atime,
|
|
|
|
'st_mtime': self.st_mtime,
|
|
|
|
'st_ctime': self.st_ctime,
|
|
|
|
}
|
|
|
|
|
|
|
|
class Directory(object):
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
def __init__(self, tfs, ro_uri, rw_uri):
|
|
|
|
self.tfs = tfs
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
self.ro_uri = ro_uri
|
|
|
|
self.rw_uri = rw_uri
|
|
|
|
assert (rw_uri or ro_uri)
|
|
|
|
self.children = {}
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
self.last_load = None
|
|
|
|
self.last_data = None
|
|
|
|
self.mtime = 0
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
def __repr__(self):
|
|
|
|
return "<Directory %s>" % (fingerprint(self.get_uri()),)
|
|
|
|
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
def maybe_refresh(self, name=None):
|
|
|
|
"""
|
|
|
|
if the previously cached data was retrieved within the cache
|
|
|
|
validity period, does nothing. otherwise refetches the data
|
|
|
|
for this directory and reloads itself
|
|
|
|
"""
|
|
|
|
now = time.time()
|
|
|
|
if self.last_load is None or (now - self.last_load) > self.tfs.cache_validity:
|
|
|
|
self.load(name)
|
|
|
|
|
|
|
|
def load(self, name=None):
|
|
|
|
now = time.time()
|
|
|
|
print 'loading', name or self
|
|
|
|
log('%s.loading(%s)' % (self, name))
|
|
|
|
url = self.tfs.compose_url("uri/%s?t=json", self.get_uri())
|
|
|
|
data = urllib.urlopen(url).read()
|
|
|
|
h = tagged_hash('cache_hash', data)
|
|
|
|
if h == self.last_data:
|
|
|
|
self.last_load = now
|
|
|
|
log('%s.load() : no change h(data)=%s' % (self, base32.b2a(h), ))
|
|
|
|
return
|
|
|
|
try:
|
|
|
|
parsed = simplejson.loads(data)
|
|
|
|
except ValueError:
|
|
|
|
log('%s.load(): unable to parse json data for dir:\n%r' % (self, data))
|
|
|
|
return
|
|
|
|
nodetype, d = parsed
|
|
|
|
assert nodetype == 'dirnode'
|
|
|
|
self.children.clear()
|
|
|
|
for cname,details in d['children'].items():
|
2008-03-07 00:43:25 +00:00
|
|
|
cname = unicode_to_utf8(cname)
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
ctype, cattrs = details
|
|
|
|
metadata = cattrs.get('metadata', {})
|
|
|
|
if ctype == 'dirnode':
|
|
|
|
cobj = self.tfs.dir_for(cname, cattrs.get('ro_uri'), cattrs.get('rw_uri'))
|
|
|
|
else:
|
|
|
|
assert ctype == "filenode"
|
|
|
|
cobj = File(cattrs.get('size'), cattrs.get('ro_uri'))
|
|
|
|
self.children[cname] = cobj, metadata
|
|
|
|
self.last_load = now
|
|
|
|
self.last_data = h
|
|
|
|
self.mtime = now
|
|
|
|
log('%s.load() loaded: \n%s' % (self, self.pprint(),))
|
|
|
|
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
def get_children(self):
|
|
|
|
return self.children.keys()
|
|
|
|
|
|
|
|
def get_child(self, name):
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
return self.children[name][0]
|
|
|
|
|
|
|
|
def add_child(self, name, child, metadata):
|
|
|
|
log('%s.add_child(%r, %r, %r)' % (self, name, child, metadata, ))
|
|
|
|
self.children[name] = child, metadata
|
|
|
|
url = self.tfs.compose_url("uri/%s/%s?t=uri", self.get_uri(), name)
|
|
|
|
child_cap = do_http('PUT', url, child.get_uri())
|
|
|
|
# XXX [ ] TODO: push metadata to tahoe node
|
|
|
|
assert child_cap == child.get_uri()
|
|
|
|
self.mtime = time.time()
|
|
|
|
log('added child %r with %r to %r' % (name, child_cap, self))
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
def remove_child(self, name):
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
log('%s.remove_child(%r)' % (self, name, ))
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
del self.children[name]
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
url = self.tfs.compose_url("uri/%s/%s", self.get_uri(), name)
|
|
|
|
resp = do_http('DELETE', url)
|
|
|
|
self.mtime = time.time()
|
|
|
|
log('child (%s) removal yielded %r' % (name, resp,))
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
def get_uri(self):
|
|
|
|
return self.rw_uri or self.ro_uri
|
|
|
|
|
|
|
|
def writable(self):
|
|
|
|
return self.rw_uri and self.rw_uri != self.ro_uri
|
|
|
|
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
def pprint(self, prefix='', printed=None, suffix=''):
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
ret = []
|
|
|
|
if printed is None:
|
|
|
|
printed = set()
|
|
|
|
writable = self.writable() and '+' or ' '
|
|
|
|
if self in printed:
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
ret.append(" %s/%s ... <%s> : %s" % (prefix, writable, fingerprint(self.get_uri()), suffix, ))
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
else:
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
ret.append("[%s] %s/%s : %s" % (fingerprint(self.get_uri()), prefix, writable, suffix, ))
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
printed.add(self)
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
for name,(child,metadata) in sorted(self.children.items()):
|
|
|
|
ret.append(child.pprint(' ' * (len(prefix)+1)+name, printed, repr(metadata)))
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
return '\n'.join(ret)
|
|
|
|
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
def get_metadata(self, name):
|
|
|
|
return self.children[name][1]
|
|
|
|
|
|
|
|
def get_stat(self, name):
|
|
|
|
child,metadata = self.children[name]
|
|
|
|
log("%s.get_stat(%s) md: %r" % (self, name, metadata))
|
|
|
|
|
|
|
|
if isinstance(child, Directory):
|
|
|
|
child.maybe_refresh(name)
|
|
|
|
mode = metadata.get('st_mode') or (stat.S_IFDIR | 0755)
|
|
|
|
s = TStat(metadata, st_mode=mode, st_nlink=1, st_mtime=child.mtime)
|
|
|
|
else:
|
|
|
|
if hasattr(child, 'tmp_fname'):
|
|
|
|
s = os.stat(child.tmp_fname)
|
|
|
|
log("%s.get_stat(%s) returning local stat of tmp file" % (self, name, ))
|
|
|
|
else:
|
|
|
|
s = TStat(metadata,
|
|
|
|
st_nlink = 1,
|
|
|
|
st_size = child.size,
|
|
|
|
st_mode = metadata.get('st_mode') or (stat.S_IFREG | 0444),
|
|
|
|
st_mtime = metadata.get('mtime') or self.mtime,
|
|
|
|
)
|
|
|
|
return s
|
|
|
|
|
|
|
|
log("%s.get_stat(%s)->%s" % (self, name, s))
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
return s
|
|
|
|
|
|
|
|
class File(object):
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
def __init__(self, size, ro_uri):
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
self.size = size
|
|
|
|
if ro_uri:
|
|
|
|
ro_uri = str(ro_uri)
|
|
|
|
self.ro_uri = ro_uri
|
|
|
|
|
|
|
|
def __repr__(self):
|
|
|
|
return "<File %s>" % (fingerprint(self.ro_uri) or [self.tmp_fname],)
|
|
|
|
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
def pprint(self, prefix='', printed=None, suffix=''):
|
|
|
|
return " %s (%s) : %s" % (prefix, self.size, suffix, )
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
def get_uri(self):
|
|
|
|
return self.ro_uri
|
|
|
|
|
|
|
|
def writable(self):
|
|
|
|
return True
|
|
|
|
|
|
|
|
class TFS(object):
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
def __init__(self, nodeurl, root_uri, cache_validity_period=DEFAULT_DIRECTORY_VALIDITY):
|
|
|
|
self.cache_validity = cache_validity_period
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
self.nodeurl = nodeurl
|
|
|
|
self.root_uri = root_uri
|
|
|
|
self.dirs = {}
|
|
|
|
|
|
|
|
self.cache = FileCache(nodeurl, os.path.expanduser('~/.tahoe/_cache'))
|
|
|
|
ro_uri = NewDirectoryURI.init_from_string(self.root_uri).get_readonly()
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
self.root = Directory(self, ro_uri, self.root_uri)
|
|
|
|
self.root.maybe_refresh('<root>')
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
def log(self, msg):
|
|
|
|
log("<TFS> %s" % (msg, ))
|
|
|
|
|
|
|
|
def pprint(self):
|
|
|
|
return self.root.pprint()
|
|
|
|
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
def compose_url(self, fmt, *args):
|
|
|
|
return self.nodeurl + (fmt % tuple(map(urllib.quote, args)))
|
|
|
|
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
def get_parent_name_and_child(self, path):
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
"""
|
|
|
|
find the parent dir node, name of child relative to that parent, and
|
|
|
|
child node within the TFS object space.
|
|
|
|
@returns: (parent, name, child) if the child is found
|
|
|
|
(parent, name, None) if the child is missing from the parent
|
|
|
|
(None, name, None) if the parent is not found
|
|
|
|
"""
|
|
|
|
if path == '/':
|
|
|
|
return
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
dirname, name = os.path.split(path)
|
|
|
|
parent = self.get_path(dirname)
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
if parent:
|
|
|
|
try:
|
|
|
|
child = parent.get_child(name)
|
|
|
|
return parent, name, child
|
|
|
|
except KeyError:
|
|
|
|
return parent, name, None
|
|
|
|
else:
|
|
|
|
return None, name, None
|
|
|
|
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
def get_path(self, path):
|
|
|
|
comps = path.strip('/').split('/')
|
|
|
|
if comps == ['']:
|
|
|
|
comps = []
|
|
|
|
cursor = self.root
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
c_name = '<root>'
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
for comp in comps:
|
|
|
|
if not isinstance(cursor, Directory):
|
|
|
|
self.log('path "%s" is not a dir' % (path,))
|
|
|
|
return None
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
cursor.maybe_refresh(c_name)
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
try:
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
cursor = cursor.get_child(comp)
|
|
|
|
c_name = comp
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
except KeyError:
|
|
|
|
self.log('path "%s" not found' % (path,))
|
|
|
|
return None
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
if isinstance(cursor, Directory):
|
|
|
|
cursor.maybe_refresh(c_name)
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
return cursor
|
|
|
|
|
|
|
|
def dir_for(self, name, ro_uri, rw_uri):
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
#self.log('dir_for(%s) [%s/%s]' % (name, fingerprint(ro_uri), fingerprint(rw_uri)))
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
if ro_uri:
|
|
|
|
ro_uri = str(ro_uri)
|
|
|
|
if rw_uri:
|
|
|
|
rw_uri = str(rw_uri)
|
|
|
|
uri = rw_uri or ro_uri
|
|
|
|
assert uri
|
|
|
|
dirobj = self.dirs.get(uri)
|
|
|
|
if not dirobj:
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
self.log('dir_for(%s) creating new Directory' % (name, ))
|
|
|
|
dirobj = Directory(self, ro_uri, rw_uri)
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
self.dirs[uri] = dirobj
|
|
|
|
return dirobj
|
|
|
|
|
|
|
|
def upload(self, fname):
|
|
|
|
self.log('upload(%r)' % (fname,))
|
|
|
|
fh = file(fname, 'rb')
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
url = self.compose_url("uri")
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
file_cap = do_http('PUT', url, fh)
|
|
|
|
self.log('uploaded to: %r' % (file_cap,))
|
|
|
|
return file_cap
|
|
|
|
|
|
|
|
def mkdir(self, path):
|
|
|
|
self.log('mkdir(%r)' % (path,))
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
parent, name, child = self.get_parent_name_and_child(path)
|
|
|
|
|
|
|
|
if child:
|
|
|
|
raise EEXIST('File exists: %s' % (name,))
|
|
|
|
if not parent:
|
|
|
|
raise ENOENT('No such file or directory: %s' % (path,))
|
|
|
|
|
|
|
|
url = self.compose_url("uri?t=mkdir")
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
new_dir_cap = do_http('PUT', url)
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
|
|
|
|
ro_uri = NewDirectoryURI.init_from_string(new_dir_cap).get_readonly()
|
|
|
|
child = Directory(self, ro_uri, new_dir_cap)
|
|
|
|
parent.add_child(name, child, {})
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
def rename(self, path, path1):
|
|
|
|
self.log('rename(%s, %s)' % (path, path1))
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
src_parent, src_name, src_child = self.get_parent_name_and_child(path)
|
|
|
|
dst_parent, dst_name, dst_child = self.get_parent_name_and_child(path1)
|
|
|
|
|
|
|
|
if not src_child or not dst_parent:
|
|
|
|
raise ENOENT('No such file or directory')
|
|
|
|
|
|
|
|
dst_parent.add_child(dst_name, src_child, {})
|
|
|
|
src_parent.remove_child(src_name)
|
|
|
|
|
|
|
|
def unlink(self, path):
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
parent, name, child = self.get_parent_name_and_child(path)
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
|
|
|
|
if child is None: # parent or child is missing
|
|
|
|
raise ENOENT('No such file or directory')
|
|
|
|
if not parent.writable():
|
|
|
|
raise EACCESS('Permission denied')
|
|
|
|
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
parent.remove_child(name)
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
|
|
|
|
def link(self, path, path1):
|
|
|
|
src = self.get_path(path)
|
|
|
|
dst_parent, dst_name, dst_child = self.get_parent_name_and_child(path1)
|
|
|
|
|
|
|
|
if not src:
|
|
|
|
raise ENOENT('No such file or directory')
|
|
|
|
if dst_parent is None:
|
|
|
|
raise ENOENT('No such file or directory')
|
|
|
|
if not dst_parent.writable():
|
|
|
|
raise EACCESS('Permission denied')
|
|
|
|
|
|
|
|
dst_parent.add_child(dst_name, src, {})
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
|
|
|
class FileCache(object):
|
|
|
|
def __init__(self, nodeurl, cachedir):
|
|
|
|
self.nodeurl = nodeurl
|
|
|
|
self.cachedir = cachedir
|
|
|
|
if not os.path.exists(self.cachedir):
|
|
|
|
os.makedirs(self.cachedir)
|
|
|
|
self.tmpdir = os.path.join(self.cachedir, 'tmp')
|
|
|
|
if not os.path.exists(self.tmpdir):
|
|
|
|
os.makedirs(self.tmpdir)
|
|
|
|
|
|
|
|
def log(self, msg):
|
|
|
|
log("<FC> %s" % (msg, ))
|
|
|
|
|
|
|
|
def get_file(self, uri):
|
|
|
|
self.log('get_file(%s)' % (uri,))
|
|
|
|
if uri.startswith("URI:LIT"):
|
|
|
|
return self.get_literal(uri)
|
|
|
|
else:
|
|
|
|
return self.get_chk(uri)
|
|
|
|
|
|
|
|
def get_literal(self, uri):
|
|
|
|
h = sha.new(uri).digest()
|
|
|
|
u = LiteralFileURI.init_from_string(uri)
|
|
|
|
fname = os.path.join(self.cachedir, '__'+base64.b32encode(h).lower())
|
|
|
|
size = len(u.data)
|
|
|
|
self.log('writing literal file %s (%s)' % (fname, size, ))
|
|
|
|
fh = open(fname, 'wb')
|
|
|
|
fh.write(u.data)
|
|
|
|
fh.close()
|
|
|
|
return fname
|
|
|
|
|
|
|
|
def get_chk(self, uri):
|
|
|
|
u = CHKFileURI.init_from_string(str(uri))
|
|
|
|
storage_index = u.storage_index
|
|
|
|
size = u.size
|
|
|
|
fname = os.path.join(self.cachedir, base64.b32encode(storage_index).lower())
|
|
|
|
if os.path.exists(fname):
|
|
|
|
fsize = os.path.getsize(fname)
|
|
|
|
if fsize == size:
|
|
|
|
return fname
|
|
|
|
else:
|
|
|
|
self.log('warning file "%s" is too short %s < %s' % (fname, fsize, size))
|
|
|
|
self.log('downloading file %s (%s)' % (fname, size, ))
|
|
|
|
fh = open(fname, 'wb')
|
|
|
|
url = "%suri/%s" % (self.nodeurl, uri)
|
|
|
|
download = urllib.urlopen(''.join([ self.nodeurl, "uri/", uri ]))
|
|
|
|
while True:
|
|
|
|
chunk = download.read(4096)
|
|
|
|
if not chunk:
|
|
|
|
break
|
|
|
|
fh.write(chunk)
|
|
|
|
fh.close()
|
|
|
|
return fname
|
|
|
|
|
|
|
|
def tmp_file(self, id):
|
|
|
|
fname = os.path.join(self.tmpdir, base64.b32encode(id).lower())
|
|
|
|
return fname
|
|
|
|
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
_tfs = None # to appease pyflakes; is set in main()
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
def print_tree():
|
|
|
|
log('tree:\n' + _tfs.pprint())
|
|
|
|
|
2008-02-20 00:16:08 +00:00
|
|
|
def main(argv):
|
|
|
|
log("\n\nmain(%s)" % (argv,))
|
|
|
|
|
|
|
|
if not argv:
|
|
|
|
argv = ['--help']
|
|
|
|
if len(argv) == 1 and argv[0] in ['-h', '--help', '--version']:
|
|
|
|
launch_tahoe_fuse(None, argv)
|
|
|
|
return -2
|
|
|
|
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
if not argv[0].startswith('-') and len(argv) > 1:
|
2008-02-20 00:16:08 +00:00
|
|
|
cap_name = argv[0]
|
|
|
|
basedir = getbasedir(cap_name)
|
|
|
|
if basedir is None:
|
|
|
|
print 'root dir named "%s" not found.' % (cap_name,)
|
|
|
|
return -2
|
|
|
|
argv = argv[1:]
|
|
|
|
else:
|
|
|
|
basedir = getbasedir() # default 'root_dir'
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
cap_name = 'root_dir'
|
|
|
|
|
|
|
|
# switch to named log file.
|
|
|
|
global logfile
|
|
|
|
fname = 'tfuse.%s.log' % (cap_name,)
|
|
|
|
log('switching to %s' % (fname,))
|
|
|
|
logfile.close()
|
|
|
|
logfile = file(fname, 'ab')
|
2008-02-20 00:16:08 +00:00
|
|
|
|
|
|
|
if argv[-1].startswith('-'):
|
|
|
|
print 'mountpoint not given'
|
|
|
|
return -2
|
|
|
|
mountpoint = os.path.abspath(argv[-1])
|
|
|
|
if not os.path.exists(mountpoint):
|
|
|
|
#raise OSError(2, 'No such file or directory: "%s"' % (mountpoint,))
|
|
|
|
print 'No such file or directory: "%s"' % (mountpoint,)
|
|
|
|
return -2
|
|
|
|
|
|
|
|
nodeurl = getnodeurl()
|
|
|
|
|
|
|
|
tfs = TFS(nodeurl, basedir)
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
print tfs.pprint()
|
|
|
|
|
|
|
|
# make tfs instance accesible to print_tree() for dbg
|
|
|
|
global _tfs
|
|
|
|
_tfs = tfs
|
|
|
|
|
2008-02-20 00:16:08 +00:00
|
|
|
launch_tahoe_fuse(tfs, argv)
|
macfuse: another tahoe fuse implementation
This is the result of various experimentation done into using python-fuse
to provide access to tahoe on the mac. It's rough in quite a few places,
and is really the result of investigation more than a thorough
implemenation of the fuse api.
upon launch, it looks for the users root_dir by opening ~/.tahoe/node.url
and ~/.tahoe/private/root_dir.cap it then proceeds to cache the directory
structure found by walking the users tahoe drive (safely in the face of
directory loops) into memory and then mounts that filesystem.
when a file is read, it calls the tahoe node to first download the file
into a cache directory (~/.tahoe/_cache) and then serves up the file
from there.
when a file is written, a temporary file is allocated within the tmp dir
of the cache, and upon close() (specifically upon release()) the file is
uploaded to the tahoe node, and the new directory entry written.
note that while the durectory structure is cached into memory only when
the filesystem is mounted, that it is 'write through' i.e. changes made
via fuse are reflected into the underlying tahoe fs, even though changes
made to the tahoe fs otherwise show up only upon restart.
in addition to opening files for read and write, the mkdir() and rename()
calls are supported. most other file system operations are not yet
supported. notably stat() metadata is not currently tracked by tahoe,
and is variably reported by this fs depending on write cache files.
also note that this version does not fully support Finder. access through
normal unix commands such as cat, cp, mv, ls etc works fine, and read
access to file from within finder (including preview images and double-
click to open) work ok. but copies to the tahoe drive from within finder
may or may not succeed, but will always report an error. This is still
under investigation.
also note that this does not include any build integration. the included
_fusemodule.so was built on mac os 10.4 against macfuse 1.3.0, and is
known to not work against 10.5-1.3.1 it's possible it may also contain
dependencies upon parts of macports used to build the python that it was
built against. this will be cleaned up later.
usage:
python tahoefuse.py /Path/to/choice/of/mountpoint
or optionally
python tahoefuse.py -ovolicon=/Path/to/icon.icns /Path/to/mountpoint
upon startup, tahoefuse will walk the tahoe directory, then print a
summary of files and folders found, and then daemonise itself. to exit,
either eject the 'drive' (note: 10.5 doesn't show it as a drive, since
it considers fuse to be a connected server instead) or unmount it via
umount /Path/to/mountpoint etc.
2008-02-15 01:35:10 +00:00
|
|
|
|
2008-02-20 00:16:08 +00:00
|
|
|
if __name__ == '__main__':
|
macfuse: slew of updates
various updates to improve the functionality of the mac fuse plugin
1. caching
previously, the experimental tahoefuse plugin pre-loaded the whole
structure of the specified mount into memory at launch time. changes
which were made through that fuse plugin would be remembered, but any
changes made through other tahoe clients would not be reflected.
now directory contents are only loaded when needed, and the data is
cached for a limited time. any use of Directory objects should first
call maybe_refresh() which will check the time since the cache was last
loaded, and if the data is older than some validity period (currently
26s) then the directory's contents will be refetched and reloaded.
this replaces the 'load_dir()' method of TFS
whenever a local change is made to a Directory object, or when the
aforementioned cache reloading notices a change in directory data, the
mtime of the directory is automatically updated.
2. stat / metadata
the retrieval of 'stat' information for getattr(), and the way that
metadata is handled, has been refactored to better reflect the fact that
metadata in tahoe is only represented by 'edges' (i.e entries in
directories) not on 'nodes' (files or directories themselves) hence a
stat lookup should be a query to the parent directory (specifically the
parent specified by the path being queried in the case that a node has
multiple parents) for details known by that directory for the given
child, rather than a query to the child itself.
the TStat utility class for returning stat information to the python-
fuse layer has been extended to accept a 'metadata' argument in its
constructor. any fields found in the metadata dict which match the
names of the stat attributes are loaded into the TStat object. the
'ctime' and 'mtime' attributes are translated to st_ctime and st_mtime
to mesh with the existing timestamp handling code. any fields specified
by kwargs to the constructor override things that might be loaded from
the metadata dict.
Directory objects now track their children as a dict mapping name to
(child_obj, metadata) tuples. This is because the metadata in tahoe
will be stored exclusively on the edges of the graph. each Directory
maintains its own mtime however, and get_stat() calls will report the
mtime of a directory based on the last modification of the Directory
object, not based on any mtime records from the parent directory's
metadata for that child. This addresses the fact that since directories
may be shared, a given parent may or may not reflect the latest changes,
however one of the Finder's behaviours is to examine the stat of a
directory, and not to bother doing a readdir() if the stat is unchanged.
i.e. unless directories report their changes in their stat info, the
Finder will not show changes within that directory.
3. refactoring
reporting of many error codes has been refactored to raise IOError
subclasses with the appropriate errno. this exploits python-fuse's
built-in mechanism for catching IOError and reporting the errno
embedded within it automatically, while simplifying the code within
the plugin.
the add_child() method on TFS was removed in favour of simply having an
add_child() method on Directory objects. this provides a more OO
approach in that Directory is responsible for maintaining its own in
memory state and also writing changes back to the node. similarly for
remove_child()
these changes, along with the new tfs.compose_url() method,
significantly simplify and improve readability of mkdir, rename methods
along with the newer link and unlink. these also get improved error
reporting.
various operations (chmod, chown, truncate, utime) are now ignored.
previously they would report an unsupported operation (EOPNOTSUPP)
but now are simply logged and ignored. this surpresses errors caused
by some client programs which try to use these operations, but at the
moment those operations are meaningless to the tahoe filesystem anyway.
4. link / unlink / rmdir
link, symlink calls are now supported, though with semantics differing
from posix, both equivalent. unlink, rmdir calls are now supported,
also equivalent.
link or symlink calls duplicate the uri of the named source and adds it
as a child of another directory according to the destination path. for
directories, this creates a 'hard' link, i.e. the same directory will
appear in multiple locations within the filesystem, and changes in
any place will be reflected everywhere. for files, by contrast, since
the uri being duplicated is an immutable CHK uri, link/symlink for files
is equivalent to a copy - though significantly cheaper. (a file copy
with the fuse plugin is likely to cause a new file to be written and
uploaded, the link command simply adds an entry referring to an
existing uri)
in testing, the 'ln' command is unable to make hard links (i.e. call
link()) for directories, though symlink ('ln -s') is supported.
either forms works equivalently for files.
unlink and rmdir both remove the specified entry from its parent
directory.
5. logging
the 'tfuse.log' file now only reports launches of the fuse plugin. once
the plugin has parsed the options, it reopens the log file with the
name of the mount, e.g. tfuse.root_dir.log, so that multiple instances
running concurrently will not interfere with each others' logging.
6. bug fixes
the tmp_file in the cache dir backing files opened for write was
intermittently failing to open the file. added O_CREAT to the os.open
call so that files will be created if missing, not throw errors.
a failure to correctly parse arguments if no mount (dir_cap) name was
given but also no fuse options were given has been fixed. now the
command 'tahoe fuse mountpoint' will correctly default to root_dir
also when running from source, arguments to tahoefuse were not handled
to correctly match the 'tahoe fuse ...' behaviour.
2008-03-01 03:12:41 +00:00
|
|
|
main(sys.argv[1:])
|