Brian Warner
ba019bfd3a
#527 : expire the cached files that are used to support Range: headers, every hour, when the file is unused and older than an hour
2008-10-30 13:39:09 -07:00
Brian Warner
c205a54965
util/cachedir.py: add a cache-directory manager class, which expires+deletes unused files after a while
2008-10-30 13:01:20 -07:00
Brian Warner
1419db6ac7
test_cli: try to fix windows again
2008-10-30 12:32:04 -07:00
Brian Warner
d97c34d622
debug/test_cli: fix error handling for catalog-shares, to make the test stop failing on windows
2008-10-30 12:06:51 -07:00
Brian Warner
b1db6d9ff2
web: add 'Repair' button to checker results when they indicate unhealthyness. Also add the object's uri to the CheckerResults instance.
2008-10-29 18:09:17 -07:00
Brian Warner
6a7d5b893b
create_node.py: add 'web.static = public_html' to the initial tahoe.cfg
2008-10-29 17:13:36 -07:00
Brian Warner
d3f2df00be
webapi: serve the /static URL tree from /public_html (configurable)
2008-10-29 15:34:31 -07:00
Brian Warner
a9101112a4
catalog-shares command: tolerate errors, log them to stderr, handle v2-immutable shares
2008-10-29 15:10:10 -07:00
Brian Warner
186b64b633
test_web.py: one more line of test coverage
2008-10-28 22:00:15 -07:00
Brian Warner
0350994133
test_web: improve test coverage of PUT DIRURL t=uri replace=false
2008-10-28 21:57:44 -07:00
Brian Warner
a5aebf0cbd
web: test (and fix) PUT DIRURL t=uri, which replaces a directory in-place with some other cap
2008-10-28 21:54:46 -07:00
Brian Warner
e1d9169ba3
web/directory.py: slight shuffle to improve test coverage
2008-10-28 21:54:06 -07:00
Brian Warner
248033d78c
test_client.py: improve test coverage a bit
2008-10-28 21:43:35 -07:00
Brian Warner
67ae5c1b2f
node.py: remove unused old_log() function
2008-10-28 21:35:58 -07:00
Brian Warner
ba26239879
node.py: remove support for the old BASEDIR/authorized_keys.PORT file
2008-10-28 21:34:20 -07:00
Brian Warner
b73c380cdb
move testutil into test/common_util.py, since it doesn't count as 'code under test' for our pyflakes numbers
2008-10-28 21:28:31 -07:00
Brian Warner
294e3fb682
util: move PollMixin to a separate file (pollmixin.py), so testutil can be moved into test/
2008-10-28 21:15:48 -07:00
Brian Warner
3dd9b79e3f
control.py: removed unused testutil.PollMixin
2008-10-28 21:03:59 -07:00
Brian Warner
4c2235fcef
web/filenode: oops, fix test failures, not everything has a storage index
2008-10-28 18:17:20 -07:00
Brian Warner
6e1db34817
web/filenode: add Accept-Ranges and ETag (for immutable files) headers to GET responses
2008-10-28 18:01:03 -07:00
Brian Warner
b1ca238176
#527 : respond to GETs with early ranges quickly, without waiting for the whole file to download. Fixes the alacrity problems with the earlier code. Still needs cache expiration.
2008-10-28 17:56:18 -07:00
Brian Warner
37e3d8e47c
#527 : support HTTP 'Range:' requests, using a cachefile. Adds filenode.read(consumer, offset, size) method. Still needs: cache expiration, reduced alacrity.
2008-10-28 13:41:04 -07:00
Brian Warner
2d4e895096
iputil.py: avoid a DNS lookup at startup (which may timeout tests when run on a partially-offline host) by using 198.41.0.4 instead of A.ROOT-SERVERS.NET
2008-10-28 13:36:46 -07:00
Brian Warner
914655c52b
interfaces.py: promote immutable.encode.NotEnoughSharesError.. it isn't just for immutable files any more
2008-10-27 13:34:49 -07:00
Brian Warner
4b48d94c52
interfaces.IMutableFileNode.download_best_version(): fix return value
2008-10-27 13:20:46 -07:00
Brian Warner
fca158e83a
dirnode lookup: use distinct NoSuchChildError instead of the generic KeyError when a child can't be found
2008-10-27 13:15:25 -07:00
Brian Warner
5fd6633289
storage: don't use colons in the corruption-advisory filename, since windows can't tolerate them
2008-10-25 19:46:33 -07:00
Brian Warner
9f21f7cf65
mutable: call remove_advise_corrupt_share when we see share corruption in mapupdate/download/check, tolerate servers that do not implement it
2008-10-24 13:21:28 -07:00
Brian Warner
db37c14ab7
storage: add remote_advise_corrupt_share, for clients to tell storage servers about share corruption that they've discovered. The server logs the report.
2008-10-24 11:52:48 -07:00
Brian Warner
6dbef907ac
mutable/servermap.py: fix needs_merge(), it was incorrectly claiming that mixed shares with distinct seqnums needed a merge, causing repair(force=False) to fail
2008-10-23 21:00:24 -07:00
Brian Warner
0383ec5a49
test_web.test_POST_DIRURL_deepcheck: confirm that /operations/HANDLE/ works with or without the slash
2008-10-23 19:17:59 -07:00
Brian Warner
750ab8fa59
web/checker_results.py: remove dead code
2008-10-23 17:17:17 -07:00
Brian Warner
cadb045288
test_web: more test coverage
2008-10-23 17:11:18 -07:00
Brian Warner
a1cfac89f9
webapi: fix t=rename from==to, it used to delete the file
2008-10-23 16:32:36 -07:00
Brian Warner
d4b4cd8ab8
test_system: update test to match web checker results
2008-10-23 16:32:02 -07:00
Brian Warner
066aedcfa1
webapi deep-check: show the root as <root>, rather than an empty path string
2008-10-23 16:03:59 -07:00
Brian Warner
54c0ffd214
mutable/checker: announce the mapupdate op on the 'recent uploads+downloads' page
2008-10-23 16:03:19 -07:00
Brian Warner
7338440d86
scripts/create_node.py: remove empty-string defaults for --introducer= and --nickname=
2008-10-23 16:02:35 -07:00
Brian Warner
c455d52453
deep-check: add webapi links to detailed per-file/dir results
2008-10-23 16:00:31 -07:00
Brian Warner
819d6b3d03
interface.py: fix typo
2008-10-23 15:59:36 -07:00
Brian Warner
76b7b9d835
webapi: make the /operations/ 't=status' qualifier optional, remove it from examples
2008-10-23 15:56:58 -07:00
Brian Warner
4025f29b8a
web/info.py: fix 'Check This Object' link, for files it was checking the parent directory by mistake
2008-10-22 10:10:56 -07:00
Brian Warner
0129e9931f
#514 : add meta-refresh=60 tag to t=status page for incomplete operations
2008-10-22 09:48:42 -07:00
Brian Warner
2ee6356c89
test_dirnode.py: oops, missed a Monitor(), unbreak tests
2008-10-22 01:50:54 -07:00
Brian Warner
1566a9474c
immutable/filenode.py: add TODO note about the #514 monitor to check(), rather than going through the checker/verifier code and adding it, since Zooko is currently working on that code
2008-10-22 01:42:37 -07:00
Brian Warner
977c6ac510
more #514 : pass a Monitor to all checker operations, make mutable-checker honor the cancel flag
2008-10-22 01:38:18 -07:00
Brian Warner
8178b10ef1
dirnode.py: check for cancel during deep-traverse operations, and don't initiate any new ones if we've been cancelled. Gets us closer to #514 .
2008-10-22 00:55:52 -07:00
Brian Warner
34ab4e3de3
more #514 log-webop status/cancel: add handle-expiration, test coverage
2008-10-21 22:13:54 -07:00
Brian Warner
0f7c1fd6fd
#514 : improve test coverage
2008-10-21 17:52:56 -07:00
Brian Warner
ad3d9207a9
Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514 .
2008-10-21 17:03:07 -07:00
Brian Warner
0832334c0d
storage.py: assert that immutable share size will fit in the 4-byte v1 container (see #346 ). The struct module in py2.4 raises an error on overflow, but py2.5 merely emits a warning
2008-10-20 10:22:08 -07:00
robk-tahoe
4785299980
gui/macapp: minor bugfixes
...
though it seemed to work before the 'fstype' passed to fuse of 'allmydata' was
today throwing errors that len(fstype) must be at most 7.
fixed a typo in changes to 'mount_filesystem()' args
bumped the delay between mounting a filesystem and 'open'ing it in Finder to
4s, as it seems to take a little longer to mount now the client and server
fuse processes need to coordinate.
2008-10-16 09:30:52 -07:00
robk-tahoe
183dd587d5
gui/macapp: slew of code cleanup; unmount filesystems on quit
...
a handful of code cleanup, renaming and refactoring. basically consolidating
'application logic' (mount/unmount fs) into the 'MacGuiApp' class (the wx.App)
and cleaning up various scoping things around that. renamed all references to
'app' to refer more clearly to the 'AppContainer' or to the guiapp.
globally renamed basedir -> nodedir
also made the guiapp keep a note of each filesystem it mounts, and unmount
them upon 'quit' so as to cleanup the user's environment before the tahoe node
vanishes from out underneath the orphaned tahoe fuse processes
2008-09-25 16:32:35 -07:00
robk-tahoe
9b10f46374
gui/macapp: make submenu of aliases for 'webopen'
...
this changes the 'open webroot' menu item to be a submenu listing all aliases
defined in ~/.tahoe. Note that the dock menu does not support submenus, so it
only offers a single 'open webroot' option for the default tahoe: alias.
I had trouble with this at first and concluded that the submenus didn't work,
and made it a distinct 'WebUI' menu in it's own right. on further inspection,
there are still problems but they seem to be something like once the dock menu
has been used, sometimes the app's main menubar menus will cease to function,
and this happens regardless of whether submenus or plain simple menus are used.
I have no idea what the peoblem is, but it's not submenu specific.
2008-09-25 09:39:19 -07:00
Zooko O'Whielacronx
57d9f6beb6
repairer: fix flaw in testutil.flip_one_bit() that Brian pointed out
2008-10-16 12:48:48 -07:00
Zooko O'Whielacronx
8a6d1e5da6
repairer: test all different kinds of corruption that can happen to share files on disk
2008-10-14 16:09:20 -07:00
Brian Warner
b0a00e0f33
util/time_format.py: accept space separator, add unit tests
2008-10-13 15:52:58 -07:00
Brian Warner
46657b797f
test_storage: use different filenames, poor stupid windows
2008-10-09 19:11:39 -07:00
Brian Warner
fd32d72b20
scripts/debug.py: emit the immutable-share version number, tolerate v2
2008-10-09 18:34:22 -07:00
Brian Warner
c289a9c31b
storage.py: improve some precondition() error messages
2008-10-09 18:14:25 -07:00
Brian Warner
7031a69bee
storage: introduce v2 immutable shares, with 8-byte offsets fields, to remove two of the three size limitations in #346 . This code handles v2 shares but does not generate them. We'll make a release with this v2-tolerance, wait a while, then make a second release that actually generates v2 shares, to avoid compatibility problems.
2008-10-09 18:13:27 -07:00
Brian Warner
a0250e320b
debug.py: oops, add missing import for ReadBucketProxy
2008-10-09 17:29:22 -07:00
Brian Warner
288d55825c
storage: split WriteBucketProxy and ReadBucketProxy out into immutable/layout.py . No behavioral changes.
2008-10-09 17:08:00 -07:00
Zooko O'Whielacronx
86e22b8add
interfaces: loosen a few max-size constraints which would limit us to a mere 1.09 TB maximum file size
...
These constraints were originally intended to protect against attacks on the
storage server protocol layer which exhaust memory in the peer. However,
defending against that sort of DoS is hard -- probably it isn't completely
achieved -- and it costs development time to think about it, and it sometimes
imposes limits on legitimate users which we don't necessarily want to impose.
So, for now we forget about limiting the amount of RAM that a foolscap peer can
cause you to start using.
2008-10-09 12:13:57 -07:00
Brian Warner
93725a6935
util/limiter: add a repr
2008-10-07 13:19:45 -07:00
Brian Warner
9d4749d546
dirnode.build_manifest: include node.list in the limiter, that's the most important thing to slow down
2008-10-07 13:19:29 -07:00
Brian Warner
cd236efc68
web/directory: t=manifest output=html: make the caps into clickable hrefs
2008-10-07 13:18:45 -07:00
Brian Warner
f7edbc1536
web/directory: factor out the get_root function
2008-10-07 13:17:42 -07:00
Brian Warner
53c9867c5c
web/directory.py: remove unused imports
2008-10-07 12:48:20 -07:00
Brian Warner
2edfe12b88
test_web: deep-size is more variable than I thought, so assert less
2008-10-06 22:11:47 -07:00
Brian Warner
3ffaded809
web: change t=manifest to return a list of (path,read/writecap) tuples, instead of a list of verifycaps. Add output=html,text,json.
2008-10-06 21:36:18 -07:00
Brian Warner
8e6d122ecf
web: rewrite t=deep-size in terms of deep-stats, update test to match inclusion of directory sizes
2008-10-06 21:35:39 -07:00
Brian Warner
18a1070e66
ftpd: hush pyflakes
2008-10-06 18:45:13 -07:00
Brian Warner
701e5cd1da
ftpd: make sure we're using a patched/fixed Twisted, to avoid confusion later
2008-10-06 18:14:11 -07:00
Brian Warner
05a8360177
ftp: change the twisted hack necessary for async-write-close, to one more agreeable to the twisted-dev folks, add a copy of the necessary patch to docs/ftp.txt
2008-10-06 18:06:05 -07:00
Brian Warner
70ae1bfc8e
ftpd: remove debug messages
2008-10-06 16:16:20 -07:00
Brian Warner
bf3d09d430
ftpd: add native_client.php -based HTTP authentication scheme
2008-10-06 16:15:11 -07:00
Brian Warner
daf4af6699
ftpd: add ftp.accounts checker, remove InMemoryPasswordChecker
2008-10-06 15:51:24 -07:00
Brian Warner
d90a3ed7f8
test_system: add test coverage for immutable download.ConsumerAdapter, remove debug messages
2008-10-06 15:50:37 -07:00
Brian Warner
bc237b3956
ftp server: initial implementation. Still needs unit tests, custom Twisted patches. For #512
2008-10-06 12:52:36 -07:00
Brian Warner
934dc6aa77
test_cli.py: remove unused imports
2008-10-06 17:42:04 -07:00
Brian Warner
dbc2337639
CLI: remove 'tahoe admin generate-keypair', since the pycryptopp ecdsa API is about to change incompatibly. We'll undo this once pycryptopp is updated
2008-10-06 17:23:20 -07:00
Brian Warner
09341a969a
dirnode: fix my remarkably-consistent 'metdadata' typo
2008-10-02 18:08:45 -07:00
Brian Warner
41bacca3f1
interfaces: fix minor typo
2008-10-02 17:52:49 -07:00
Brian Warner
d0bdf9a611
dirnode: add get_child_and_metadata_at_path
2008-10-02 17:52:03 -07:00
Brian Warner
9c505e49c2
stop using 'as' as an identifier: as with 'with', 'as' has become a reserved word in python 2.6
2008-10-02 17:27:49 -07:00
Brian Warner
034364c80b
scripts/admin: split up generate_keypair code so that unit tests can use it more easily
2008-10-01 16:52:38 -07:00
Brian Warner
cd26f58305
#518 : replace various BASEDIR/* config files with a single BASEDIR/tahoe.cfg, with backwards-compatibility of course
2008-09-30 16:21:49 -07:00
Brian Warner
935d185696
tolerate simplejson-2.0.0 and newer, which frequently return bytestrings instead of unicode objects. Closes #523
2008-09-30 15:21:06 -07:00
Brian Warner
78d58875a8
test_node: improve coverage of advertised_ip_addresses a bit
2008-09-29 23:08:16 -07:00
Brian Warner
1e3500ae5f
testutil.PollMixin: set default timeout (to 100s), emit a more helpful error when the timeout is hit
2008-09-29 22:23:09 -07:00
Zooko O'Whielacronx
2e8b89e9e4
repair: fix test to map from storage index to directory structure properly (thanks, cygwin buildbot, for being so kloodgey that you won't accept random binary filenames and thus making me notice this bug)
2008-09-26 15:49:13 -07:00
Zooko O'Whielacronx
cd0af638f3
repairer: assert that the test code isn't accidentally allowing the repairer code which is being tested to do impossible things
2008-09-26 15:23:53 -07:00
Zooko O'Whielacronx
ebee8ee9f4
repairer: enhance the repairer tests
...
Make sure the file can actually be downloaded afterward, that it used one of the
deleted and then repaired shares to do so, and that it repairs from multiple
deletions at once (without using more than a reasonable amount of calls to
storage server allocate).
2008-09-26 10:47:19 -07:00
Brian Warner
98c8e25709
netstring: add required_trailer= argument
2008-09-26 09:57:54 -07:00
Brian Warner
27a30b90d7
test_netstring.py: move netstring tests to a separate file
2008-09-26 09:55:26 -07:00
Brian Warner
e8cf581e3f
move netstring() and split_netstring() into a separate util.netstring module
2008-09-25 21:38:24 -07:00
Zooko O'Whielacronx
39fa9018ef
repairer: remove a test that doesn't apply to the repair-from-corruption case
2008-09-25 15:09:54 -07:00
Zooko O'Whielacronx
73f92ea571
repairer: add a test that repairer fixes corrupted shares (in addition to the test that it fixes deleted shares)
2008-09-25 15:07:12 -07:00
Zooko O'Whielacronx
a3b3d2b7bc
repairer: fix swapped docstrings; thanks Brian
2008-09-25 11:24:36 -07:00
Zooko O'Whielacronx
30111811e3
trivial: remove unused imports; thanks, pyflakes
2008-09-25 11:04:22 -07:00
Zooko O'Whielacronx
a363994c4c
trivial: remove unused imports -- thanks, pyflakes
2008-09-25 10:34:53 -07:00
Zooko O'Whielacronx
1e8d37cc2d
repairer: add basic test of repairer, move tests of immutable checker/repairer from test_system to test_immutable_checker, remove obsolete test helper code from test_filenode
...
Hm... "Checker" ought to be renamed to "CheckerRepairer" or "Repairer" at some point...
2008-09-25 10:16:53 -07:00
Zooko O'Whielacronx
39f305e44f
setup: remove a few minimal unit tests from test_filenode which have been obviated by much better tests in test_mutable and test_system
2008-09-25 09:15:44 -07:00
robk-tahoe
1fd085ced7
gui/macapp: rough cut of ui tweaks; configurability, auto-mount
...
chatting with peter, two things the mac gui needed were the ability to mount
the 'allmydata drive' automatically upon launching the app, and open the
Finder to reveal it. (also a request to hide the debug 'open webroot' stuff)
this (somewhat rough) patch implements all the above as default behaviour
it also contains a quick configuration mechanism for the gui - rather than a
preferences gui, running with a more 'tahoe' styled mechanism, the contents
of a few optional files can modify the default behaviour, specifically file
in ~/.tahoe/gui.conf control behaviour as follows:
auto-mount (bool): if set (the default) then the mac app will, upon launch
automatically mount the 'tahoe:' alias with the display name 'Allmydata'
using a mountpoint of ~/.tahoe/mnt/__auto__
auto-open (bool): if set (the default) then upon mounting a file system
(including the auto-mount if set) finder will be opened to the mountpoint
of the filesystem, which essentially reveals the newly mounted drive in a
Finder window
show-webopen (bool): if set (false by default) then the 'open webroot'
action will be made available in both the dock and file menus of the app
daemon-timout (int): sets the daemon-timeout option passed into tahoe fuse
when a filesystem is mounted. this defaults to 5 min
files of type (int) much, naturally contain a parsable int representation.
files of type (bool) are considered true if their (case-insensitive) contents
are any of ['y', 'yes', 'true', 'on', '1'] and considered false otherwise.
2008-09-25 07:12:24 -07:00
robk-tahoe
63ba26a445
gui/macapp: improve 'about' box
...
adds exactly 1 metric dollop of professionalism to the previously
rather amateurish looking about box.
2008-09-25 06:54:15 -07:00
robk-tahoe
90e3f37173
manhole: be more tolerant of authorized_keys. files in .tahoe
...
both peter and I independently tried to do the same thing to eliminate the
authorized_keys file which was causing problems with the broken mac build
(c.f. #522 ) namely mv authorized_keys.8223{,.bak} but the node is, ahem,
let's say 'intolerant' of the trailing .bak - rather than disable the
manhole as one might expect, it instead causes the node to explode on
startup. this patch makes it skip over anything that doesn't pass the
'parse this trailing stuff as an int' test.
2008-09-24 20:11:49 -07:00
robk-tahoe
4d9f499de1
macapp: changes to support aliases, updated tahoefuse command line options
...
the tahoefuse command line options changed to support the runtests harness,
and as part of that gained support for named aliases via --alias
this changes the mac app's invocation of tahoefuse to match that, and also
changes the gui to present the list of defined aliases as valid mounts
this replaces the previous logic which examined the ~/.tahoe/private directory
looking for files ending in '.cap' - an ad-hoc alias mechanism.
if a file is found matching ~/.tahoe/private/ALIASNAME.icns then that will still
be passed to tahoefuse as the icon to display for that filesystem. if no such
file is found, the allmydata icon will be used by default.
the '-olocal' option is passed to tahoefuse. this is potentially contentious.
specifically this is telling the OS that this is a 'local' filesystem, which is
intended to be used to locally attached devices. however leopard (OSX 10.5)
will only display non-local filesystems in the Finder's side bar if they are of
fs types specifically known by Finder to be network file systems (nfs, cifs,
webdav, afp) hence the -olocal flag is the only way on leopard to cause finder
to display the mounted filesystem in the sidebar, but it displays as a 'device'.
there is a potential (i.e. the fuse docs carry warnings) that this may cause
vague and unspecified undesirable behaviour.
(c.f. http://code.google.com/p/macfuse/wiki/FAQ specifically Q4.3 and Q4.1)
2008-09-24 18:01:28 -07:00
robk-tahoe
ed9873c2a9
CLI: rework webopen, and moreover its tests w.r.t. path handling
...
in the recent reconciliation of webopen patches, I wound up adjusting
webopen to 'pass through' the state of the trailing slash on the given
argument to the resultant url passed to the browser. this change
removes the requirement that arguments must be directories, and allows
webopen to be used with files. it also broke the tests that assumed
that webopen would always normalise the url to have a trailing slash.
in fixing the tests, I realised that, IMHO, there's something deeply
awry with the way tahoe handles paths; specifically in the combination
of '/' being the name of the root path within an alias, but a leading
slash on paths, e.g. 'alias:/path', is catagorically incorrect. i.e.
'tahoe:' == 'tahoe:/' == '/'
but 'tahoe:/foo' is an invalid path, and must be 'tahoe:foo'
I wound up making the internals of webopen simply spot a 'path' of
'/' and smash it to '', which 'fixes' webopen to match the behaviour
of tahoe's path handling elsewhere, but that special case sort of
points to the weirdness.
(fwiw, I personally found the fact that the leading / in a path was
disallowed to be weird - I'm just used to seeing paths qualified by
the leading / I guess - so in a debate about normalising path handling
I'd vote to include the /)
2008-09-24 09:45:23 -07:00
robk-tahoe
2c9a854f10
CLI: reconcile webopen changes
...
I think this is largely attributable to a cleanup patch I'd made
which never got committed upstream somehow, but at any rate various
conflicting changes to webopen had been made. This cleans up the
conflicts therein, and hopefully brings 'tahoe webopen' in line with
other cli commands.
2008-09-24 08:20:02 -07:00
robk-tahoe
8f28d64d77
cli: cleanup webopen command
...
moved the body of webopen out of cli.py into tahoe_webopen.py
made its invocation consistent with the other cli commands, most
notably replacing its 'vdrive path' with the same alias parsing,
allowing usage such as 'tahoe webopen private:Pictures/xti'
2008-06-18 13:19:40 -07:00
robk-tahoe
0042881736
macapp: changed to remove 'Tahoe' from .app name
...
Change the build product from 'Allmydata Tahoe' to 'Allmydata'
more inkeeping with the branding of the Allmydata product
2008-06-10 17:31:45 -07:00
Brian Warner
53f53b7c82
add --syslog argument to 'tahoe start' and 'tahoe restart', used to pass --syslog to twistd for non-Tahoe nodes (like cpu-watcher)
2008-09-24 18:03:02 -07:00
Brian Warner
54fce5c066
webapi: survive slashes in filenames better: make t=info and t=delete to work, and let t=rename fix the problem
2008-09-24 13:35:05 -07:00
Zooko O'Whielacronx
7339e09c72
setup: when detecting platform, ask the Python Standard Library's platform.dist() before executing lsb_release, and cache the result in global (module) variables
...
This should make it sufficiently fast, while still giving a better answer on Ubuntu than platform.dist() currently does, and also falling back to lsb_release if platform.dist() says that it doesn't know.
2008-09-24 11:09:22 -07:00
Brian Warner
dd9171eb72
node.py: add BASEDIR/keepalive_timeout and BASEDIR/disconnect_timeout, to set/enable the foolscap timers, for #521
2008-09-24 10:51:12 -07:00
Zooko O'Whielacronx
ff08bab0c4
immutable: remove unused imports (thanks, pyflakes)
2008-09-23 12:26:10 -07:00
Zooko O'Whielacronx
fa302544fa
immutable: refactor immutable filenodes and comparison thereof
...
* the two kinds of immutable filenode now have a common base class
* they store only an instance of their URI, not both an instance and a string
* they delegate comparison to that instance
2008-09-23 11:52:49 -07:00
Zooko O'Whielacronx
2ef977d85a
setup: try parsing /etc/lsb-release first, then invoking lsb_release, because the latter takes half-a-second on my workstation, which is too long
...
Also because in some cases the former will work and the latter won't.
This patch also tightens the regexes so it won't match random junk.
2008-09-23 10:14:31 -07:00
Zooko O'Whielacronx
37dd91238b
setup: fix a cut-and-paste error in the fallback to parsing /etc/lsb-release
2008-09-23 09:55:51 -07:00
Zooko O'Whielacronx
8a35d2ed83
setup: if executing lsb_release doesn't work, fall back to parsing /etc/lsb-release before falling back to platform.dist()
...
An explanatio of why we do it this way is in the docstring.
2008-09-23 09:28:58 -07:00
Zooko O'Whielacronx
e2b1ce13f5
setup: if invoking lsb_release doesn't work (which it doesn't on our etch buildslave), then fall back to the Python Standard Library's platform.dist() function
2008-09-23 08:48:20 -07:00
Zooko O'Whielacronx
8874e27da0
setup: fix bug in recent patch to use allmydata.get_package_versions() to tell the foolscap app-version-tracking what's what
2008-09-22 17:13:47 -07:00
Zooko O'Whielacronx
992339b97e
setup: when using the foolscap "what versions are here?" feature, use allmydata.get_package_versions() instead of specifically importing allmydata, pycryptopp, and zfec
2008-09-22 17:03:51 -07:00
Zooko O'Whielacronx
5075c12725
setup: simplify the implementation of allmydata.get_package_versions() and add "platform" which is a human-oriented summary of the underlying operating system and machine
2008-09-22 16:53:54 -07:00
Brian Warner
c6999d219e
use foolscap's new app_versions API, require foolscap-0.3.1
2008-09-20 11:38:53 -07:00
Brian Warner
063f85d157
BASEDIR/nickname is now UTF-8 encoded
2008-09-20 11:37:13 -07:00
Brian Warner
0a59991ce6
various: use util.log.err instead of twisted.log.err, so we get both Incidents and trial-test-flunking
2008-09-20 10:35:45 -07:00
Brian Warner
8854d0c1b5
util.log: send log.err to Twisted too, so that Trial tests are flunked
2008-09-20 10:34:27 -07:00
Brian Warner
0799e09910
test_cli: disable generate-keypair test on OS-X, pycryptopp still has a bug
2008-09-19 12:38:55 -07:00
Brian Warner
6607fdc586
CLI: add 'tahoe admin generate-keypair' command
2008-09-18 17:11:33 -07:00
Brian Warner
99d5a8d8b9
web: add 'more info' pages for files and directories, move URI/checker-buttons/deep-size/etc off to them
2008-09-17 22:00:41 -07:00
Peter Secor
c253655f66
conf_wiz.py - updating version numbers in file, should really get these from a TAG or conf file
2008-09-16 17:45:47 -07:00
Brian Warner
51612cd1be
webish: add an extra newline to JSON output
2008-09-15 13:43:14 -07:00
Brian Warner
f00d1961d2
startstop_node.py: find twistd in our supportlib if we had to build Twisted as a setuptools dependency. This is a form of cgalvan's #505 patch, simplified because now 'setup.py trial' takes care of sys.path and PYTHONPATH
2008-09-11 19:51:38 -07:00
Brian Warner
ca273eeaf1
web: fix output=JSON, add buttons for repair/json to the 'run deep-check' form
2008-09-10 14:11:37 -07:00
Brian Warner
f570ad7ba5
disallow deep-check on non-directories, simplifies the code a bit
2008-09-10 13:44:58 -07:00
Brian Warner
4bb88fd2ee
dirnode: refactor recursive-traversal methods, add stats to deep_check() method results and t=deep-check webapi
2008-09-10 01:45:04 -07:00
Brian Warner
f6eeb3161f
dirnode: cleanup, make get_verifier() always return a URI instance, not a string
2008-09-10 01:37:55 -07:00
Brian Warner
28a9f8f076
test_system: check t=deep-stats too
2008-09-09 23:54:57 -07:00
Brian Warner
12ff8a6eae
test_system: add deep-check-JSON tests, fix a bug
2008-09-09 23:14:16 -07:00
Brian Warner
ce7fcbde36
test_system: oops, re-enable some tests that got bypassed
2008-09-09 23:02:45 -07:00
Brian Warner
ba336aed3e
test_system: add deep-stats test
2008-09-09 22:56:34 -07:00
Brian Warner
80d8f3e862
hush pyflakes
2008-09-09 19:50:17 -07:00
Brian Warner
1d2d6a35a6
checker results: add output=JSON to webapi, add tests, clean up APIs
...
to make the internal ones use binary strings (nodeid, storage index) and
the web/JSON ones use base32-encoded strings. The immutable verifier is
still incomplete (it returns imaginary healty results).
2008-09-09 19:45:17 -07:00
Brian Warner
04513e3ac5
immutable verifier: provide some dummy results so deep-check works, make the tests ignore these results until we finish it off
2008-09-09 18:08:27 -07:00
Brian Warner
7fb3308498
mutable checker: even more tests. Everything in ICheckerResults should be covered now, except for immutable-verify which is incomplete
2008-09-09 17:57:06 -07:00
Brian Warner
84a5778507
checker results: more tests, update interface docs
2008-09-09 17:30:10 -07:00
Brian Warner
9676586732
mutable checker: oops, fix redefinition of 'healthy' (numshares < N, not numshares < k, which is 'recoverable' not 'healthy')
2008-09-09 17:28:53 -07:00
Brian Warner
f895e39d48
checker results: more tests, more results. immutable verifier tests are disabled until they emit more complete results
2008-09-09 17:15:46 -07:00
Brian Warner
90b934eb71
checker: add tests, add stub for immutable check_and_repair
2008-09-09 16:34:49 -07:00
Brian Warner
137750eca6
interfaces.py: minor improvement to IDirectoryNode.set_node
2008-09-09 16:34:16 -07:00
Brian Warner
918b0543b9
test_system: make log() tolerate the format= form
2008-09-07 20:03:36 -07:00
Brian Warner
af2231563e
immutable/checker: make log() tolerate the format= form
2008-09-07 20:03:08 -07:00
Brian Warner
3408d552cd
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 12:44:56 -07:00
Brian Warner
e2c484a761
introducer: add get_nickname_for_peerid
2008-09-05 22:07:00 -07:00
Brian Warner
bab799297f
testutil.PollMixin: use a custom exception (and convert it) to avoid the ugly 'stash' cycle
2008-09-02 20:32:51 -07:00
Zooko O'Whielacronx
037c9157dc
setup: simplify parsing of python version number
2008-08-28 17:00:45 -07:00
Zooko O'Whielacronx
6801d5f572
setup: emit the version of python in the list of versions
2008-08-28 15:04:54 -07:00
Brian Warner
3b06aa6a79
servermap: don't log late arrivals, and don't log DeadReferenceError at log.WEIRD
2008-08-26 17:37:29 -07:00
Brian Warner
1668401c16
mutable: make mutable-repair work for non-verifier runs, add tests
2008-08-26 16:34:54 -07:00
Zooko O'Whielacronx
f7b4c45d46
mutable: remove work-around for a flaw in an older version of foolscap
...
We now require "foolscap[secure_connections] >= 0.3.0", per [source:_auto_deps.py].
2008-08-26 08:50:55 -07:00
Brian Warner
a94af879ff
logging: add 'unique-message-ids' (or 'umids') to each WEIRD-or-higher log.msg call, to make it easier to correlate log message with source code
2008-08-25 18:57:59 -07:00
Brian Warner
735aa895b9
logging cleanups: lower DeadReferenceError from WEIRD (which provokes Incidents) to merely UNUSUAL, don't pre-format Failures in others
2008-08-25 17:51:55 -07:00
Zooko O'Whielacronx
def9fc8cf0
checker: make the log() function of SimpleCHKFileVerifier compatible with the log() function of its superclasses and subclasses
2008-08-25 14:44:07 -07:00
Brian Warner
eb695f1c58
mutable/checker: log a WEIRD-level event when we see a hash failure, to trigger an Incident
2008-08-12 20:50:20 -07:00
Brian Warner
97852cd626
immutable checker: add a status_report field
2008-08-12 20:35:30 -07:00
Brian Warner
e82d77c3cf
mutable/servermap: lower the priority of many log messages
2008-08-12 20:35:06 -07:00
Brian Warner
e5252fd5b2
web/deep-check: show the webapi runtime at the bottom of the page
2008-08-12 20:34:26 -07:00
Brian Warner
7a206421b4
CLI: tolerate blank lines in the aliases file
2008-08-12 19:50:50 -07:00
Brian Warner
bcbb51fe4a
test_web: workaround broken HEAD behavior in twisted-2.5.0 and earlier
2008-08-12 19:45:20 -07:00
Brian Warner
6af4619928
test_web: oops, actually use HEAD (instead of GET) in the HEAD test
2008-08-12 19:04:51 -07:00
Brian Warner
4d6b504883
web: use get_size_of_best_version for HEAD requests, provide correct content-type
2008-08-12 19:04:10 -07:00
Brian Warner
d43baa2ad7
mutable: add get_size_of_best_version to the interface, to simplify the web HEAD code, and tests
2008-08-12 19:02:52 -07:00
Brian Warner
014c9b5969
CLI: add 'tahoe debug corrupt-share', and use it for deep-verify tests, and fix non-deep web checker API to pass verify=true into node
2008-08-12 17:05:01 -07:00
Brian Warner
c80e352951
IFilesystemNode: add get_storage_index(), it makes tests easier
2008-08-12 16:14:07 -07:00
Brian Warner
5c0bfd7681
test_system: rename Checker to ImmutableChecker, to make room for a mutable one
2008-08-12 15:59:32 -07:00
Brian Warner
67e7333955
'tahoe debug dump-share': add --offsets, to show section offsets
2008-08-12 14:46:56 -07:00
Brian Warner
022fcf3db8
test_cli: oops, fix tests after recent stdout/stderr cleanup
2008-08-12 14:46:34 -07:00
Brian Warner
acb0297806
scripts/debug: split out dump_immutable_share
2008-08-12 13:55:17 -07:00
Brian Warner
533afd039f
scripts/debug: clean up use of stdout/stderr
2008-08-12 13:52:42 -07:00
Brian Warner
1711b5591f
CLI: move the 'repl' command to 'tahoe debug repl'
2008-08-12 13:40:17 -07:00
Brian Warner
bb33e3e4c2
CLI: move all debug commands (dump-share, dump-cap, find-shares, catalog-shares) into a 'debug' subcommand, and improve --help output
2008-08-12 13:37:32 -07:00
Brian Warner
3cf70697e8
hush a pyflakes warning
2008-08-11 21:24:23 -07:00
Brian Warner
08e229f29a
web/directory: enable verify=true in t=deep-check
2008-08-11 21:24:09 -07:00
Brian Warner
1306b11a76
dirnode: add some deep-check logging
2008-08-11 21:23:38 -07:00
Brian Warner
79576cf75b
checker_results.problems: don't str the whole Failure, just extract the reason string
2008-08-11 21:23:06 -07:00
Brian Warner
d106e411af
checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted
2008-08-11 21:03:26 -07:00
Brian Warner
17d7f7d983
mutable/checker: rearrange a bit, change checker-results to have a status_report string
2008-08-11 20:20:33 -07:00
Brian Warner
a15ae52122
mutable/servermap: add summarize_version
2008-08-11 20:19:30 -07:00
Brian Warner
376df2eb5a
CLI: make 'tahoe webopen' command accept aliases like 'tahoe ls'
2008-08-11 18:20:23 -07:00
Brian Warner
0d5c8468e9
storage: include disk-free information in the stats-gatherer output
2008-08-06 14:06:02 -07:00
Brian Warner
c7c57bd85c
mutable: more repair tests, one with force=True to check out merging
2008-08-06 12:06:07 -07:00
Brian Warner
a68c9364b8
test/common: add ShouldFailMixin
2008-08-06 12:05:52 -07:00
Brian Warner
572115a405
test_mutable: add comment about minimal-bandwidth repairer, comma lack of
2008-08-06 10:38:50 -07:00
Brian Warner
cdbfccd23e
test_mutable: factor out common setup code
2008-08-06 10:38:04 -07:00
Brian Warner
dd6ec73efa
mutable: start adding Repair tests, fix a simple bug
2008-08-05 23:12:39 -07:00
Brian Warner
3ad0168fbf
test_system: factor out find_shares/replace_shares to a common class, so they can be used by other tests
2008-08-05 18:49:58 -07:00
Brian Warner
56943bc5e2
web: add 'report incident' button at the bottom of the welcome page
2008-08-05 12:09:21 -07:00
Brian Warner
45d12baba2
test_cli: more coverage for 'tahoe put' modifying a mutable file in-place, by filename, closes #441
2008-08-04 13:26:43 -07:00
Brian Warner
4fb03c2510
check_grid.py: update to match new CLI: 'put - TARGET' instead of 'put TARGET'
2008-08-01 19:48:56 -07:00
Brian Warner
b9f4006b96
test_cli: remove windows-worrying newlines from test data
2008-08-01 19:47:34 -07:00
Brian Warner
35afe726c1
test_cli.py: factor out CLITestMixin
2008-08-01 19:29:38 -07:00
Brian Warner
f7d2fcc233
CLI: change one-arg forms of 'tahoe put' to make an unlinked file, fix replace-mutable #441
2008-08-01 19:27:29 -07:00
Brian Warner
2eec38279d
CLI: add create-alias command, to merge mkdir and add-alias into a single (secure-from-argv-snooping) step
2008-08-01 19:10:41 -07:00
Brian Warner
dc57ab4d36
test_cli: add system-based tests for PUT, including a mutable put that fails/todo ( #441 )
2008-08-01 15:10:09 -07:00
Brian Warner
071cbec865
tests: simplify CLI tests that use stdin, now that runner supports it
2008-08-01 15:05:14 -07:00
Brian Warner
f181a0458a
CLI: simplify argument-passing, use options= for everthing, including stdout
2008-08-01 11:46:24 -07:00
Zooko O'Whielacronx
6d3ec390b0
tests: add test that verifier notices any (randomly chosen) bit flipped in the verifiable part of any (randomly chosen) share
...
The currently verifier doesn't (usually) pass this randomized test, hence the TODO.
2008-07-30 17:20:15 -07:00
Zooko O'Whielacronx
86160c3fc9
tests: test that checker doesn't cause reads on the storage servers
...
It would still pass the test if it noticed a corrupted share. (It won't
notice, of course.) But it is required to do its work without causing storage
servers to read blocks from the filesystem.
2008-07-30 16:54:20 -07:00
Zooko O'Whielacronx
29255568df
storage: make storage servers declare oldest supported version == 1.0, and storage clients declare oldest supported version == 1.0
...
See comments in patch for intended semantics.
2008-07-30 15:51:07 -07:00
Zooko O'Whielacronx
dbd1c2397e
tests: use the handy dandy TestCase.mktemp() function from trial to give unique and nicely named directories for each testcase
2008-07-30 15:49:20 -07:00
Zooko O'Whielacronx
6aabaa0797
tests: don't use SignalMixin
...
It seems like we no longer need it, and it screws up something internal in
trial which causes trial's TestCase.mktemp() method to exhibit wrong behavior
(always using a certain test method name instead of using the current test
method name), and I wish to use TestCase.mktemp().
Of course, it is possible that the buildbot is about to tell me that we do
still require SignalMixin on some of our platforms...
2008-07-30 15:35:36 -07:00
Zooko O'Whielacronx
56282613c5
tests: add test_system.Checker which tests basic checking (without verification) functionality
2008-07-28 16:43:17 -07:00
Zooko O'Whielacronx
9d14341c7d
test: add testutil.flip_one_bit which flips a randomly chosen bit of the input string
2008-07-28 16:42:17 -07:00
Zooko O'Whielacronx
fe522e31a4
tests: make it so that you can use common.py's SystemTestMixin.set_up_nodes() more than once with the same introducer
2008-07-28 16:40:29 -07:00
Brian Warner
cd4d3cdeea
download.py: set up self._paused before registering the producer, since they might call pauseProducing right away
2008-07-28 14:57:31 -07:00
Brian Warner
9e22353773
test/common.py: use pre-computed Tub certificates for the system-test mixin, to speed such tests up by maybe 15%. The goal is to encourage more full-grid tests.
2008-07-28 12:44:21 -07:00
Brian Warner
914783acdf
web: add /status/?t=json, with active upload/download ops. Addresses #493 .
2008-07-25 17:41:10 -07:00
Brian Warner
f8221c2c66
web: make t=json stats pages use text/plain, instead of leaving it at text/html
2008-07-25 17:24:27 -07:00
Brian Warner
48c64b7a29
test_system.py: factor SystemTestMixin out of SystemTest
2008-07-25 15:33:49 -07:00
Brian Warner
10bbc740e9
test_system.py: modify system-test setup code in preparation for merge with common.SystemTestMixin
2008-07-25 15:29:31 -07:00
Brian Warner
766cdd9c52
test_system.py: move SystemTestMixin out into common.py, where further improvements will occur
2008-07-25 15:17:58 -07:00
Brian Warner
e317fcc08c
test_system.py: create SystemTestMixin, with less cruft, for faster system-like tests
2008-07-25 15:13:00 -07:00
Brian Warner
afda2a43e4
storage: remove update_write_enabler method, it won't serve the desired purpose, and I have a better scheme in mind. See #489 for details
2008-07-21 17:28:28 -07:00
Zooko O'Whielacronx
9461887e0a
immutable file download: make the ciphertext hash tree mandatory
...
This fixes #491 (URIs do not refer to unique files in Allmydata Tahoe).
Fortunately all of the versions of Tahoe currently in use are already producing
this ciphertext hash tree when uploading, so there is no
backwards-compatibility problem with having the downloader require it to be
present.
2008-07-21 09:31:02 -07:00
Brian Warner
edec917060
test_web: test that save=true filename=unicode doesn't crash
2008-07-18 18:58:57 -07:00
Brian Warner
572c848d98
web: for GET save=true, don't interpret the filename= arg with any character set, just copy the bytes back into the Content-Disposition header. This seems to make it maximally compatible with Firefox and IE7
2008-07-18 18:06:50 -07:00
Brian Warner
879fefe5f3
first pass at a mutable repairer. not tested at all yet, but of course all existing tests pass
2008-07-17 21:09:23 -07:00
Brian Warner
3e95681bad
interfaces: add IRepairable
2008-07-17 17:32:17 -07:00
Brian Warner
923c9c242a
oops, fix import/pyflakes problems
2008-07-17 17:06:20 -07:00
Brian Warner
67db0a4967
deep-check: add webapi, add 'DEEP-CHECK' button to wui, add tests, rearrange checker API a bit
2008-07-17 16:47:09 -07:00
Brian Warner
69156aeb28
dirnode deep-check: add tests of cycles, fix failures
2008-07-17 14:37:04 -07:00
Brian Warner
acf3180fac
dirnode deep-check: rearrange traversal approach, simplify code a bit
2008-07-17 14:25:04 -07:00
Brian Warner
9289433ba3
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-16 18:20:57 -07:00
Brian Warner
7b9fff388d
immutable download: remove dead LiteralDownloader, now that we use filenodes for download
2008-07-16 16:31:47 -07:00
Brian Warner
3e9322bcb6
checker: re-enable checker web results (although they just say 'Healthy' right now)
2008-07-16 15:42:56 -07:00
Brian Warner
7394607141
move encode/upload/download/checker.py into a new immutable/ directory. No behavior changes expected.
2008-07-16 13:14:39 -07:00
Brian Warner
94e619c1f6
overhaul checker invocation
...
Removed the Checker service, removed checker results storage (both in-memory
and the tiny stub of sqlite-based storage). Added ICheckable, all
check/verify is now done by calling the check() method on filenodes and
dirnodes (immutable files, literal files, mutable files, and directory
instances).
Checker results are returned in a Results instance, with an html() method for
display. Checker results have been temporarily removed from the wui directory
listing until we make some other fixes.
Also fixed client.create_node_from_uri() to create LiteralFileNodes properly,
since they have different checking behavior. Previously we were creating full
FileNodes with LIT uris inside, which were downloadable but not checkable.
2008-07-15 17:23:25 -07:00
Brian Warner
fd465b4aaf
download: fix stopProducing failure ('self._paused_at not defined'), add tests
2008-07-14 15:25:21 -07:00
Brian Warner
c46f5bc634
storage: rename the latency key names so they sort properly
2008-07-11 21:51:02 -07:00
Brian Warner
bffe14ef3e
add 1%,10% percentiles to the storage server latency output
2008-07-11 21:34:36 -07:00
Brian Warner
60725ed065
storage: add add_lease/update_write_enabler to remote API, revamp lease handling
2008-07-09 18:06:55 -07:00
Brian Warner
de8ce0cb5b
welcome page: add link to statistics page
2008-07-09 17:37:22 -07:00
Brian Warner
a7aa6f8686
implement a mutable checker+verifier. No repair yet. Part of #205 .
2008-07-07 17:36:00 -07:00
Brian Warner
a0de01e971
servermap.shares_available: report N too, not just k
2008-07-07 15:06:15 -07:00
Brian Warner
45405d85c4
interfaces: add verify= and repair= args to check()
2008-07-07 14:37:36 -07:00
Brian Warner
9b6f23ac05
'tahoe dump-share': show verify-cap too
2008-07-07 14:11:02 -07:00
Brian Warner
2074c92dd1
test_mutable.py: add tests for no-servers conditions, closes #463 .
2008-07-07 12:18:10 -07:00
Brian Warner
bbc63e5d3f
web/root.py: fix minor typo
2008-07-07 00:18:16 -07:00