Brian Warner
7b9fff388d
immutable download: remove dead LiteralDownloader, now that we use filenodes for download
2008-07-16 16:31:47 -07:00
Brian Warner
3e9322bcb6
checker: re-enable checker web results (although they just say 'Healthy' right now)
2008-07-16 15:42:56 -07:00
Brian Warner
7394607141
move encode/upload/download/checker.py into a new immutable/ directory. No behavior changes expected.
2008-07-16 13:14:39 -07:00
Brian Warner
94e619c1f6
overhaul checker invocation
...
Removed the Checker service, removed checker results storage (both in-memory
and the tiny stub of sqlite-based storage). Added ICheckable, all
check/verify is now done by calling the check() method on filenodes and
dirnodes (immutable files, literal files, mutable files, and directory
instances).
Checker results are returned in a Results instance, with an html() method for
display. Checker results have been temporarily removed from the wui directory
listing until we make some other fixes.
Also fixed client.create_node_from_uri() to create LiteralFileNodes properly,
since they have different checking behavior. Previously we were creating full
FileNodes with LIT uris inside, which were downloadable but not checkable.
2008-07-15 17:23:25 -07:00
Brian Warner
fd465b4aaf
download: fix stopProducing failure ('self._paused_at not defined'), add tests
2008-07-14 15:25:21 -07:00
Brian Warner
c46f5bc634
storage: rename the latency key names so they sort properly
2008-07-11 21:51:02 -07:00
Brian Warner
bffe14ef3e
add 1%,10% percentiles to the storage server latency output
2008-07-11 21:34:36 -07:00
Brian Warner
60725ed065
storage: add add_lease/update_write_enabler to remote API, revamp lease handling
2008-07-09 18:06:55 -07:00
Brian Warner
de8ce0cb5b
welcome page: add link to statistics page
2008-07-09 17:37:22 -07:00
Brian Warner
a7aa6f8686
implement a mutable checker+verifier. No repair yet. Part of #205 .
2008-07-07 17:36:00 -07:00
Brian Warner
a0de01e971
servermap.shares_available: report N too, not just k
2008-07-07 15:06:15 -07:00
Brian Warner
45405d85c4
interfaces: add verify= and repair= args to check()
2008-07-07 14:37:36 -07:00
Brian Warner
9b6f23ac05
'tahoe dump-share': show verify-cap too
2008-07-07 14:11:02 -07:00
Brian Warner
2074c92dd1
test_mutable.py: add tests for no-servers conditions, closes #463 .
2008-07-07 12:18:10 -07:00
Brian Warner
bbc63e5d3f
web/root.py: fix minor typo
2008-07-07 00:18:16 -07:00
Brian Warner
91c7e0f689
mutable: if there are no servers available, return an error, rather than hanging forever
2008-07-07 00:17:41 -07:00
Brian Warner
7dc8ff0263
re-enable incident-reporting, but disable it in unit tests, because they take 150% longer, and the leftover trailing timers cause errors
2008-07-06 23:49:08 -07:00
Brian Warner
13d0ba72a4
disable incident reporting: unit tests are failing because of the leftover post-incident timer
2008-07-02 18:24:00 -07:00
Brian Warner
d58591ceba
start using Foolscap's 'incident-logging' feature, which requires foolscap-0.2.9
2008-07-02 17:40:29 -07:00
Brian Warner
dfe235bcba
test_storage: oops, update the test to match the leave-incoming/ change
2008-06-26 11:36:17 -07:00
Brian Warner
02ce46cfb4
catalog-shares: ignore incoming/
2008-06-26 11:23:44 -07:00
Brian Warner
edabd2f004
storage.py: leave the storage/shares/incoming/ directory in place when the bucket is closed
2008-06-26 11:07:57 -07:00
Brian Warner
af4e4dc847
storage: record latency stats in a flat dict, not nested, to conform to RIStatsProvider
2008-06-24 17:21:18 -07:00
Brian Warner
dba0ad3d28
test_mutable.py: add more tests of post-mapupdate corruption, to support #474 testing
2008-06-24 11:08:10 -07:00
Brian Warner
4e5b9ee63e
introducer: move the relevant interfaces out to introducer/interfaces.py
2008-06-18 17:04:41 -07:00
Brian Warner
fa453c9b5e
introducer: add old (V1) introducer code, add test framework for compatibility testing
2008-06-18 16:58:34 -07:00
Brian Warner
9ed9eb2048
check_grid.py: make it work, move node start/stop responsibility from Makefile to script
2008-06-19 12:39:52 -07:00
Brian Warner
50d945ea8f
start work on 'check-grid' target, for use in an automated prodnet tester. Just a docstring so far.
2008-06-18 12:57:47 -07:00
Brian Warner
28f4652b96
break introducer up into separate modules in the new allmydata.introducer package
2008-06-18 12:24:16 -07:00
Brian Warner
5bdff74e5b
web: stop using absolute links (or url.here) in forms and pages, since they break behind proxies. Partially addresses #461
2008-06-17 19:49:40 -07:00
Brian Warner
c09c342718
test_storage: yet more coverage
2008-06-17 17:44:10 -07:00
Brian Warner
6b55b8b022
test_storage.py: improve test coverage
2008-06-17 17:01:42 -07:00
Brian Warner
8e52b3d97d
storage.ReadBucketProxy: avoid double-start, this only affected tests
2008-06-17 17:01:29 -07:00
Brian Warner
b62b3974d3
storage: remove unused method
2008-06-17 11:36:32 -07:00
Brian Warner
2154b5751b
test_storage: add coverage for discard_storage
2008-06-16 17:52:40 -07:00
Brian Warner
75e662cc46
test_storage: add coverage for readonly_storage
2008-06-16 17:52:13 -07:00
Brian Warner
166e68f2a3
web /statistics : show 0 instead of None for mutable files
2008-06-16 16:37:04 -07:00
Brian Warner
1ce3b77dde
storage: improve stats, make them accessible via webport /statistics
2008-06-16 16:35:59 -07:00
Brian Warner
6b7ff02e36
storage: measure latency-per-operation, calculate mean/median/percentiles
2008-06-16 15:21:55 -07:00
robk-tahoe
212c19fc45
webish: re-fix the addSlash issue in directory rendering
...
having fixed the relevant unit test not to assert incorrect behaviour of
the node when rendering directories, it's now safe to fix the latter.
2008-06-11 15:35:27 -07:00
robk-tahoe
b803158de7
test_web: fix webapi test around redirection issues
...
this fixes the test_web test test_POST_upload_mutable which chdir's into
a /foo subdirectory, but then later asserts that the web ui redirects the
user back to /foo, which should really be /foo/ since it's a directory.
2008-06-11 15:19:17 -07:00
Brian Warner
4c681847f8
directory.py: revert that addSlash, it causes a test to fail and we don't understand it yet, and we don't need the fix quite yet
2008-06-11 12:59:28 -07:00
robk-tahoe
021a9e370f
webish: fix directory url redirection
...
when rendering a directory in webish, if the url did not correctly include
a trailing slash '/' then the browser will be redirected to a url which
does. this causes any relative links (e.g. the 'other representations'
which are links to files relative to the current directory) to be correctly
intrpreted as relative to the directory in question by the browser.
uses twisted's addSlash mechanism, which is designed for this purpose :-)
2008-06-11 12:46:28 -07:00
Zooko O'Whielacronx
65618ba5a2
wapi/wui: add a trailing slash to the targets of hyperlinks of children of the current directory when those targets are directories
...
This is intended to fix #458 '"other representations" broken in webish ui'.
2008-06-10 19:59:38 -07:00
Brian Warner
baf3b6868e
storage.py: remove unused import
2008-06-10 13:05:44 -07:00
Brian Warner
bc04b8528a
upload: abort the bucket upon any write error, and do it with callRemoteOnly to avoid double errors
2008-06-10 11:55:28 -07:00
Brian Warner
814922a9a1
storage: ignore shares in incoming/, to make clients use other servers during simultaneous uploads
2008-06-10 11:53:10 -07:00
Brian Warner
0aa6d24d62
web: add test for unicode POST when the name comes from name=, not the filename attribute
2008-06-03 17:09:39 -07:00
Brian Warner
9f5075e83d
web: make nickname more visible in the welcome page, closes #361
2008-06-03 15:02:10 -07:00
Brian Warner
9f59ecafbb
webish: make /cap/ equivalent to /uri/, accepting both with the same meanings. Closes #428
2008-06-03 14:34:00 -07:00
Brian Warner
32c89a8d59
web: transform FileTooLargeError into a friendlier '413 Request Entity Too Large' error
2008-06-03 00:03:16 -07:00
Brian Warner
f4496bd553
mutable/publish.py: raise FileTooLargeError instead of an ugly assertion when the SDMF restrictions are exceeded
2008-06-03 00:02:10 -07:00
Brian Warner
5289064dcf
move FileTooLargeError out to a common location
2008-06-03 00:01:15 -07:00
Brian Warner
beb502ac18
CLI: make 'tahoe webopen' use the 'tahoe:' alias properly, instead of the old --dir-cap option
2008-06-02 17:55:54 -07:00
Brian Warner
3e8d58767f
CLI: improve docs w.r.t. aliases, add examples to 'tahoe put' and 'tahoe get' help output. Addresses part of #431
2008-06-02 17:54:56 -07:00
Brian Warner
8c37b8e3af
Don't allow uploads of large files (about 12GiB or larger), since they're doomed to be corrupted. Closes #439
2008-06-02 16:57:01 -07:00
Brian Warner
b6c2f80170
test_system: even more 'cp -r' coverage
2008-05-21 18:40:49 -07:00
Brian Warner
4824d058cc
test_system.py: improve 'cp -r' coverage: exercise copy from tahoe to local disk
2008-05-21 18:36:25 -07:00
Brian Warner
0f8f213baf
CLI: remove the '-r' shortcut for --dir-cap, to make it available for cp -r
2008-05-21 17:55:32 -07:00
Brian Warner
457248f023
tahoe_cp: rewrite, make --recursive work
2008-05-21 17:35:21 -07:00
Brian Warner
15fbe05092
tahoe_ls: improve error message when the target is missing
2008-05-21 17:34:52 -07:00
Brian Warner
282e22abcf
scripts/common_http.py: add comment about why we need .seek
2008-05-21 17:31:19 -07:00
Brian Warner
7ae432abf8
tahoe_cp.py: fix pyflakes complaint
2008-05-21 12:09:13 -07:00
Brian Warner
7afd869854
cli: initial implementation of 'cp -r', probably doesn't work yet
2008-05-21 11:49:22 -07:00
Brian Warner
fe44b8fb4b
util.dictutil: move DictOfSets out to a separate utility module
2008-05-21 09:43:49 -07:00
Brian Warner
0df663b7ae
CLI: implement the easy part of cp (no -r, only two arguments)
2008-05-20 16:56:03 -07:00
Brian Warner
3ac4a734e5
web t=json: add 'mutable' key to the information dict
2008-05-20 15:40:49 -07:00
Brian Warner
9c992c0d06
test_web: improve test coverage of web.common utility code
2008-05-20 15:21:46 -07:00
Brian Warner
8dded7819c
web: fix JSON output for mutable files
2008-05-20 15:14:19 -07:00
Brian Warner
1d8e452184
CLI: add 'list-aliases', factor out get_aliases
2008-05-20 14:36:04 -07:00
Brian Warner
1236bc3408
CLI: add 'ln', just like move but without the delete
2008-05-20 13:30:31 -07:00
Brian Warner
f09f92c371
test_system.py: remove dead code
2008-05-20 13:20:48 -07:00
Brian Warner
3803d591cf
tahoe_ls: improve output formatting
2008-05-20 13:07:50 -07:00
Brian Warner
abe4addbdb
CLI mv: if we can't created the new child, don't delete the old one
2008-05-20 12:49:47 -07:00
Brian Warner
a8ed3208ba
CLI ls: add --readonly-uri to display readonly URIs for all children
2008-05-20 12:49:11 -07:00
Brian Warner
e889463f0c
CLI: add put --mutable, enhance ls to show mutable vs immutable as rw/r-
2008-05-20 12:36:55 -07:00
Brian Warner
e323019708
web: handle PUT mutable=true properly
2008-05-20 12:36:02 -07:00
Brian Warner
d72f75019f
test_web: add HEAD coverage
2008-05-20 11:47:43 -07:00
Brian Warner
b0e306a753
web: more mutable-file coverage
2008-05-20 11:35:47 -07:00
Brian Warner
6b51841ec8
web: even more test coverage
2008-05-20 11:33:14 -07:00
Brian Warner
2eadabbf82
web: improve test coverage, remove some dead code
2008-05-20 11:13:12 -07:00
Brian Warner
88457fbc32
test_web/test_system: improve test coverage
2008-05-19 23:28:52 -07:00
Brian Warner
506ce2dfd0
test_system: fix windows test failure, open files with binary mode
2008-05-19 20:24:12 -07:00
Brian Warner
8e92dfcb50
overhaul CLI: not quite complete but it works a lot better than it used to. The new scheme uses 'tahoe add-alias' and rsync/scp-style 'alias:foo/bar.txt' arguments
2008-05-19 19:28:50 -07:00
Brian Warner
9662e6d986
test/common: oops, forgot the FakeMutableFileNode.get_readonly fix
2008-05-19 18:52:19 -07:00
Brian Warner
ef929e51f9
web: improve test coverage
2008-05-19 18:38:39 -07:00
Brian Warner
fe8fdfac64
web/filenode: remove unreachable code
2008-05-19 18:38:28 -07:00
Brian Warner
daebb42060
web/directory: fix rw_uri output in t=JSON to reflect mutable files properly
2008-05-19 18:37:28 -07:00
Brian Warner
d0685d42ff
web: factor out identical renderHTTP methods
2008-05-19 15:19:25 -07:00
Brian Warner
242b98f750
webish: censor URLs that start with /file and /named too
2008-05-19 14:34:13 -07:00
Brian Warner
fdb4149bad
test_web: remove leftover import to hush pyflakes
2008-05-19 14:28:39 -07:00
Brian Warner
b1d2922ea3
test_system: match webapi change, making new files now returns 201 Created, not 200 OK
2008-05-19 14:09:31 -07:00
Brian Warner
40172047a0
filenode: add is_mutable to non-MutableFileNode classes
2008-05-19 13:03:00 -07:00
Brian Warner
f9cd30d9bc
webish: complete rewrite, break into smaller pieces, auto-create directories, improve error handling
2008-05-19 12:57:04 -07:00
Brian Warner
b8c0217ad5
test_web: implement API changes from the recent webapi.txt overhaul
2008-05-19 12:56:02 -07:00
Brian Warner
869e1ed41a
test_web.py: minor cleanups, improved error reporting
2008-05-19 12:33:39 -07:00
Brian Warner
3360fd24fc
test_web.py: localdir=/localfile= is going away, so remove the tests that exercise it
2008-05-19 12:32:09 -07:00
Brian Warner
b2f8c22fb8
debug dump-share: print size of encprivkey and signature too
2008-05-19 14:24:41 -07:00
Brian Warner
154dca10a0
servermap.py: add size_of_version(), to retrieve the size of a mutable file version
2008-05-19 14:16:02 -07:00
Brian Warner
87c1e8e066
dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child
2008-05-16 16:09:47 -07:00
Brian Warner
1121df6664
testutil.shouldFail: mention the 'which' string in substring failures too
2008-05-16 16:08:38 -07:00
Brian Warner
304abfee32
webish: add /file links, change directory page to use them. This fixes filenames in wget. Closes #221 .
2008-05-14 14:32:21 -07:00
Brian Warner
5098297a2b
introweb.py: tolerate foolscap>=0.2.6, which changed the internals of .locationHints
2008-05-14 13:55:47 -07:00
Brian Warner
ff0b9e2549
test_mutable.py: test replacing a file that has one new outlier share present: closes #272
2008-05-14 13:10:41 -07:00
Brian Warner
26187bfc81
use a weakref cache in the client to manage singleton filenodes/dirnodes, to avoid autocollision. Should close #391 .
2008-05-08 18:02:55 -07:00
Brian Warner
8f4e409677
check_memory: oops, fix HTTP-based modes: trailing slash in node.url vs no slash
2008-05-08 18:01:31 -07:00
Brian Warner
7323ced8e4
test_dirnode: improve coverage of not-mutable-error a bit
2008-05-08 16:53:35 -07:00
Brian Warner
411f578c19
check_memory: don't use fixed webports, they cause occasional test failures
2008-05-08 16:49:27 -07:00
Brian Warner
fabdc28c06
deep-stats: add file-size histogram
2008-05-08 16:19:42 -07:00
Brian Warner
a1ea3d9b37
test_dirnode: the size of the directories varies, tolerate it as low as 500 bytes
2008-05-08 13:42:42 -07:00
Brian Warner
e6ae7a2c60
dirnode: refactor deep-stats a bit
2008-05-08 13:33:07 -07:00
Brian Warner
6c00a70dbc
dirnode: add a deep_stats(), like deep-size but with more information. webish adds t=deeps-size too.
2008-05-08 13:21:14 -07:00
Brian Warner
462ef2a0ac
run a stats provider even if there's no gatherer, since the HTTP /statistics page is then useful. Only run the once-per-second load-monitor if there is a gatherer configured
2008-05-08 11:37:30 -07:00
Brian Warner
3cb361e233
dirnode: use the concurrency limiter in t=manifest and t=deep-size, allow 10 retrievals in parallel
2008-05-07 18:36:37 -07:00
Brian Warner
d76cf6f748
limiter: fix bug that ignores the limit= argument
2008-05-07 18:35:53 -07:00
Brian Warner
15a9f3d01b
test_util: improve ConcurrencyLimiter tests
2008-05-07 17:00:47 -07:00
Brian Warner
c7e441309d
add a basic concurrency limiter utility
2008-05-07 16:53:30 -07:00
Brian Warner
1e4b904b1a
web status: abbreviate file size in the upload+download summary page
2008-05-07 13:53:58 -07:00
Zooko O'Whielacronx
c1184f51e4
docs: fix a few stale comments in code
2008-05-07 08:39:03 -07:00
Brian Warner
fc774803ce
test_stats.py: improve test coverage
2008-04-30 11:52:31 -07:00
Brian Warner
a5a7ba24ef
stats: add tests for CPUUsageMonitor, modify it a bit to facilitate testing
2008-04-30 11:39:13 -07:00
Zooko O'Whielacronx
fc271a0ee9
setup: don't catch ImportError when importing _auto_deps in allmydata/__init__.py
...
Nowadays pkg_resources is a runtime requirement, and if there is something screwed up in the installation, we want an explicit ImportError exception as early as possible.
2008-04-30 13:22:04 -07:00
Brian Warner
88951548ee
test_mutable: update notify_publish() to match new signature
2008-04-29 18:24:57 -07:00
Brian Warner
4b3adb3fcf
mutable stats: track mutable bytes published too
2008-04-29 18:20:05 -07:00
Brian Warner
3be921174b
stats: add CPU-percentage monitor, with 1min/5min/15min moving-window averages, using time.clock()
2008-04-29 18:12:53 -07:00
Zooko O'Whielacronx
f842656ec2
wui: reorganize the welcome.xhtml page
...
Jake Edge tried Tahoe out and didn't notice the /status page. Hopefully with this new organization people like he will see that link more easily. This also addresses drewp's suggestion that the controls appear above the list of servers instead of below. (I think that was his suggestion.) I also reordered the controls.
2008-04-29 15:10:14 -07:00
Brian Warner
351a7f4632
offloaded: add 'resumes' counter to stats: how many upload requests are resuming an interrupted upload?
2008-04-28 13:27:55 -07:00
ben
c85869fc08
support freebsd 6
2008-04-28 00:41:40 -07:00
Brian Warner
186492e620
introducer: only record one announcement per (tubid,service) tuple. Fixes #343 .
2008-04-23 15:05:39 -07:00
Brian Warner
55dfb697a4
client: don't start the IntroducerClient until the Tub is ready, otherwise we will sometimes connect to the introducer (or other clients) before we've done Tub.setLocation, which loses some information on the introducer status page
2008-04-23 14:52:34 -07:00
Brian Warner
a557d44877
test_mutable: test that all servers refusing our share means a publish fails
2008-04-22 18:53:20 -07:00
Brian Warner
13eb00145e
mutable: improve test coverage in Retrieve, when shares change after mapupdate
2008-04-22 17:25:14 -07:00
Brian Warner
acdd98432d
hush pyflakes warning about code that got moved in the recent StallMixin refactoring
2008-04-22 17:14:26 -07:00
Brian Warner
a7632a345e
mutable/servermap: improve test coverage
2008-04-22 16:47:52 -07:00
Brian Warner
81ab1ec785
testutil: factor stall() out into a common location
2008-04-22 16:47:15 -07:00
Brian Warner
57d5341715
test_introducer.py: don't log nodeids as binary goop
2008-04-22 12:54:16 -07:00
Brian Warner
9ad6ed3777
web/status: remove trailing whitespace
2008-04-22 11:51:14 -07:00
Brian Warner
e6074f5dfc
mutable: test write failures, uncoordinated write detection
2008-04-22 11:49:53 -07:00
Brian Warner
7e1f9761e8
mutable: remove some dead code, rearrange use of populate_pubkey
2008-04-21 17:51:22 -07:00
Brian Warner
db4ddfc1bd
mutable: improve testing to exercise the non-cached case
2008-04-21 17:50:50 -07:00
Brian Warner
f39f8ae032
check_speed: rearrange a bit to make it easy to disable certain portions
2008-04-21 17:29:59 -07:00
Brian Warner
8377244c46
mutable status: make the 'total' timing value optional: unit tests fail otherwise
2008-04-21 17:29:12 -07:00
Brian Warner
6af124dc3e
mutable read: enable the cache (written during mapupdate, read during retrieve). This speeds up small-file reads by about 30% over a link with an average 25ms RTT
2008-04-21 17:27:50 -07:00
Brian Warner
e8b0f3d3cb
mutable status: use google-chart-API to draw server response times for servermap update
2008-04-21 16:16:55 -07:00
Brian Warner
a04c2866b1
mutable: improve test coverage slightly
2008-04-21 15:10:50 -07:00
Brian Warner
f16f48a909
download status: add time spent paused by the client (when we're serving over a slow HTTP link)
2008-04-21 12:19:17 -07:00
Zooko O'Whielacronx
78829dcf08
key_generator: make default key size be a constructor argument instead of a class variable, pass default key size of 522 (the smallest that we can do) in unit tests to make them faster
2008-04-22 12:28:18 -07:00
Brian Warner
c375b1503a
mutable/servermap: improve test coverage
2008-04-18 20:13:28 -07:00
Brian Warner
09dcfeae22
mutable: improve test coverage, fix bug in privkey fetching, add .finished to stats, remove dead code
2008-04-18 19:55:12 -07:00
Brian Warner
fc7b75e55f
mutable.node: avoid reentrancy problems in Deferred code on twisted2.5, by adding an eventual-send call
2008-04-18 00:43:29 -07:00
