Zooko O'Whielacronx
718537ab8c
tests: mark test_runner as coded in utf-8 instead of ascii
2009-05-07 15:31:51 -07:00
Zooko O'Whielacronx
04ff9a3f97
tests: raise timeout on test_runner.RunNode.test_introducer from 120s to 240s, since it hit the 120s time-out on François Lenny-armv5tel
2009-05-07 14:50:12 -07:00
Zooko O'Whielacronx
6350634ee3
tests: remove obsolete test (it tests functionality that is long gone) which occasionally ERRORs now that we have more metadata (since [20090411225205-92b7f-7adfb89cb4db4ac7d28427934dea3d2c108f6476])
2009-04-12 19:36:21 -07:00
Zooko O'Whielacronx
6b9899ade1
trivial: remove trailing whitespace and unused import
2009-04-11 19:17:42 -07:00
Zooko O'Whielacronx
9729753692
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API
...
Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628 .
2009-04-11 15:52:05 -07:00
Brian Warner
922b3034be
web: make sure that PUT /uri?mutable=false really means immutable, fixes #675
2009-04-07 19:13:40 -07:00
Zooko O'Whielacronx
37c6e77764
wui: edit some of the human-readable parts of the wui such as button labels
...
(The word "parent" suggests that you can go up a directory hierarchy -- perhaps that word is vestigial.)
2009-04-07 11:54:59 -07:00
Zooko O'Whielacronx
8b1cd154da
wui: fix bug in which empty directory is marked as "unreadable", add test, remove exclamation point
2009-04-07 11:28:34 -07:00
Zooko O'Whielacronx
de8475389b
hashtree: fix tests of depth_of
2009-04-03 16:56:14 -07:00
Zooko O'Whielacronx
a07e746d9c
trivial: whitespace
2009-04-03 16:33:02 -07:00
Zooko O'Whielacronx
b12a7f9ee8
leases, time_format: modify time stamping in lease description
...
* emit lease expiry date in ISO-8601'ish format as well as Brian's format
* rename iso_utc_time_to_localseconds() to iso_utc_time_to_seconds()
* add iso_utc_date()
* simplify the body of iso_utc_time_to_seconds()
2009-04-03 15:59:04 -07:00
Brian Warner
466014f66f
hashtree: fix O(N**2) behavior, to improve fatal alacrity problems in a 10GB file ( #670 ). Also improve docstring.
2009-03-31 13:21:27 -07:00
Brian Warner
bd93430c53
expirer: include crawler progress in the JSON status output
2009-03-24 13:51:37 -07:00
Brian Warner
01e2032669
hashutil: add constant-time comparison function, to avoid timing attacks when python's short-circuiting data-dependent == operator is used to, say, check a write-enabler
2009-03-22 20:20:55 -07:00
Brian Warner
6599eae6f9
WUI: fix display of empty directories, it threw an exception before
2009-03-20 16:58:09 -07:00
Brian Warner
3b65607926
expirer: tolerate empty buckets, refactor bucketsize-counting code a bit, don't increment -mutable/-immutable counters unless we actually know the sharetype
2009-03-20 12:18:16 -07:00
Brian Warner
8645738c77
storage: improve wording of status message
2009-03-19 11:48:37 -07:00
Brian Warner
66fe8ca24c
parse_date: insist that it returns an int
2009-03-19 11:44:43 -07:00
Brian Warner
186b6a8c01
storage status: report expiration-cutoff-date like 19-Mar-2009 (as opposed to the tahoe.cfg input format of 2009-03-19), for redundancy: someone who gets the month and day switched will have a better chance to spot the problem in the storage-status output if it's in a different format
2009-03-19 11:07:56 -07:00
Brian Warner
f0071c2571
expirer: clean up constructor args, add tahoe.cfg controls, use cutoff_date instead of date_cutoff
2009-03-18 18:00:09 -07:00
Brian Warner
babcf632da
util/time_format: new routine to parse dates like 2009-03-18, switch expirer to use it. I'd prefer to use 18-Mar-2009, but it is surprisingly non-trivial to build a parser that will take UTC dates instead of local dates
2009-03-18 17:58:14 -07:00
Brian Warner
8eaee28550
expirer: change setup, config options, in preparation for adding tahoe.cfg controls
2009-03-18 17:21:38 -07:00
Brian Warner
fffab0d724
expirer: track mutable-vs-immutable sharecounts and sizes, report them on the web status page for comparison
2009-03-18 13:25:04 -07:00
Brian Warner
406fdba61f
add utility function to parse durations, for lease-expiration config
2009-03-17 00:01:17 -07:00
Brian Warner
24ab5ec26f
expirer: add mode to expire only-mutable or only-immutable shares
2009-03-16 23:51:18 -07:00
Brian Warner
c7254c5f1d
GC: add date-cutoff -based expiration, add proposed docs
2009-03-16 22:10:41 -07:00
Brian Warner
2b525a42d3
test_deepcheck: remove the 10s timeout: our dapper buildslave requires 30s, and the reduced timeout was only there because this tests fails by timeout rather than explicitly
2009-03-15 17:51:34 -07:00
Brian Warner
f0b0ad1c8f
tahoe cp -r: add --caps-only flag, to write filecaps into local files instead of actual file contents. Used only for debugging and as a quick tree-comparison tool.
2009-03-15 16:19:58 -07:00
Brian Warner
6e57576f2e
dirnode deep_traverse: insert a turn break (fireEventually) at least once every 100 files, otherwise a CHK followed by more than 158 LITs can overflow the stack, sort of like #237 .
2009-03-13 16:31:35 -07:00
Brian Warner
ddc9a7ae74
consolidator: re-use more directories, add total directories seen-vs-used counts
2009-03-12 20:48:01 -07:00
Brian Warner
1c24707f19
consolidator: add more verbose traversal of directories
2009-03-12 16:29:00 -07:00
Brian Warner
760688a224
consolidate: create multiple numbered backups of the original Archives directory, not just the first time
2009-03-12 16:04:27 -07:00
Brian Warner
c4c6a62954
add 'tahoe debug consolidate' command, to merge directories created by repeated 'tahoe cp -r' or the allmydata win32 backup tool, into the form that would have been created by 'tahoe backup'.
2009-03-12 13:56:06 -07:00
Brian Warner
1ccd426a34
expirer: fix prediction math, thanks to Zandr for the catch
2009-03-09 13:42:17 -07:00
Brian Warner
a68ad06254
storage.expirer: exercise the last missing line of webstatus code
2009-03-08 20:38:28 -07:00
Brian Warner
6d6049430c
expirer: tolerate corrupt shares, add them to the state and history for future examination
2009-03-08 20:08:40 -07:00
Brian Warner
1a98521c3d
storage/immutable: raise a specific error upon seeing a bad version number, instead of using assert. Also wrap to 80cols.
2009-03-08 20:07:32 -07:00
Brian Warner
02b40ec499
storage: add test coverage for BucketReader.__repr__
2009-03-08 20:05:27 -07:00
Brian Warner
6d7319c588
storage/mutable: raise a specific error upon seeing bad magic, instead of using assert
2009-03-08 19:02:01 -07:00
Brian Warner
5675b4e7e0
expirer: make web display a bit more consistent
2009-03-07 16:14:42 -07:00
Brian Warner
2c2c3c2295
test_cli.py: assert that 'ls' on an unrecoverable file now gives a better error message
2009-03-07 05:08:15 -07:00
Brian Warner
67d7600781
test_web: test behavior of broken-dirnode GET, both html and json
2009-03-07 04:57:07 -07:00
Brian Warner
9c7fafd5af
test_cli: use explicit (and stable) testdirs, instead of using self.mktemp
2009-03-07 03:04:28 -07:00
Brian Warner
091cfd04ca
test_storage: solaris doesn't appear to give the same block count as other platforms, so don't assert as much about 'diskbytes' recovered
2009-03-07 02:45:18 -07:00
Brian Warner
0dee2a6036
storage: add a lease-checker-and-expirer crawler, plus web status page.
...
This walks slowly through all shares, examining their leases, deciding which
are still valid and which have expired. Once enabled, it will then remove the
expired leases, and delete shares which no longer have any valid leases. Note
that there is not yet a tahoe.cfg option to enable lease-deletion: the
current code is read-only. A subsequent patch will add a tahoe.cfg knob to
control this, as well as docs. Some other minor items included in this patch:
tahoe debug dump-share has a new --leases-only flag
storage sharefile/leaseinfo code is cleaned up
storage web status page (/storage) has more info, more tests coverage
space-left measurement on OS-X should be more accurate (it was off by 2048x)
(use stat .f_frsize instead of f_bsize)
2009-03-06 22:45:17 -07:00
Brian Warner
c5ecb7b3ba
test_cli: validate non-HTML error response of 'tahoe get' on an unrecoverable file
2009-03-03 22:11:46 -07:00
Brian Warner
f42e3bb107
web: full patch for HTML-vs-plaintext traceback renderings, improve test coverage of exception rendering
2009-03-03 21:56:30 -07:00
Brian Warner
53f50fcc78
test/common: rearrange shouldHTTPError
2009-03-03 19:40:59 -07:00
Brian Warner
67571eb033
add more information to NotEnoughSharesError, split out new exceptions for no-servers and no-source-of-ueb-hash
2009-03-03 19:37:15 -07:00
Brian Warner
f15c0be5af
test_web: move shouldHTTPError into common.WebErrorMixin
2009-03-03 16:56:20 -07:00
Brian Warner
559c369574
test_web: clean up shouldHTTPError methods
2009-03-03 16:46:04 -07:00
Brian Warner
8c3013c4f7
rrefutil: add check_remote utility function
2009-02-27 00:59:57 -07:00
Brian Warner
1b3e635936
rrefutil: add trap_remote utility and friends
2009-02-27 00:55:24 -07:00
Brian Warner
8251572e01
web: improve layout of storage status with a table
2009-02-26 19:58:38 -07:00
Brian Warner
112dc35563
crawler: add ETA to get_progress()
2009-02-26 19:42:48 -07:00
Brian Warner
63b19e567c
web: fix the ERROR: line to work the same in python2.4 and 2.5
2009-02-25 01:46:21 -07:00
Brian Warner
7d4901197a
test_cli/test_web: fix spurious test failure on solaris (maybe python2.4?) due to variations in the way that exceptions are stringified
2009-02-25 00:01:28 -07:00
Brian Warner
a3c1fe35d9
CLI: modify 'tahoe manifest' and 'tahoe deep-check' to report ERROR: properly. For #590 .
2009-02-24 23:44:15 -07:00
Brian Warner
fd4ceb6a87
webapi: modify streaming deep-manifest/deep-checker to emit an ERROR: line if they encounter an unrecoverable+untraversable directory. For #590 .
2009-02-24 23:13:35 -07:00
Brian Warner
7f8708cb7c
scripts/common: fix alias handling on windows again, emit slightly nicer error message in response to an unknown alias
2009-02-24 22:21:36 -07:00
Brian Warner
51e3baeefc
test_deepcheck: switch deep-check tests to use no-network too. This cuts the runtime down by about 50%
2009-02-24 21:04:57 -07:00
Brian Warner
6cf4096156
test_deepcheck: convert MutableChecker to no-network GridTest
2009-02-24 20:00:10 -07:00
Brian Warner
2e7f64d392
tests/no_network: move GET into the GridTestMixin class
2009-02-24 18:33:00 -07:00
Brian Warner
6f981e4eab
common_web.py: oops, add .fields to the other FakeRequest
2009-02-24 18:04:59 -07:00
Brian Warner
376eab0c04
test_cli: exercise the recent tolerate-'c:\dir\file.txt' fix in scripts/common, recorded in a separate match to make it easier to merge the fix to prod
2009-02-24 17:56:20 -07:00
Brian Warner
8d8250e44a
test/common_web.py: add a .fields attribute to our FakeRequest, since we support versions of Nevow that are old enough to not do it themselves
2009-02-24 17:20:50 -07:00
Brian Warner
2810de32b1
test_web: add (disabled) test to see what happens when deep-check encounters an unrecoverable directory. We still need code changes to improve this behavior.
2009-02-24 15:40:17 -07:00
Brian Warner
7cfbb9c832
test_repairer: change to use faster no_network.GridTestMixin, split Verifier tests into separate cases, refactor judgement funcs into shared methods
2009-02-23 22:15:06 -07:00
Brian Warner
2be729b1e4
test_repairer: change Repairer to use much-faster no_network.GridTestMixin. As a side-effect, fix what I think was a bug: some of the assert-minimal-effort-expended checks were mixing write counts and allocate counts
2009-02-23 17:42:27 -07:00
Brian Warner
cc3c1ae8df
test/no_network.py: add a basic stats provider
2009-02-23 17:39:37 -07:00
Brian Warner
f35c659b2f
tests: stop using setUpClass/tearDownClass, since they've been deprecated in Twisted-8.2.0
2009-02-23 14:43:12 -07:00
Brian Warner
bc91689f8e
test_checker: improve test coverage for checker results
2009-02-23 14:19:43 -07:00
Alberto Berti
d9649052d4
Fixed tests again so they will pass on windows.
2009-02-22 18:35:02 -07:00
Brian Warner
c4f8005867
remove more RuntimeError from unit tests, for #639
2009-02-22 17:28:55 -07:00
Brian Warner
5c3d7d8e73
stop using RuntimeError in unit tests, for #639
2009-02-22 17:27:22 -07:00
Alberto Berti
b945c275c2
Added tests for the cse when listdir is an iterator
2009-02-22 16:43:56 -07:00
Alberto Berti
cd0bd97954
Fixed tests so that they pass also on buildbots.
2009-02-22 16:43:11 -07:00
Alberto Berti
b2f8fe04c6
Use failUnlessEqual instead of failUnless(a == b)
2009-02-22 16:42:14 -07:00
Alberto Berti
5d629d0050
Added tests for the --exclude* options of backup command.
2009-02-22 10:51:06 -07:00
Alberto Berti
ff6a77cc6b
Added tests for the fixed alias related command's synopsis
2009-02-22 10:37:32 -07:00
Brian Warner
c85eccbf64
test_web.py: increase test coverage of web.status.plural()
2009-02-21 18:01:16 -07:00
Brian Warner
77f3b83d68
crawler: fix performance problems: only save state once per timeslice (not after every bucket), don't start the crawler until 5 minutes after node startup
2009-02-21 14:56:49 -07:00
Brian Warner
a04d3b8fe8
test_system: oops, don't assume that all files in storage/ are in a deep storage/shares/prefix/si/shnum path, since now the crawler pickle has a short path
2009-02-21 00:17:10 -07:00
Brian Warner
3c6471c717
test_storage: startService the server, as is now the standard practice
2009-02-20 21:57:55 -07:00
Brian Warner
1077826357
BucketCountingCrawler: rename status and state keys to use 'bucket' instead of 'share', because the former is more accurate
2009-02-20 21:46:06 -07:00
Brian Warner
b3cd4952bd
storage: add bucket-counting share crawler, add its output (number of files+directories maintained by a storage server) and status to the webapi /storage page
2009-02-20 21:04:08 -07:00
Brian Warner
73e05bf967
crawler: add get_progress, clean up get_state
2009-02-20 18:27:43 -07:00
Brian Warner
2e45619844
web/storage: make sure we can handle platforms without os.statvfs too
2009-02-20 16:03:53 -07:00
Brian Warner
c6a061e600
crawler: provide for one-shot crawlers, which stop after their first full cycle, for share-upgraders and database-populaters
2009-02-20 15:19:11 -07:00
Brian Warner
00677ff9a5
web: add Storage status page, improve tests
2009-02-20 14:29:26 -07:00
Brian Warner
c6c30ac5d4
webapi: pass client through constructor arguments, remove IClient, should make it easier to test web renderers in isolation
2009-02-20 12:15:54 -07:00
Brian Warner
e357b8a679
test/no_network: do startService on the storage servers, make it easier to customize the storage servers
2009-02-19 20:22:54 -07:00
Brian Warner
ef4ff21ae7
crawler: modify API to support upcoming bucket-counting crawler
2009-02-19 19:31:42 -07:00
Brian Warner
9bc08158c6
test_backupdb: make the not-a-database file larger, since the older sqlite-2.3.2 on OS-X is easily fooled
2009-02-19 18:04:09 -07:00
Brian Warner
2f751332e8
test_crawler: hush pyflakes
2009-02-19 14:23:40 -07:00
Brian Warner
ff9811211d
test_crawler: disable the percentage-of-cpu-used test, since it is too unreliable on our slow buildslaves. But leave the code in place for developers to run by hand.
2009-02-19 14:16:54 -07:00
Brian Warner
790a10d1b2
reliability.py: fix the numpy conversion, it was completely broken. Thanks to Terrell Russell for the help.
2009-02-19 13:55:15 -07:00
Brian Warner
268cd538e2
reliability: switch to NumPy, since Numeric is deprecated
2009-02-19 01:44:35 -07:00
Brian Warner
f10aa59e5b
test_crawler: don't require >=1 cycle on cygwin
2009-02-19 00:58:18 -07:00
Brian Warner
b949ea4f32
crawler: fix problems on windows and our slow cygwin slave
2009-02-18 22:24:31 -07:00
