Kevin Reid
35d3671f4e
Link all Tahoe web pages to the /tahoe_css stylesheet which already exists.
2009-05-03 13:25:33 -07:00
Kevin Reid
2f3842954b
Fix broken link from Provisioning to Reliability page.
2009-05-01 12:10:50 -07:00
Brian Warner
de83526acd
immutable/encode.py: tolerate immediate _remove_shareholder by copying the
...
landlord list before iterating over it. This can probably only happen in unit
tests, but cleaning it up makes certain test failures easier to analyze.
2009-05-22 11:44:24 -07:00
Brian Warner
1863aee0aa
switch to using RemoteException instead of 'wrapped' RemoteReferences. Should fix #653 , the rref-EQ problem
2009-05-21 17:46:32 -07:00
Brian Warner
c9803d5217
switch all foolscap imports to use foolscap.api or foolscap.logging
2009-05-21 17:38:23 -07:00
Brian Warner
b542d5aca6
test_runner.py: fix minor typo
2009-05-19 20:36:20 -07:00
Brian Warner
9f0bc04d32
test_introducer.py: add a test for the python2.4.0/2.4.1 bug in base64.b32decode
2009-05-18 20:41:01 -07:00
Brian Warner
79437baade
immutable WriteBucketProxy: use pipeline to speed up uploads by overlapping roundtrips, for #392
2009-05-18 16:44:22 -07:00
Brian Warner
e76c6b606f
util/pipeline.py: new utility class to manage size-limited work pipelines, for #392
2009-05-18 16:43:26 -07:00
Zooko O'Whielacronx
718537ab8c
tests: mark test_runner as coded in utf-8 instead of ascii
2009-05-07 15:31:51 -07:00
Zooko O'Whielacronx
04ff9a3f97
tests: raise timeout on test_runner.RunNode.test_introducer from 120s to 240s, since it hit the 120s time-out on François Lenny-armv5tel
2009-05-07 14:50:12 -07:00
Zooko O'Whielacronx
77b578928e
trivial: fix comment
2009-04-13 10:41:38 -07:00
Zooko O'Whielacronx
6350634ee3
tests: remove obsolete test (it tests functionality that is long gone) which occasionally ERRORs now that we have more metadata (since [20090411225205-92b7f-7adfb89cb4db4ac7d28427934dea3d2c108f6476])
2009-04-12 19:36:21 -07:00
Zooko O'Whielacronx
6b9899ade1
trivial: remove trailing whitespace and unused import
2009-04-11 19:17:42 -07:00
Zooko O'Whielacronx
9729753692
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API
...
Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628 .
2009-04-11 15:52:05 -07:00
Zooko O'Whielacronx
5b02896307
rollback [20090226150237-b2345-1e916a746a7f4627b050f02f0e442fae5caf69d4] for 1.4.0 release; #645
2009-04-11 11:19:06 -07:00
Brian Warner
54952e9bea
#622 : disable 'Repair' button on check-results page until we make it work correctly
2009-04-09 16:59:59 -07:00
Brian Warner
922b3034be
web: make sure that PUT /uri?mutable=false really means immutable, fixes #675
2009-04-07 19:13:40 -07:00
Zooko O'Whielacronx
37c6e77764
wui: edit some of the human-readable parts of the wui such as button labels
...
(The word "parent" suggests that you can go up a directory hierarchy -- perhaps that word is vestigial.)
2009-04-07 11:54:59 -07:00
Zooko O'Whielacronx
8b1cd154da
wui: fix bug in which empty directory is marked as "unreadable", add test, remove exclamation point
2009-04-07 11:28:34 -07:00
Brian Warner
40c78297e8
hashtree.py: another micro-optimization, expand on a comment
2009-04-03 21:35:27 -07:00
Zooko O'Whielacronx
de8475389b
hashtree: fix tests of depth_of
2009-04-03 16:56:14 -07:00
Zooko O'Whielacronx
3ac5f427ba
trivial: source code metadata
2009-04-03 16:33:15 -07:00
Zooko O'Whielacronx
a07e746d9c
trivial: whitespace
2009-04-03 16:33:02 -07:00
Zooko O'Whielacronx
d381bdc905
hashtree: a couple of tiny improvements
...
* calculate depth-first with math instead of traversing the actual tree
* don't mark a node with a red dot if you instead compare it with an extant hash value (tiny optimization)
* edit a comment about checking the root node
2009-04-03 16:30:37 -07:00
Zooko O'Whielacronx
b12a7f9ee8
leases, time_format: modify time stamping in lease description
...
* emit lease expiry date in ISO-8601'ish format as well as Brian's format
* rename iso_utc_time_to_localseconds() to iso_utc_time_to_seconds()
* add iso_utc_date()
* simplify the body of iso_utc_time_to_seconds()
2009-04-03 15:59:04 -07:00
Brian Warner
466014f66f
hashtree: fix O(N**2) behavior, to improve fatal alacrity problems in a 10GB file ( #670 ). Also improve docstring.
2009-03-31 13:21:27 -07:00
Brian Warner
bd93430c53
expirer: include crawler progress in the JSON status output
2009-03-24 13:51:37 -07:00
Brian Warner
b5d4972a7a
move GC docs out of proposed/, since it's all implemented now. Add reference to configuration.txt . Add expire.*= suggestions to tahoe.cfg .
2009-03-23 16:08:20 -07:00
Brian Warner
5e8c31c3b6
storage: use constant-time comparison for write-enablers and lease-secrets
2009-03-22 20:21:28 -07:00
Brian Warner
01e2032669
hashutil: add constant-time comparison function, to avoid timing attacks when python's short-circuiting data-dependent == operator is used to, say, check a write-enabler
2009-03-22 20:20:55 -07:00
Brian Warner
6599eae6f9
WUI: fix display of empty directories, it threw an exception before
2009-03-20 16:58:09 -07:00
Brian Warner
4f4d748fd9
storage webstatus: insert spaces when we're configured to expire multiple sharetypes
2009-03-20 15:44:50 -07:00
Brian Warner
3b65607926
expirer: tolerate empty buckets, refactor bucketsize-counting code a bit, don't increment -mutable/-immutable counters unless we actually know the sharetype
2009-03-20 12:18:16 -07:00
Brian Warner
8645738c77
storage: improve wording of status message
2009-03-19 11:48:37 -07:00
Brian Warner
66fe8ca24c
parse_date: insist that it returns an int
2009-03-19 11:44:43 -07:00
Brian Warner
186b6a8c01
storage status: report expiration-cutoff-date like 19-Mar-2009 (as opposed to the tahoe.cfg input format of 2009-03-19), for redundancy: someone who gets the month and day switched will have a better chance to spot the problem in the storage-status output if it's in a different format
2009-03-19 11:07:56 -07:00
Brian Warner
f0071c2571
expirer: clean up constructor args, add tahoe.cfg controls, use cutoff_date instead of date_cutoff
2009-03-18 18:00:09 -07:00
Brian Warner
babcf632da
util/time_format: new routine to parse dates like 2009-03-18, switch expirer to use it. I'd prefer to use 18-Mar-2009, but it is surprisingly non-trivial to build a parser that will take UTC dates instead of local dates
2009-03-18 17:58:14 -07:00
Brian Warner
8eaee28550
expirer: change setup, config options, in preparation for adding tahoe.cfg controls
2009-03-18 17:21:38 -07:00
Brian Warner
fffab0d724
expirer: track mutable-vs-immutable sharecounts and sizes, report them on the web status page for comparison
2009-03-18 13:25:04 -07:00
Brian Warner
406fdba61f
add utility function to parse durations, for lease-expiration config
2009-03-17 00:01:17 -07:00
Brian Warner
24ab5ec26f
expirer: add mode to expire only-mutable or only-immutable shares
2009-03-16 23:51:18 -07:00
Brian Warner
475e7b8804
tahoe_ls.py: add comment about error cases to improve
2009-03-16 22:12:06 -07:00
Brian Warner
c7254c5f1d
GC: add date-cutoff -based expiration, add proposed docs
2009-03-16 22:10:41 -07:00
Brian Warner
2b525a42d3
test_deepcheck: remove the 10s timeout: our dapper buildslave requires 30s, and the reduced timeout was only there because this tests fails by timeout rather than explicitly
2009-03-15 17:51:34 -07:00
Brian Warner
f0b0ad1c8f
tahoe cp -r: add --caps-only flag, to write filecaps into local files instead of actual file contents. Used only for debugging and as a quick tree-comparison tool.
2009-03-15 16:19:58 -07:00
Brian Warner
6e57576f2e
dirnode deep_traverse: insert a turn break (fireEventually) at least once every 100 files, otherwise a CHK followed by more than 158 LITs can overflow the stack, sort of like #237 .
2009-03-13 16:31:35 -07:00
Brian Warner
e59164e0bb
consolidate: remove pointless 'else' after for loop
2009-03-13 01:27:51 -07:00
Brian Warner
ae64ddf8e6
consolidate: add eta, flush stdout
2009-03-13 01:24:51 -07:00
Brian Warner
06bf443e91
consolidate: tolerate unicode dirnames
2009-03-12 23:54:02 -07:00
Brian Warner
1a741fdb03
dirnode.py: when doing deep-traverse, walk each directory in alphabetical order, to make things like 'manifest' more predictable
2009-03-12 23:50:46 -07:00
Brian Warner
a18f8d4cc7
consolidator: add progress to scan-old-directory passes
2009-03-12 22:47:28 -07:00
Brian Warner
ddb2f1a6e3
consolidator: fix cycle detection to not trigger on merely shared directories, add snapshot counter to progress
2009-03-12 21:22:29 -07:00
Brian Warner
ddc9a7ae74
consolidator: re-use more directories, add total directories seen-vs-used counts
2009-03-12 20:48:01 -07:00
Brian Warner
809ec25ffa
tahoe_backup.py: tolerate more time formats
2009-03-12 18:16:00 -07:00
Brian Warner
1c24707f19
consolidator: add more verbose traversal of directories
2009-03-12 16:29:00 -07:00
Brian Warner
760688a224
consolidate: create multiple numbered backups of the original Archives directory, not just the first time
2009-03-12 16:04:27 -07:00
Brian Warner
c4c6a62954
add 'tahoe debug consolidate' command, to merge directories created by repeated 'tahoe cp -r' or the allmydata win32 backup tool, into the form that would have been created by 'tahoe backup'.
2009-03-12 13:56:06 -07:00
Brian Warner
969b758bf5
tahoe_cp.py: return 0 for success, instead of None
2009-03-12 13:53:45 -07:00
Brian Warner
1ccd426a34
expirer: fix prediction math, thanks to Zandr for the catch
2009-03-09 13:42:17 -07:00
Brian Warner
8708045a98
storage.expirer: oops, fix upgrade-handler code
2009-03-08 20:55:16 -07:00
Brian Warner
df3f7f93e0
storage.expirer: handle upgrades better
2009-03-08 20:42:20 -07:00
Brian Warner
a68ad06254
storage.expirer: exercise the last missing line of webstatus code
2009-03-08 20:38:28 -07:00
Brian Warner
6d6049430c
expirer: tolerate corrupt shares, add them to the state and history for future examination
2009-03-08 20:08:40 -07:00
Brian Warner
1a98521c3d
storage/immutable: raise a specific error upon seeing a bad version number, instead of using assert. Also wrap to 80cols.
2009-03-08 20:07:32 -07:00
Brian Warner
02b40ec499
storage: add test coverage for BucketReader.__repr__
2009-03-08 20:05:27 -07:00
Brian Warner
6d7319c588
storage/mutable: raise a specific error upon seeing bad magic, instead of using assert
2009-03-08 19:02:01 -07:00
Zooko O'Whielacronx
7e7167f2e5
nodeadmin: node stops itself if a hotline file hasn't been touched in 120 seconds now, instead of in 60 seconds
...
A test failed on draco (MacPPC) because it took 67.1 seconds to get around to running the test, and the node had already stopped itself when the hotline file was 60 seconds old.
2009-03-07 20:50:39 -07:00
Brian Warner
5675b4e7e0
expirer: make web display a bit more consistent
2009-03-07 16:14:42 -07:00
Brian Warner
df045650e0
web/storage.py: tolerate unknown-future displays, I'm not sure why LeaseCrawler.test_unpredictable_future didn't catch this
2009-03-07 16:02:43 -07:00
Brian Warner
192c161be7
tahoe_cp.py: improve error reporting slightly: don't json-interpret HTTP errors, pass through tahoe webapi error messages
2009-03-07 05:40:51 -07:00
Brian Warner
2c2c3c2295
test_cli.py: assert that 'ls' on an unrecoverable file now gives a better error message
2009-03-07 05:08:15 -07:00
Brian Warner
67d7600781
test_web: test behavior of broken-dirnode GET, both html and json
2009-03-07 04:57:07 -07:00
Brian Warner
950200fece
web: when a dirnode can't be read, emit a regular HTML page but with the child-table and upload-forms replaced with an apologetic message. Make sure to include the 'get info' links so the user can do a filecheck
2009-03-07 04:56:01 -07:00
Brian Warner
badd79671c
web/common: split out exception-to-explanation+code mapping to a separate humanize_failure() function, so it can be used by other code. Add explanation for mutable UnrecoverableFileError.
2009-03-07 04:54:08 -07:00
Brian Warner
9c7fafd5af
test_cli: use explicit (and stable) testdirs, instead of using self.mktemp
2009-03-07 03:04:28 -07:00
Brian Warner
091cfd04ca
test_storage: solaris doesn't appear to give the same block count as other platforms, so don't assert as much about 'diskbytes' recovered
2009-03-07 02:45:18 -07:00
Brian Warner
24ebdb3954
tahoe_cp.py: don't be verbose unless --verbose is passed: default is one line of success
2009-03-07 02:44:21 -07:00
Brian Warner
0dee2a6036
storage: add a lease-checker-and-expirer crawler, plus web status page.
...
This walks slowly through all shares, examining their leases, deciding which
are still valid and which have expired. Once enabled, it will then remove the
expired leases, and delete shares which no longer have any valid leases. Note
that there is not yet a tahoe.cfg option to enable lease-deletion: the
current code is read-only. A subsequent patch will add a tahoe.cfg knob to
control this, as well as docs. Some other minor items included in this patch:
tahoe debug dump-share has a new --leases-only flag
storage sharefile/leaseinfo code is cleaned up
storage web status page (/storage) has more info, more tests coverage
space-left measurement on OS-X should be more accurate (it was off by 2048x)
(use stat .f_frsize instead of f_bsize)
2009-03-06 22:45:17 -07:00
Zooko O'Whielacronx
e9199a89ab
trivial: remove unused import detected by pyflakes, and remove trailing whitespace
2009-03-05 16:32:04 -07:00
Zooko O'Whielacronx
5e90d82a02
trivial: use more specific function for ascii-encoding storage index
2009-02-22 11:57:51 -07:00
Brian Warner
c5ecb7b3ba
test_cli: validate non-HTML error response of 'tahoe get' on an unrecoverable file
2009-03-03 22:11:46 -07:00
Brian Warner
f42e3bb107
web: full patch for HTML-vs-plaintext traceback renderings, improve test coverage of exception rendering
2009-03-03 21:56:30 -07:00
Brian Warner
90226f335f
web/common.py: use 'Accept:' header to control HTML-vs-text/plain traceback renderings
2009-03-03 21:54:57 -07:00
Brian Warner
53f50fcc78
test/common: rearrange shouldHTTPError
2009-03-03 19:40:59 -07:00
Brian Warner
c4bda3daa3
web: move plural() to common.py
2009-03-03 19:40:19 -07:00
Brian Warner
67571eb033
add more information to NotEnoughSharesError, split out new exceptions for no-servers and no-source-of-ueb-hash
2009-03-03 19:37:15 -07:00
Brian Warner
f15c0be5af
test_web: move shouldHTTPError into common.WebErrorMixin
2009-03-03 16:56:20 -07:00
Brian Warner
559c369574
test_web: clean up shouldHTTPError methods
2009-03-03 16:46:04 -07:00
Brian Warner
400c04c19a
immutable checker add-lease: catch remote IndexError here too
2009-02-27 01:17:24 -07:00
Brian Warner
99ebf6eaf5
servermap add-lease: fix the code that's supposed to catch remote IndexErrors, I forgot that they present as ServerFailures instead. This should stop the deluge of Incidents that occur when you do add-lease against 1.3.0 servers
2009-02-27 01:04:26 -07:00
Brian Warner
8c3013c4f7
rrefutil: add check_remote utility function
2009-02-27 00:59:57 -07:00
Brian Warner
1b3e635936
rrefutil: add trap_remote utility and friends
2009-02-27 00:55:24 -07:00
Brian Warner
8251572e01
web: improve layout of storage status with a table
2009-02-26 19:58:38 -07:00
Brian Warner
112dc35563
crawler: add ETA to get_progress()
2009-02-26 19:42:48 -07:00
Alberto Berti
3035dfb8ed
Fix for bug #645 , correct path handling logic so that it works from sshfs
2009-02-26 09:02:37 -07:00
Zooko O'Whielacronx
9ab4aa9016
wui: s/Provisioning/Reliability/ ; suggested by Terrell
2009-02-25 21:09:04 -07:00
Brian Warner
7c35f01073
startstop_node.py: when restarting, tolerate a stale pid too
2009-02-25 19:52:45 -07:00
Brian Warner
63b19e567c
web: fix the ERROR: line to work the same in python2.4 and 2.5
2009-02-25 01:46:21 -07:00
Brian Warner
7d4901197a
test_cli/test_web: fix spurious test failure on solaris (maybe python2.4?) due to variations in the way that exceptions are stringified
2009-02-25 00:01:28 -07:00
Brian Warner
a3c1fe35d9
CLI: modify 'tahoe manifest' and 'tahoe deep-check' to report ERROR: properly. For #590 .
2009-02-24 23:44:15 -07:00
Brian Warner
fd4ceb6a87
webapi: modify streaming deep-manifest/deep-checker to emit an ERROR: line if they encounter an unrecoverable+untraversable directory. For #590 .
2009-02-24 23:13:35 -07:00
Brian Warner
7f8708cb7c
scripts/common: fix alias handling on windows again, emit slightly nicer error message in response to an unknown alias
2009-02-24 22:21:36 -07:00
Brian Warner
fdb0200eca
#165 : make 'tahoe restart --force' the default behavior: warn but do not stop if restart is used on something that wasn't a running node, and always try to start it afterwards. This is particularly important for #315 (restart -m), because otherwise a single not-already-running node will prevent all nodes from being restarted, resulting in longer downtime than necessary
2009-02-24 21:42:13 -07:00
Brian Warner
51e3baeefc
test_deepcheck: switch deep-check tests to use no-network too. This cuts the runtime down by about 50%
2009-02-24 21:04:57 -07:00
Brian Warner
6cf4096156
test_deepcheck: convert MutableChecker to no-network GridTest
2009-02-24 20:00:10 -07:00
Brian Warner
2e7f64d392
tests/no_network: move GET into the GridTestMixin class
2009-02-24 18:33:00 -07:00
Brian Warner
6f981e4eab
common_web.py: oops, add .fields to the other FakeRequest
2009-02-24 18:04:59 -07:00
Brian Warner
376eab0c04
test_cli: exercise the recent tolerate-'c:\dir\file.txt' fix in scripts/common, recorded in a separate match to make it easier to merge the fix to prod
2009-02-24 17:56:20 -07:00
Brian Warner
a0aea06a9e
scripts/common: on windows, tolerate paths like 'c:\dir\file.txt', by treating single-letter aliases on windows/cygwin as non-aliases
2009-02-24 17:55:22 -07:00
Brian Warner
8d8250e44a
test/common_web.py: add a .fields attribute to our FakeRequest, since we support versions of Nevow that are old enough to not do it themselves
2009-02-24 17:20:50 -07:00
Alberto Berti
e1c925b373
Two small fixes on documentation for cli backup command.
2009-02-24 16:36:34 -07:00
Brian Warner
2810de32b1
test_web: add (disabled) test to see what happens when deep-check encounters an unrecoverable directory. We still need code changes to improve this behavior.
2009-02-24 15:40:17 -07:00
Alberto Berti
7e8958671b
Add elapsed timestamp to cli backup command final summary.
2009-02-24 11:14:25 -07:00
Brian Warner
7cfbb9c832
test_repairer: change to use faster no_network.GridTestMixin, split Verifier tests into separate cases, refactor judgement funcs into shared methods
2009-02-23 22:15:06 -07:00
Brian Warner
f95e9b5964
immutable/checker.py: trap ShareVersionIncompatible too. Also, use f.check
...
instead of examining the value returned by f.trap, because the latter appears
to squash exception types down into their base classes (i.e. since
ShareVersionIncompatible is a subclass of LayoutInvalid,
f.trap(Failure(ShareVersionIncompatible)) == LayoutInvalid).
All this resulted in 'incompatible' shares being misclassified as 'corrupt'.
2009-02-23 22:14:05 -07:00
Brian Warner
9af9d8ae35
immutable/layout.py: wrap to 80 cols, no functional changes
2009-02-23 18:58:37 -07:00
Brian Warner
2be729b1e4
test_repairer: change Repairer to use much-faster no_network.GridTestMixin. As a side-effect, fix what I think was a bug: some of the assert-minimal-effort-expended checks were mixing write counts and allocate counts
2009-02-23 17:42:27 -07:00
Brian Warner
cc3c1ae8df
test/no_network.py: add a basic stats provider
2009-02-23 17:39:37 -07:00
Brian Warner
f35c659b2f
tests: stop using setUpClass/tearDownClass, since they've been deprecated in Twisted-8.2.0
2009-02-23 14:43:12 -07:00
Brian Warner
bc91689f8e
test_checker: improve test coverage for checker results
2009-02-23 14:19:43 -07:00
Alberto Berti
d9649052d4
Fixed tests again so they will pass on windows.
2009-02-22 18:35:02 -07:00
Brian Warner
15ee186499
scripts: stop using RuntimeError, for #639
2009-02-22 17:31:06 -07:00
Brian Warner
c23d051d3f
mutable/publish: stop using RuntimeError, for #639
2009-02-22 17:30:56 -07:00
Brian Warner
c4f8005867
remove more RuntimeError from unit tests, for #639
2009-02-22 17:28:55 -07:00
Brian Warner
5c3d7d8e73
stop using RuntimeError in unit tests, for #639
2009-02-22 17:27:22 -07:00
Brian Warner
d95b01a214
ftpd/sftpd: stop using RuntimeError, for #639
2009-02-22 17:24:26 -07:00
Alberto Berti
b945c275c2
Added tests for the cse when listdir is an iterator
2009-02-22 16:43:56 -07:00
Alberto Berti
cd0bd97954
Fixed tests so that they pass also on buildbots.
2009-02-22 16:43:11 -07:00
Alberto Berti
b2f8fe04c6
Use failUnlessEqual instead of failUnless(a == b)
2009-02-22 16:42:14 -07:00
Alberto Berti
89a20fe4dd
Better implementation of filtering algorithm.
2009-02-22 16:40:49 -07:00
Alberto Berti
2fbca1596b
Removed '.hgrags' from vcs excludes
2009-02-22 16:39:46 -07:00
Alberto Berti
5d629d0050
Added tests for the --exclude* options of backup command.
2009-02-22 10:51:06 -07:00
Alberto Berti
5d57da93fd
Added --exclude, --exclude-from and --exclude-vcs options to backup command.
...
It is still impossible to specify absolute exclusion path, only
relative. I must check with tar or rsync how they allow them to be
specified.
2009-02-22 11:08:29 -07:00
Alberto Berti
c54fdd61b0
Raise a more explanatory exception for errors encountered during backup processing.
2009-02-22 11:02:52 -07:00
Alberto Berti
ff6a77cc6b
Added tests for the fixed alias related command's synopsis
2009-02-22 10:37:32 -07:00
Alberto Berti
ffab4002aa
Add missing synopsis and descriptions for alias commands.
2009-02-20 18:31:06 -07:00
Brian Warner
c85eccbf64
test_web.py: increase test coverage of web.status.plural()
2009-02-21 18:01:16 -07:00
Brian Warner
77f3b83d68
crawler: fix performance problems: only save state once per timeslice (not after every bucket), don't start the crawler until 5 minutes after node startup
2009-02-21 14:56:49 -07:00
Brian Warner
a04d3b8fe8
test_system: oops, don't assume that all files in storage/ are in a deep storage/shares/prefix/si/shnum path, since now the crawler pickle has a short path
2009-02-21 00:17:10 -07:00
Brian Warner
b9c4f4bdf6
crawler: tolerate low-resolution system clocks (i.e. windows)
2009-02-21 00:15:33 -07:00
Brian Warner
106d31b112
BucketCountingCrawler: store just the count, not cycle+count, since it's too easy to make usage mistakes otherwise
2009-02-20 21:58:31 -07:00
Brian Warner
3c6471c717
test_storage: startService the server, as is now the standard practice
2009-02-20 21:57:55 -07:00
Brian Warner
f934289d2d
crawler: load state from the pickle in init, rather than waiting until startService, so get_state() can be called early
2009-02-20 21:57:20 -07:00
Brian Warner
1077826357
BucketCountingCrawler: rename status and state keys to use 'bucket' instead of 'share', because the former is more accurate
2009-02-20 21:46:06 -07:00
Brian Warner
d2d297f12f
storage: also report space-free-for-root and space-free-for-nonroot, since that helps users understand the space-left-for-tahoe number better
2009-02-20 21:28:56 -07:00
Brian Warner
b3cd4952bd
storage: add bucket-counting share crawler, add its output (number of files+directories maintained by a storage server) and status to the webapi /storage page
2009-02-20 21:04:08 -07:00
Brian Warner
d14f00c537
storage: move si_b2a/si_a2b/storage_index_to_dir out of server.py and into common.py
2009-02-20 21:03:09 -07:00
Brian Warner
73e05bf967
crawler: add get_progress, clean up get_state
2009-02-20 18:27:43 -07:00
Brian Warner
2e45619844
web/storage: make sure we can handle platforms without os.statvfs too
2009-02-20 16:03:53 -07:00
Brian Warner
c6a061e600
crawler: provide for one-shot crawlers, which stop after their first full cycle, for share-upgraders and database-populaters
2009-02-20 15:19:11 -07:00
Brian Warner
00677ff9a5
web: add Storage status page, improve tests
2009-02-20 14:29:26 -07:00
Brian Warner
ff6907a557
storage: include reserved_space in stats
2009-02-20 14:29:20 -07:00
Brian Warner
4b8cc42f18
web/check_results: sort share identifiers in the sharemap display
2009-02-20 12:29:22 -07:00
Brian Warner
c6c30ac5d4
webapi: pass client through constructor arguments, remove IClient, should make it easier to test web renderers in isolation
2009-02-20 12:15:54 -07:00
Brian Warner
e357b8a679
test/no_network: do startService on the storage servers, make it easier to customize the storage servers
2009-02-19 20:22:54 -07:00
Brian Warner
ef4ff21ae7
crawler: modify API to support upcoming bucket-counting crawler
2009-02-19 19:31:42 -07:00
Brian Warner
9bc08158c6
test_backupdb: make the not-a-database file larger, since the older sqlite-2.3.2 on OS-X is easily fooled
2009-02-19 18:04:09 -07:00
Brian Warner
26fbe9df95
web/reliability: add parameter descriptions, adapted from a patch from Terrell Russell.
2009-02-19 16:29:18 -07:00
Brian Warner
2f751332e8
test_crawler: hush pyflakes
2009-02-19 14:23:40 -07:00
Brian Warner
ff9811211d
test_crawler: disable the percentage-of-cpu-used test, since it is too unreliable on our slow buildslaves. But leave the code in place for developers to run by hand.
2009-02-19 14:16:54 -07:00
Brian Warner
790a10d1b2
reliability.py: fix the numpy conversion, it was completely broken. Thanks to Terrell Russell for the help.
2009-02-19 13:55:15 -07:00
Brian Warner
268cd538e2
reliability: switch to NumPy, since Numeric is deprecated
2009-02-19 01:44:35 -07:00
Brian Warner
f10aa59e5b
test_crawler: don't require >=1 cycle on cygwin
2009-02-19 00:58:18 -07:00
Brian Warner
dfd72c6483
crawler: use fileutil.move_info_place in preference to our own version
2009-02-18 23:13:42 -07:00
Brian Warner
89f041ac83
fileutil: add move_into_place(), to perform the standard unix trick of atomically replacing a file, with a fallback for windows
2009-02-18 23:13:10 -07:00
Brian Warner
b949ea4f32
crawler: fix problems on windows and our slow cygwin slave
2009-02-18 22:24:31 -07:00
Brian Warner
193889f793
#633 : first version of a rate-limited interruptable share-crawler
2009-02-18 21:46:33 -07:00
Brian Warner
4646451de6
change StorageServer to take nodeid in the constructor, instead of assigning it later, since it's cleaner and because the original problem (Tubs not being ready until later) went away
2009-02-18 16:23:01 -07:00
Brian Warner
b398a69f14
test_system: split off checker tests to test_deepcheck.py, this file is too big
2009-02-18 15:42:34 -07:00
Brian Warner
ef53da2b12
break storage.py into smaller pieces in storage/*.py . No behavioral changes.
2009-02-18 14:46:55 -07:00
Brian Warner
a0c5f92cbd
immutable/layout: minor change to repr name
2009-02-18 14:46:48 -07:00
Brian Warner
2346d8621d
interfaces.py: allow add/renew/cancel-lease to return Any, so that 1.3.1 clients (the first to use these calls) can tolerate future storage servers which might return something other than None
2009-02-18 13:29:03 -07:00
Brian Warner
bce4a5385b
add --add-lease to 'tahoe check', 'tahoe deep-check', and webapi.
2009-02-17 19:32:43 -07:00
Brian Warner
e9563ebc02
change RIStorageServer.remote_add_lease to exit silently in case of no-such-bucket, instead of raising IndexError, because that makes the upcoming --add-lease feature faster and less noisy
2009-02-17 19:30:53 -07:00
Brian Warner
fde2289e7b
CLI #590 : convert 'tahoe deep-check' to streaming form, improve display, add tests
2009-02-17 17:15:11 -07:00
Brian Warner
0e78b2587c
interfaces.py: document behavior of add_lease/renew_lease/cancel_lease, before I change it
2009-02-17 13:48:09 -07:00
Brian Warner
d52eadfc58
test_backupdb: improve error messages if the test fails
2009-02-17 11:08:38 -07:00
Brian Warner
476a5c8fac
webapi #590 : add streaming deep-check. Still need a CLI tool to use it.
2009-02-16 23:35:53 -07:00
Brian Warner
8579e25059
test_web.Grid: change the CHECK() function to make it easier to test t= values with hyphens in them
2009-02-16 23:00:34 -07:00
Brian Warner
522641bb32
test_web: improve checker-results coverage with a no-network -based test, enhance no-network harness to assist, fix some bugs in web/check_results.py that were exposed
2009-02-16 22:12:42 -07:00
Brian Warner
059f24adca
web: fix handling of reliability page when Numeric is not available
2009-02-16 19:56:58 -07:00
Brian Warner
f2f8ede11f
test/no_network: update comments with setup timing: no_network takes 50ms, SystemTestMixin takes 2s (on my laptop)
2009-02-16 18:06:43 -07:00
Brian Warner
040cb39613
test_upload: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary
2009-02-16 17:44:57 -07:00
Brian Warner
911abcc34b
test_download: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary
2009-02-16 17:36:58 -07:00
Brian Warner
e9427ea777
test_dirnode.py: convert Deleter to new no-network gridtest
2009-02-16 17:23:48 -07:00
Brian Warner
267d91f797
test_cli.py: modify to use the new 'no-network' gridtest instead of SystemTestMixin, which speeds it up from 73s to 43s on my system
2009-02-16 17:20:05 -07:00
Brian Warner
191fd8556a
tests: fix no_network framework to work with upload/download and checker
2009-02-16 17:19:47 -07:00
Brian Warner
0409bc5628
client.py: improve docstring
2009-02-16 17:15:32 -07:00
Brian Warner
770f976a85
test_cli: add test coverage for help strings
2009-02-16 15:08:33 -07:00
Brian Warner
ebe0f2ac08
test/no_network: new test harness, like system-test but doesn't use the network so it's faster
2009-02-16 14:58:44 -07:00
Brian Warner
8ff76c6269
interfaces.py: minor docstring edit
2009-02-16 14:58:16 -07:00
Brian Warner
df90dd8e73
test_cli: improve test coverage slightly
2009-02-15 21:04:51 -07:00
Brian Warner
3235b9630b
test_util: get almost full test coverage of dictutil, starting with the original pyutil tests as a base. The remaining three uncovered lines involve funny cases of ValueOrderedDict that I can't figure out how to get at
2009-02-15 20:32:10 -07:00
Brian Warner
cc27b218f5
provisioning/reliability: add tests, hush pyflakes, remove dead code, fix web links
2009-02-15 16:24:51 -07:00
Brian Warner
e1380b132b
util/statistics: add tests, fix mean_repair_cost
2009-02-15 16:23:26 -07:00
Brian Warner
812f23585d
test_repairer: hush pyflakes
2009-02-15 16:23:10 -07:00
Brian Warner
e2efd911a3
build a 'reliability' web page, with a simulation of file decay and repair over time
2009-02-13 17:42:34 -07:00
Shawn Willden
51ab76875b
More lossmodel work, on repair.
2009-01-15 20:56:48 -07:00
