Brian Warner
699510c8f1
PollMixin: snoop trial's error observer to halt the test early if an error is seen. This turns a lot of timeouts into fast failures.
2009-06-22 19:07:31 -07:00
Brian Warner
711c09bc5d
clean up storage_broker interface: should fix #732
2009-06-21 16:51:19 -07:00
Brian Warner
f14004eeb3
hush pyflakes with recent FileTooLarge removal
2009-06-21 16:17:57 -07:00
kevan
efcc45951d
change max filesize limit tests
...
Instead of testing to see that the previous SDMF filesize limit was being
obeyed, we now test to make sure that we can insert files larger than that
limit.
2009-06-20 14:28:22 -07:00
kevan
db939750a8
remove upper limit on SDMF filesize
2009-06-20 14:31:30 -07:00
Brian Warner
a6caae9b5d
immutable/download: instrument do-you-have-block responses to investigate #732
2009-06-20 21:12:09 -07:00
Brian Warner
2c5aa12151
test_util: add known-answer tests for hashutil tags
2009-06-17 21:57:09 -07:00
Zooko O'Whielacronx
6a93caf7a3
tests: bump up a timeout that expired on Zandr's box
2009-06-13 12:57:03 -07:00
Zooko O'Whielacronx
cc2953e663
util: hooray! A clean implementation of this simple utility! Black Dew pointed out that the inverse of time.gmtime() is hidden in the "calendar" module.
2009-06-13 09:01:12 -07:00
Zooko O'Whielacronx
45928315f6
util: Brian's horrible hack to figure out how much localtime and utctime differ. Now we'll see if it works on Windows.
2009-06-12 13:45:56 -07:00
Zooko O'Whielacronx
4a30c5899c
util: oops, time.tzset() doesn't work on Windows -- hopefully the new "London" unit test passes on Windows when we skip tzset() on platforms that don't have it
2009-06-11 17:09:20 -07:00
Zooko O'Whielacronx
8978cb0738
util: fix time_format.iso_utc_time_to_seconds() so that it works even in London
2009-06-11 15:11:29 -07:00
Zooko O'Whielacronx
0ed65fba34
tests: significantly increase timeouts that triggered on Zandr's ARM box
2009-06-10 09:10:43 -07:00
Zooko O'Whielacronx
69a3b23b34
test: multiple by 10 or so all timeouts that Zandr's ARM box just overran
2009-06-10 05:56:39 -07:00
Zooko O'Whielacronx
bdfb80834f
tests: bump up timeout on a test that timed out on draco
2009-06-09 21:46:28 -07:00
Zooko O'Whielacronx
974295f963
tests: raise the timeout for test_cli since Zandr's ARM machine totally burst through the old one
2009-06-09 14:05:09 -07:00
Brian Warner
2e35648901
test_cli.Backup: increase timeout massively, it takes 1200s on zandr's ARM linkstation
2009-06-08 22:28:01 -07:00
Zooko O'Whielacronx
3cfc00f5cf
tests: double the timeouts on some tests which time-out on Francois's box
2009-06-08 19:17:53 -07:00
Zooko O'Whielacronx
0bc116b3a2
tests: bump up timeouts so that the tests can finish before timeout on Francois's little arm box
2009-06-08 15:55:57 -07:00
Zooko O'Whielacronx
c3e9d36d97
tests: increase timeouts on some other tests that timed-out on Francois's arm box
2009-06-05 07:34:37 -07:00
Zooko O'Whielacronx
63c8f0c424
tests: bump up the timeout on a bunch of tests that took longer than the default timeout (120s) on François Lenny-armv5tel
2009-06-04 20:14:44 -07:00
Zooko O'Whielacronx
8c18ac38a9
backup: remove the --no-backupdb command, the handling of "can't import sqlite", and the related tests, and change an error message to more correctly indicate failure to load the database from disk rather than failure to import sqlite module
...
Fixes #728 .
2009-06-04 10:31:31 -07:00
Zooko O'Whielacronx
2841d6a266
setup: add pysqlite and sqlite to get_package_versions()
2009-06-04 08:37:28 -07:00
Brian Warner
35b3f7f426
more refactoring: move get_all_serverids() and get_nickname_for_serverid() from Client to storage_broker
2009-06-01 20:07:50 -07:00
Brian Warner
b1290633b8
more storage_broker refactoring: downloader gets a broker instead of a client,
...
use Client.get_storage_broker() accessor instead of direct attribute access.
2009-06-01 19:25:11 -07:00
Brian Warner
3a1196701f
test_runner.py: remove test_client_no_noise: the issue in question is
...
ticketed in http://divmod.org/trac/ticket/2830 and doesn't need a Tahoe-side
change, plus this test fails on win32 for unrelated reasons (and test_client
is the place to think about the win32 issue).
2009-06-01 15:50:07 -07:00
Brian Warner
4177a3616b
remove plaintext-hashing code from the helper interface, to close #722
...
and deny the Helper the ability to mount a partial-information-guessing
attack. This will probably break compatibility between new clients and very
old (pre-1.0) helpers.
2009-06-01 15:49:16 -07:00
Brian Warner
c516361fd2
start to factor server-connection-management into a distinct 'StorageServerFarmBroker' object, separate from the client and the introducer. This is the starting point for #467 : static server selection
2009-06-01 14:06:04 -07:00
Brian Warner
d29281c9c5
mutable: catch and display first error, so code bugs which break all servers get displayed better
2009-06-01 14:04:07 -07:00
Kevin Reid
23441389a5
Modify markup of Tahoe web pages to be more amenable to styling; some minor changes of wording.
2009-05-26 16:25:45 -07:00
Kevin Reid
a71c914e50
Tweak wording in directory page: not-read-only is "modifiable", mention creating a directory _in this directory_.
2009-05-26 16:24:14 -07:00
Kevin Reid
3e80676a29
Comment on duplication of code/markup found during styling project.
2009-05-03 13:34:42 -07:00
Kevin Reid
d0b4fd4689
Add CSS styles to spiff up the Tahoe WUI's appearance, particularly the welcome page and directories.
2009-05-03 13:31:42 -07:00
Kevin Reid
35d3671f4e
Link all Tahoe web pages to the /tahoe_css stylesheet which already exists.
2009-05-03 13:25:33 -07:00
Kevin Reid
2f3842954b
Fix broken link from Provisioning to Reliability page.
2009-05-01 12:10:50 -07:00
Brian Warner
de83526acd
immutable/encode.py: tolerate immediate _remove_shareholder by copying the
...
landlord list before iterating over it. This can probably only happen in unit
tests, but cleaning it up makes certain test failures easier to analyze.
2009-05-22 11:44:24 -07:00
Brian Warner
1863aee0aa
switch to using RemoteException instead of 'wrapped' RemoteReferences. Should fix #653 , the rref-EQ problem
2009-05-21 17:46:32 -07:00
Brian Warner
c9803d5217
switch all foolscap imports to use foolscap.api or foolscap.logging
2009-05-21 17:38:23 -07:00
Brian Warner
b542d5aca6
test_runner.py: fix minor typo
2009-05-19 20:36:20 -07:00
Brian Warner
9f0bc04d32
test_introducer.py: add a test for the python2.4.0/2.4.1 bug in base64.b32decode
2009-05-18 20:41:01 -07:00
Brian Warner
79437baade
immutable WriteBucketProxy: use pipeline to speed up uploads by overlapping roundtrips, for #392
2009-05-18 16:44:22 -07:00
Brian Warner
e76c6b606f
util/pipeline.py: new utility class to manage size-limited work pipelines, for #392
2009-05-18 16:43:26 -07:00
Zooko O'Whielacronx
718537ab8c
tests: mark test_runner as coded in utf-8 instead of ascii
2009-05-07 15:31:51 -07:00
Zooko O'Whielacronx
04ff9a3f97
tests: raise timeout on test_runner.RunNode.test_introducer from 120s to 240s, since it hit the 120s time-out on François Lenny-armv5tel
2009-05-07 14:50:12 -07:00
Zooko O'Whielacronx
77b578928e
trivial: fix comment
2009-04-13 10:41:38 -07:00
Zooko O'Whielacronx
6350634ee3
tests: remove obsolete test (it tests functionality that is long gone) which occasionally ERRORs now that we have more metadata (since [20090411225205-92b7f-7adfb89cb4db4ac7d28427934dea3d2c108f6476])
2009-04-12 19:36:21 -07:00
Zooko O'Whielacronx
6b9899ade1
trivial: remove trailing whitespace and unused import
2009-04-11 19:17:42 -07:00
Zooko O'Whielacronx
9729753692
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API
...
Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628 .
2009-04-11 15:52:05 -07:00
Zooko O'Whielacronx
5b02896307
rollback [20090226150237-b2345-1e916a746a7f4627b050f02f0e442fae5caf69d4] for 1.4.0 release; #645
2009-04-11 11:19:06 -07:00
Brian Warner
54952e9bea
#622 : disable 'Repair' button on check-results page until we make it work correctly
2009-04-09 16:59:59 -07:00
Brian Warner
922b3034be
web: make sure that PUT /uri?mutable=false really means immutable, fixes #675
2009-04-07 19:13:40 -07:00
Zooko O'Whielacronx
37c6e77764
wui: edit some of the human-readable parts of the wui such as button labels
...
(The word "parent" suggests that you can go up a directory hierarchy -- perhaps that word is vestigial.)
2009-04-07 11:54:59 -07:00
Zooko O'Whielacronx
8b1cd154da
wui: fix bug in which empty directory is marked as "unreadable", add test, remove exclamation point
2009-04-07 11:28:34 -07:00
Brian Warner
40c78297e8
hashtree.py: another micro-optimization, expand on a comment
2009-04-03 21:35:27 -07:00
Zooko O'Whielacronx
de8475389b
hashtree: fix tests of depth_of
2009-04-03 16:56:14 -07:00
Zooko O'Whielacronx
3ac5f427ba
trivial: source code metadata
2009-04-03 16:33:15 -07:00
Zooko O'Whielacronx
a07e746d9c
trivial: whitespace
2009-04-03 16:33:02 -07:00
Zooko O'Whielacronx
d381bdc905
hashtree: a couple of tiny improvements
...
* calculate depth-first with math instead of traversing the actual tree
* don't mark a node with a red dot if you instead compare it with an extant hash value (tiny optimization)
* edit a comment about checking the root node
2009-04-03 16:30:37 -07:00
Zooko O'Whielacronx
b12a7f9ee8
leases, time_format: modify time stamping in lease description
...
* emit lease expiry date in ISO-8601'ish format as well as Brian's format
* rename iso_utc_time_to_localseconds() to iso_utc_time_to_seconds()
* add iso_utc_date()
* simplify the body of iso_utc_time_to_seconds()
2009-04-03 15:59:04 -07:00
Brian Warner
466014f66f
hashtree: fix O(N**2) behavior, to improve fatal alacrity problems in a 10GB file ( #670 ). Also improve docstring.
2009-03-31 13:21:27 -07:00
Brian Warner
bd93430c53
expirer: include crawler progress in the JSON status output
2009-03-24 13:51:37 -07:00
Brian Warner
b5d4972a7a
move GC docs out of proposed/, since it's all implemented now. Add reference to configuration.txt . Add expire.*= suggestions to tahoe.cfg .
2009-03-23 16:08:20 -07:00
Brian Warner
5e8c31c3b6
storage: use constant-time comparison for write-enablers and lease-secrets
2009-03-22 20:21:28 -07:00
Brian Warner
01e2032669
hashutil: add constant-time comparison function, to avoid timing attacks when python's short-circuiting data-dependent == operator is used to, say, check a write-enabler
2009-03-22 20:20:55 -07:00
Brian Warner
6599eae6f9
WUI: fix display of empty directories, it threw an exception before
2009-03-20 16:58:09 -07:00
Brian Warner
4f4d748fd9
storage webstatus: insert spaces when we're configured to expire multiple sharetypes
2009-03-20 15:44:50 -07:00
Brian Warner
3b65607926
expirer: tolerate empty buckets, refactor bucketsize-counting code a bit, don't increment -mutable/-immutable counters unless we actually know the sharetype
2009-03-20 12:18:16 -07:00
Brian Warner
8645738c77
storage: improve wording of status message
2009-03-19 11:48:37 -07:00
Brian Warner
66fe8ca24c
parse_date: insist that it returns an int
2009-03-19 11:44:43 -07:00
Brian Warner
186b6a8c01
storage status: report expiration-cutoff-date like 19-Mar-2009 (as opposed to the tahoe.cfg input format of 2009-03-19), for redundancy: someone who gets the month and day switched will have a better chance to spot the problem in the storage-status output if it's in a different format
2009-03-19 11:07:56 -07:00
Brian Warner
f0071c2571
expirer: clean up constructor args, add tahoe.cfg controls, use cutoff_date instead of date_cutoff
2009-03-18 18:00:09 -07:00
Brian Warner
babcf632da
util/time_format: new routine to parse dates like 2009-03-18, switch expirer to use it. I'd prefer to use 18-Mar-2009, but it is surprisingly non-trivial to build a parser that will take UTC dates instead of local dates
2009-03-18 17:58:14 -07:00
Brian Warner
8eaee28550
expirer: change setup, config options, in preparation for adding tahoe.cfg controls
2009-03-18 17:21:38 -07:00
Brian Warner
fffab0d724
expirer: track mutable-vs-immutable sharecounts and sizes, report them on the web status page for comparison
2009-03-18 13:25:04 -07:00
Brian Warner
406fdba61f
add utility function to parse durations, for lease-expiration config
2009-03-17 00:01:17 -07:00
Brian Warner
24ab5ec26f
expirer: add mode to expire only-mutable or only-immutable shares
2009-03-16 23:51:18 -07:00
Brian Warner
475e7b8804
tahoe_ls.py: add comment about error cases to improve
2009-03-16 22:12:06 -07:00
Brian Warner
c7254c5f1d
GC: add date-cutoff -based expiration, add proposed docs
2009-03-16 22:10:41 -07:00
Brian Warner
2b525a42d3
test_deepcheck: remove the 10s timeout: our dapper buildslave requires 30s, and the reduced timeout was only there because this tests fails by timeout rather than explicitly
2009-03-15 17:51:34 -07:00
Brian Warner
f0b0ad1c8f
tahoe cp -r: add --caps-only flag, to write filecaps into local files instead of actual file contents. Used only for debugging and as a quick tree-comparison tool.
2009-03-15 16:19:58 -07:00
Brian Warner
6e57576f2e
dirnode deep_traverse: insert a turn break (fireEventually) at least once every 100 files, otherwise a CHK followed by more than 158 LITs can overflow the stack, sort of like #237 .
2009-03-13 16:31:35 -07:00
Brian Warner
e59164e0bb
consolidate: remove pointless 'else' after for loop
2009-03-13 01:27:51 -07:00
Brian Warner
ae64ddf8e6
consolidate: add eta, flush stdout
2009-03-13 01:24:51 -07:00
Brian Warner
06bf443e91
consolidate: tolerate unicode dirnames
2009-03-12 23:54:02 -07:00
Brian Warner
1a741fdb03
dirnode.py: when doing deep-traverse, walk each directory in alphabetical order, to make things like 'manifest' more predictable
2009-03-12 23:50:46 -07:00
Brian Warner
a18f8d4cc7
consolidator: add progress to scan-old-directory passes
2009-03-12 22:47:28 -07:00
Brian Warner
ddb2f1a6e3
consolidator: fix cycle detection to not trigger on merely shared directories, add snapshot counter to progress
2009-03-12 21:22:29 -07:00
Brian Warner
ddc9a7ae74
consolidator: re-use more directories, add total directories seen-vs-used counts
2009-03-12 20:48:01 -07:00
Brian Warner
809ec25ffa
tahoe_backup.py: tolerate more time formats
2009-03-12 18:16:00 -07:00
Brian Warner
1c24707f19
consolidator: add more verbose traversal of directories
2009-03-12 16:29:00 -07:00
Brian Warner
760688a224
consolidate: create multiple numbered backups of the original Archives directory, not just the first time
2009-03-12 16:04:27 -07:00
Brian Warner
c4c6a62954
add 'tahoe debug consolidate' command, to merge directories created by repeated 'tahoe cp -r' or the allmydata win32 backup tool, into the form that would have been created by 'tahoe backup'.
2009-03-12 13:56:06 -07:00
Brian Warner
969b758bf5
tahoe_cp.py: return 0 for success, instead of None
2009-03-12 13:53:45 -07:00
Brian Warner
1ccd426a34
expirer: fix prediction math, thanks to Zandr for the catch
2009-03-09 13:42:17 -07:00
Brian Warner
8708045a98
storage.expirer: oops, fix upgrade-handler code
2009-03-08 20:55:16 -07:00
Brian Warner
df3f7f93e0
storage.expirer: handle upgrades better
2009-03-08 20:42:20 -07:00
Brian Warner
a68ad06254
storage.expirer: exercise the last missing line of webstatus code
2009-03-08 20:38:28 -07:00
Brian Warner
6d6049430c
expirer: tolerate corrupt shares, add them to the state and history for future examination
2009-03-08 20:08:40 -07:00
Brian Warner
1a98521c3d
storage/immutable: raise a specific error upon seeing a bad version number, instead of using assert. Also wrap to 80cols.
2009-03-08 20:07:32 -07:00
Brian Warner
02b40ec499
storage: add test coverage for BucketReader.__repr__
2009-03-08 20:05:27 -07:00
