ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2025-01-04 20:24:12 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
4,107 Commits 185 Branches 90 Tags 89 MiB
0cf320c2ab
Commit Graph

1238 Commits

Author SHA1 Message Date
kevan
ca4de9ee97 Add unit tests for the Adder in dirnode.py 2009-07-18 12:50:49 -07:00
Brian Warner
531cc7899f rename NewDirectoryNode to DirectoryNode, NewDirectoryURI to DirectoryURI 2009-07-17 17:15:49 -05:00
Brian Warner
1192b61dfe upload: fix #758 recursion-loop in peer-selection when servers report errors. 
The bug was in the code that handles a third-or-later pass, and was
previously untested.
 2009-07-17 00:07:09 -05:00
Brian Warner
efa608a135 'tahoe stats': tolerate empty directories. Closes #693. 2009-07-15 00:51:09 -07:00
Brian Warner
d8ba8c2eb5 Allow tests to pass with -OO by turning some AssertionErrors (the ones that 
we actually exercise during tests) into more specific exceptions, so they
don't get optimized away. The best rule to follow is probably this: if an
exception is worth testing, then it's part of the API, and AssertionError
should never be part of the API. Closes #749.
 2009-07-14 23:45:10 -07:00
Zooko O'Whielacronx
10ad3ad7d6 wui: fix test for welcome page 2009-07-14 20:34:29 -07:00
kevan
44c652fbbe Modify bench_dirnode.py to use CachingDict. 2009-07-05 15:31:42 -07:00
kevan
325793457d Add tests for CachingDict, _pack_contents, _unpack_contents 2009-07-03 20:43:28 -07:00
Zooko O'Whielacronx
efafcfb91a directories: keep track of your position as you decode netstring after netstring from an input buffer instead of copying the trailing part 
This makes decoding linear in the number of netstrings instead of O(N^2).
 2009-07-04 19:51:09 -07:00
Zooko O'Whielacronx
4206a2c1c7 directories: in bench_dirnode.py, use a real CacheDirectoryManager instead of a fake one (because CacheDirectoryManager is a significant user of CPU and/or time) 2009-07-06 20:41:19 -07:00
Zooko O'Whielacronx
0df5b446eb directories: make the profiling behavior of bench_dirnode.py accessible by adding '--profile' to the cmdline 2009-07-06 20:30:35 -07:00
Zooko O'Whielacronx
e414c73877 directories: update the directory benchmarks to exercise the unpack-and-repack functionality, and add optional profiling 2009-07-05 09:29:53 -07:00
Zooko O'Whielacronx
859d7ba8b0 test/benchmark: benchmark the time to pack and unpack dirnodes 
See also the prof_benchmarks() function in this file which will run the benchmarks under profiling.
 2009-07-04 15:43:00 -07:00
Brian Warner
ef1b6ae8e3 Tolerate unknown URI types in directory structures. Part of #683. 
The idea is that future versions of Tahoe will add new URI types that this
version won't recognize, but might store them in directories that we *can*
read. We should handle these "objects from the future" as best we can.
Previous releases of Tahoe would just explode. With this change, we'll
continue to be able to work with everything else in the directory.

The code change is to wrap anything we don't recognize as an UnknownNode
instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how
to render these (mostly by leaving fields blank), deep-check knows to skip
over them, deep-stats counts them in "count-unknown". You can rename and
delete these things, but you can't add new ones (because we wouldn't know how
to generate a readcap to put into the dirnode's rocap slot, and because this
lets us catch typos better).
 2009-07-02 18:07:49 -07:00
Brian Warner
4a46e91192 test_client.py: minor refactoring of BASECONFIG usage 2009-07-02 17:54:50 -07:00
Brian Warner
6237aeabd7 create_node_from_uri: take both writecap+readcap, move logic out of dirnode.py 2009-07-02 15:25:37 -07:00
Brian Warner
b640baf500 cli: webopen: when called with no arguments, open the Welcome page 2009-07-01 13:05:48 -07:00
Brian Warner
e330abc3c3 mutable repairer: skip repair of readcaps instead of throwing an exception. 
This should improve the behavior of #625 a bit: at least all the files will
get repaired.
 2009-06-30 18:13:43 -07:00
Brian Warner
52fa421430 use 522-bit RSA keys in all unit tests (except one) 
This reduces the total test time on my laptop from 400s to 283s.
* src/allmydata/test/test_system.py (SystemTest.test_mutable._test_debug):
  Remove assertion about container_size/data_size, this changes with keysize
  and was too variable anyways.
* src/allmydata/mutable/filenode.py (MutableFileNode.create): add keysize=
* src/allmydata/dirnode.py (NewDirectoryNode.create): same
* src/allmydata/client.py (Client.DEFAULT_MUTABLE_KEYSIZE): add default,
  this overrides the one in MutableFileNode
 2009-06-29 15:31:24 -07:00
Brian Warner
c6ae255847 remove trailing whitespace 2009-06-29 13:03:58 -07:00
Brian Warner
2947da457e hush pyflakes 2009-06-24 19:18:09 -07:00
Brian Warner
bd6ecc9f44 Split out NoSharesError, stop adding attributes to NotEnoughSharesError, change humanize_failure to include the original exception string, update tests, behave better if humanize_failure fails. 2009-06-24 19:17:07 -07:00
Brian Warner
837733641f remove introducer/old.py, will create something similar when the RIIntroducer changes 2009-06-24 12:40:38 -07:00
Brian Warner
f573bb2929 check_grid.py: print stderr when a subcommand fails 2009-06-22 19:28:33 -07:00
Brian Warner
8df15e9f30 big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging 2009-06-22 19:10:47 -07:00
Brian Warner
699510c8f1 PollMixin: snoop trial's error observer to halt the test early if an error is seen. This turns a lot of timeouts into fast failures. 2009-06-22 19:07:31 -07:00
Brian Warner
711c09bc5d clean up storage_broker interface: should fix #732 2009-06-21 16:51:19 -07:00
Brian Warner
f14004eeb3 hush pyflakes with recent FileTooLarge removal 2009-06-21 16:17:57 -07:00
kevan
efcc45951d change max filesize limit tests 
Instead of testing to see that the previous SDMF filesize limit was being
obeyed, we now test to make sure that we can insert files larger than that
limit.
 2009-06-20 14:28:22 -07:00
Brian Warner
2c5aa12151 test_util: add known-answer tests for hashutil tags 2009-06-17 21:57:09 -07:00
Zooko O'Whielacronx
6a93caf7a3 tests: bump up a timeout that expired on Zandr's box 2009-06-13 12:57:03 -07:00
Zooko O'Whielacronx
45928315f6 util: Brian's horrible hack to figure out how much localtime and utctime differ. Now we'll see if it works on Windows. 2009-06-12 13:45:56 -07:00
Zooko O'Whielacronx
4a30c5899c util: oops, time.tzset() doesn't work on Windows -- hopefully the new "London" unit test passes on Windows when we skip tzset() on platforms that don't have it 2009-06-11 17:09:20 -07:00
Zooko O'Whielacronx
8978cb0738 util: fix time_format.iso_utc_time_to_seconds() so that it works even in London 2009-06-11 15:11:29 -07:00
Zooko O'Whielacronx
0ed65fba34 tests: significantly increase timeouts that triggered on Zandr's ARM box 2009-06-10 09:10:43 -07:00
Zooko O'Whielacronx
69a3b23b34 test: multiple by 10 or so all timeouts that Zandr's ARM box just overran 2009-06-10 05:56:39 -07:00
Zooko O'Whielacronx
bdfb80834f tests: bump up timeout on a test that timed out on draco 2009-06-09 21:46:28 -07:00
Zooko O'Whielacronx
974295f963 tests: raise the timeout for test_cli since Zandr's ARM machine totally burst through the old one 2009-06-09 14:05:09 -07:00
Brian Warner
2e35648901 test_cli.Backup: increase timeout massively, it takes 1200s on zandr's ARM linkstation 2009-06-08 22:28:01 -07:00
Zooko O'Whielacronx
3cfc00f5cf tests: double the timeouts on some tests which time-out on Francois's box 2009-06-08 19:17:53 -07:00
Zooko O'Whielacronx
0bc116b3a2 tests: bump up timeouts so that the tests can finish before timeout on Francois's little arm box 2009-06-08 15:55:57 -07:00
Zooko O'Whielacronx
c3e9d36d97 tests: increase timeouts on some other tests that timed-out on Francois's arm box 2009-06-05 07:34:37 -07:00
Zooko O'Whielacronx
63c8f0c424 tests: bump up the timeout on a bunch of tests that took longer than the default timeout (120s) on François Lenny-armv5tel 2009-06-04 20:14:44 -07:00
Zooko O'Whielacronx
8c18ac38a9 backup: remove the --no-backupdb command, the handling of "can't import sqlite", and the related tests, and change an error message to more correctly indicate failure to load the database from disk rather than failure to import sqlite module 
Fixes #728.
 2009-06-04 10:31:31 -07:00
Brian Warner
35b3f7f426 more refactoring: move get_all_serverids() and get_nickname_for_serverid() from Client to storage_broker 2009-06-01 20:07:50 -07:00
Brian Warner
b1290633b8 more storage_broker refactoring: downloader gets a broker instead of a client, 
use Client.get_storage_broker() accessor instead of direct attribute access.
 2009-06-01 19:25:11 -07:00
Brian Warner
3a1196701f test_runner.py: remove test_client_no_noise: the issue in question is 
ticketed in http://divmod.org/trac/ticket/2830 and doesn't need a Tahoe-side
change, plus this test fails on win32 for unrelated reasons (and test_client
is the place to think about the win32 issue).
 2009-06-01 15:50:07 -07:00
Brian Warner
c516361fd2 start to factor server-connection-management into a distinct 'StorageServerFarmBroker' object, separate from the client and the introducer. This is the starting point for #467: static server selection 2009-06-01 14:06:04 -07:00
Kevin Reid
23441389a5 Modify markup of Tahoe web pages to be more amenable to styling; some minor changes of wording. 2009-05-26 16:25:45 -07:00
Kevin Reid
a71c914e50 Tweak wording in directory page: not-read-only is "modifiable", mention creating a directory _in this directory_. 2009-05-26 16:24:14 -07:00
Brian Warner
1863aee0aa switch to using RemoteException instead of 'wrapped' RemoteReferences. Should fix #653, the rref-EQ problem 2009-05-21 17:46:32 -07:00
Brian Warner
c9803d5217 switch all foolscap imports to use foolscap.api or foolscap.logging 2009-05-21 17:38:23 -07:00
Brian Warner
b542d5aca6 test_runner.py: fix minor typo 2009-05-19 20:36:20 -07:00
Brian Warner
9f0bc04d32 test_introducer.py: add a test for the python2.4.0/2.4.1 bug in base64.b32decode 2009-05-18 20:41:01 -07:00
Brian Warner
e76c6b606f util/pipeline.py: new utility class to manage size-limited work pipelines, for #392 2009-05-18 16:43:26 -07:00
Zooko O'Whielacronx
718537ab8c tests: mark test_runner as coded in utf-8 instead of ascii 2009-05-07 15:31:51 -07:00
Zooko O'Whielacronx
04ff9a3f97 tests: raise timeout on test_runner.RunNode.test_introducer from 120s to 240s, since it hit the 120s time-out on François Lenny-armv5tel 2009-05-07 14:50:12 -07:00
Zooko O'Whielacronx
6350634ee3 tests: remove obsolete test (it tests functionality that is long gone) which occasionally ERRORs now that we have more metadata (since [20090411225205-92b7f-7adfb89cb4db4ac7d28427934dea3d2c108f6476]) 2009-04-12 19:36:21 -07:00
Zooko O'Whielacronx
6b9899ade1 trivial: remove trailing whitespace and unused import 2009-04-11 19:17:42 -07:00
Zooko O'Whielacronx
9729753692 dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API 
Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values.  See ticket #628.
 2009-04-11 15:52:05 -07:00
Brian Warner
922b3034be web: make sure that PUT /uri?mutable=false really means immutable, fixes #675 2009-04-07 19:13:40 -07:00
Zooko O'Whielacronx
37c6e77764 wui: edit some of the human-readable parts of the wui such as button labels 
(The word "parent" suggests that you can go up a directory hierarchy -- perhaps that word is vestigial.)
 2009-04-07 11:54:59 -07:00
Zooko O'Whielacronx
8b1cd154da wui: fix bug in which empty directory is marked as "unreadable", add test, remove exclamation point 2009-04-07 11:28:34 -07:00
Zooko O'Whielacronx
de8475389b hashtree: fix tests of depth_of 2009-04-03 16:56:14 -07:00
Zooko O'Whielacronx
a07e746d9c trivial: whitespace 2009-04-03 16:33:02 -07:00
Zooko O'Whielacronx
b12a7f9ee8 leases, time_format: modify time stamping in lease description 
* emit lease expiry date in ISO-8601'ish format as well as Brian's format
 * rename iso_utc_time_to_localseconds() to iso_utc_time_to_seconds()
 * add iso_utc_date()
 * simplify the body of iso_utc_time_to_seconds()
 2009-04-03 15:59:04 -07:00
Brian Warner
466014f66f hashtree: fix O(N**2) behavior, to improve fatal alacrity problems in a 10GB file (#670). Also improve docstring. 2009-03-31 13:21:27 -07:00
Brian Warner
bd93430c53 expirer: include crawler progress in the JSON status output 2009-03-24 13:51:37 -07:00
Brian Warner
01e2032669 hashutil: add constant-time comparison function, to avoid timing attacks when python's short-circuiting data-dependent == operator is used to, say, check a write-enabler 2009-03-22 20:20:55 -07:00
Brian Warner
6599eae6f9 WUI: fix display of empty directories, it threw an exception before 2009-03-20 16:58:09 -07:00
Brian Warner
3b65607926 expirer: tolerate empty buckets, refactor bucketsize-counting code a bit, don't increment -mutable/-immutable counters unless we actually know the sharetype 2009-03-20 12:18:16 -07:00
Brian Warner
8645738c77 storage: improve wording of status message 2009-03-19 11:48:37 -07:00
Brian Warner
66fe8ca24c parse_date: insist that it returns an int 2009-03-19 11:44:43 -07:00
Brian Warner
186b6a8c01 storage status: report expiration-cutoff-date like 19-Mar-2009 (as opposed to the tahoe.cfg input format of 2009-03-19), for redundancy: someone who gets the month and day switched will have a better chance to spot the problem in the storage-status output if it's in a different format 2009-03-19 11:07:56 -07:00
Brian Warner
f0071c2571 expirer: clean up constructor args, add tahoe.cfg controls, use cutoff_date instead of date_cutoff 2009-03-18 18:00:09 -07:00
Brian Warner
babcf632da util/time_format: new routine to parse dates like 2009-03-18, switch expirer to use it. I'd prefer to use 18-Mar-2009, but it is surprisingly non-trivial to build a parser that will take UTC dates instead of local dates 2009-03-18 17:58:14 -07:00
Brian Warner
8eaee28550 expirer: change setup, config options, in preparation for adding tahoe.cfg controls 2009-03-18 17:21:38 -07:00
Brian Warner
fffab0d724 expirer: track mutable-vs-immutable sharecounts and sizes, report them on the web status page for comparison 2009-03-18 13:25:04 -07:00
Brian Warner
406fdba61f add utility function to parse durations, for lease-expiration config 2009-03-17 00:01:17 -07:00
Brian Warner
24ab5ec26f expirer: add mode to expire only-mutable or only-immutable shares 2009-03-16 23:51:18 -07:00
Brian Warner
c7254c5f1d GC: add date-cutoff -based expiration, add proposed docs 2009-03-16 22:10:41 -07:00
Brian Warner
2b525a42d3 test_deepcheck: remove the 10s timeout: our dapper buildslave requires 30s, and the reduced timeout was only there because this tests fails by timeout rather than explicitly 2009-03-15 17:51:34 -07:00
Brian Warner
f0b0ad1c8f tahoe cp -r: add --caps-only flag, to write filecaps into local files instead of actual file contents. Used only for debugging and as a quick tree-comparison tool. 2009-03-15 16:19:58 -07:00
Brian Warner
6e57576f2e dirnode deep_traverse: insert a turn break (fireEventually) at least once every 100 files, otherwise a CHK followed by more than 158 LITs can overflow the stack, sort of like #237. 2009-03-13 16:31:35 -07:00
Brian Warner
ddc9a7ae74 consolidator: re-use more directories, add total directories seen-vs-used counts 2009-03-12 20:48:01 -07:00
Brian Warner
1c24707f19 consolidator: add more verbose traversal of directories 2009-03-12 16:29:00 -07:00
Brian Warner
760688a224 consolidate: create multiple numbered backups of the original Archives directory, not just the first time 2009-03-12 16:04:27 -07:00
Brian Warner
c4c6a62954 add 'tahoe debug consolidate' command, to merge directories created by repeated 'tahoe cp -r' or the allmydata win32 backup tool, into the form that would have been created by 'tahoe backup'. 2009-03-12 13:56:06 -07:00
Brian Warner
1ccd426a34 expirer: fix prediction math, thanks to Zandr for the catch 2009-03-09 13:42:17 -07:00
Brian Warner
a68ad06254 storage.expirer: exercise the last missing line of webstatus code 2009-03-08 20:38:28 -07:00
Brian Warner
6d6049430c expirer: tolerate corrupt shares, add them to the state and history for future examination 2009-03-08 20:08:40 -07:00
Brian Warner
1a98521c3d storage/immutable: raise a specific error upon seeing a bad version number, instead of using assert. Also wrap to 80cols. 2009-03-08 20:07:32 -07:00
Brian Warner
02b40ec499 storage: add test coverage for BucketReader.__repr__ 2009-03-08 20:05:27 -07:00
Brian Warner
6d7319c588 storage/mutable: raise a specific error upon seeing bad magic, instead of using assert 2009-03-08 19:02:01 -07:00
Brian Warner
5675b4e7e0 expirer: make web display a bit more consistent 2009-03-07 16:14:42 -07:00
Brian Warner
2c2c3c2295 test_cli.py: assert that 'ls' on an unrecoverable file now gives a better error message 2009-03-07 05:08:15 -07:00
Brian Warner
67d7600781 test_web: test behavior of broken-dirnode GET, both html and json 2009-03-07 04:57:07 -07:00
Brian Warner
9c7fafd5af test_cli: use explicit (and stable) testdirs, instead of using self.mktemp 2009-03-07 03:04:28 -07:00
Brian Warner
091cfd04ca test_storage: solaris doesn't appear to give the same block count as other platforms, so don't assert as much about 'diskbytes' recovered 2009-03-07 02:45:18 -07:00
Brian Warner
0dee2a6036 storage: add a lease-checker-and-expirer crawler, plus web status page. 
This walks slowly through all shares, examining their leases, deciding which
are still valid and which have expired. Once enabled, it will then remove the
expired leases, and delete shares which no longer have any valid leases. Note
that there is not yet a tahoe.cfg option to enable lease-deletion: the
current code is read-only. A subsequent patch will add a tahoe.cfg knob to
control this, as well as docs. Some other minor items included in this patch:

 tahoe debug dump-share has a new --leases-only flag
 storage sharefile/leaseinfo code is cleaned up
 storage web status page (/storage) has more info, more tests coverage
 space-left measurement on OS-X should be more accurate (it was off by 2048x)
  (use stat .f_frsize instead of f_bsize)
 2009-03-06 22:45:17 -07:00