ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2025-01-04 20:24:12 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
4,246 Commits 185 Branches 90 Tags 89 MiB
9741b9655f
Commit Graph

1297 Commits

Author SHA1 Message Date
Zooko O'Whielacronx
c3e9d36d97 tests: increase timeouts on some other tests that timed-out on Francois's arm box 2009-06-05 07:34:37 -07:00
Zooko O'Whielacronx
63c8f0c424 tests: bump up the timeout on a bunch of tests that took longer than the default timeout (120s) on François Lenny-armv5tel 2009-06-04 20:14:44 -07:00
Zooko O'Whielacronx
8c18ac38a9 backup: remove the --no-backupdb command, the handling of "can't import sqlite", and the related tests, and change an error message to more correctly indicate failure to load the database from disk rather than failure to import sqlite module 
Fixes #728.
 2009-06-04 10:31:31 -07:00
Brian Warner
35b3f7f426 more refactoring: move get_all_serverids() and get_nickname_for_serverid() from Client to storage_broker 2009-06-01 20:07:50 -07:00
Brian Warner
b1290633b8 more storage_broker refactoring: downloader gets a broker instead of a client, 
use Client.get_storage_broker() accessor instead of direct attribute access.
 2009-06-01 19:25:11 -07:00
Brian Warner
3a1196701f test_runner.py: remove test_client_no_noise: the issue in question is 
ticketed in http://divmod.org/trac/ticket/2830 and doesn't need a Tahoe-side
change, plus this test fails on win32 for unrelated reasons (and test_client
is the place to think about the win32 issue).
 2009-06-01 15:50:07 -07:00
Brian Warner
c516361fd2 start to factor server-connection-management into a distinct 'StorageServerFarmBroker' object, separate from the client and the introducer. This is the starting point for #467: static server selection 2009-06-01 14:06:04 -07:00
Kevin Reid
23441389a5 Modify markup of Tahoe web pages to be more amenable to styling; some minor changes of wording. 2009-05-26 16:25:45 -07:00
Kevin Reid
a71c914e50 Tweak wording in directory page: not-read-only is "modifiable", mention creating a directory _in this directory_. 2009-05-26 16:24:14 -07:00
Brian Warner
1863aee0aa switch to using RemoteException instead of 'wrapped' RemoteReferences. Should fix #653, the rref-EQ problem 2009-05-21 17:46:32 -07:00
Brian Warner
c9803d5217 switch all foolscap imports to use foolscap.api or foolscap.logging 2009-05-21 17:38:23 -07:00
Brian Warner
b542d5aca6 test_runner.py: fix minor typo 2009-05-19 20:36:20 -07:00
Brian Warner
9f0bc04d32 test_introducer.py: add a test for the python2.4.0/2.4.1 bug in base64.b32decode 2009-05-18 20:41:01 -07:00
Brian Warner
e76c6b606f util/pipeline.py: new utility class to manage size-limited work pipelines, for #392 2009-05-18 16:43:26 -07:00
Zooko O'Whielacronx
718537ab8c tests: mark test_runner as coded in utf-8 instead of ascii 2009-05-07 15:31:51 -07:00
Zooko O'Whielacronx
04ff9a3f97 tests: raise timeout on test_runner.RunNode.test_introducer from 120s to 240s, since it hit the 120s time-out on François Lenny-armv5tel 2009-05-07 14:50:12 -07:00
Zooko O'Whielacronx
6350634ee3 tests: remove obsolete test (it tests functionality that is long gone) which occasionally ERRORs now that we have more metadata (since [20090411225205-92b7f-7adfb89cb4db4ac7d28427934dea3d2c108f6476]) 2009-04-12 19:36:21 -07:00
Zooko O'Whielacronx
6b9899ade1 trivial: remove trailing whitespace and unused import 2009-04-11 19:17:42 -07:00
Zooko O'Whielacronx
9729753692 dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API 
Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values.  See ticket #628.
 2009-04-11 15:52:05 -07:00
Brian Warner
922b3034be web: make sure that PUT /uri?mutable=false really means immutable, fixes #675 2009-04-07 19:13:40 -07:00
Zooko O'Whielacronx
37c6e77764 wui: edit some of the human-readable parts of the wui such as button labels 
(The word "parent" suggests that you can go up a directory hierarchy -- perhaps that word is vestigial.)
 2009-04-07 11:54:59 -07:00
Zooko O'Whielacronx
8b1cd154da wui: fix bug in which empty directory is marked as "unreadable", add test, remove exclamation point 2009-04-07 11:28:34 -07:00
Zooko O'Whielacronx
de8475389b hashtree: fix tests of depth_of 2009-04-03 16:56:14 -07:00
Zooko O'Whielacronx
a07e746d9c trivial: whitespace 2009-04-03 16:33:02 -07:00
Zooko O'Whielacronx
b12a7f9ee8 leases, time_format: modify time stamping in lease description 
* emit lease expiry date in ISO-8601'ish format as well as Brian's format
 * rename iso_utc_time_to_localseconds() to iso_utc_time_to_seconds()
 * add iso_utc_date()
 * simplify the body of iso_utc_time_to_seconds()
 2009-04-03 15:59:04 -07:00
Brian Warner
466014f66f hashtree: fix O(N**2) behavior, to improve fatal alacrity problems in a 10GB file (#670). Also improve docstring. 2009-03-31 13:21:27 -07:00
Brian Warner
bd93430c53 expirer: include crawler progress in the JSON status output 2009-03-24 13:51:37 -07:00
Brian Warner
01e2032669 hashutil: add constant-time comparison function, to avoid timing attacks when python's short-circuiting data-dependent == operator is used to, say, check a write-enabler 2009-03-22 20:20:55 -07:00
Brian Warner
6599eae6f9 WUI: fix display of empty directories, it threw an exception before 2009-03-20 16:58:09 -07:00
Brian Warner
3b65607926 expirer: tolerate empty buckets, refactor bucketsize-counting code a bit, don't increment -mutable/-immutable counters unless we actually know the sharetype 2009-03-20 12:18:16 -07:00
Brian Warner
8645738c77 storage: improve wording of status message 2009-03-19 11:48:37 -07:00
Brian Warner
66fe8ca24c parse_date: insist that it returns an int 2009-03-19 11:44:43 -07:00
Brian Warner
186b6a8c01 storage status: report expiration-cutoff-date like 19-Mar-2009 (as opposed to the tahoe.cfg input format of 2009-03-19), for redundancy: someone who gets the month and day switched will have a better chance to spot the problem in the storage-status output if it's in a different format 2009-03-19 11:07:56 -07:00
Brian Warner
f0071c2571 expirer: clean up constructor args, add tahoe.cfg controls, use cutoff_date instead of date_cutoff 2009-03-18 18:00:09 -07:00
Brian Warner
babcf632da util/time_format: new routine to parse dates like 2009-03-18, switch expirer to use it. I'd prefer to use 18-Mar-2009, but it is surprisingly non-trivial to build a parser that will take UTC dates instead of local dates 2009-03-18 17:58:14 -07:00
Brian Warner
8eaee28550 expirer: change setup, config options, in preparation for adding tahoe.cfg controls 2009-03-18 17:21:38 -07:00
Brian Warner
fffab0d724 expirer: track mutable-vs-immutable sharecounts and sizes, report them on the web status page for comparison 2009-03-18 13:25:04 -07:00
Brian Warner
406fdba61f add utility function to parse durations, for lease-expiration config 2009-03-17 00:01:17 -07:00
Brian Warner
24ab5ec26f expirer: add mode to expire only-mutable or only-immutable shares 2009-03-16 23:51:18 -07:00
Brian Warner
c7254c5f1d GC: add date-cutoff -based expiration, add proposed docs 2009-03-16 22:10:41 -07:00
Brian Warner
2b525a42d3 test_deepcheck: remove the 10s timeout: our dapper buildslave requires 30s, and the reduced timeout was only there because this tests fails by timeout rather than explicitly 2009-03-15 17:51:34 -07:00
Brian Warner
f0b0ad1c8f tahoe cp -r: add --caps-only flag, to write filecaps into local files instead of actual file contents. Used only for debugging and as a quick tree-comparison tool. 2009-03-15 16:19:58 -07:00
Brian Warner
6e57576f2e dirnode deep_traverse: insert a turn break (fireEventually) at least once every 100 files, otherwise a CHK followed by more than 158 LITs can overflow the stack, sort of like #237. 2009-03-13 16:31:35 -07:00
Brian Warner
ddc9a7ae74 consolidator: re-use more directories, add total directories seen-vs-used counts 2009-03-12 20:48:01 -07:00
Brian Warner
1c24707f19 consolidator: add more verbose traversal of directories 2009-03-12 16:29:00 -07:00
Brian Warner
760688a224 consolidate: create multiple numbered backups of the original Archives directory, not just the first time 2009-03-12 16:04:27 -07:00
Brian Warner
c4c6a62954 add 'tahoe debug consolidate' command, to merge directories created by repeated 'tahoe cp -r' or the allmydata win32 backup tool, into the form that would have been created by 'tahoe backup'. 2009-03-12 13:56:06 -07:00
Brian Warner
1ccd426a34 expirer: fix prediction math, thanks to Zandr for the catch 2009-03-09 13:42:17 -07:00
Brian Warner
a68ad06254 storage.expirer: exercise the last missing line of webstatus code 2009-03-08 20:38:28 -07:00
Brian Warner
6d6049430c expirer: tolerate corrupt shares, add them to the state and history for future examination 2009-03-08 20:08:40 -07:00
Brian Warner
1a98521c3d storage/immutable: raise a specific error upon seeing a bad version number, instead of using assert. Also wrap to 80cols. 2009-03-08 20:07:32 -07:00
Brian Warner
02b40ec499 storage: add test coverage for BucketReader.__repr__ 2009-03-08 20:05:27 -07:00
Brian Warner
6d7319c588 storage/mutable: raise a specific error upon seeing bad magic, instead of using assert 2009-03-08 19:02:01 -07:00
Brian Warner
5675b4e7e0 expirer: make web display a bit more consistent 2009-03-07 16:14:42 -07:00
Brian Warner
2c2c3c2295 test_cli.py: assert that 'ls' on an unrecoverable file now gives a better error message 2009-03-07 05:08:15 -07:00
Brian Warner
67d7600781 test_web: test behavior of broken-dirnode GET, both html and json 2009-03-07 04:57:07 -07:00
Brian Warner
9c7fafd5af test_cli: use explicit (and stable) testdirs, instead of using self.mktemp 2009-03-07 03:04:28 -07:00
Brian Warner
091cfd04ca test_storage: solaris doesn't appear to give the same block count as other platforms, so don't assert as much about 'diskbytes' recovered 2009-03-07 02:45:18 -07:00
Brian Warner
0dee2a6036 storage: add a lease-checker-and-expirer crawler, plus web status page. 
This walks slowly through all shares, examining their leases, deciding which
are still valid and which have expired. Once enabled, it will then remove the
expired leases, and delete shares which no longer have any valid leases. Note
that there is not yet a tahoe.cfg option to enable lease-deletion: the
current code is read-only. A subsequent patch will add a tahoe.cfg knob to
control this, as well as docs. Some other minor items included in this patch:

 tahoe debug dump-share has a new --leases-only flag
 storage sharefile/leaseinfo code is cleaned up
 storage web status page (/storage) has more info, more tests coverage
 space-left measurement on OS-X should be more accurate (it was off by 2048x)
  (use stat .f_frsize instead of f_bsize)
 2009-03-06 22:45:17 -07:00
Brian Warner
c5ecb7b3ba test_cli: validate non-HTML error response of 'tahoe get' on an unrecoverable file 2009-03-03 22:11:46 -07:00
Brian Warner
f42e3bb107 web: full patch for HTML-vs-plaintext traceback renderings, improve test coverage of exception rendering 2009-03-03 21:56:30 -07:00
Brian Warner
53f50fcc78 test/common: rearrange shouldHTTPError 2009-03-03 19:40:59 -07:00
Brian Warner
67571eb033 add more information to NotEnoughSharesError, split out new exceptions for no-servers and no-source-of-ueb-hash 2009-03-03 19:37:15 -07:00
Brian Warner
f15c0be5af test_web: move shouldHTTPError into common.WebErrorMixin 2009-03-03 16:56:20 -07:00
Brian Warner
559c369574 test_web: clean up shouldHTTPError methods 2009-03-03 16:46:04 -07:00
Brian Warner
8c3013c4f7 rrefutil: add check_remote utility function 2009-02-27 00:59:57 -07:00
Brian Warner
1b3e635936 rrefutil: add trap_remote utility and friends 2009-02-27 00:55:24 -07:00
Brian Warner
8251572e01 web: improve layout of storage status with a table 2009-02-26 19:58:38 -07:00
Brian Warner
112dc35563 crawler: add ETA to get_progress() 2009-02-26 19:42:48 -07:00
Brian Warner
63b19e567c web: fix the ERROR: line to work the same in python2.4 and 2.5 2009-02-25 01:46:21 -07:00
Brian Warner
7d4901197a test_cli/test_web: fix spurious test failure on solaris (maybe python2.4?) due to variations in the way that exceptions are stringified 2009-02-25 00:01:28 -07:00
Brian Warner
a3c1fe35d9 CLI: modify 'tahoe manifest' and 'tahoe deep-check' to report ERROR: properly. For #590. 2009-02-24 23:44:15 -07:00
Brian Warner
fd4ceb6a87 webapi: modify streaming deep-manifest/deep-checker to emit an ERROR: line if they encounter an unrecoverable+untraversable directory. For #590. 2009-02-24 23:13:35 -07:00
Brian Warner
7f8708cb7c scripts/common: fix alias handling on windows again, emit slightly nicer error message in response to an unknown alias 2009-02-24 22:21:36 -07:00
Brian Warner
51e3baeefc test_deepcheck: switch deep-check tests to use no-network too. This cuts the runtime down by about 50% 2009-02-24 21:04:57 -07:00
Brian Warner
6cf4096156 test_deepcheck: convert MutableChecker to no-network GridTest 2009-02-24 20:00:10 -07:00
Brian Warner
2e7f64d392 tests/no_network: move GET into the GridTestMixin class 2009-02-24 18:33:00 -07:00
Brian Warner
6f981e4eab common_web.py: oops, add .fields to the other FakeRequest 2009-02-24 18:04:59 -07:00
Brian Warner
376eab0c04 test_cli: exercise the recent tolerate-'c:\dir\file.txt' fix in scripts/common, recorded in a separate match to make it easier to merge the fix to prod 2009-02-24 17:56:20 -07:00
Brian Warner
8d8250e44a test/common_web.py: add a .fields attribute to our FakeRequest, since we support versions of Nevow that are old enough to not do it themselves 2009-02-24 17:20:50 -07:00
Brian Warner
2810de32b1 test_web: add (disabled) test to see what happens when deep-check encounters an unrecoverable directory. We still need code changes to improve this behavior. 2009-02-24 15:40:17 -07:00
Brian Warner
7cfbb9c832 test_repairer: change to use faster no_network.GridTestMixin, split Verifier tests into separate cases, refactor judgement funcs into shared methods 2009-02-23 22:15:06 -07:00
Brian Warner
2be729b1e4 test_repairer: change Repairer to use much-faster no_network.GridTestMixin. As a side-effect, fix what I think was a bug: some of the assert-minimal-effort-expended checks were mixing write counts and allocate counts 2009-02-23 17:42:27 -07:00
Brian Warner
cc3c1ae8df test/no_network.py: add a basic stats provider 2009-02-23 17:39:37 -07:00
Brian Warner
f35c659b2f tests: stop using setUpClass/tearDownClass, since they've been deprecated in Twisted-8.2.0 2009-02-23 14:43:12 -07:00
Brian Warner
bc91689f8e test_checker: improve test coverage for checker results 2009-02-23 14:19:43 -07:00
Alberto Berti
d9649052d4 Fixed tests again so they will pass on windows. 2009-02-22 18:35:02 -07:00
Brian Warner
c4f8005867 remove more RuntimeError from unit tests, for #639 2009-02-22 17:28:55 -07:00
Brian Warner
5c3d7d8e73 stop using RuntimeError in unit tests, for #639 2009-02-22 17:27:22 -07:00
Alberto Berti
b945c275c2 Added tests for the cse when listdir is an iterator 2009-02-22 16:43:56 -07:00
Alberto Berti
cd0bd97954 Fixed tests so that they pass also on buildbots. 2009-02-22 16:43:11 -07:00
Alberto Berti
b2f8fe04c6 Use failUnlessEqual instead of failUnless(a == b) 2009-02-22 16:42:14 -07:00
Alberto Berti
5d629d0050 Added tests for the --exclude* options of backup command. 2009-02-22 10:51:06 -07:00
Alberto Berti
ff6a77cc6b Added tests for the fixed alias related command's synopsis 2009-02-22 10:37:32 -07:00
Brian Warner
c85eccbf64 test_web.py: increase test coverage of web.status.plural() 2009-02-21 18:01:16 -07:00
Brian Warner
77f3b83d68 crawler: fix performance problems: only save state once per timeslice (not after every bucket), don't start the crawler until 5 minutes after node startup 2009-02-21 14:56:49 -07:00
Brian Warner
a04d3b8fe8 test_system: oops, don't assume that all files in storage/ are in a deep storage/shares/prefix/si/shnum path, since now the crawler pickle has a short path 2009-02-21 00:17:10 -07:00
Brian Warner
3c6471c717 test_storage: startService the server, as is now the standard practice 2009-02-20 21:57:55 -07:00
Brian Warner
1077826357 BucketCountingCrawler: rename status and state keys to use 'bucket' instead of 'share', because the former is more accurate 2009-02-20 21:46:06 -07:00
Brian Warner
b3cd4952bd storage: add bucket-counting share crawler, add its output (number of files+directories maintained by a storage server) and status to the webapi /storage page 2009-02-20 21:04:08 -07:00
Brian Warner
73e05bf967 crawler: add get_progress, clean up get_state 2009-02-20 18:27:43 -07:00
Brian Warner
2e45619844 web/storage: make sure we can handle platforms without os.statvfs too 2009-02-20 16:03:53 -07:00
Brian Warner
c6a061e600 crawler: provide for one-shot crawlers, which stop after their first full cycle, for share-upgraders and database-populaters 2009-02-20 15:19:11 -07:00
Brian Warner
00677ff9a5 web: add Storage status page, improve tests 2009-02-20 14:29:26 -07:00
Brian Warner
c6c30ac5d4 webapi: pass client through constructor arguments, remove IClient, should make it easier to test web renderers in isolation 2009-02-20 12:15:54 -07:00
Brian Warner
e357b8a679 test/no_network: do startService on the storage servers, make it easier to customize the storage servers 2009-02-19 20:22:54 -07:00
Brian Warner
ef4ff21ae7 crawler: modify API to support upcoming bucket-counting crawler 2009-02-19 19:31:42 -07:00
Brian Warner
9bc08158c6 test_backupdb: make the not-a-database file larger, since the older sqlite-2.3.2 on OS-X is easily fooled 2009-02-19 18:04:09 -07:00
Brian Warner
2f751332e8 test_crawler: hush pyflakes 2009-02-19 14:23:40 -07:00
Brian Warner
ff9811211d test_crawler: disable the percentage-of-cpu-used test, since it is too unreliable on our slow buildslaves. But leave the code in place for developers to run by hand. 2009-02-19 14:16:54 -07:00
Brian Warner
790a10d1b2 reliability.py: fix the numpy conversion, it was completely broken. Thanks to Terrell Russell for the help. 2009-02-19 13:55:15 -07:00
Brian Warner
268cd538e2 reliability: switch to NumPy, since Numeric is deprecated 2009-02-19 01:44:35 -07:00
Brian Warner
f10aa59e5b test_crawler: don't require >=1 cycle on cygwin 2009-02-19 00:58:18 -07:00
Brian Warner
b949ea4f32 crawler: fix problems on windows and our slow cygwin slave 2009-02-18 22:24:31 -07:00
Brian Warner
193889f793 #633: first version of a rate-limited interruptable share-crawler 2009-02-18 21:46:33 -07:00
Brian Warner
4646451de6 change StorageServer to take nodeid in the constructor, instead of assigning it later, since it's cleaner and because the original problem (Tubs not being ready until later) went away 2009-02-18 16:23:01 -07:00
Brian Warner
b398a69f14 test_system: split off checker tests to test_deepcheck.py, this file is too big 2009-02-18 15:42:34 -07:00
Brian Warner
ef53da2b12 break storage.py into smaller pieces in storage/*.py . No behavioral changes. 2009-02-18 14:46:55 -07:00
Brian Warner
bce4a5385b add --add-lease to 'tahoe check', 'tahoe deep-check', and webapi. 2009-02-17 19:32:43 -07:00
Brian Warner
e9563ebc02 change RIStorageServer.remote_add_lease to exit silently in case of no-such-bucket, instead of raising IndexError, because that makes the upcoming --add-lease feature faster and less noisy 2009-02-17 19:30:53 -07:00
Brian Warner
fde2289e7b CLI #590: convert 'tahoe deep-check' to streaming form, improve display, add tests 2009-02-17 17:15:11 -07:00
Brian Warner
d52eadfc58 test_backupdb: improve error messages if the test fails 2009-02-17 11:08:38 -07:00
Brian Warner
476a5c8fac webapi #590: add streaming deep-check. Still need a CLI tool to use it. 2009-02-16 23:35:53 -07:00
Brian Warner
8579e25059 test_web.Grid: change the CHECK() function to make it easier to test t= values with hyphens in them 2009-02-16 23:00:34 -07:00
Brian Warner
522641bb32 test_web: improve checker-results coverage with a no-network -based test, enhance no-network harness to assist, fix some bugs in web/check_results.py that were exposed 2009-02-16 22:12:42 -07:00
Brian Warner
f2f8ede11f test/no_network: update comments with setup timing: no_network takes 50ms, SystemTestMixin takes 2s (on my laptop) 2009-02-16 18:06:43 -07:00
Brian Warner
040cb39613 test_upload: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary 2009-02-16 17:44:57 -07:00
Brian Warner
911abcc34b test_download: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary 2009-02-16 17:36:58 -07:00
Brian Warner
e9427ea777 test_dirnode.py: convert Deleter to new no-network gridtest 2009-02-16 17:23:48 -07:00
Brian Warner
267d91f797 test_cli.py: modify to use the new 'no-network' gridtest instead of SystemTestMixin, which speeds it up from 73s to 43s on my system 2009-02-16 17:20:05 -07:00
Brian Warner
191fd8556a tests: fix no_network framework to work with upload/download and checker 2009-02-16 17:19:47 -07:00
Brian Warner
770f976a85 test_cli: add test coverage for help strings 2009-02-16 15:08:33 -07:00
Brian Warner
ebe0f2ac08 test/no_network: new test harness, like system-test but doesn't use the network so it's faster 2009-02-16 14:58:44 -07:00
Brian Warner
df90dd8e73 test_cli: improve test coverage slightly 2009-02-15 21:04:51 -07:00
Brian Warner
3235b9630b test_util: get almost full test coverage of dictutil, starting with the original pyutil tests as a base. The remaining three uncovered lines involve funny cases of ValueOrderedDict that I can't figure out how to get at 2009-02-15 20:32:10 -07:00
Brian Warner
cc27b218f5 provisioning/reliability: add tests, hush pyflakes, remove dead code, fix web links 2009-02-15 16:24:51 -07:00
Brian Warner
e1380b132b util/statistics: add tests, fix mean_repair_cost 2009-02-15 16:23:26 -07:00
Brian Warner
812f23585d test_repairer: hush pyflakes 2009-02-15 16:23:10 -07:00
Shawn Willden
21832280da Statistics module 
Added a statistics module for calculating various facets of
share survival statistics.
 2009-01-13 20:12:35 -07:00
Brian Warner
60b06b8d32 test_dirnode: #625 run deep-check on a readonly dirnode too 2009-02-13 14:53:37 -07:00
Zooko O'Whielacronx
2d45d192d1 immutable repairer: add tests of how it handles short reads 2009-02-12 16:48:53 -07:00
Zooko O'Whielacronx
8303f90da0 nodeadmin: node stops itself if a hotline file hasn't been touched in 60 seconds now, instead of in 40 seconds 
A test failed on draco (MacPPC) because it took 49 seconds to get around to running the test, and the node had already stopped itself when the hotline file was 40 seconds old.
 2009-02-12 15:21:31 -07:00
Zooko O'Whielacronx
76d7cc4404 immutable repairer: errback any pending readers of DownUpConnectorwhen it runs out of bytes, and test that fact 2009-02-11 20:11:29 -07:00
Zooko O'Whielacronx
7eb260a9cf versioning: include an "appname" in the application version string in the versioning protocol, and make that appname be controlled by setup.py 
It is currently hardcoded in setup.py to be 'allmydata-tahoe'.  Ticket #556 is to make it configurable by a runtime command-line argument to setup.py: "--appname=foo", but I suddenly wondered if we really wanted that and at the same time realized that we don't need that for tahoe-1.3.0 release, so this patch just hardcodes it in setup.py.
setup.py inspects a file named 'src/allmydata/_appname.py' and assert that it contains the string "__appname__ = 'allmydata-tahoe'", and creates it if it isn't already present.  src/allmydata/__init__.py import _appname and reads __appname__ from it.  The rest of the Python code imports allmydata and inspects "allmydata.__appname__", although actually every use it uses "allmydata.__full_version__" instead, where "allmydata.__full_version__" is created in src/allmydata/__init__.py to be:

__full_version__ = __appname + '-' + str(__version__).

All the code that emits an "application version string" when describing what version of a protocol it supports (introducer server, storage server, upload helper), or when describing itself in general (introducer client), usese allmydata.__full_version__.

This fixes ticket #556 at least well enough for tahoe-1.3.0 release.
 2009-02-11 17:18:16 -07:00
Brian Warner
5f288842d5 test_repairer: disable repair-from-corruption tests until other things are improved well enough to make it useful 2009-02-11 15:01:59 -07:00
Zooko O'Whielacronx
079c1851a9 trivial: whitespace 2009-02-11 11:19:35 -07:00
Brian Warner
7d907f35aa storage: make add-lease work, change default ownernum=1 since 0 is reserved to mean 'no lease here' 2009-02-10 23:39:38 -07:00
Brian Warner
13a3ef5ec1 #620: storage: allow mutable shares to be deleted, with a writev where new_length=0 2009-02-10 23:37:56 -07:00
Brian Warner
6a5f28f47d test_cli.Backup: insert some stalls to make sure two successive backups get distinct timestamps, avoiding intermittent failures 2009-02-10 20:37:09 -07:00
Brian Warner
f7a263eb0b #619: make 'tahoe backup' complain and refuse to run if sqlite is unavailable and --no-backupdb is not passed 2009-02-10 18:49:10 -07:00
Brian Warner
3b1c8ceb86 tests: increase the default timeout for SystemTestMixin -based tests to 300 seconds, since our slower buildslaves sometimes take longer than the default 120s 2009-02-10 14:44:12 -07:00
Zooko O'Whielacronx
ef1bfdd2bf immutable: repairer: add a simple test to exercise the "leftover" code path, fix the bug (and rename the variable "leftover" to "extra") 2009-02-10 12:12:45 -07:00
Zooko O'Whielacronx
75e4e67ed7 immutable: tighten preconditions -- you can write empty strings or read zero bytes, and add the first simple unit test of DownUpConnector 2009-02-10 00:56:47 -07:00
Zooko O'Whielacronx
d436c6235d immutable: tests: the real WRITE_LEEWAY is 35 (it was a mistake to move it from 10 to 35 earlier -- I had seen a failure in which it took 35 times as many writes as I thought were optimal, but I misread and thought it took only 20 times as many) 2009-02-09 23:53:48 -07:00
Zooko O'Whielacronx
65c12b24b8 immutable: tests: sigh, raise, again the limit of how many extra writes you can do and still pass this test 
Obviously requiring the code under test to perform within some limit isn't very meaningful if we raise the limit whenever the test goes outside of it.
But I still don't want to remove the test code which measures how many writes (and, elsewhere, how many reads) a client does in order to fulfill these duties.
Let this number -- now 20 -- stand as an approximation of the inefficiency of our code divided by my mental model of how many operations are actually optimal for these duties.
 2009-02-09 20:09:31 -07:00
Zooko O'Whielacronx
5738d94ccd immutable: tests: assert that verifier gives a clean bill of health after corruption and repair (the previous patch mistakenly did this only after deletion and repair), and also test whether deleting seven other shares and then downloading works. Also count the number of shares stored in the local filesystem. 2009-02-09 20:08:41 -07:00
Zooko O'Whielacronx
bbf48f4a88 immutable: test: add a test after attempting to repair from corruption: does a full verify run give the file a clean bill of health? If not, the you haven't successfully repaired it. 
This will make the repairer tests more consistent -- less accidentally passing due to getting lucky.
 2009-02-09 19:01:49 -07:00
Zooko O'Whielacronx
4091f73f34 immutable: tests: put shares back to their pristine condition in between each test of corrupting-and-repairing them 
This is important, because if the repairer doesn't completely repair all kinds of corruption (as the current one doesn't), then the successive tests get messed up by assuming that the shares were uncorrupted when the test first set about to corrupt them.
 2009-02-09 18:29:56 -07:00
Brian Warner
b197c5bb8c test_runner: skip all spawnProcess-using tests on cygwin, since spawnProcess just hangs forever 2009-02-09 02:34:00 -07:00
Brian Warner
19902a0f16 test_runner.py: revert the CreateNode section to using runner() inline, rather than spawning a process, to get more precise coverage 2009-02-09 02:26:17 -07:00
Brian Warner
89683a3906 test_upload: add test of maximum-immutable-share-size, to complete the last item of #538 2009-02-08 19:41:27 -07:00
Brian Warner
d8e00c37b5 test_cli.Backup: capture stderr when sqlite is unavailable 2009-02-07 15:14:40 -07:00
Brian Warner
213ef3d64b test_repairer: wrap comments to 80cols, my laptop does not have a wide screen. No functional changes. 2009-02-07 14:06:26 -07:00
Brian Warner
50886ea494 test/common.py: in share-layout-reading code, use '>L' consistently, since '>l' doesn't specify the signedness and the windows tests appear to be failing with an endianness-like problem (version==0x01000000). Also use binary mode when editing sharefiles 2009-02-07 13:38:17 -07:00
Brian Warner
1fd8f023ab test_cli: increase timeout on test_backup, since our dapper buildslave is really slow 2009-02-06 02:17:53 -07:00
Brian Warner
9933aa0d49 test_backupdb.py: reset the check-timers after one step, otherwise a slow host can false-fail 2009-02-06 01:33:05 -07:00
Brian Warner
814fec5495 test_cli.backup: oops, fix test to work even when sqlite is unavailable 2009-02-05 22:10:42 -07:00
Brian Warner
ca32db5b39 #598: add cli+backupdb tests, improve user display, update docs, move docs out of proposed/ 2009-02-05 22:07:01 -07:00
Brian Warner
4be2e11d11 add sqlite-based backupdb, for #598 and others (including 'tahoe cp'). Not enabled yet. 2009-02-05 18:17:56 -07:00
Brian Warner
4e572a5f24 webapi: add verifycap (spelled 'verify_url') to the t=json output on files and directories. Closes #559. 2009-02-03 19:22:48 -07:00
Brian Warner
cfce8b5eab #598: first cut of 'tahoe backup' command: no backupdb, but yes shared-unchanged-directories and Archives/TIMESTAMP and Latest/ 2009-02-02 21:09:02 -07:00
Zooko O'Whielacronx
e08e8d3f41 storage: disable test_large_share again: my linux laptop has less than 4 GiB free 2009-01-30 22:16:49 -07:00
Zooko O'Whielacronx
f0682cf43e setup: more verbose assertion failure in test_runner 2009-01-29 10:49:06 -07:00
Zooko O'Whielacronx
204629be43 storage: enable the test of a share larger than 2 GiB; this will probably be too expensive on Mac OS X, but I think it won't be on all the other platforms ; I will probably set it to SkipTest if the sys.platform is Mac after seeing the results of this buildbot run 2009-01-28 16:33:12 -07:00
Zooko O'Whielacronx
8d7fe2adb3 trivial: remove unused imports noticed by pyflakes 2009-01-27 15:11:48 -07:00
Zooko O'Whielacronx
64d12a9402 setup: fix test_runner to invoke bin/tahoe.exe instead of bin/tahoe if on Windows (this is what happens when a user invokes bin/tahoe on Windows) 2009-01-27 14:37:17 -07:00
Zooko O'Whielacronx
1ae08acc8f setup: fix test_runner to assert that lines which are output to stderr must end with a punctuation mark (apparently re.search("x$", "x\r\n") does not match. :-() 2009-01-27 14:35:05 -07:00
Zooko O'Whielacronx
3cd4722c9b setup: fix test_runner.RunNode.test_baddir -- it was left incomplete and broken by a recent patch 2009-01-27 14:32:45 -07:00
Zooko O'Whielacronx
688bb7a453 setup: go ahead and check for noise in test_client_no_noise 2009-01-26 17:46:16 -07:00
Zooko O'Whielacronx
8d2a3e78ca setup: add a test for a warning message from importing nevow, marked as TODO, comment-out the assertion of no-noise inside other test_runner tests 2009-01-26 17:30:46 -07:00
Zooko O'Whielacronx
3e4450cf3f setup: find a "bin/tahoe" executable to test based on allmydata.__file__ instead of based on the CWD 
This means that the tests still work if you are executing them from a CWD other than the src dir -- *if* the "bin/tahoe" is found at os.path.dirname(os.path.dirname(os.path.dirname(allmydata.__file__))).
If no file is found at that location, then just skip the tests of executing the "tahoe" executable, because we don't want to accidentally run those tests against an executable from a different version of tahoe.
 2009-01-23 18:34:37 -07:00
Brian Warner
1de4110b35 #509: remove non-streaming 'tahoe manifest' CLI form 2009-01-23 17:00:02 -07:00
Brian Warner
6a04103cc2 #509 CLI: add 'tahoe manifest --stream' 2009-01-23 16:33:21 -07:00
Brian Warner
5a0e3f3a40 #509: test_system.py: add test for streamed-manifest 2009-01-23 16:32:47 -07:00
Brian Warner
9ade197c3b test_system: rearrange DeepCheckWebGood to make it easier to add CLI tests 2009-01-23 16:13:06 -07:00
Brian Warner
26260374e9 #590: add webish t=stream-manifest 2009-01-22 22:01:36 -07:00
Brian Warner
72adeccf2d dirnode: add get_repair_cap() 2009-01-22 21:44:49 -07:00
Brian Warner
d8b3505cf5 filenode: add get_repair_cap(), which uses the read-write filecap for immutable files, and the verifycap for immutable files 2009-01-22 21:38:36 -07:00
Zooko O'Whielacronx
b77c89ac60 setup: add test that the tests are testing the right source code 
This is a test of #145, and I think that now the code passes this test.
 2009-01-22 15:52:40 -07:00
Zooko O'Whielacronx
967102e1b9 trivial: removed unused imports noticed by pyflakes 2009-01-22 15:52:13 -07:00
Zooko O'Whielacronx
210da5cd9a setup: change test_runner to invoke "bin/tahoe" in a subprocess instead of executing runner.runner() 
This is necessary because loading allmydata code now depends on PYTHONPATH manipulation which is done in the "bin/tahoe" script.  Unfortunately it makes test_runner slower since it launches and waits for many subprocesses.
 2009-01-22 15:38:18 -07:00
Zooko O'Whielacronx
7d91dc5efe rollback the feature of making "ambient upload authority" configurable 
This reverses some, but not all, of the changes that were committed in the following set of patches.

rolling back:

Sun Jan 18 09:54:30 MST 2009  toby.murray
  * add 'web.ambient_upload_authority' as a paramater to tahoe.cfg

    M ./src/allmydata/client.py -1 +3
    M ./src/allmydata/test/common.py -7 +9
    A ./src/allmydata/test/test_ambient_upload_authority.py
    M ./src/allmydata/web/root.py +12
    M ./src/allmydata/webish.py -1 +4
Sun Jan 18 09:56:08 MST 2009  zooko@zooko.com
  * trivial: whitespace
  I ran emacs's "M-x whitespace-cleanup" on the files that Toby's recent patch had touched that had trailing whitespace on some lines.

    M ./src/allmydata/test/test_ambient_upload_authority.py -9 +8
    M ./src/allmydata/web/root.py -2 +1
    M ./src/allmydata/webish.py -2 +1
Mon Jan 19 14:16:19 MST 2009  zooko@zooko.com
  * trivial: remove unused import noticed by pyflakes

    M ./src/allmydata/test/test_ambient_upload_authority.py -1
Mon Jan 19 21:38:35 MST 2009  toby.murray
  * doc: describe web.ambient_upload_authority

    M ./docs/configuration.txt +14
    M ./docs/frontends/webapi.txt +11
Mon Jan 19 21:38:57 MST 2009  zooko@zooko.com
  * doc: add Toby Murray to the CREDITS

    M ./CREDITS +4
 2009-01-20 20:47:35 -07:00
Zooko O'Whielacronx
6bbbb375e1 setup: fix test_system to require tahoe under its package==distribution name "allmydata-tahoe" instead of its module name "allmydata" 2009-01-20 12:38:09 -07:00
Zooko O'Whielacronx
845b21a5ad trivial: remove unused import noticed by pyflakes 2009-01-19 15:16:19 -07:00
Zooko O'Whielacronx
4771e1c9d5 trivial: whitespace 
Ran "M-x whitespace-cleanup" on files that Toby's recent patch touched, even though they didn't have trailing whitespace.
 2009-01-18 10:58:15 -07:00
Zooko O'Whielacronx
ce7c448bc0 trivial: whitespace 
I ran emacs's "M-x whitespace-cleanup" on the files that Toby's recent patch had touched that had trailing whitespace on some lines.
 2009-01-18 10:56:08 -07:00
toby.murray
66f83c7356 add 'web.ambient_upload_authority' as a paramater to tahoe.cfg 2009-01-18 10:54:30 -07:00
Zooko O'Whielacronx
f9ee60a88e cli: tests: skip symlink test if there is no os.symlink 2009-01-14 18:10:10 -07:00
Brian Warner
2fe099a0b3 mutable: move recent operation history management code (MutableWatcher) into history.py, have History provide stats 2009-01-14 17:36:20 -07:00
Brian Warner
cc50e2f4aa upload: use WriteBucketProxy_v2 when uploading a large file (with shares larger than 4GiB). This finally closes #346. I think we can now handle immutable files up to 48EiB. 2009-01-12 20:14:42 -07:00