Brian Warner
559c369574
test_web: clean up shouldHTTPError methods
2009-03-03 16:46:04 -07:00
Brian Warner
8c3013c4f7
rrefutil: add check_remote utility function
2009-02-27 00:59:57 -07:00
Brian Warner
1b3e635936
rrefutil: add trap_remote utility and friends
2009-02-27 00:55:24 -07:00
Brian Warner
8251572e01
web: improve layout of storage status with a table
2009-02-26 19:58:38 -07:00
Brian Warner
112dc35563
crawler: add ETA to get_progress()
2009-02-26 19:42:48 -07:00
Brian Warner
63b19e567c
web: fix the ERROR: line to work the same in python2.4 and 2.5
2009-02-25 01:46:21 -07:00
Brian Warner
7d4901197a
test_cli/test_web: fix spurious test failure on solaris (maybe python2.4?) due to variations in the way that exceptions are stringified
2009-02-25 00:01:28 -07:00
Brian Warner
a3c1fe35d9
CLI: modify 'tahoe manifest' and 'tahoe deep-check' to report ERROR: properly. For #590 .
2009-02-24 23:44:15 -07:00
Brian Warner
fd4ceb6a87
webapi: modify streaming deep-manifest/deep-checker to emit an ERROR: line if they encounter an unrecoverable+untraversable directory. For #590 .
2009-02-24 23:13:35 -07:00
Brian Warner
7f8708cb7c
scripts/common: fix alias handling on windows again, emit slightly nicer error message in response to an unknown alias
2009-02-24 22:21:36 -07:00
Brian Warner
51e3baeefc
test_deepcheck: switch deep-check tests to use no-network too. This cuts the runtime down by about 50%
2009-02-24 21:04:57 -07:00
Brian Warner
6cf4096156
test_deepcheck: convert MutableChecker to no-network GridTest
2009-02-24 20:00:10 -07:00
Brian Warner
2e7f64d392
tests/no_network: move GET into the GridTestMixin class
2009-02-24 18:33:00 -07:00
Brian Warner
6f981e4eab
common_web.py: oops, add .fields to the other FakeRequest
2009-02-24 18:04:59 -07:00
Brian Warner
376eab0c04
test_cli: exercise the recent tolerate-'c:\dir\file.txt' fix in scripts/common, recorded in a separate match to make it easier to merge the fix to prod
2009-02-24 17:56:20 -07:00
Brian Warner
8d8250e44a
test/common_web.py: add a .fields attribute to our FakeRequest, since we support versions of Nevow that are old enough to not do it themselves
2009-02-24 17:20:50 -07:00
Brian Warner
2810de32b1
test_web: add (disabled) test to see what happens when deep-check encounters an unrecoverable directory. We still need code changes to improve this behavior.
2009-02-24 15:40:17 -07:00
Brian Warner
7cfbb9c832
test_repairer: change to use faster no_network.GridTestMixin, split Verifier tests into separate cases, refactor judgement funcs into shared methods
2009-02-23 22:15:06 -07:00
Brian Warner
2be729b1e4
test_repairer: change Repairer to use much-faster no_network.GridTestMixin. As a side-effect, fix what I think was a bug: some of the assert-minimal-effort-expended checks were mixing write counts and allocate counts
2009-02-23 17:42:27 -07:00
Brian Warner
cc3c1ae8df
test/no_network.py: add a basic stats provider
2009-02-23 17:39:37 -07:00
Brian Warner
f35c659b2f
tests: stop using setUpClass/tearDownClass, since they've been deprecated in Twisted-8.2.0
2009-02-23 14:43:12 -07:00
Brian Warner
bc91689f8e
test_checker: improve test coverage for checker results
2009-02-23 14:19:43 -07:00
Alberto Berti
d9649052d4
Fixed tests again so they will pass on windows.
2009-02-22 18:35:02 -07:00
Brian Warner
c4f8005867
remove more RuntimeError from unit tests, for #639
2009-02-22 17:28:55 -07:00
Brian Warner
5c3d7d8e73
stop using RuntimeError in unit tests, for #639
2009-02-22 17:27:22 -07:00
Alberto Berti
b945c275c2
Added tests for the cse when listdir is an iterator
2009-02-22 16:43:56 -07:00
Alberto Berti
cd0bd97954
Fixed tests so that they pass also on buildbots.
2009-02-22 16:43:11 -07:00
Alberto Berti
b2f8fe04c6
Use failUnlessEqual instead of failUnless(a == b)
2009-02-22 16:42:14 -07:00
Alberto Berti
5d629d0050
Added tests for the --exclude* options of backup command.
2009-02-22 10:51:06 -07:00
Alberto Berti
ff6a77cc6b
Added tests for the fixed alias related command's synopsis
2009-02-22 10:37:32 -07:00
Brian Warner
c85eccbf64
test_web.py: increase test coverage of web.status.plural()
2009-02-21 18:01:16 -07:00
Brian Warner
77f3b83d68
crawler: fix performance problems: only save state once per timeslice (not after every bucket), don't start the crawler until 5 minutes after node startup
2009-02-21 14:56:49 -07:00
Brian Warner
a04d3b8fe8
test_system: oops, don't assume that all files in storage/ are in a deep storage/shares/prefix/si/shnum path, since now the crawler pickle has a short path
2009-02-21 00:17:10 -07:00
Brian Warner
3c6471c717
test_storage: startService the server, as is now the standard practice
2009-02-20 21:57:55 -07:00
Brian Warner
1077826357
BucketCountingCrawler: rename status and state keys to use 'bucket' instead of 'share', because the former is more accurate
2009-02-20 21:46:06 -07:00
Brian Warner
b3cd4952bd
storage: add bucket-counting share crawler, add its output (number of files+directories maintained by a storage server) and status to the webapi /storage page
2009-02-20 21:04:08 -07:00
Brian Warner
73e05bf967
crawler: add get_progress, clean up get_state
2009-02-20 18:27:43 -07:00
Brian Warner
2e45619844
web/storage: make sure we can handle platforms without os.statvfs too
2009-02-20 16:03:53 -07:00
Brian Warner
c6a061e600
crawler: provide for one-shot crawlers, which stop after their first full cycle, for share-upgraders and database-populaters
2009-02-20 15:19:11 -07:00
Brian Warner
00677ff9a5
web: add Storage status page, improve tests
2009-02-20 14:29:26 -07:00
Brian Warner
c6c30ac5d4
webapi: pass client through constructor arguments, remove IClient, should make it easier to test web renderers in isolation
2009-02-20 12:15:54 -07:00
Brian Warner
e357b8a679
test/no_network: do startService on the storage servers, make it easier to customize the storage servers
2009-02-19 20:22:54 -07:00
Brian Warner
ef4ff21ae7
crawler: modify API to support upcoming bucket-counting crawler
2009-02-19 19:31:42 -07:00
Brian Warner
9bc08158c6
test_backupdb: make the not-a-database file larger, since the older sqlite-2.3.2 on OS-X is easily fooled
2009-02-19 18:04:09 -07:00
Brian Warner
2f751332e8
test_crawler: hush pyflakes
2009-02-19 14:23:40 -07:00
Brian Warner
ff9811211d
test_crawler: disable the percentage-of-cpu-used test, since it is too unreliable on our slow buildslaves. But leave the code in place for developers to run by hand.
2009-02-19 14:16:54 -07:00
Brian Warner
790a10d1b2
reliability.py: fix the numpy conversion, it was completely broken. Thanks to Terrell Russell for the help.
2009-02-19 13:55:15 -07:00
Brian Warner
268cd538e2
reliability: switch to NumPy, since Numeric is deprecated
2009-02-19 01:44:35 -07:00
Brian Warner
f10aa59e5b
test_crawler: don't require >=1 cycle on cygwin
2009-02-19 00:58:18 -07:00
Brian Warner
b949ea4f32
crawler: fix problems on windows and our slow cygwin slave
2009-02-18 22:24:31 -07:00
Brian Warner
193889f793
#633 : first version of a rate-limited interruptable share-crawler
2009-02-18 21:46:33 -07:00
Brian Warner
4646451de6
change StorageServer to take nodeid in the constructor, instead of assigning it later, since it's cleaner and because the original problem (Tubs not being ready until later) went away
2009-02-18 16:23:01 -07:00
Brian Warner
b398a69f14
test_system: split off checker tests to test_deepcheck.py, this file is too big
2009-02-18 15:42:34 -07:00
Brian Warner
ef53da2b12
break storage.py into smaller pieces in storage/*.py . No behavioral changes.
2009-02-18 14:46:55 -07:00
Brian Warner
bce4a5385b
add --add-lease to 'tahoe check', 'tahoe deep-check', and webapi.
2009-02-17 19:32:43 -07:00
Brian Warner
e9563ebc02
change RIStorageServer.remote_add_lease to exit silently in case of no-such-bucket, instead of raising IndexError, because that makes the upcoming --add-lease feature faster and less noisy
2009-02-17 19:30:53 -07:00
Brian Warner
fde2289e7b
CLI #590 : convert 'tahoe deep-check' to streaming form, improve display, add tests
2009-02-17 17:15:11 -07:00
Brian Warner
d52eadfc58
test_backupdb: improve error messages if the test fails
2009-02-17 11:08:38 -07:00
Brian Warner
476a5c8fac
webapi #590 : add streaming deep-check. Still need a CLI tool to use it.
2009-02-16 23:35:53 -07:00
Brian Warner
8579e25059
test_web.Grid: change the CHECK() function to make it easier to test t= values with hyphens in them
2009-02-16 23:00:34 -07:00
Brian Warner
522641bb32
test_web: improve checker-results coverage with a no-network -based test, enhance no-network harness to assist, fix some bugs in web/check_results.py that were exposed
2009-02-16 22:12:42 -07:00
Brian Warner
f2f8ede11f
test/no_network: update comments with setup timing: no_network takes 50ms, SystemTestMixin takes 2s (on my laptop)
2009-02-16 18:06:43 -07:00
Brian Warner
040cb39613
test_upload: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary
2009-02-16 17:44:57 -07:00
Brian Warner
911abcc34b
test_download: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary
2009-02-16 17:36:58 -07:00
Brian Warner
e9427ea777
test_dirnode.py: convert Deleter to new no-network gridtest
2009-02-16 17:23:48 -07:00
Brian Warner
267d91f797
test_cli.py: modify to use the new 'no-network' gridtest instead of SystemTestMixin, which speeds it up from 73s to 43s on my system
2009-02-16 17:20:05 -07:00
Brian Warner
191fd8556a
tests: fix no_network framework to work with upload/download and checker
2009-02-16 17:19:47 -07:00
Brian Warner
770f976a85
test_cli: add test coverage for help strings
2009-02-16 15:08:33 -07:00
Brian Warner
ebe0f2ac08
test/no_network: new test harness, like system-test but doesn't use the network so it's faster
2009-02-16 14:58:44 -07:00
Brian Warner
df90dd8e73
test_cli: improve test coverage slightly
2009-02-15 21:04:51 -07:00
Brian Warner
3235b9630b
test_util: get almost full test coverage of dictutil, starting with the original pyutil tests as a base. The remaining three uncovered lines involve funny cases of ValueOrderedDict that I can't figure out how to get at
2009-02-15 20:32:10 -07:00
Brian Warner
cc27b218f5
provisioning/reliability: add tests, hush pyflakes, remove dead code, fix web links
2009-02-15 16:24:51 -07:00
Brian Warner
e1380b132b
util/statistics: add tests, fix mean_repair_cost
2009-02-15 16:23:26 -07:00
Brian Warner
812f23585d
test_repairer: hush pyflakes
2009-02-15 16:23:10 -07:00
Shawn Willden
21832280da
Statistics module
...
Added a statistics module for calculating various facets of
share survival statistics.
2009-01-13 20:12:35 -07:00
Brian Warner
60b06b8d32
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 14:53:37 -07:00
Zooko O'Whielacronx
2d45d192d1
immutable repairer: add tests of how it handles short reads
2009-02-12 16:48:53 -07:00
Zooko O'Whielacronx
8303f90da0
nodeadmin: node stops itself if a hotline file hasn't been touched in 60 seconds now, instead of in 40 seconds
...
A test failed on draco (MacPPC) because it took 49 seconds to get around to running the test, and the node had already stopped itself when the hotline file was 40 seconds old.
2009-02-12 15:21:31 -07:00
Zooko O'Whielacronx
76d7cc4404
immutable repairer: errback any pending readers of DownUpConnectorwhen it runs out of bytes, and test that fact
2009-02-11 20:11:29 -07:00
Zooko O'Whielacronx
7eb260a9cf
versioning: include an "appname" in the application version string in the versioning protocol, and make that appname be controlled by setup.py
...
It is currently hardcoded in setup.py to be 'allmydata-tahoe'. Ticket #556 is to make it configurable by a runtime command-line argument to setup.py: "--appname=foo", but I suddenly wondered if we really wanted that and at the same time realized that we don't need that for tahoe-1.3.0 release, so this patch just hardcodes it in setup.py.
setup.py inspects a file named 'src/allmydata/_appname.py' and assert that it contains the string "__appname__ = 'allmydata-tahoe'", and creates it if it isn't already present. src/allmydata/__init__.py import _appname and reads __appname__ from it. The rest of the Python code imports allmydata and inspects "allmydata.__appname__", although actually every use it uses "allmydata.__full_version__" instead, where "allmydata.__full_version__" is created in src/allmydata/__init__.py to be:
__full_version__ = __appname + '-' + str(__version__).
All the code that emits an "application version string" when describing what version of a protocol it supports (introducer server, storage server, upload helper), or when describing itself in general (introducer client), usese allmydata.__full_version__.
This fixes ticket #556 at least well enough for tahoe-1.3.0 release.
2009-02-11 17:18:16 -07:00
Brian Warner
5f288842d5
test_repairer: disable repair-from-corruption tests until other things are improved well enough to make it useful
2009-02-11 15:01:59 -07:00
Zooko O'Whielacronx
079c1851a9
trivial: whitespace
2009-02-11 11:19:35 -07:00
Brian Warner
7d907f35aa
storage: make add-lease work, change default ownernum=1 since 0 is reserved to mean 'no lease here'
2009-02-10 23:39:38 -07:00
Brian Warner
13a3ef5ec1
#620 : storage: allow mutable shares to be deleted, with a writev where new_length=0
2009-02-10 23:37:56 -07:00
Brian Warner
6a5f28f47d
test_cli.Backup: insert some stalls to make sure two successive backups get distinct timestamps, avoiding intermittent failures
2009-02-10 20:37:09 -07:00
Brian Warner
f7a263eb0b
#619 : make 'tahoe backup' complain and refuse to run if sqlite is unavailable and --no-backupdb is not passed
2009-02-10 18:49:10 -07:00
Brian Warner
3b1c8ceb86
tests: increase the default timeout for SystemTestMixin -based tests to 300 seconds, since our slower buildslaves sometimes take longer than the default 120s
2009-02-10 14:44:12 -07:00
Zooko O'Whielacronx
ef1bfdd2bf
immutable: repairer: add a simple test to exercise the "leftover" code path, fix the bug (and rename the variable "leftover" to "extra")
2009-02-10 12:12:45 -07:00
Zooko O'Whielacronx
75e4e67ed7
immutable: tighten preconditions -- you can write empty strings or read zero bytes, and add the first simple unit test of DownUpConnector
2009-02-10 00:56:47 -07:00
Zooko O'Whielacronx
d436c6235d
immutable: tests: the real WRITE_LEEWAY is 35 (it was a mistake to move it from 10 to 35 earlier -- I had seen a failure in which it took 35 times as many writes as I thought were optimal, but I misread and thought it took only 20 times as many)
2009-02-09 23:53:48 -07:00
Zooko O'Whielacronx
65c12b24b8
immutable: tests: sigh, raise, again the limit of how many extra writes you can do and still pass this test
...
Obviously requiring the code under test to perform within some limit isn't very meaningful if we raise the limit whenever the test goes outside of it.
But I still don't want to remove the test code which measures how many writes (and, elsewhere, how many reads) a client does in order to fulfill these duties.
Let this number -- now 20 -- stand as an approximation of the inefficiency of our code divided by my mental model of how many operations are actually optimal for these duties.
2009-02-09 20:09:31 -07:00
Zooko O'Whielacronx
5738d94ccd
immutable: tests: assert that verifier gives a clean bill of health after corruption and repair (the previous patch mistakenly did this only after deletion and repair), and also test whether deleting seven other shares and then downloading works. Also count the number of shares stored in the local filesystem.
2009-02-09 20:08:41 -07:00
Zooko O'Whielacronx
bbf48f4a88
immutable: test: add a test after attempting to repair from corruption: does a full verify run give the file a clean bill of health? If not, the you haven't successfully repaired it.
...
This will make the repairer tests more consistent -- less accidentally passing due to getting lucky.
2009-02-09 19:01:49 -07:00
Zooko O'Whielacronx
4091f73f34
immutable: tests: put shares back to their pristine condition in between each test of corrupting-and-repairing them
...
This is important, because if the repairer doesn't completely repair all kinds of corruption (as the current one doesn't), then the successive tests get messed up by assuming that the shares were uncorrupted when the test first set about to corrupt them.
2009-02-09 18:29:56 -07:00
Brian Warner
b197c5bb8c
test_runner: skip all spawnProcess-using tests on cygwin, since spawnProcess just hangs forever
2009-02-09 02:34:00 -07:00
Brian Warner
19902a0f16
test_runner.py: revert the CreateNode section to using runner() inline, rather than spawning a process, to get more precise coverage
2009-02-09 02:26:17 -07:00
Brian Warner
89683a3906
test_upload: add test of maximum-immutable-share-size, to complete the last item of #538
2009-02-08 19:41:27 -07:00
Brian Warner
d8e00c37b5
test_cli.Backup: capture stderr when sqlite is unavailable
2009-02-07 15:14:40 -07:00
Brian Warner
213ef3d64b
test_repairer: wrap comments to 80cols, my laptop does not have a wide screen. No functional changes.
2009-02-07 14:06:26 -07:00
Brian Warner
50886ea494
test/common.py: in share-layout-reading code, use '>L' consistently, since '>l' doesn't specify the signedness and the windows tests appear to be failing with an endianness-like problem (version==0x01000000). Also use binary mode when editing sharefiles
2009-02-07 13:38:17 -07:00
