Commit Graph

3792 Commits

Author SHA1 Message Date
Brian Warner
b9c4f4bdf6 crawler: tolerate low-resolution system clocks (i.e. windows) 2009-02-21 00:15:33 -07:00
Brian Warner
106d31b112 BucketCountingCrawler: store just the count, not cycle+count, since it's too easy to make usage mistakes otherwise 2009-02-20 21:58:31 -07:00
Brian Warner
3c6471c717 test_storage: startService the server, as is now the standard practice 2009-02-20 21:57:55 -07:00
Brian Warner
f934289d2d crawler: load state from the pickle in init, rather than waiting until startService, so get_state() can be called early 2009-02-20 21:57:20 -07:00
Brian Warner
1077826357 BucketCountingCrawler: rename status and state keys to use 'bucket' instead of 'share', because the former is more accurate 2009-02-20 21:46:06 -07:00
Brian Warner
d2d297f12f storage: also report space-free-for-root and space-free-for-nonroot, since that helps users understand the space-left-for-tahoe number better 2009-02-20 21:28:56 -07:00
Brian Warner
b3cd4952bd storage: add bucket-counting share crawler, add its output (number of files+directories maintained by a storage server) and status to the webapi /storage page 2009-02-20 21:04:08 -07:00
Brian Warner
d14f00c537 storage: move si_b2a/si_a2b/storage_index_to_dir out of server.py and into common.py 2009-02-20 21:03:09 -07:00
Brian Warner
73e05bf967 crawler: add get_progress, clean up get_state 2009-02-20 18:27:43 -07:00
Brian Warner
2e45619844 web/storage: make sure we can handle platforms without os.statvfs too 2009-02-20 16:03:53 -07:00
Brian Warner
c6a061e600 crawler: provide for one-shot crawlers, which stop after their first full cycle, for share-upgraders and database-populaters 2009-02-20 15:19:11 -07:00
Brian Warner
00677ff9a5 web: add Storage status page, improve tests 2009-02-20 14:29:26 -07:00
Brian Warner
ff6907a557 storage: include reserved_space in stats 2009-02-20 14:29:20 -07:00
Brian Warner
4b8cc42f18 web/check_results: sort share identifiers in the sharemap display 2009-02-20 12:29:22 -07:00
Brian Warner
c6c30ac5d4 webapi: pass client through constructor arguments, remove IClient, should make it easier to test web renderers in isolation 2009-02-20 12:15:54 -07:00
Brian Warner
e357b8a679 test/no_network: do startService on the storage servers, make it easier to customize the storage servers 2009-02-19 20:22:54 -07:00
Brian Warner
ef4ff21ae7 crawler: modify API to support upcoming bucket-counting crawler 2009-02-19 19:31:42 -07:00
Brian Warner
9bc08158c6 test_backupdb: make the not-a-database file larger, since the older sqlite-2.3.2 on OS-X is easily fooled 2009-02-19 18:04:09 -07:00
Brian Warner
26fbe9df95 web/reliability: add parameter descriptions, adapted from a patch from Terrell Russell. 2009-02-19 16:29:18 -07:00
Brian Warner
2f751332e8 test_crawler: hush pyflakes 2009-02-19 14:23:40 -07:00
Brian Warner
ff9811211d test_crawler: disable the percentage-of-cpu-used test, since it is too unreliable on our slow buildslaves. But leave the code in place for developers to run by hand. 2009-02-19 14:16:54 -07:00
Brian Warner
790a10d1b2 reliability.py: fix the numpy conversion, it was completely broken. Thanks to Terrell Russell for the help. 2009-02-19 13:55:15 -07:00
Brian Warner
268cd538e2 reliability: switch to NumPy, since Numeric is deprecated 2009-02-19 01:44:35 -07:00
Brian Warner
589a757e25 setup.py: fix pyflakes complaints 2009-02-19 01:36:43 -07:00
Brian Warner
c2a2e93089 move show-tool-versions out of setup.py and into a separate script in misc/ , since setuptools is trying to build and install a bunch of stuff first 2009-02-19 01:35:58 -07:00
Brian Warner
f10aa59e5b test_crawler: don't require >=1 cycle on cygwin 2009-02-19 00:58:18 -07:00
Brian Warner
461b12c1d3 setup.py: add show_tool_versions command, for the benefit of a new buildbot step 2009-02-19 00:24:36 -07:00
Brian Warner
50b55474d4 setup.py: wrap to 80 cols, no functional changes 2009-02-18 23:57:51 -07:00
Brian Warner
dfd72c6483 crawler: use fileutil.move_info_place in preference to our own version 2009-02-18 23:13:42 -07:00
Brian Warner
89f041ac83 fileutil: add move_into_place(), to perform the standard unix trick of atomically replacing a file, with a fallback for windows 2009-02-18 23:13:10 -07:00
Brian Warner
b949ea4f32 crawler: fix problems on windows and our slow cygwin slave 2009-02-18 22:24:31 -07:00
Brian Warner
193889f793 #633: first version of a rate-limited interruptable share-crawler 2009-02-18 21:46:33 -07:00
Brian Warner
4646451de6 change StorageServer to take nodeid in the constructor, instead of assigning it later, since it's cleaner and because the original problem (Tubs not being ready until later) went away 2009-02-18 16:23:01 -07:00
Brian Warner
b398a69f14 test_system: split off checker tests to test_deepcheck.py, this file is too big 2009-02-18 15:42:34 -07:00
Brian Warner
ef53da2b12 break storage.py into smaller pieces in storage/*.py . No behavioral changes. 2009-02-18 14:46:55 -07:00
Brian Warner
a0c5f92cbd immutable/layout: minor change to repr name 2009-02-18 14:46:48 -07:00
Brian Warner
c86acc42f4 docs: add lease-tradeoffs diagram 2009-02-18 14:41:37 -07:00
Brian Warner
2346d8621d interfaces.py: allow add/renew/cancel-lease to return Any, so that 1.3.1 clients (the first to use these calls) can tolerate future storage servers which might return something other than None 2009-02-18 13:29:03 -07:00
Brian Warner
505220f27d docs/debian.txt: minor edit 2009-02-17 21:22:12 -07:00
Brian Warner
bce4a5385b add --add-lease to 'tahoe check', 'tahoe deep-check', and webapi. 2009-02-17 19:32:43 -07:00
Brian Warner
e9563ebc02 change RIStorageServer.remote_add_lease to exit silently in case of no-such-bucket, instead of raising IndexError, because that makes the upcoming --add-lease feature faster and less noisy 2009-02-17 19:30:53 -07:00
Brian Warner
fde2289e7b CLI #590: convert 'tahoe deep-check' to streaming form, improve display, add tests 2009-02-17 17:15:11 -07:00
Brian Warner
0e78b2587c interfaces.py: document behavior of add_lease/renew_lease/cancel_lease, before I change it 2009-02-17 13:48:09 -07:00
Brian Warner
d52eadfc58 test_backupdb: improve error messages if the test fails 2009-02-17 11:08:38 -07:00
Brian Warner
476a5c8fac webapi #590: add streaming deep-check. Still need a CLI tool to use it. 2009-02-16 23:35:53 -07:00
Brian Warner
8579e25059 test_web.Grid: change the CHECK() function to make it easier to test t= values with hyphens in them 2009-02-16 23:00:34 -07:00
Brian Warner
522641bb32 test_web: improve checker-results coverage with a no-network -based test, enhance no-network harness to assist, fix some bugs in web/check_results.py that were exposed 2009-02-16 22:12:42 -07:00
Brian Warner
059f24adca web: fix handling of reliability page when Numeric is not available 2009-02-16 19:56:58 -07:00
Brian Warner
f2f8ede11f test/no_network: update comments with setup timing: no_network takes 50ms, SystemTestMixin takes 2s (on my laptop) 2009-02-16 18:06:43 -07:00
Brian Warner
040cb39613 test_upload: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary 2009-02-16 17:44:57 -07:00