wilcoxjg
67ad0175cd
server.py: get_latencies now reports percentiles _only_ if there are sufficient observations for the interpretation of the percentile to be unambiguous.
...
interfaces.py: modified the return type of RIStatsProvider.get_stats to allow for None as a return value
NEWS.rst, stats.py: documentation of change to get_latencies
stats.rst: now documents percentile modification in get_latencies
test_storage.py: test_latencies now expects None in output categories that contain too few samples for the associated percentile to be unambiguously reported.
fixes #1392
2011-05-27 05:01:35 -07:00
Zooko O'Whielacronx
123a1a3f04
storage: use fileutil's version of get_disk_stats() and get_available_space(), use mockery/fakery in tests, enable large share test on platforms with sparse files and if > 4 GiB of disk space is currently available
2010-09-10 08:36:29 -08:00
Zooko O'Whielacronx
0f94923f22
immutable: use PrefixingLogMixin to organize logging in Tahoe2PeerSelector and add more detailed messages about peer
2010-07-19 01:20:00 -07:00
Kevan Carstensen
6374f4307f
storage/immutable.py: make remote_abort btell the storage server about aborted buckets.
2010-07-15 16:21:05 -07:00
david-sarah
e76092e16c
Change relative imports to absolute
2010-02-26 01:14:33 -07:00
Brian Warner
731d15e56f
hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900 .
2010-01-14 14:15:29 -08:00
Brian Warner
1059db51f2
server.py: undo my bogus 'correction' of David-Sarah's comment fix
...
and move it to a better line
2009-11-30 21:46:07 -05:00
Brian Warner
91e7cb484b
storage.py: update comment
2009-11-30 11:59:13 -08:00
david-sarah
ef002c935a
storage server: detect disk space usage on Windows too ( fixes #637 )
2009-11-20 21:56:44 -08:00
Brian Warner
c9803d5217
switch all foolscap imports to use foolscap.api or foolscap.logging
2009-05-21 17:38:23 -07:00
Brian Warner
5e8c31c3b6
storage: use constant-time comparison for write-enablers and lease-secrets
2009-03-22 20:21:28 -07:00
Brian Warner
3b65607926
expirer: tolerate empty buckets, refactor bucketsize-counting code a bit, don't increment -mutable/-immutable counters unless we actually know the sharetype
2009-03-20 12:18:16 -07:00
Brian Warner
f0071c2571
expirer: clean up constructor args, add tahoe.cfg controls, use cutoff_date instead of date_cutoff
2009-03-18 18:00:09 -07:00
Brian Warner
8eaee28550
expirer: change setup, config options, in preparation for adding tahoe.cfg controls
2009-03-18 17:21:38 -07:00
Brian Warner
fffab0d724
expirer: track mutable-vs-immutable sharecounts and sizes, report them on the web status page for comparison
2009-03-18 13:25:04 -07:00
Brian Warner
24ab5ec26f
expirer: add mode to expire only-mutable or only-immutable shares
2009-03-16 23:51:18 -07:00
Brian Warner
c7254c5f1d
GC: add date-cutoff -based expiration, add proposed docs
2009-03-16 22:10:41 -07:00
Brian Warner
1ccd426a34
expirer: fix prediction math, thanks to Zandr for the catch
2009-03-09 13:42:17 -07:00
Brian Warner
8708045a98
storage.expirer: oops, fix upgrade-handler code
2009-03-08 20:55:16 -07:00
Brian Warner
df3f7f93e0
storage.expirer: handle upgrades better
2009-03-08 20:42:20 -07:00
Brian Warner
6d6049430c
expirer: tolerate corrupt shares, add them to the state and history for future examination
2009-03-08 20:08:40 -07:00
Brian Warner
1a98521c3d
storage/immutable: raise a specific error upon seeing a bad version number, instead of using assert. Also wrap to 80cols.
2009-03-08 20:07:32 -07:00
Brian Warner
6d7319c588
storage/mutable: raise a specific error upon seeing bad magic, instead of using assert
2009-03-08 19:02:01 -07:00
Brian Warner
5675b4e7e0
expirer: make web display a bit more consistent
2009-03-07 16:14:42 -07:00
Brian Warner
0dee2a6036
storage: add a lease-checker-and-expirer crawler, plus web status page.
...
This walks slowly through all shares, examining their leases, deciding which
are still valid and which have expired. Once enabled, it will then remove the
expired leases, and delete shares which no longer have any valid leases. Note
that there is not yet a tahoe.cfg option to enable lease-deletion: the
current code is read-only. A subsequent patch will add a tahoe.cfg knob to
control this, as well as docs. Some other minor items included in this patch:
tahoe debug dump-share has a new --leases-only flag
storage sharefile/leaseinfo code is cleaned up
storage web status page (/storage) has more info, more tests coverage
space-left measurement on OS-X should be more accurate (it was off by 2048x)
(use stat .f_frsize instead of f_bsize)
2009-03-06 22:45:17 -07:00
Zooko O'Whielacronx
e9199a89ab
trivial: remove unused import detected by pyflakes, and remove trailing whitespace
2009-03-05 16:32:04 -07:00
Zooko O'Whielacronx
5e90d82a02
trivial: use more specific function for ascii-encoding storage index
2009-02-22 11:57:51 -07:00
Brian Warner
112dc35563
crawler: add ETA to get_progress()
2009-02-26 19:42:48 -07:00
Brian Warner
77f3b83d68
crawler: fix performance problems: only save state once per timeslice (not after every bucket), don't start the crawler until 5 minutes after node startup
2009-02-21 14:56:49 -07:00
Brian Warner
b9c4f4bdf6
crawler: tolerate low-resolution system clocks (i.e. windows)
2009-02-21 00:15:33 -07:00
Brian Warner
106d31b112
BucketCountingCrawler: store just the count, not cycle+count, since it's too easy to make usage mistakes otherwise
2009-02-20 21:58:31 -07:00
Brian Warner
f934289d2d
crawler: load state from the pickle in init, rather than waiting until startService, so get_state() can be called early
2009-02-20 21:57:20 -07:00
Brian Warner
1077826357
BucketCountingCrawler: rename status and state keys to use 'bucket' instead of 'share', because the former is more accurate
2009-02-20 21:46:06 -07:00
Brian Warner
d2d297f12f
storage: also report space-free-for-root and space-free-for-nonroot, since that helps users understand the space-left-for-tahoe number better
2009-02-20 21:28:56 -07:00
Brian Warner
b3cd4952bd
storage: add bucket-counting share crawler, add its output (number of files+directories maintained by a storage server) and status to the webapi /storage page
2009-02-20 21:04:08 -07:00
Brian Warner
d14f00c537
storage: move si_b2a/si_a2b/storage_index_to_dir out of server.py and into common.py
2009-02-20 21:03:09 -07:00
Brian Warner
73e05bf967
crawler: add get_progress, clean up get_state
2009-02-20 18:27:43 -07:00
Brian Warner
2e45619844
web/storage: make sure we can handle platforms without os.statvfs too
2009-02-20 16:03:53 -07:00
Brian Warner
c6a061e600
crawler: provide for one-shot crawlers, which stop after their first full cycle, for share-upgraders and database-populaters
2009-02-20 15:19:11 -07:00
Brian Warner
ff6907a557
storage: include reserved_space in stats
2009-02-20 14:29:20 -07:00
Brian Warner
ef4ff21ae7
crawler: modify API to support upcoming bucket-counting crawler
2009-02-19 19:31:42 -07:00
Brian Warner
dfd72c6483
crawler: use fileutil.move_info_place in preference to our own version
2009-02-18 23:13:42 -07:00
Brian Warner
b949ea4f32
crawler: fix problems on windows and our slow cygwin slave
2009-02-18 22:24:31 -07:00
Brian Warner
193889f793
#633 : first version of a rate-limited interruptable share-crawler
2009-02-18 21:46:33 -07:00
Brian Warner
4646451de6
change StorageServer to take nodeid in the constructor, instead of assigning it later, since it's cleaner and because the original problem (Tubs not being ready until later) went away
2009-02-18 16:23:01 -07:00
Brian Warner
ef53da2b12
break storage.py into smaller pieces in storage/*.py . No behavioral changes.
2009-02-18 14:46:55 -07:00
