Commit Graph

254 Commits

Author SHA1 Message Date
Brian Warner
7fb3308498 mutable checker: even more tests. Everything in ICheckerResults should be covered now, except for immutable-verify which is incomplete 2008-09-09 17:57:06 -07:00
Brian Warner
9676586732 mutable checker: oops, fix redefinition of 'healthy' (numshares < N, not numshares < k, which is 'recoverable' not 'healthy') 2008-09-09 17:28:53 -07:00
Brian Warner
f895e39d48 checker results: more tests, more results. immutable verifier tests are disabled until they emit more complete results 2008-09-09 17:15:46 -07:00
Brian Warner
3408d552cd checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays 2008-09-07 12:44:56 -07:00
Brian Warner
3b06aa6a79 servermap: don't log late arrivals, and don't log DeadReferenceError at log.WEIRD 2008-08-26 17:37:29 -07:00
Brian Warner
1668401c16 mutable: make mutable-repair work for non-verifier runs, add tests 2008-08-26 16:34:54 -07:00
Zooko O'Whielacronx
f7b4c45d46 mutable: remove work-around for a flaw in an older version of foolscap
We now require "foolscap[secure_connections] >= 0.3.0", per [source:_auto_deps.py].
2008-08-26 08:50:55 -07:00
Brian Warner
a94af879ff logging: add 'unique-message-ids' (or 'umids') to each WEIRD-or-higher log.msg call, to make it easier to correlate log message with source code 2008-08-25 18:57:59 -07:00
Brian Warner
735aa895b9 logging cleanups: lower DeadReferenceError from WEIRD (which provokes Incidents) to merely UNUSUAL, don't pre-format Failures in others 2008-08-25 17:51:55 -07:00
Brian Warner
eb695f1c58 mutable/checker: log a WEIRD-level event when we see a hash failure, to trigger an Incident 2008-08-12 20:50:20 -07:00
Brian Warner
e82d77c3cf mutable/servermap: lower the priority of many log messages 2008-08-12 20:35:06 -07:00
Brian Warner
d43baa2ad7 mutable: add get_size_of_best_version to the interface, to simplify the web HEAD code, and tests 2008-08-12 19:02:52 -07:00
Brian Warner
79576cf75b checker_results.problems: don't str the whole Failure, just extract the reason string 2008-08-11 21:23:06 -07:00
Brian Warner
d106e411af checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted 2008-08-11 21:03:26 -07:00
Brian Warner
17d7f7d983 mutable/checker: rearrange a bit, change checker-results to have a status_report string 2008-08-11 20:20:33 -07:00
Brian Warner
a15ae52122 mutable/servermap: add summarize_version 2008-08-11 20:19:30 -07:00
Brian Warner
c7c57bd85c mutable: more repair tests, one with force=True to check out merging 2008-08-06 12:06:07 -07:00
Brian Warner
dd6ec73efa mutable: start adding Repair tests, fix a simple bug 2008-08-05 23:12:39 -07:00
Brian Warner
879fefe5f3 first pass at a mutable repairer. not tested at all yet, but of course all existing tests pass 2008-07-17 21:09:23 -07:00
Brian Warner
923c9c242a oops, fix import/pyflakes problems 2008-07-17 17:06:20 -07:00
Brian Warner
67db0a4967 deep-check: add webapi, add 'DEEP-CHECK' button to wui, add tests, rearrange checker API a bit 2008-07-17 16:47:09 -07:00
Brian Warner
9289433ba3 first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests 2008-07-16 18:20:57 -07:00
Brian Warner
3e9322bcb6 checker: re-enable checker web results (although they just say 'Healthy' right now) 2008-07-16 15:42:56 -07:00
Brian Warner
7394607141 move encode/upload/download/checker.py into a new immutable/ directory. No behavior changes expected. 2008-07-16 13:14:39 -07:00
Brian Warner
94e619c1f6 overhaul checker invocation
Removed the Checker service, removed checker results storage (both in-memory
and the tiny stub of sqlite-based storage). Added ICheckable, all
check/verify is now done by calling the check() method on filenodes and
dirnodes (immutable files, literal files, mutable files, and directory
instances).

Checker results are returned in a Results instance, with an html() method for
display. Checker results have been temporarily removed from the wui directory
listing until we make some other fixes.

Also fixed client.create_node_from_uri() to create LiteralFileNodes properly,
since they have different checking behavior. Previously we were creating full
FileNodes with LIT uris inside, which were downloadable but not checkable.
2008-07-15 17:23:25 -07:00
Brian Warner
a7aa6f8686 implement a mutable checker+verifier. No repair yet. Part of #205. 2008-07-07 17:36:00 -07:00
Brian Warner
a0de01e971 servermap.shares_available: report N too, not just k 2008-07-07 15:06:15 -07:00
Brian Warner
91c7e0f689 mutable: if there are no servers available, return an error, rather than hanging forever 2008-07-07 00:17:41 -07:00
Brian Warner
f4496bd553 mutable/publish.py: raise FileTooLargeError instead of an ugly assertion when the SDMF restrictions are exceeded 2008-06-03 00:02:10 -07:00
Brian Warner
fe44b8fb4b util.dictutil: move DictOfSets out to a separate utility module 2008-05-21 09:43:49 -07:00
Brian Warner
154dca10a0 servermap.py: add size_of_version(), to retrieve the size of a mutable file version 2008-05-19 14:16:02 -07:00
Brian Warner
4b3adb3fcf mutable stats: track mutable bytes published too 2008-04-29 18:20:05 -07:00
Brian Warner
13eb00145e mutable: improve test coverage in Retrieve, when shares change after mapupdate 2008-04-22 17:25:14 -07:00
Brian Warner
a7632a345e mutable/servermap: improve test coverage 2008-04-22 16:47:52 -07:00
Brian Warner
e6074f5dfc mutable: test write failures, uncoordinated write detection 2008-04-22 11:49:53 -07:00
Brian Warner
7e1f9761e8 mutable: remove some dead code, rearrange use of populate_pubkey 2008-04-21 17:51:22 -07:00
Brian Warner
6af124dc3e mutable read: enable the cache (written during mapupdate, read during retrieve). This speeds up small-file reads by about 30% over a link with an average 25ms RTT 2008-04-21 17:27:50 -07:00
Brian Warner
e8b0f3d3cb mutable status: use google-chart-API to draw server response times for servermap update 2008-04-21 16:16:55 -07:00
Brian Warner
a04c2866b1 mutable: improve test coverage slightly 2008-04-21 15:10:50 -07:00
Brian Warner
09dcfeae22 mutable: improve test coverage, fix bug in privkey fetching, add .finished to stats, remove dead code 2008-04-18 19:55:12 -07:00
Brian Warner
fc7b75e55f mutable.node: avoid reentrancy problems in Deferred code on twisted2.5, by adding an eventual-send call 2008-04-18 00:43:29 -07:00
Zooko O'Whielacronx
a504f3395f let MutableFileNode.__repr__() return something useful even if it isn't not initialize yet
This is because I sometimes log or examine a repr of an object from a function which is being called from that object's own __init__()...
(I'm committing this patch in order to test our patch management infrastructure.)
2008-04-17 21:32:38 -07:00
Brian Warner
004d3c14bf mutable: implement MutableFileNode.modify, plus tests 2008-04-17 19:12:42 -07:00
Brian Warner
a379690b04 mutable: replace MutableFileNode API, update tests. Changed all callers to use overwrite(), but that will change soon 2008-04-17 17:51:38 -07:00
Brian Warner
ee218c262d mutable WIP: use fireOnOneErrback when using a DeferredList 2008-04-17 13:11:48 -07:00
Brian Warner
e1838ba217 mutable WIP: clean up status handling, shrink the code a lot, improve test coverage 2008-04-17 13:02:22 -07:00
Brian Warner
a1670497a8 mutable WIP: add servermap update status pages 2008-04-16 19:05:41 -07:00
Brian Warner
f35f20e10b mutable WIP: publish status doesn't know its size early enough to update the stats_provider 2008-04-16 17:55:17 -07:00
Brian Warner
749c42fa2c mutable WIP: re-enable publish/retrieve status 2008-04-16 17:49:06 -07:00
Brian Warner
4f07d96667 mutable WIP: improve logging a bit 2008-04-16 15:22:30 -07:00
Brian Warner
c0c8d72b44 mutable WIP: rewrite ServerMap data structure, add tests 2008-04-16 14:49:47 -07:00
Brian Warner
1b4b4cbd4a mutable WIP: rename NotEnoughPeersError to NotEnoughSharesError 2008-04-15 16:08:32 -07:00
Brian Warner
ca14b99397 mutable WIP: if corrupt shares cause a retrieve to fail, restart it once, ignoring those shares and using different ones 2008-04-15 15:58:02 -07:00
Brian Warner
d4230d1781 mutable WIP: split mutable.py into separate files. All tests pass. 2008-04-11 14:31:16 -07:00