Brian Warner
7ea0a1316a
mutable publish: reinstate the foolscap-reference-token-bug workaround, both for the original reasons and because of an apparent new foolscap bug that's triggered by reference tokens. See #541 for details.
2008-11-25 14:27:35 -07:00
Brian Warner
6f6556da22
mutable: respect the new tahoe.cfg 'shares.needed' and 'shares.total' settings
2008-11-19 14:05:01 -07:00
Brian Warner
dfa2408157
checker: add is_recoverable() to checker results, make our stub immutable-verifier not throw an exception on unrecoverable files, add tests
2008-11-06 22:35:47 -07:00
Brian Warner
b1db6d9ff2
web: add 'Repair' button to checker results when they indicate unhealthyness. Also add the object's uri to the CheckerResults instance.
2008-10-29 18:09:17 -07:00
Brian Warner
914655c52b
interfaces.py: promote immutable.encode.NotEnoughSharesError.. it isn't just for immutable files any more
2008-10-27 13:34:49 -07:00
Brian Warner
9f21f7cf65
mutable: call remove_advise_corrupt_share when we see share corruption in mapupdate/download/check, tolerate servers that do not implement it
2008-10-24 13:21:28 -07:00
Brian Warner
6dbef907ac
mutable/servermap.py: fix needs_merge(), it was incorrectly claiming that mixed shares with distinct seqnums needed a merge, causing repair(force=False) to fail
2008-10-23 21:00:24 -07:00
Brian Warner
54c0ffd214
mutable/checker: announce the mapupdate op on the 'recent uploads+downloads' page
2008-10-23 16:03:19 -07:00
Brian Warner
977c6ac510
more #514 : pass a Monitor to all checker operations, make mutable-checker honor the cancel flag
2008-10-22 01:38:18 -07:00
Brian Warner
0a59991ce6
various: use util.log.err instead of twisted.log.err, so we get both Incidents and trial-test-flunking
2008-09-20 10:35:45 -07:00
Brian Warner
f570ad7ba5
disallow deep-check on non-directories, simplifies the code a bit
2008-09-10 13:44:58 -07:00
Brian Warner
1d2d6a35a6
checker results: add output=JSON to webapi, add tests, clean up APIs
...
to make the internal ones use binary strings (nodeid, storage index) and
the web/JSON ones use base32-encoded strings. The immutable verifier is
still incomplete (it returns imaginary healty results).
2008-09-09 19:45:17 -07:00
Brian Warner
7fb3308498
mutable checker: even more tests. Everything in ICheckerResults should be covered now, except for immutable-verify which is incomplete
2008-09-09 17:57:06 -07:00
Brian Warner
9676586732
mutable checker: oops, fix redefinition of 'healthy' (numshares < N, not numshares < k, which is 'recoverable' not 'healthy')
2008-09-09 17:28:53 -07:00
Brian Warner
f895e39d48
checker results: more tests, more results. immutable verifier tests are disabled until they emit more complete results
2008-09-09 17:15:46 -07:00
Brian Warner
3408d552cd
checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays
2008-09-07 12:44:56 -07:00
Brian Warner
3b06aa6a79
servermap: don't log late arrivals, and don't log DeadReferenceError at log.WEIRD
2008-08-26 17:37:29 -07:00
Brian Warner
1668401c16
mutable: make mutable-repair work for non-verifier runs, add tests
2008-08-26 16:34:54 -07:00
Zooko O'Whielacronx
f7b4c45d46
mutable: remove work-around for a flaw in an older version of foolscap
...
We now require "foolscap[secure_connections] >= 0.3.0", per [source:_auto_deps.py].
2008-08-26 08:50:55 -07:00
Brian Warner
a94af879ff
logging: add 'unique-message-ids' (or 'umids') to each WEIRD-or-higher log.msg call, to make it easier to correlate log message with source code
2008-08-25 18:57:59 -07:00
Brian Warner
735aa895b9
logging cleanups: lower DeadReferenceError from WEIRD (which provokes Incidents) to merely UNUSUAL, don't pre-format Failures in others
2008-08-25 17:51:55 -07:00
Brian Warner
eb695f1c58
mutable/checker: log a WEIRD-level event when we see a hash failure, to trigger an Incident
2008-08-12 20:50:20 -07:00
Brian Warner
e82d77c3cf
mutable/servermap: lower the priority of many log messages
2008-08-12 20:35:06 -07:00
Brian Warner
d43baa2ad7
mutable: add get_size_of_best_version to the interface, to simplify the web HEAD code, and tests
2008-08-12 19:02:52 -07:00
Brian Warner
79576cf75b
checker_results.problems: don't str the whole Failure, just extract the reason string
2008-08-11 21:23:06 -07:00
Brian Warner
d106e411af
checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted
2008-08-11 21:03:26 -07:00
Brian Warner
17d7f7d983
mutable/checker: rearrange a bit, change checker-results to have a status_report string
2008-08-11 20:20:33 -07:00
Brian Warner
a15ae52122
mutable/servermap: add summarize_version
2008-08-11 20:19:30 -07:00
Brian Warner
c7c57bd85c
mutable: more repair tests, one with force=True to check out merging
2008-08-06 12:06:07 -07:00
Brian Warner
dd6ec73efa
mutable: start adding Repair tests, fix a simple bug
2008-08-05 23:12:39 -07:00
Brian Warner
879fefe5f3
first pass at a mutable repairer. not tested at all yet, but of course all existing tests pass
2008-07-17 21:09:23 -07:00
Brian Warner
923c9c242a
oops, fix import/pyflakes problems
2008-07-17 17:06:20 -07:00
Brian Warner
67db0a4967
deep-check: add webapi, add 'DEEP-CHECK' button to wui, add tests, rearrange checker API a bit
2008-07-17 16:47:09 -07:00
Brian Warner
9289433ba3
first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests
2008-07-16 18:20:57 -07:00
Brian Warner
3e9322bcb6
checker: re-enable checker web results (although they just say 'Healthy' right now)
2008-07-16 15:42:56 -07:00
Brian Warner
7394607141
move encode/upload/download/checker.py into a new immutable/ directory. No behavior changes expected.
2008-07-16 13:14:39 -07:00
Brian Warner
94e619c1f6
overhaul checker invocation
...
Removed the Checker service, removed checker results storage (both in-memory
and the tiny stub of sqlite-based storage). Added ICheckable, all
check/verify is now done by calling the check() method on filenodes and
dirnodes (immutable files, literal files, mutable files, and directory
instances).
Checker results are returned in a Results instance, with an html() method for
display. Checker results have been temporarily removed from the wui directory
listing until we make some other fixes.
Also fixed client.create_node_from_uri() to create LiteralFileNodes properly,
since they have different checking behavior. Previously we were creating full
FileNodes with LIT uris inside, which were downloadable but not checkable.
2008-07-15 17:23:25 -07:00
Brian Warner
a7aa6f8686
implement a mutable checker+verifier. No repair yet. Part of #205 .
2008-07-07 17:36:00 -07:00
Brian Warner
a0de01e971
servermap.shares_available: report N too, not just k
2008-07-07 15:06:15 -07:00
Brian Warner
91c7e0f689
mutable: if there are no servers available, return an error, rather than hanging forever
2008-07-07 00:17:41 -07:00
Brian Warner
f4496bd553
mutable/publish.py: raise FileTooLargeError instead of an ugly assertion when the SDMF restrictions are exceeded
2008-06-03 00:02:10 -07:00
Brian Warner
fe44b8fb4b
util.dictutil: move DictOfSets out to a separate utility module
2008-05-21 09:43:49 -07:00
Brian Warner
154dca10a0
servermap.py: add size_of_version(), to retrieve the size of a mutable file version
2008-05-19 14:16:02 -07:00
Brian Warner
4b3adb3fcf
mutable stats: track mutable bytes published too
2008-04-29 18:20:05 -07:00
Brian Warner
13eb00145e
mutable: improve test coverage in Retrieve, when shares change after mapupdate
2008-04-22 17:25:14 -07:00
Brian Warner
a7632a345e
mutable/servermap: improve test coverage
2008-04-22 16:47:52 -07:00
Brian Warner
e6074f5dfc
mutable: test write failures, uncoordinated write detection
2008-04-22 11:49:53 -07:00
Brian Warner
7e1f9761e8
mutable: remove some dead code, rearrange use of populate_pubkey
2008-04-21 17:51:22 -07:00
Brian Warner
6af124dc3e
mutable read: enable the cache (written during mapupdate, read during retrieve). This speeds up small-file reads by about 30% over a link with an average 25ms RTT
2008-04-21 17:27:50 -07:00
Brian Warner
e8b0f3d3cb
mutable status: use google-chart-API to draw server response times for servermap update
2008-04-21 16:16:55 -07:00
