mirror of
https://github.com/tahoe-lafs/tahoe-lafs.git
synced 2025-01-11 07:23:04 +00:00
7a504dd99f
17 Commits
Author | SHA1 | Message | Date | |
---|---|---|---|---|
Itamar Turner-Trauring
|
d238242d73 | Port to Python 3. | ||
David-Sarah Hopwood
|
4563ba456b |
Remove ResponseCache in favor of MDMFSlotReadProxy's cache. closes #1240.
This contains several merged patches. Individual messages follow, latest first: * Fix a warning from check-miscaptures. * In retrieve.py, explicitly test whether a key is in self.servermap.proxies rather than catching KeyError. * Added a new comment to the MDMF version of the test I removed, explaining the removal of the SDMF version. * Removed test_corrupt_all_block_hash_tree_late, since the entire block_hash_tree is cached in the servermap for an SDMF file. * Fixed several tests that require files larger than the servermap cache. * Remove unused test_response_cache_memory_leak(). * Exercise the cache. * Test infrastructure for counting cache misses on MDMF files. * Removed the ResponseCache. Instead, the MDMFSlotReadProxy initialized by ServerMap is kept around so Retrieve can access it. The ReadProxy has a cache of the first 1000 bytes initially read from each share by the ServerMap. We're able to satisfy a number of requests out of this cache, so roundtrips are reduced from 84 to 60 in test_deepcheck_mdmf. There is still some mystery about under what conditions the cache has fewer than 1000 bytes. Also this breaks some existing unit tests that depend on the inner behavior of ResponseCache. * The servermap.proxies (a cache of SlotReadProxies) is now keyed by (verinfo,serverid,shnum) rather than just (serverid,shnum) * Minor cosmetic changes * Added a test failure if the number of cache misses is too high. Author: Andrew Miller <amiller@dappervision.com> Signed-off-by: David-Sarah Hopwood <davidsarah@jacaranda.org> |
||
Brian Warner
|
5bae4a1bd2 |
Mutable repair: use new MODE_REPAIR to query all servers *and* get privkey
This fixes bug #1689. Repair was using MODE_READ to build the servermap, which doesn't try hard enough to grab the privkey, and also doesn't guarantee sending queries to all servers. This patch adds a new MODE_REPAIR which does both, and does a separate, distinct mapupdate to start wth repair cycle, instead of relying upon the (MODE_CHECK) mapupdate leftover from the filecheck that triggered the repair. |
||
Brian Warner
|
c56839478e | mutable/layout.py: raise BadShareError instead of assert() | ||
Brian Warner
|
5bf1ffbc87 |
IServer refactoring: pass IServer instances around, instead of peerids
refs #1363 This collapses 88 small incremental changes (each of which passes all tests) into one big patch. The development process for the long path started with adding some temporary scaffolding, changing one method at a time, then removing the scaffolding. The individual pieces are as follows, in reverse chronological order (the first patch is at the end of this comment): commit 9bbe4174fd0d98a6cf47a8ef96e85d9ef34b2f9a Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 16:05:00 2011 -0400 immutable/downloader/status.py: correct comment src/allmydata/immutable/downloader/status.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) commit 72146a7c7c91eac2f7c3ceb801eb7a1721376889 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 15:46:20 2011 -0400 remove temporary ServerMap._storage_broker src/allmydata/mutable/checker.py | 2 +- src/allmydata/mutable/filenode.py | 2 +- src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 5 ++--- src/allmydata/test/test_mutable.py | 8 ++++---- 5 files changed, 9 insertions(+), 10 deletions(-) commit d703096b41632c47d76414b12672e076a422ff5c Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 15:37:05 2011 -0400 remove temporary storage_broker.get_server_for_id() src/allmydata/storage_client.py | 3 --- src/allmydata/test/no_network.py | 13 ------------- 2 files changed, 0 insertions(+), 16 deletions(-) commit 620cc5d80882ef6f7decfd26af8a6c7c1ddf80d1 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:50:06 2011 -0400 API of Retrieve._try_to_validate_privkey(), trying to remove reader.server src/allmydata/mutable/retrieve.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) commit 92f43f856f4a8b36c207d1b190ed8699b5a4ecb4 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:48:08 2011 -0400 API of Retrieve._validate_block(), trying to remove reader.server src/allmydata/mutable/retrieve.py | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) commit 572d5070761861a2190349d1ed8d85dbc25698a5 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:36:58 2011 -0400 API of Retrieve._mark_bad_share(), trying to remove reader.server src/allmydata/mutable/retrieve.py | 21 +++++++++------------ 1 files changed, 9 insertions(+), 12 deletions(-) commit a793ff00c0de1e2eec7b46288fdf388c7a2bec89 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:06:13 2011 -0400 remove now-unused get_rref_for_serverid() src/allmydata/mutable/servermap.py | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) commit 1b9827cc9366bf90b93297fdd6832f2ad0480ce7 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:03:09 2011 -0400 Retrieve: stop adding .serverid attributes to readers src/allmydata/mutable/retrieve.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) commit 5d4e9d491b19e49d2e443a1dfff2c672842c36ef Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:03:34 2011 -0400 return value of Retrieve(verify=True) src/allmydata/mutable/checker.py | 11 ++++++----- src/allmydata/mutable/retrieve.py | 3 +-- 2 files changed, 7 insertions(+), 7 deletions(-) commit e9ab7978c384e1f677cb7779dc449b1044face82 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:54:23 2011 -0400 Retrieve._bad_shares (but not return value, used by Verifier) src/allmydata/mutable/retrieve.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) commit 2d91926de233ec5c881f30e36b4a30ad92ab42a9 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:51:23 2011 -0400 Publish: stop adding .serverid attributes to writers src/allmydata/mutable/publish.py | 9 ++------- 1 files changed, 2 insertions(+), 7 deletions(-) commit 47c7a0105dec7cbf4f7e0a3ce800bbb85b15df4a Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:56:33 2011 -0400 API of get_write_enabler() src/allmydata/mutable/filenode.py | 7 ++++--- src/allmydata/mutable/publish.py | 4 ++-- src/allmydata/test/no_network.py | 3 +++ 3 files changed, 9 insertions(+), 5 deletions(-) commit 9196a5c6590fdbfd660325ea8358b345887d3db0 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:46:24 2011 -0400 API of get_(renewal|cancel)_secret() src/allmydata/mutable/filenode.py | 14 ++++++++------ src/allmydata/mutable/publish.py | 8 ++++---- src/allmydata/mutable/servermap.py | 5 ++--- 3 files changed, 14 insertions(+), 13 deletions(-) commit de7c1552f8c163eff5b6d820b5fb3b21c1b47cb5 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:41:52 2011 -0400 API of CorruptShareError. Also comment out some related+unused test_web.py code src/allmydata/mutable/common.py | 13 +++++-------- src/allmydata/mutable/retrieve.py | 10 +++++----- src/allmydata/mutable/servermap.py | 8 +++----- src/allmydata/test/common.py | 13 ++++++++----- 4 files changed, 21 insertions(+), 23 deletions(-) commit 2c1c314046b620c16f1e66d030c150d768b7d01e Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:01:46 2011 -0400 API of ServerMap.mark_bad_share() src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/retrieve.py | 6 +++--- src/allmydata/mutable/servermap.py | 6 ++---- src/allmydata/test/test_mutable.py | 3 +-- 4 files changed, 7 insertions(+), 10 deletions(-) commit 1bed349030779fd0c378ae4e821384f953c6f6ff Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:11:17 2011 -0400 API+name of ServerMap.shares_on_server() : only for tests, so debug_ prefix src/allmydata/mutable/servermap.py | 7 ++----- src/allmydata/test/test_mutable.py | 6 +++--- 2 files changed, 5 insertions(+), 8 deletions(-) commit 2d32e448677d6b818692e801045d4115b29abf21 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:07:10 2011 -0400 API of ServerMap.all_servers_for_version() src/allmydata/mutable/servermap.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit 48f3204d1889c3e7179578125c4bdef515af3d6a Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:04:50 2011 -0400 internals of ServerMap methods that use make_versionmap(), remove temp copy src/allmydata/mutable/servermap.py | 28 +++++++++---------------- 1 files changed, 10 insertions(+), 18 deletions(-) commit 5c3da77b6c777a145bd5ddfaa4db849dc9495548 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:01:28 2011 -0400 API of ServerMap.make_versionmap() src/allmydata/mutable/checker.py | 4 ++-- src/allmydata/mutable/retrieve.py | 5 ++--- src/allmydata/mutable/servermap.py | 4 ++-- src/allmydata/test/test_mutable.py | 7 ++++--- 4 files changed, 10 insertions(+), 10 deletions(-) commit b6882ece49afb4c507d118af2db346fa329209dc Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 10:53:38 2011 -0400 make a copy of ServerMap.make_versionmap() (_make_versionmap2) for internal use src/allmydata/mutable/servermap.py | 18 +++++++++++++----- 1 files changed, 13 insertions(+), 5 deletions(-) commit 963f8e63faf32b950eb1b8103cd2ff16fe8f0151 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:45:58 2011 -0400 API of RetrieveStatus.add_problem() src/allmydata/mutable/retrieve.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 4976d29ffae565a048851601c29013bbae2976d8 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:45:05 2011 -0400 API of RetrieveStatus.add_fetch_timing() src/allmydata/mutable/retrieve.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit d057d3bbba72663ee148a8b916bc2d52be2e3982 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:44:04 2011 -0400 API of Retrieve.notify_server_corruption() src/allmydata/mutable/retrieve.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit 8a2a81e46671c860610e0e96d6add1a57551f22d Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:42:32 2011 -0400 remove unused _outstanding_queries src/allmydata/mutable/retrieve.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) commit 56d12cc9968d03ccd53764455c671122c4f391d1 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:40:57 2011 -0400 change Retrieve.remaining_sharemap src/allmydata/mutable/retrieve.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit 4f0b7af4821f43290bfc70f2b1fc30149ad81281 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 10:40:18 2011 -0400 accessor for PublishStatus._problems src/allmydata/mutable/publish.py | 4 +++- src/allmydata/web/status.py | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) commit 627087cf66d0b8cc519f4d551a967a7bd9b6a741 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 10:36:39 2011 -0400 accessor for RetrieveStatus._problems src/allmydata/mutable/retrieve.py | 8 ++++++-- src/allmydata/web/status.py | 2 +- 2 files changed, 7 insertions(+), 3 deletions(-) commit ca7dea81f03801b1c7353fc00ecba689268109cf Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:35:32 2011 -0400 add .server to "reader", so we can get at it later src/allmydata/mutable/retrieve.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 6ef516e24908ec195af084a7550d1921a5e983b0 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:32:32 2011 -0400 temporarily give Retrieve a _storage_broker, so it can map serverids to servers src/allmydata/mutable/checker.py | 3 ++- src/allmydata/mutable/filenode.py | 6 ++++-- src/allmydata/mutable/retrieve.py | 5 +++-- src/allmydata/test/test_mutable.py | 4 ++-- 4 files changed, 11 insertions(+), 7 deletions(-) commit afe08e4dd3f4ff9ff7e8a2a8d28b181e3625bcc9 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:21:51 2011 -0400 mutable/retrieve.py: s/peer/server/ src/allmydata/mutable/retrieve.py | 82 +++++++++++++------------- src/allmydata/test/test_mutable.py | 6 +- 2 files changed, 44 insertions(+), 44 deletions(-) commit 910afcb5d7f274880f68dd6cdb5b05f2bbc29adc Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:16:01 2011 -0400 web.status.PublishStatusPage: add comment, I think .problems isn't exercised src/allmydata/web/status.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) commit 311466dd8c931bbba40d590ade867704282e7f1a Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:48:16 2011 -0400 API of PublishStatus.add_per_server_time() src/allmydata/mutable/publish.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 2df5faa1b6cbfbaded520d2320305a62fe961118 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:46:37 2011 -0400 more simplifications src/allmydata/mutable/publish.py | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) commit 6ac4544a3da385f2aad9392f906b90192f4f919a Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:44:08 2011 -0400 API of ServerMap.version_on_server() src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 4 ++-- src/allmydata/test/test_mutable.py | 5 ++--- 3 files changed, 5 insertions(+), 6 deletions(-) commit 3e187e322511072e4683329df6b2c6c733a66dba Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:16:32 2011 -0400 API of ServerMap.make_sharemap() src/allmydata/mutable/servermap.py | 4 ++-- src/allmydata/test/test_mutable.py | 7 ++++--- src/allmydata/web/status.py | 4 ++-- 3 files changed, 8 insertions(+), 7 deletions(-) commit 318feed8437bdd8d4943c6569d38f7b54b6313cc Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:36:19 2011 -0400 small cleanups src/allmydata/mutable/publish.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit bd459ed5714e1db5a7163935c54b7b0b56db8349 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:33:39 2011 -0400 API of ServerMap.add_new_share() src/allmydata/mutable/publish.py | 4 ++-- src/allmydata/mutable/servermap.py | 6 ++---- 2 files changed, 4 insertions(+), 6 deletions(-) commit f2804fb6ed11d80088e0da8ed48e6c2922f2ffef Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:30:26 2011 -0400 API of ServerMap.get_bad_shares() src/allmydata/mutable/publish.py | 3 +-- src/allmydata/mutable/servermap.py | 9 ++++----- 2 files changed, 5 insertions(+), 7 deletions(-) commit 965074a47b3ce1431cb46d9a233840afcf9105f5 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:26:58 2011 -0400 more small cleanups src/allmydata/mutable/publish.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit 38020da34f034f8889947dd3dc05e087ffff7106 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:18:47 2011 -0400 change Publish.bad_share_checkstrings src/allmydata/mutable/publish.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit 5efebcbd2ee0c2f299ea86f7591d856c0f265304 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:16:31 2011 -0400 change internals of Publish.update_goal() src/allmydata/mutable/publish.py | 8 +++----- 1 files changed, 3 insertions(+), 5 deletions(-) commit e91b55ff4c2a69165b71f2c7b217ac319ff4c527 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:11:42 2011 -0400 get rid of Publish.connections src/allmydata/mutable/publish.py | 27 +++++---------------------- 1 files changed, 5 insertions(+), 22 deletions(-) commit 64e9a53b3229ebe2f9ebf7ed502d539311d0e037 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:05:32 2011 -0400 change Publish.bad_servers src/allmydata/mutable/publish.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) commit b85a934bef315a06bcfe00c9c12a3627fed2b918 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:03:07 2011 -0400 Publish.bad_servers: fix bug, this should be a set of serverids, not writers src/allmydata/mutable/publish.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) commit 605ea15ec15ed671513819003ccd211cdb9761e0 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:00:21 2011 -0400 change .placed src/allmydata/mutable/publish.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit f7aba37b1b345d5b6d5cb16e3b3f6f3c1afb658e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 22:59:22 2011 -0400 temporarily stash IServer as .server on the "writer" object src/allmydata/mutable/publish.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) commit f9b551d788e7db1f187fce5ab98ab5d5fe4e1c36 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 22:48:18 2011 -0400 change Publish.goal and API of log_goal() to use IServer, not serverid src/allmydata/mutable/publish.py | 48 ++++++++++++++-------------- 1 files changed, 24 insertions(+), 24 deletions(-) commit 75f20616558e4900b8b1f685dd99aa838de6d452 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:27:02 2011 -0400 API of ServerMap.get_known_shares() src/allmydata/mutable/publish.py | 16 ++++++++++------ src/allmydata/mutable/servermap.py | 7 ++----- 2 files changed, 12 insertions(+), 11 deletions(-) commit 1c38c9d37bb08221b4418762234b1a62397b3b4b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:20:29 2011 -0400 Publish.full_serverlist src/allmydata/mutable/publish.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) commit b6cbd215a04b9cde31a7d92a97a7f048622b16f1 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:12:31 2011 -0400 API of ServerMap.all_servers() src/allmydata/mutable/servermap.py | 19 ++++++------------- 1 files changed, 6 insertions(+), 13 deletions(-) commit e63cd0315fae65357b1727ec6d5ff3c6e0d27c98 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:10:18 2011 -0400 remove ServerMap.connections, set_rref_for_serverid() src/allmydata/mutable/servermap.py | 11 +---------- 1 files changed, 1 insertions(+), 10 deletions(-) commit 4df52db2f80eb12eefa5d57103c24893cde89553 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:04:06 2011 -0400 API of ServerMap.mark_server_reachable() src/allmydata/mutable/servermap.py | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-) commit 69c715bde77944dc25181b3dbbeb042c816f9a1b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:03:21 2011 -0400 API of ServerMap.mark_server_unreachable() src/allmydata/mutable/servermap.py | 9 +++------ 1 files changed, 3 insertions(+), 6 deletions(-) commit 3d784d60eec1c508858e3a617e4411ffbcc3c1fa Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:02:03 2011 -0400 API of status.set_privkey_from() src/allmydata/mutable/servermap.py | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) commit 544ed3ea29bed7e66da7fd29ca3f6f076f27a9e6 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:01:15 2011 -0400 API of status.add_per_server_time() src/allmydata/mutable/servermap.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) commit fffe5008b6320bd1e04c3c68389a2bf2ee383fa8 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:59:02 2011 -0400 remove unused .versionmap src/allmydata/mutable/servermap.py | 7 ------- 1 files changed, 0 insertions(+), 7 deletions(-) commit 2816562e090d2294179db3588dafcca18de1bc2b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:57:51 2011 -0400 remove serverid from all log messages. Also one unused lambda. src/allmydata/mutable/servermap.py | 30 +++++++++++++------------- 1 files changed, 15 insertions(+), 15 deletions(-) commit 28fa6b1a2738fa98c1f1dbd3d0e01ae98912d11f Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:54:30 2011 -0400 removed unused _readers src/allmydata/mutable/servermap.py | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) commit a8e4ed3d645ab592d1add6a1e69b6d1ebfb77817 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:54:16 2011 -0400 remove unused _sharemap src/allmydata/mutable/servermap.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) commit 3f072e55cf1d0700f9fffe23f8f3a475725df588 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:49:03 2011 -0400 _must_query src/allmydata/mutable/servermap.py | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) commit c599a059b8df3f5785e4bf89fb6ecc6d8dcd708b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:48:05 2011 -0400 _queries_outstanding src/allmydata/mutable/servermap.py | 16 +++++++--------- 1 files changed, 7 insertions(+), 9 deletions(-) commit 7743759f98ac2c07926b2fdbd80bf52dfab33085 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:46:17 2011 -0400 _empty_servers src/allmydata/mutable/servermap.py | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) commit 6bb1825916828a713a32cdf7f7411fa3ea2e1e5d Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:45:39 2011 -0400 _good_servers src/allmydata/mutable/servermap.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit 1768fab1b51d8dd93ecabbaaabfadfa20cf6c3d4 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:44:59 2011 -0400 _bad_servers src/allmydata/mutable/servermap.py | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) commit dccbaef30f0ba714c746bf6d4a1a803c36e17b65 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:41:54 2011 -0400 API of _try_to_set_pubkey() src/allmydata/mutable/servermap.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) commit 0481ea70042ba3575f15eac7fd0780f8ece580cc Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:35:02 2011 -0400 API of notify_server_corruption() src/allmydata/mutable/servermap.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit bea9cba18fb3b9c11bb22f18356a263ecec7351e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:34:09 2011 -0400 API of _got_signature_one_share() src/allmydata/mutable/servermap.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) commit 1520123583cf78650706e114b15bb5b0ac1f4a14 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:32:33 2011 -0400 API of _try_to_validate_privkey() src/allmydata/mutable/servermap.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) commit 938852c9c8519c7a078f58a9b1f4dd8ec8b6715e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:31:48 2011 -0400 API and internals of _add_lease_failed() src/allmydata/mutable/servermap.py | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) commit 3843dba367e3c19e176a622ab853cb51d2472ddf Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:30:37 2011 -0400 API of _privkey_query_failed() src/allmydata/mutable/servermap.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 2219a710e1633cd57d0ca0786490de87b3e19ba7 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:29:43 2011 -0400 fix bug in call to _privkey_query_failed, unrelated to refactoring src/allmydata/mutable/servermap.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) commit ae615bec7d0d1b269710b6902797b12f9592ad62 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:27:17 2011 -0400 API of _got_corrupt_share() src/allmydata/mutable/servermap.py | 17 +++++++++-------- 1 files changed, 9 insertions(+), 8 deletions(-) commit cb51c95a6f4e077278157a77dab060c8c1ad7a81 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:23:16 2011 -0400 API of _got_results() src/allmydata/mutable/servermap.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) commit bac9154fe0af18f226999a58ffc2362d8cf4b802 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:19:19 2011 -0400 API of _query_failed() src/allmydata/mutable/servermap.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit fdc29a8ca95d4b5c503e5382b9e5d4d02141ba12 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:17:20 2011 -0400 API of _do_read() src/allmydata/mutable/servermap.py | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) commit e7e9e338f28d004aa4d423d11c65f1e271ac7322 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:20:21 2011 -0400 API of _do_query() src/allmydata/mutable/servermap.py | 15 +++++++-------- 1 files changed, 7 insertions(+), 8 deletions(-) commit 330625b9dac4cdbe72a11464a893065b9aeed453 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:43:05 2011 -0400 next step: first batch of updates to ServermapUpdater updates: most method-local variables in update() API of _build_initial_querylist() API of _send_initial_requests() .full_serverlist .extra_servers src/allmydata/mutable/servermap.py | 39 ++++++++++++++------------ 1 files changed, 21 insertions(+), 18 deletions(-) commit 4aadc584fa7dcb2daa86b048c81dee0049ba26d9 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:07:00 2011 -0400 internal change: index _bad_shares with IServer src/allmydata/mutable/servermap.py | 20 ++++++++++---------- 1 files changed, 10 insertions(+), 10 deletions(-) commit 16d4e6fa82a9907dbdc92094213387c6a4164e41 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:20:47 2011 +0100 internal change: index _known_shares with IServer instead of serverid callers are unchanged src/allmydata/mutable/servermap.py | 42 +++++++++++++++---------- 1 files changed, 25 insertions(+), 17 deletions(-) commit ceeb5f4938cc814a0c75d1b8f4018aed965c2176 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:11:43 2011 +0100 accessors and name cleanup for servermap.Servermap.last_update_mode/time src/allmydata/mutable/filenode.py | 6 +++--- src/allmydata/mutable/publish.py | 4 ++-- src/allmydata/mutable/servermap.py | 17 +++++++++++------ 3 files changed, 16 insertions(+), 11 deletions(-) commit 8d3cbda82661c0a7e5c3d3b65cf7a5d5ab7e32c0 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:11:14 2011 +0100 accessors and name cleanup for servermap.Servermap.problems src/allmydata/mutable/servermap.py | 21 +++++++++++++-------- src/allmydata/test/test_mutable.py | 6 +++--- 2 files changed, 16 insertions(+), 11 deletions(-) commit 348f57988f79389db0aab7672e6eaa9a6d8e3219 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:10:41 2011 +0100 accessors and name cleanup for servermap.Servermap.bad_shares src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 30 ++++++++++++++----------- 2 files changed, 18 insertions(+), 14 deletions(-) commit 520c9368134673cdf76c653c5e1bb91c2ab5d51e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:10:05 2011 +0100 accessors and name cleanup for servermap.Servermap.servermap . src/allmydata/mutable/publish.py | 14 +++++---- src/allmydata/mutable/servermap.py | 38 ++++++++++++++----------- 2 files changed, 29 insertions(+), 23 deletions(-) commit b8b8dc38287a91dbdf494426ac801d9381ce5841 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:08:02 2011 +0100 fix reachable_servers src/allmydata/mutable/checker.py | 3 ++- src/allmydata/mutable/publish.py | 4 +++- src/allmydata/mutable/servermap.py | 12 ++++++++++-- 3 files changed, 15 insertions(+), 4 deletions(-) commit cb0cfd1adfefad357c187aaaf690c3df68b622bc Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:06:03 2011 +0100 fix Servermap.unreachable_servers src/allmydata/mutable/servermap.py | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-) commit 2d9ea79b94bd4db674d40386fda90825785ac495 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:03:48 2011 +0100 give ServerMap a StorageFarmBroker, temporary this makes it possible for the ServerMap to accept bare serverids and still build data structures with IServers src/allmydata/mutable/checker.py | 2 +- src/allmydata/mutable/filenode.py | 2 +- src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 5 +++-- src/allmydata/test/test_mutable.py | 8 ++++---- 5 files changed, 10 insertions(+), 9 deletions(-) commit 718d1aeff6fded893f65397806d22ece928b0dd4 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 13:43:30 2011 -0400 add StorageFarmBroker.get_server_for_id(), temporary helper This will go away once we're passing IServers everywhere. src/allmydata/storage_client.py | 2 ++ src/allmydata/test/no_network.py | 13 +++++++++++++ 2 files changed, 15 insertions(+), 0 deletions(-) commit ece20231d7fda0d503704842a4aa068dfbc2e54e Author: Brian Warner <warner@lothar.com> Date: Sun Oct 2 01:11:50 2011 +0100 add proper accessors for Servermap.connections, to make refactoring easier src/allmydata/mutable/publish.py | 6 +++--- src/allmydata/mutable/retrieve.py | 10 +++++----- src/allmydata/mutable/servermap.py | 17 +++++++++++------ 3 files changed, 19 insertions(+), 14 deletions(-) commit 3b943d6bf302ff702668081a612fc4fe2604cf9c Author: Brian Warner <warner@lothar.com> Date: Fri Sep 23 10:34:30 2011 -0700 mutable/servermap.py and neighbors: s/peer/server/ src/allmydata/mutable/checker.py | 22 +- src/allmydata/mutable/publish.py | 204 +++++++------- src/allmydata/mutable/servermap.py | 402 +++++++++++++------------- src/allmydata/test/test_mutable.py | 18 +- 4 files changed, 323 insertions(+), 323 deletions(-) IServer refactoring: pass IServer instances around, instead of peerids refs #1363 This collapses 88 small incremental changes (each of which passes all tests) into one big patch. The development process for the long path started with adding some temporary scaffolding, changing one method at a time, then removing the scaffolding. The individual pieces are as follows, in reverse chronological order (the first patch is at the end of this comment): commit 9bbe4174fd0d98a6cf47a8ef96e85d9ef34b2f9a Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 16:05:00 2011 -0400 immutable/downloader/status.py: correct comment src/allmydata/immutable/downloader/status.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) commit 72146a7c7c91eac2f7c3ceb801eb7a1721376889 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 15:46:20 2011 -0400 remove temporary ServerMap._storage_broker src/allmydata/mutable/checker.py | 2 +- src/allmydata/mutable/filenode.py | 2 +- src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 5 ++--- src/allmydata/test/test_mutable.py | 8 ++++---- 5 files changed, 9 insertions(+), 10 deletions(-) commit d703096b41632c47d76414b12672e076a422ff5c Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 15:37:05 2011 -0400 remove temporary storage_broker.get_server_for_id() src/allmydata/storage_client.py | 3 --- src/allmydata/test/no_network.py | 13 ------------- 2 files changed, 0 insertions(+), 16 deletions(-) commit 620cc5d80882ef6f7decfd26af8a6c7c1ddf80d1 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:50:06 2011 -0400 API of Retrieve._try_to_validate_privkey(), trying to remove reader.server src/allmydata/mutable/retrieve.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) commit 92f43f856f4a8b36c207d1b190ed8699b5a4ecb4 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:48:08 2011 -0400 API of Retrieve._validate_block(), trying to remove reader.server src/allmydata/mutable/retrieve.py | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) commit 572d5070761861a2190349d1ed8d85dbc25698a5 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:36:58 2011 -0400 API of Retrieve._mark_bad_share(), trying to remove reader.server src/allmydata/mutable/retrieve.py | 21 +++++++++------------ 1 files changed, 9 insertions(+), 12 deletions(-) commit a793ff00c0de1e2eec7b46288fdf388c7a2bec89 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:06:13 2011 -0400 remove now-unused get_rref_for_serverid() src/allmydata/mutable/servermap.py | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) commit 1b9827cc9366bf90b93297fdd6832f2ad0480ce7 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:03:09 2011 -0400 Retrieve: stop adding .serverid attributes to readers src/allmydata/mutable/retrieve.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) commit 5d4e9d491b19e49d2e443a1dfff2c672842c36ef Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:03:34 2011 -0400 return value of Retrieve(verify=True) src/allmydata/mutable/checker.py | 11 ++++++----- src/allmydata/mutable/retrieve.py | 3 +-- 2 files changed, 7 insertions(+), 7 deletions(-) commit e9ab7978c384e1f677cb7779dc449b1044face82 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:54:23 2011 -0400 Retrieve._bad_shares (but not return value, used by Verifier) src/allmydata/mutable/retrieve.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) commit 2d91926de233ec5c881f30e36b4a30ad92ab42a9 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:51:23 2011 -0400 Publish: stop adding .serverid attributes to writers src/allmydata/mutable/publish.py | 9 ++------- 1 files changed, 2 insertions(+), 7 deletions(-) commit 47c7a0105dec7cbf4f7e0a3ce800bbb85b15df4a Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:56:33 2011 -0400 API of get_write_enabler() src/allmydata/mutable/filenode.py | 7 ++++--- src/allmydata/mutable/publish.py | 4 ++-- src/allmydata/test/no_network.py | 3 +++ 3 files changed, 9 insertions(+), 5 deletions(-) commit 9196a5c6590fdbfd660325ea8358b345887d3db0 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:46:24 2011 -0400 API of get_(renewal|cancel)_secret() src/allmydata/mutable/filenode.py | 14 ++++++++------ src/allmydata/mutable/publish.py | 8 ++++---- src/allmydata/mutable/servermap.py | 5 ++--- 3 files changed, 14 insertions(+), 13 deletions(-) commit de7c1552f8c163eff5b6d820b5fb3b21c1b47cb5 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:41:52 2011 -0400 API of CorruptShareError. Also comment out some related+unused test_web.py code src/allmydata/mutable/common.py | 13 +++++-------- src/allmydata/mutable/retrieve.py | 10 +++++----- src/allmydata/mutable/servermap.py | 8 +++----- src/allmydata/test/common.py | 13 ++++++++----- 4 files changed, 21 insertions(+), 23 deletions(-) commit 2c1c314046b620c16f1e66d030c150d768b7d01e Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:01:46 2011 -0400 API of ServerMap.mark_bad_share() src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/retrieve.py | 6 +++--- src/allmydata/mutable/servermap.py | 6 ++---- src/allmydata/test/test_mutable.py | 3 +-- 4 files changed, 7 insertions(+), 10 deletions(-) commit 1bed349030779fd0c378ae4e821384f953c6f6ff Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:11:17 2011 -0400 API+name of ServerMap.shares_on_server() : only for tests, so debug_ prefix src/allmydata/mutable/servermap.py | 7 ++----- src/allmydata/test/test_mutable.py | 6 +++--- 2 files changed, 5 insertions(+), 8 deletions(-) commit 2d32e448677d6b818692e801045d4115b29abf21 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:07:10 2011 -0400 API of ServerMap.all_servers_for_version() src/allmydata/mutable/servermap.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit 48f3204d1889c3e7179578125c4bdef515af3d6a Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:04:50 2011 -0400 internals of ServerMap methods that use make_versionmap(), remove temp copy src/allmydata/mutable/servermap.py | 28 +++++++++---------------- 1 files changed, 10 insertions(+), 18 deletions(-) commit 5c3da77b6c777a145bd5ddfaa4db849dc9495548 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:01:28 2011 -0400 API of ServerMap.make_versionmap() src/allmydata/mutable/checker.py | 4 ++-- src/allmydata/mutable/retrieve.py | 5 ++--- src/allmydata/mutable/servermap.py | 4 ++-- src/allmydata/test/test_mutable.py | 7 ++++--- 4 files changed, 10 insertions(+), 10 deletions(-) commit b6882ece49afb4c507d118af2db346fa329209dc Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 10:53:38 2011 -0400 make a copy of ServerMap.make_versionmap() (_make_versionmap2) for internal use src/allmydata/mutable/servermap.py | 18 +++++++++++++----- 1 files changed, 13 insertions(+), 5 deletions(-) commit 963f8e63faf32b950eb1b8103cd2ff16fe8f0151 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:45:58 2011 -0400 API of RetrieveStatus.add_problem() src/allmydata/mutable/retrieve.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 4976d29ffae565a048851601c29013bbae2976d8 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:45:05 2011 -0400 API of RetrieveStatus.add_fetch_timing() src/allmydata/mutable/retrieve.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit d057d3bbba72663ee148a8b916bc2d52be2e3982 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:44:04 2011 -0400 API of Retrieve.notify_server_corruption() src/allmydata/mutable/retrieve.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit 8a2a81e46671c860610e0e96d6add1a57551f22d Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:42:32 2011 -0400 remove unused _outstanding_queries src/allmydata/mutable/retrieve.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) commit 56d12cc9968d03ccd53764455c671122c4f391d1 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:40:57 2011 -0400 change Retrieve.remaining_sharemap src/allmydata/mutable/retrieve.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit 4f0b7af4821f43290bfc70f2b1fc30149ad81281 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 10:40:18 2011 -0400 accessor for PublishStatus._problems src/allmydata/mutable/publish.py | 4 +++- src/allmydata/web/status.py | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) commit 627087cf66d0b8cc519f4d551a967a7bd9b6a741 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 10:36:39 2011 -0400 accessor for RetrieveStatus._problems src/allmydata/mutable/retrieve.py | 8 ++++++-- src/allmydata/web/status.py | 2 +- 2 files changed, 7 insertions(+), 3 deletions(-) commit ca7dea81f03801b1c7353fc00ecba689268109cf Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:35:32 2011 -0400 add .server to "reader", so we can get at it later src/allmydata/mutable/retrieve.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 6ef516e24908ec195af084a7550d1921a5e983b0 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:32:32 2011 -0400 temporarily give Retrieve a _storage_broker, so it can map serverids to servers src/allmydata/mutable/checker.py | 3 ++- src/allmydata/mutable/filenode.py | 6 ++++-- src/allmydata/mutable/retrieve.py | 5 +++-- src/allmydata/test/test_mutable.py | 4 ++-- 4 files changed, 11 insertions(+), 7 deletions(-) commit afe08e4dd3f4ff9ff7e8a2a8d28b181e3625bcc9 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:21:51 2011 -0400 mutable/retrieve.py: s/peer/server/ src/allmydata/mutable/retrieve.py | 82 +++++++++++++------------- src/allmydata/test/test_mutable.py | 6 +- 2 files changed, 44 insertions(+), 44 deletions(-) commit 910afcb5d7f274880f68dd6cdb5b05f2bbc29adc Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:16:01 2011 -0400 web.status.PublishStatusPage: add comment, I think .problems isn't exercised src/allmydata/web/status.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) commit 311466dd8c931bbba40d590ade867704282e7f1a Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:48:16 2011 -0400 API of PublishStatus.add_per_server_time() src/allmydata/mutable/publish.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 2df5faa1b6cbfbaded520d2320305a62fe961118 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:46:37 2011 -0400 more simplifications src/allmydata/mutable/publish.py | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) commit 6ac4544a3da385f2aad9392f906b90192f4f919a Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:44:08 2011 -0400 API of ServerMap.version_on_server() src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 4 ++-- src/allmydata/test/test_mutable.py | 5 ++--- 3 files changed, 5 insertions(+), 6 deletions(-) commit 3e187e322511072e4683329df6b2c6c733a66dba Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:16:32 2011 -0400 API of ServerMap.make_sharemap() src/allmydata/mutable/servermap.py | 4 ++-- src/allmydata/test/test_mutable.py | 7 ++++--- src/allmydata/web/status.py | 4 ++-- 3 files changed, 8 insertions(+), 7 deletions(-) commit 318feed8437bdd8d4943c6569d38f7b54b6313cc Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:36:19 2011 -0400 small cleanups src/allmydata/mutable/publish.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit bd459ed5714e1db5a7163935c54b7b0b56db8349 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:33:39 2011 -0400 API of ServerMap.add_new_share() src/allmydata/mutable/publish.py | 4 ++-- src/allmydata/mutable/servermap.py | 6 ++---- 2 files changed, 4 insertions(+), 6 deletions(-) commit f2804fb6ed11d80088e0da8ed48e6c2922f2ffef Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:30:26 2011 -0400 API of ServerMap.get_bad_shares() src/allmydata/mutable/publish.py | 3 +-- src/allmydata/mutable/servermap.py | 9 ++++----- 2 files changed, 5 insertions(+), 7 deletions(-) commit 965074a47b3ce1431cb46d9a233840afcf9105f5 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:26:58 2011 -0400 more small cleanups src/allmydata/mutable/publish.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit 38020da34f034f8889947dd3dc05e087ffff7106 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:18:47 2011 -0400 change Publish.bad_share_checkstrings src/allmydata/mutable/publish.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit 5efebcbd2ee0c2f299ea86f7591d856c0f265304 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:16:31 2011 -0400 change internals of Publish.update_goal() src/allmydata/mutable/publish.py | 8 +++----- 1 files changed, 3 insertions(+), 5 deletions(-) commit e91b55ff4c2a69165b71f2c7b217ac319ff4c527 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:11:42 2011 -0400 get rid of Publish.connections src/allmydata/mutable/publish.py | 27 +++++---------------------- 1 files changed, 5 insertions(+), 22 deletions(-) commit 64e9a53b3229ebe2f9ebf7ed502d539311d0e037 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:05:32 2011 -0400 change Publish.bad_servers src/allmydata/mutable/publish.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) commit b85a934bef315a06bcfe00c9c12a3627fed2b918 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:03:07 2011 -0400 Publish.bad_servers: fix bug, this should be a set of serverids, not writers src/allmydata/mutable/publish.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) commit 605ea15ec15ed671513819003ccd211cdb9761e0 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:00:21 2011 -0400 change .placed src/allmydata/mutable/publish.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit f7aba37b1b345d5b6d5cb16e3b3f6f3c1afb658e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 22:59:22 2011 -0400 temporarily stash IServer as .server on the "writer" object src/allmydata/mutable/publish.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) commit f9b551d788e7db1f187fce5ab98ab5d5fe4e1c36 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 22:48:18 2011 -0400 change Publish.goal and API of log_goal() to use IServer, not serverid src/allmydata/mutable/publish.py | 48 ++++++++++++++-------------- 1 files changed, 24 insertions(+), 24 deletions(-) commit 75f20616558e4900b8b1f685dd99aa838de6d452 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:27:02 2011 -0400 API of ServerMap.get_known_shares() src/allmydata/mutable/publish.py | 16 ++++++++++------ src/allmydata/mutable/servermap.py | 7 ++----- 2 files changed, 12 insertions(+), 11 deletions(-) commit 1c38c9d37bb08221b4418762234b1a62397b3b4b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:20:29 2011 -0400 Publish.full_serverlist src/allmydata/mutable/publish.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) commit b6cbd215a04b9cde31a7d92a97a7f048622b16f1 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:12:31 2011 -0400 API of ServerMap.all_servers() src/allmydata/mutable/servermap.py | 19 ++++++------------- 1 files changed, 6 insertions(+), 13 deletions(-) commit e63cd0315fae65357b1727ec6d5ff3c6e0d27c98 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:10:18 2011 -0400 remove ServerMap.connections, set_rref_for_serverid() src/allmydata/mutable/servermap.py | 11 +---------- 1 files changed, 1 insertions(+), 10 deletions(-) commit 4df52db2f80eb12eefa5d57103c24893cde89553 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:04:06 2011 -0400 API of ServerMap.mark_server_reachable() src/allmydata/mutable/servermap.py | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-) commit 69c715bde77944dc25181b3dbbeb042c816f9a1b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:03:21 2011 -0400 API of ServerMap.mark_server_unreachable() src/allmydata/mutable/servermap.py | 9 +++------ 1 files changed, 3 insertions(+), 6 deletions(-) commit 3d784d60eec1c508858e3a617e4411ffbcc3c1fa Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:02:03 2011 -0400 API of status.set_privkey_from() src/allmydata/mutable/servermap.py | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) commit 544ed3ea29bed7e66da7fd29ca3f6f076f27a9e6 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:01:15 2011 -0400 API of status.add_per_server_time() src/allmydata/mutable/servermap.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) commit fffe5008b6320bd1e04c3c68389a2bf2ee383fa8 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:59:02 2011 -0400 remove unused .versionmap src/allmydata/mutable/servermap.py | 7 ------- 1 files changed, 0 insertions(+), 7 deletions(-) commit 2816562e090d2294179db3588dafcca18de1bc2b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:57:51 2011 -0400 remove serverid from all log messages. Also one unused lambda. src/allmydata/mutable/servermap.py | 30 +++++++++++++------------- 1 files changed, 15 insertions(+), 15 deletions(-) commit 28fa6b1a2738fa98c1f1dbd3d0e01ae98912d11f Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:54:30 2011 -0400 removed unused _readers src/allmydata/mutable/servermap.py | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) commit a8e4ed3d645ab592d1add6a1e69b6d1ebfb77817 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:54:16 2011 -0400 remove unused _sharemap src/allmydata/mutable/servermap.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) commit 3f072e55cf1d0700f9fffe23f8f3a475725df588 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:49:03 2011 -0400 _must_query src/allmydata/mutable/servermap.py | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) commit c599a059b8df3f5785e4bf89fb6ecc6d8dcd708b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:48:05 2011 -0400 _queries_outstanding src/allmydata/mutable/servermap.py | 16 +++++++--------- 1 files changed, 7 insertions(+), 9 deletions(-) commit 7743759f98ac2c07926b2fdbd80bf52dfab33085 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:46:17 2011 -0400 _empty_servers src/allmydata/mutable/servermap.py | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) commit 6bb1825916828a713a32cdf7f7411fa3ea2e1e5d Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:45:39 2011 -0400 _good_servers src/allmydata/mutable/servermap.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit 1768fab1b51d8dd93ecabbaaabfadfa20cf6c3d4 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:44:59 2011 -0400 _bad_servers src/allmydata/mutable/servermap.py | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) commit dccbaef30f0ba714c746bf6d4a1a803c36e17b65 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:41:54 2011 -0400 API of _try_to_set_pubkey() src/allmydata/mutable/servermap.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) commit 0481ea70042ba3575f15eac7fd0780f8ece580cc Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:35:02 2011 -0400 API of notify_server_corruption() src/allmydata/mutable/servermap.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit bea9cba18fb3b9c11bb22f18356a263ecec7351e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:34:09 2011 -0400 API of _got_signature_one_share() src/allmydata/mutable/servermap.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) commit 1520123583cf78650706e114b15bb5b0ac1f4a14 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:32:33 2011 -0400 API of _try_to_validate_privkey() src/allmydata/mutable/servermap.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) commit 938852c9c8519c7a078f58a9b1f4dd8ec8b6715e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:31:48 2011 -0400 API and internals of _add_lease_failed() src/allmydata/mutable/servermap.py | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) commit 3843dba367e3c19e176a622ab853cb51d2472ddf Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:30:37 2011 -0400 API of _privkey_query_failed() src/allmydata/mutable/servermap.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 2219a710e1633cd57d0ca0786490de87b3e19ba7 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:29:43 2011 -0400 fix bug in call to _privkey_query_failed, unrelated to refactoring src/allmydata/mutable/servermap.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) commit ae615bec7d0d1b269710b6902797b12f9592ad62 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:27:17 2011 -0400 API of _got_corrupt_share() src/allmydata/mutable/servermap.py | 17 +++++++++-------- 1 files changed, 9 insertions(+), 8 deletions(-) commit cb51c95a6f4e077278157a77dab060c8c1ad7a81 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:23:16 2011 -0400 API of _got_results() src/allmydata/mutable/servermap.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) commit bac9154fe0af18f226999a58ffc2362d8cf4b802 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:19:19 2011 -0400 API of _query_failed() src/allmydata/mutable/servermap.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit fdc29a8ca95d4b5c503e5382b9e5d4d02141ba12 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:17:20 2011 -0400 API of _do_read() src/allmydata/mutable/servermap.py | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) commit e7e9e338f28d004aa4d423d11c65f1e271ac7322 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:20:21 2011 -0400 API of _do_query() src/allmydata/mutable/servermap.py | 15 +++++++-------- 1 files changed, 7 insertions(+), 8 deletions(-) commit 330625b9dac4cdbe72a11464a893065b9aeed453 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:43:05 2011 -0400 next step: first batch of updates to ServermapUpdater updates: most method-local variables in update() API of _build_initial_querylist() API of _send_initial_requests() .full_serverlist .extra_servers src/allmydata/mutable/servermap.py | 39 ++++++++++++++------------ 1 files changed, 21 insertions(+), 18 deletions(-) commit 4aadc584fa7dcb2daa86b048c81dee0049ba26d9 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:07:00 2011 -0400 internal change: index _bad_shares with IServer src/allmydata/mutable/servermap.py | 20 ++++++++++---------- 1 files changed, 10 insertions(+), 10 deletions(-) commit 16d4e6fa82a9907dbdc92094213387c6a4164e41 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:20:47 2011 +0100 internal change: index _known_shares with IServer instead of serverid callers are unchanged src/allmydata/mutable/servermap.py | 42 +++++++++++++++---------- 1 files changed, 25 insertions(+), 17 deletions(-) commit ceeb5f4938cc814a0c75d1b8f4018aed965c2176 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:11:43 2011 +0100 accessors and name cleanup for servermap.Servermap.last_update_mode/time src/allmydata/mutable/filenode.py | 6 +++--- src/allmydata/mutable/publish.py | 4 ++-- src/allmydata/mutable/servermap.py | 17 +++++++++++------ 3 files changed, 16 insertions(+), 11 deletions(-) commit 8d3cbda82661c0a7e5c3d3b65cf7a5d5ab7e32c0 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:11:14 2011 +0100 accessors and name cleanup for servermap.Servermap.problems src/allmydata/mutable/servermap.py | 21 +++++++++++++-------- src/allmydata/test/test_mutable.py | 6 +++--- 2 files changed, 16 insertions(+), 11 deletions(-) commit 348f57988f79389db0aab7672e6eaa9a6d8e3219 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:10:41 2011 +0100 accessors and name cleanup for servermap.Servermap.bad_shares src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 30 ++++++++++++++----------- 2 files changed, 18 insertions(+), 14 deletions(-) commit 520c9368134673cdf76c653c5e1bb91c2ab5d51e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:10:05 2011 +0100 accessors and name cleanup for servermap.Servermap.servermap . src/allmydata/mutable/publish.py | 14 +++++---- src/allmydata/mutable/servermap.py | 38 ++++++++++++++----------- 2 files changed, 29 insertions(+), 23 deletions(-) commit b8b8dc38287a91dbdf494426ac801d9381ce5841 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:08:02 2011 +0100 fix reachable_servers src/allmydata/mutable/checker.py | 3 ++- src/allmydata/mutable/publish.py | 4 +++- src/allmydata/mutable/servermap.py | 12 ++++++++++-- 3 files changed, 15 insertions(+), 4 deletions(-) commit cb0cfd1adfefad357c187aaaf690c3df68b622bc Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:06:03 2011 +0100 fix Servermap.unreachable_servers src/allmydata/mutable/servermap.py | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-) commit 2d9ea79b94bd4db674d40386fda90825785ac495 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:03:48 2011 +0100 give ServerMap a StorageFarmBroker, temporary this makes it possible for the ServerMap to accept bare serverids and still build data structures with IServers src/allmydata/mutable/checker.py | 2 +- src/allmydata/mutable/filenode.py | 2 +- src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 5 +++-- src/allmydata/test/test_mutable.py | 8 ++++---- 5 files changed, 10 insertions(+), 9 deletions(-) commit 718d1aeff6fded893f65397806d22ece928b0dd4 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 13:43:30 2011 -0400 add StorageFarmBroker.get_server_for_id(), temporary helper This will go away once we're passing IServers everywhere. src/allmydata/storage_client.py | 2 ++ src/allmydata/test/no_network.py | 13 +++++++++++++ 2 files changed, 15 insertions(+), 0 deletions(-) commit ece20231d7fda0d503704842a4aa068dfbc2e54e Author: Brian Warner <warner@lothar.com> Date: Sun Oct 2 01:11:50 2011 +0100 add proper accessors for Servermap.connections, to make refactoring easier src/allmydata/mutable/publish.py | 6 +++--- src/allmydata/mutable/retrieve.py | 10 +++++----- src/allmydata/mutable/servermap.py | 17 +++++++++++------ 3 files changed, 19 insertions(+), 14 deletions(-) commit 3b943d6bf302ff702668081a612fc4fe2604cf9c Author: Brian Warner <warner@lothar.com> Date: Fri Sep 23 10:34:30 2011 -0700 mutable/servermap.py and neighbors: s/peer/server/ src/allmydata/mutable/checker.py | 22 +- src/allmydata/mutable/publish.py | 204 +++++++------- src/allmydata/mutable/servermap.py | 402 +++++++++++++------------- src/allmydata/test/test_mutable.py | 18 +- 4 files changed, 323 insertions(+), 323 deletions(-) |
||
Zooko O'Whielacronx
|
299e8ad579 |
replace remaining .html docs with .rst docs
Remove install.html (long since deprecated). Also replace some obsolete references to install.html with references to quickstart.rst. Fix some broken internal references within docs/historical/historical_known_issues.txt. Thanks to Ravi Pinjala and Patrick McDonald. refs #1227 |
||
david-sarah
|
4061258c85 | make ResponseCache smarter to avoid memory leaks: don't record timestamps, use DataSpans to merge entries, and clear the cache when we see a new seqnum. refs #1045, #1229 | ||
david-sarah
|
6057bc02cc | Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements. | ||
Brian Warner
|
731d15e56f | hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900. | ||
Brian Warner
|
d8ba8c2eb5 |
Allow tests to pass with -OO by turning some AssertionErrors (the ones that
we actually exercise during tests) into more specific exceptions, so they don't get optimized away. The best rule to follow is probably this: if an exception is worth testing, then it's part of the API, and AssertionError should never be part of the API. Closes #749. |
||
Brian Warner
|
d29281c9c5 | mutable: catch and display first error, so code bugs which break all servers get displayed better | ||
Brian Warner
|
fe44b8fb4b | util.dictutil: move DictOfSets out to a separate utility module | ||
Brian Warner
|
13eb00145e | mutable: improve test coverage in Retrieve, when shares change after mapupdate | ||
Brian Warner
|
e6074f5dfc | mutable: test write failures, uncoordinated write detection | ||
Brian Warner
|
a04c2866b1 | mutable: improve test coverage slightly | ||
Brian Warner
|
4f07d96667 | mutable WIP: improve logging a bit | ||
Brian Warner
|
d4230d1781 | mutable WIP: split mutable.py into separate files. All tests pass. |