3248 Commits

Author	SHA1	Message	Date
Brian Warner	7da112ccef	Rewrite download-status-timeline visualizer ('viz') with d3.js ... * use d3.js v2.4.6 * add a "toggle misc events" button, to get hash/bitmap-checking details * only draw data that's on screen, for speed * add fragment-arg to fetch timeline data.json from somewhere else	2011-10-31 23:18:21 -07:00
Brian Warner	5bf1ffbc87	IServer refactoring: pass IServer instances around, instead of peerids ... refs #1363 This collapses 88 small incremental changes (each of which passes all tests) into one big patch. The development process for the long path started with adding some temporary scaffolding, changing one method at a time, then removing the scaffolding. The individual pieces are as follows, in reverse chronological order (the first patch is at the end of this comment): commit 9bbe4174fd0d98a6cf47a8ef96e85d9ef34b2f9a Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 16:05:00 2011 -0400 immutable/downloader/status.py: correct comment src/allmydata/immutable/downloader/status.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) commit 72146a7c7c91eac2f7c3ceb801eb7a1721376889 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 15:46:20 2011 -0400 remove temporary ServerMap._storage_broker src/allmydata/mutable/checker.py | 2 +- src/allmydata/mutable/filenode.py | 2 +- src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 5 ++--- src/allmydata/test/test_mutable.py | 8 ++++---- 5 files changed, 9 insertions(+), 10 deletions(-) commit d703096b41632c47d76414b12672e076a422ff5c Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 15:37:05 2011 -0400 remove temporary storage_broker.get_server_for_id() src/allmydata/storage_client.py | 3 --- src/allmydata/test/no_network.py | 13 ------------- 2 files changed, 0 insertions(+), 16 deletions(-) commit 620cc5d80882ef6f7decfd26af8a6c7c1ddf80d1 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:50:06 2011 -0400 API of Retrieve._try_to_validate_privkey(), trying to remove reader.server src/allmydata/mutable/retrieve.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) commit 92f43f856f4a8b36c207d1b190ed8699b5a4ecb4 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:48:08 2011 -0400 API of Retrieve._validate_block(), trying to remove reader.server src/allmydata/mutable/retrieve.py | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) commit 572d5070761861a2190349d1ed8d85dbc25698a5 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:36:58 2011 -0400 API of Retrieve._mark_bad_share(), trying to remove reader.server src/allmydata/mutable/retrieve.py | 21 +++++++++------------ 1 files changed, 9 insertions(+), 12 deletions(-) commit a793ff00c0de1e2eec7b46288fdf388c7a2bec89 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:06:13 2011 -0400 remove now-unused get_rref_for_serverid() src/allmydata/mutable/servermap.py | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) commit 1b9827cc9366bf90b93297fdd6832f2ad0480ce7 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:03:09 2011 -0400 Retrieve: stop adding .serverid attributes to readers src/allmydata/mutable/retrieve.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) commit 5d4e9d491b19e49d2e443a1dfff2c672842c36ef Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:03:34 2011 -0400 return value of Retrieve(verify=True) src/allmydata/mutable/checker.py | 11 ++++++----- src/allmydata/mutable/retrieve.py | 3 +-- 2 files changed, 7 insertions(+), 7 deletions(-) commit e9ab7978c384e1f677cb7779dc449b1044face82 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:54:23 2011 -0400 Retrieve._bad_shares (but not return value, used by Verifier) src/allmydata/mutable/retrieve.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) commit 2d91926de233ec5c881f30e36b4a30ad92ab42a9 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:51:23 2011 -0400 Publish: stop adding .serverid attributes to writers src/allmydata/mutable/publish.py | 9 ++------- 1 files changed, 2 insertions(+), 7 deletions(-) commit 47c7a0105dec7cbf4f7e0a3ce800bbb85b15df4a Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:56:33 2011 -0400 API of get_write_enabler() src/allmydata/mutable/filenode.py | 7 ++++--- src/allmydata/mutable/publish.py | 4 ++-- src/allmydata/test/no_network.py | 3 +++ 3 files changed, 9 insertions(+), 5 deletions(-) commit 9196a5c6590fdbfd660325ea8358b345887d3db0 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:46:24 2011 -0400 API of get_(renewal|cancel)_secret() src/allmydata/mutable/filenode.py | 14 ++++++++------ src/allmydata/mutable/publish.py | 8 ++++---- src/allmydata/mutable/servermap.py | 5 ++--- 3 files changed, 14 insertions(+), 13 deletions(-) commit de7c1552f8c163eff5b6d820b5fb3b21c1b47cb5 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:41:52 2011 -0400 API of CorruptShareError. Also comment out some related+unused test_web.py code src/allmydata/mutable/common.py | 13 +++++-------- src/allmydata/mutable/retrieve.py | 10 +++++----- src/allmydata/mutable/servermap.py | 8 +++----- src/allmydata/test/common.py | 13 ++++++++----- 4 files changed, 21 insertions(+), 23 deletions(-) commit 2c1c314046b620c16f1e66d030c150d768b7d01e Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:01:46 2011 -0400 API of ServerMap.mark_bad_share() src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/retrieve.py | 6 +++--- src/allmydata/mutable/servermap.py | 6 ++---- src/allmydata/test/test_mutable.py | 3 +-- 4 files changed, 7 insertions(+), 10 deletions(-) commit 1bed349030779fd0c378ae4e821384f953c6f6ff Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:11:17 2011 -0400 API+name of ServerMap.shares_on_server() : only for tests, so debug_ prefix src/allmydata/mutable/servermap.py | 7 ++----- src/allmydata/test/test_mutable.py | 6 +++--- 2 files changed, 5 insertions(+), 8 deletions(-) commit 2d32e448677d6b818692e801045d4115b29abf21 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:07:10 2011 -0400 API of ServerMap.all_servers_for_version() src/allmydata/mutable/servermap.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit 48f3204d1889c3e7179578125c4bdef515af3d6a Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:04:50 2011 -0400 internals of ServerMap methods that use make_versionmap(), remove temp copy src/allmydata/mutable/servermap.py | 28 +++++++++---------------- 1 files changed, 10 insertions(+), 18 deletions(-) commit 5c3da77b6c777a145bd5ddfaa4db849dc9495548 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:01:28 2011 -0400 API of ServerMap.make_versionmap() src/allmydata/mutable/checker.py | 4 ++-- src/allmydata/mutable/retrieve.py | 5 ++--- src/allmydata/mutable/servermap.py | 4 ++-- src/allmydata/test/test_mutable.py | 7 ++++--- 4 files changed, 10 insertions(+), 10 deletions(-) commit b6882ece49afb4c507d118af2db346fa329209dc Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 10:53:38 2011 -0400 make a copy of ServerMap.make_versionmap() (_make_versionmap2) for internal use src/allmydata/mutable/servermap.py | 18 +++++++++++++----- 1 files changed, 13 insertions(+), 5 deletions(-) commit 963f8e63faf32b950eb1b8103cd2ff16fe8f0151 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:45:58 2011 -0400 API of RetrieveStatus.add_problem() src/allmydata/mutable/retrieve.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 4976d29ffae565a048851601c29013bbae2976d8 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:45:05 2011 -0400 API of RetrieveStatus.add_fetch_timing() src/allmydata/mutable/retrieve.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit d057d3bbba72663ee148a8b916bc2d52be2e3982 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:44:04 2011 -0400 API of Retrieve.notify_server_corruption() src/allmydata/mutable/retrieve.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit 8a2a81e46671c860610e0e96d6add1a57551f22d Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:42:32 2011 -0400 remove unused _outstanding_queries src/allmydata/mutable/retrieve.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) commit 56d12cc9968d03ccd53764455c671122c4f391d1 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:40:57 2011 -0400 change Retrieve.remaining_sharemap src/allmydata/mutable/retrieve.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit 4f0b7af4821f43290bfc70f2b1fc30149ad81281 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 10:40:18 2011 -0400 accessor for PublishStatus._problems src/allmydata/mutable/publish.py | 4 +++- src/allmydata/web/status.py | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) commit 627087cf66d0b8cc519f4d551a967a7bd9b6a741 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 10:36:39 2011 -0400 accessor for RetrieveStatus._problems src/allmydata/mutable/retrieve.py | 8 ++++++-- src/allmydata/web/status.py | 2 +- 2 files changed, 7 insertions(+), 3 deletions(-) commit ca7dea81f03801b1c7353fc00ecba689268109cf Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:35:32 2011 -0400 add .server to "reader", so we can get at it later src/allmydata/mutable/retrieve.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 6ef516e24908ec195af084a7550d1921a5e983b0 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:32:32 2011 -0400 temporarily give Retrieve a _storage_broker, so it can map serverids to servers src/allmydata/mutable/checker.py | 3 ++- src/allmydata/mutable/filenode.py | 6 ++++-- src/allmydata/mutable/retrieve.py | 5 +++-- src/allmydata/test/test_mutable.py | 4 ++-- 4 files changed, 11 insertions(+), 7 deletions(-) commit afe08e4dd3f4ff9ff7e8a2a8d28b181e3625bcc9 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:21:51 2011 -0400 mutable/retrieve.py: s/peer/server/ src/allmydata/mutable/retrieve.py | 82 +++++++++++++------------- src/allmydata/test/test_mutable.py | 6 +- 2 files changed, 44 insertions(+), 44 deletions(-) commit 910afcb5d7f274880f68dd6cdb5b05f2bbc29adc Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:16:01 2011 -0400 web.status.PublishStatusPage: add comment, I think .problems isn't exercised src/allmydata/web/status.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) commit 311466dd8c931bbba40d590ade867704282e7f1a Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:48:16 2011 -0400 API of PublishStatus.add_per_server_time() src/allmydata/mutable/publish.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 2df5faa1b6cbfbaded520d2320305a62fe961118 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:46:37 2011 -0400 more simplifications src/allmydata/mutable/publish.py | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) commit 6ac4544a3da385f2aad9392f906b90192f4f919a Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:44:08 2011 -0400 API of ServerMap.version_on_server() src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 4 ++-- src/allmydata/test/test_mutable.py | 5 ++--- 3 files changed, 5 insertions(+), 6 deletions(-) commit 3e187e322511072e4683329df6b2c6c733a66dba Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:16:32 2011 -0400 API of ServerMap.make_sharemap() src/allmydata/mutable/servermap.py | 4 ++-- src/allmydata/test/test_mutable.py | 7 ++++--- src/allmydata/web/status.py | 4 ++-- 3 files changed, 8 insertions(+), 7 deletions(-) commit 318feed8437bdd8d4943c6569d38f7b54b6313cc Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:36:19 2011 -0400 small cleanups src/allmydata/mutable/publish.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit bd459ed5714e1db5a7163935c54b7b0b56db8349 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:33:39 2011 -0400 API of ServerMap.add_new_share() src/allmydata/mutable/publish.py | 4 ++-- src/allmydata/mutable/servermap.py | 6 ++---- 2 files changed, 4 insertions(+), 6 deletions(-) commit f2804fb6ed11d80088e0da8ed48e6c2922f2ffef Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:30:26 2011 -0400 API of ServerMap.get_bad_shares() src/allmydata/mutable/publish.py | 3 +-- src/allmydata/mutable/servermap.py | 9 ++++----- 2 files changed, 5 insertions(+), 7 deletions(-) commit 965074a47b3ce1431cb46d9a233840afcf9105f5 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:26:58 2011 -0400 more small cleanups src/allmydata/mutable/publish.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit 38020da34f034f8889947dd3dc05e087ffff7106 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:18:47 2011 -0400 change Publish.bad_share_checkstrings src/allmydata/mutable/publish.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit 5efebcbd2ee0c2f299ea86f7591d856c0f265304 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:16:31 2011 -0400 change internals of Publish.update_goal() src/allmydata/mutable/publish.py | 8 +++----- 1 files changed, 3 insertions(+), 5 deletions(-) commit e91b55ff4c2a69165b71f2c7b217ac319ff4c527 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:11:42 2011 -0400 get rid of Publish.connections src/allmydata/mutable/publish.py | 27 +++++---------------------- 1 files changed, 5 insertions(+), 22 deletions(-) commit 64e9a53b3229ebe2f9ebf7ed502d539311d0e037 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:05:32 2011 -0400 change Publish.bad_servers src/allmydata/mutable/publish.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) commit b85a934bef315a06bcfe00c9c12a3627fed2b918 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:03:07 2011 -0400 Publish.bad_servers: fix bug, this should be a set of serverids, not writers src/allmydata/mutable/publish.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) commit 605ea15ec15ed671513819003ccd211cdb9761e0 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:00:21 2011 -0400 change .placed src/allmydata/mutable/publish.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit f7aba37b1b345d5b6d5cb16e3b3f6f3c1afb658e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 22:59:22 2011 -0400 temporarily stash IServer as .server on the "writer" object src/allmydata/mutable/publish.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) commit f9b551d788e7db1f187fce5ab98ab5d5fe4e1c36 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 22:48:18 2011 -0400 change Publish.goal and API of log_goal() to use IServer, not serverid src/allmydata/mutable/publish.py | 48 ++++++++++++++-------------- 1 files changed, 24 insertions(+), 24 deletions(-) commit 75f20616558e4900b8b1f685dd99aa838de6d452 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:27:02 2011 -0400 API of ServerMap.get_known_shares() src/allmydata/mutable/publish.py | 16 ++++++++++------ src/allmydata/mutable/servermap.py | 7 ++----- 2 files changed, 12 insertions(+), 11 deletions(-) commit 1c38c9d37bb08221b4418762234b1a62397b3b4b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:20:29 2011 -0400 Publish.full_serverlist src/allmydata/mutable/publish.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) commit b6cbd215a04b9cde31a7d92a97a7f048622b16f1 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:12:31 2011 -0400 API of ServerMap.all_servers() src/allmydata/mutable/servermap.py | 19 ++++++------------- 1 files changed, 6 insertions(+), 13 deletions(-) commit e63cd0315fae65357b1727ec6d5ff3c6e0d27c98 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:10:18 2011 -0400 remove ServerMap.connections, set_rref_for_serverid() src/allmydata/mutable/servermap.py | 11 +---------- 1 files changed, 1 insertions(+), 10 deletions(-) commit 4df52db2f80eb12eefa5d57103c24893cde89553 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:04:06 2011 -0400 API of ServerMap.mark_server_reachable() src/allmydata/mutable/servermap.py | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-) commit 69c715bde77944dc25181b3dbbeb042c816f9a1b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:03:21 2011 -0400 API of ServerMap.mark_server_unreachable() src/allmydata/mutable/servermap.py | 9 +++------ 1 files changed, 3 insertions(+), 6 deletions(-) commit 3d784d60eec1c508858e3a617e4411ffbcc3c1fa Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:02:03 2011 -0400 API of status.set_privkey_from() src/allmydata/mutable/servermap.py | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) commit 544ed3ea29bed7e66da7fd29ca3f6f076f27a9e6 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:01:15 2011 -0400 API of status.add_per_server_time() src/allmydata/mutable/servermap.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) commit fffe5008b6320bd1e04c3c68389a2bf2ee383fa8 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:59:02 2011 -0400 remove unused .versionmap src/allmydata/mutable/servermap.py | 7 ------- 1 files changed, 0 insertions(+), 7 deletions(-) commit 2816562e090d2294179db3588dafcca18de1bc2b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:57:51 2011 -0400 remove serverid from all log messages. Also one unused lambda. src/allmydata/mutable/servermap.py | 30 +++++++++++++------------- 1 files changed, 15 insertions(+), 15 deletions(-) commit 28fa6b1a2738fa98c1f1dbd3d0e01ae98912d11f Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:54:30 2011 -0400 removed unused _readers src/allmydata/mutable/servermap.py | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) commit a8e4ed3d645ab592d1add6a1e69b6d1ebfb77817 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:54:16 2011 -0400 remove unused _sharemap src/allmydata/mutable/servermap.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) commit 3f072e55cf1d0700f9fffe23f8f3a475725df588 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:49:03 2011 -0400 _must_query src/allmydata/mutable/servermap.py | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) commit c599a059b8df3f5785e4bf89fb6ecc6d8dcd708b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:48:05 2011 -0400 _queries_outstanding src/allmydata/mutable/servermap.py | 16 +++++++--------- 1 files changed, 7 insertions(+), 9 deletions(-) commit 7743759f98ac2c07926b2fdbd80bf52dfab33085 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:46:17 2011 -0400 _empty_servers src/allmydata/mutable/servermap.py | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) commit 6bb1825916828a713a32cdf7f7411fa3ea2e1e5d Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:45:39 2011 -0400 _good_servers src/allmydata/mutable/servermap.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit 1768fab1b51d8dd93ecabbaaabfadfa20cf6c3d4 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:44:59 2011 -0400 _bad_servers src/allmydata/mutable/servermap.py | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) commit dccbaef30f0ba714c746bf6d4a1a803c36e17b65 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:41:54 2011 -0400 API of _try_to_set_pubkey() src/allmydata/mutable/servermap.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) commit 0481ea70042ba3575f15eac7fd0780f8ece580cc Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:35:02 2011 -0400 API of notify_server_corruption() src/allmydata/mutable/servermap.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit bea9cba18fb3b9c11bb22f18356a263ecec7351e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:34:09 2011 -0400 API of _got_signature_one_share() src/allmydata/mutable/servermap.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) commit 1520123583cf78650706e114b15bb5b0ac1f4a14 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:32:33 2011 -0400 API of _try_to_validate_privkey() src/allmydata/mutable/servermap.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) commit 938852c9c8519c7a078f58a9b1f4dd8ec8b6715e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:31:48 2011 -0400 API and internals of _add_lease_failed() src/allmydata/mutable/servermap.py | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) commit 3843dba367e3c19e176a622ab853cb51d2472ddf Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:30:37 2011 -0400 API of _privkey_query_failed() src/allmydata/mutable/servermap.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 2219a710e1633cd57d0ca0786490de87b3e19ba7 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:29:43 2011 -0400 fix bug in call to _privkey_query_failed, unrelated to refactoring src/allmydata/mutable/servermap.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) commit ae615bec7d0d1b269710b6902797b12f9592ad62 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:27:17 2011 -0400 API of _got_corrupt_share() src/allmydata/mutable/servermap.py | 17 +++++++++-------- 1 files changed, 9 insertions(+), 8 deletions(-) commit cb51c95a6f4e077278157a77dab060c8c1ad7a81 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:23:16 2011 -0400 API of _got_results() src/allmydata/mutable/servermap.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) commit bac9154fe0af18f226999a58ffc2362d8cf4b802 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:19:19 2011 -0400 API of _query_failed() src/allmydata/mutable/servermap.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit fdc29a8ca95d4b5c503e5382b9e5d4d02141ba12 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:17:20 2011 -0400 API of _do_read() src/allmydata/mutable/servermap.py | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) commit e7e9e338f28d004aa4d423d11c65f1e271ac7322 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:20:21 2011 -0400 API of _do_query() src/allmydata/mutable/servermap.py | 15 +++++++-------- 1 files changed, 7 insertions(+), 8 deletions(-) commit 330625b9dac4cdbe72a11464a893065b9aeed453 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:43:05 2011 -0400 next step: first batch of updates to ServermapUpdater updates: most method-local variables in update() API of _build_initial_querylist() API of _send_initial_requests() .full_serverlist .extra_servers src/allmydata/mutable/servermap.py | 39 ++++++++++++++------------ 1 files changed, 21 insertions(+), 18 deletions(-) commit 4aadc584fa7dcb2daa86b048c81dee0049ba26d9 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:07:00 2011 -0400 internal change: index _bad_shares with IServer src/allmydata/mutable/servermap.py | 20 ++++++++++---------- 1 files changed, 10 insertions(+), 10 deletions(-) commit 16d4e6fa82a9907dbdc92094213387c6a4164e41 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:20:47 2011 +0100 internal change: index _known_shares with IServer instead of serverid callers are unchanged src/allmydata/mutable/servermap.py | 42 +++++++++++++++---------- 1 files changed, 25 insertions(+), 17 deletions(-) commit ceeb5f4938cc814a0c75d1b8f4018aed965c2176 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:11:43 2011 +0100 accessors and name cleanup for servermap.Servermap.last_update_mode/time src/allmydata/mutable/filenode.py | 6 +++--- src/allmydata/mutable/publish.py | 4 ++-- src/allmydata/mutable/servermap.py | 17 +++++++++++------ 3 files changed, 16 insertions(+), 11 deletions(-) commit 8d3cbda82661c0a7e5c3d3b65cf7a5d5ab7e32c0 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:11:14 2011 +0100 accessors and name cleanup for servermap.Servermap.problems src/allmydata/mutable/servermap.py | 21 +++++++++++++-------- src/allmydata/test/test_mutable.py | 6 +++--- 2 files changed, 16 insertions(+), 11 deletions(-) commit 348f57988f79389db0aab7672e6eaa9a6d8e3219 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:10:41 2011 +0100 accessors and name cleanup for servermap.Servermap.bad_shares src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 30 ++++++++++++++----------- 2 files changed, 18 insertions(+), 14 deletions(-) commit 520c9368134673cdf76c653c5e1bb91c2ab5d51e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:10:05 2011 +0100 accessors and name cleanup for servermap.Servermap.servermap . src/allmydata/mutable/publish.py | 14 +++++---- src/allmydata/mutable/servermap.py | 38 ++++++++++++++----------- 2 files changed, 29 insertions(+), 23 deletions(-) commit b8b8dc38287a91dbdf494426ac801d9381ce5841 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:08:02 2011 +0100 fix reachable_servers src/allmydata/mutable/checker.py | 3 ++- src/allmydata/mutable/publish.py | 4 +++- src/allmydata/mutable/servermap.py | 12 ++++++++++-- 3 files changed, 15 insertions(+), 4 deletions(-) commit cb0cfd1adfefad357c187aaaf690c3df68b622bc Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:06:03 2011 +0100 fix Servermap.unreachable_servers src/allmydata/mutable/servermap.py | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-) commit 2d9ea79b94bd4db674d40386fda90825785ac495 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:03:48 2011 +0100 give ServerMap a StorageFarmBroker, temporary this makes it possible for the ServerMap to accept bare serverids and still build data structures with IServers src/allmydata/mutable/checker.py | 2 +- src/allmydata/mutable/filenode.py | 2 +- src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 5 +++-- src/allmydata/test/test_mutable.py | 8 ++++---- 5 files changed, 10 insertions(+), 9 deletions(-) commit 718d1aeff6fded893f65397806d22ece928b0dd4 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 13:43:30 2011 -0400 add StorageFarmBroker.get_server_for_id(), temporary helper This will go away once we're passing IServers everywhere. src/allmydata/storage_client.py | 2 ++ src/allmydata/test/no_network.py | 13 +++++++++++++ 2 files changed, 15 insertions(+), 0 deletions(-) commit ece20231d7fda0d503704842a4aa068dfbc2e54e Author: Brian Warner <warner@lothar.com> Date: Sun Oct 2 01:11:50 2011 +0100 add proper accessors for Servermap.connections, to make refactoring easier src/allmydata/mutable/publish.py | 6 +++--- src/allmydata/mutable/retrieve.py | 10 +++++----- src/allmydata/mutable/servermap.py | 17 +++++++++++------ 3 files changed, 19 insertions(+), 14 deletions(-) commit 3b943d6bf302ff702668081a612fc4fe2604cf9c Author: Brian Warner <warner@lothar.com> Date: Fri Sep 23 10:34:30 2011 -0700 mutable/servermap.py and neighbors: s/peer/server/ src/allmydata/mutable/checker.py | 22 +- src/allmydata/mutable/publish.py | 204 +++++++------- src/allmydata/mutable/servermap.py | 402 +++++++++++++------------- src/allmydata/test/test_mutable.py | 18 +- 4 files changed, 323 insertions(+), 323 deletions(-) IServer refactoring: pass IServer instances around, instead of peerids refs #1363 This collapses 88 small incremental changes (each of which passes all tests) into one big patch. The development process for the long path started with adding some temporary scaffolding, changing one method at a time, then removing the scaffolding. The individual pieces are as follows, in reverse chronological order (the first patch is at the end of this comment): commit 9bbe4174fd0d98a6cf47a8ef96e85d9ef34b2f9a Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 16:05:00 2011 -0400 immutable/downloader/status.py: correct comment src/allmydata/immutable/downloader/status.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) commit 72146a7c7c91eac2f7c3ceb801eb7a1721376889 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 15:46:20 2011 -0400 remove temporary ServerMap._storage_broker src/allmydata/mutable/checker.py | 2 +- src/allmydata/mutable/filenode.py | 2 +- src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 5 ++--- src/allmydata/test/test_mutable.py | 8 ++++---- 5 files changed, 9 insertions(+), 10 deletions(-) commit d703096b41632c47d76414b12672e076a422ff5c Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 15:37:05 2011 -0400 remove temporary storage_broker.get_server_for_id() src/allmydata/storage_client.py | 3 --- src/allmydata/test/no_network.py | 13 ------------- 2 files changed, 0 insertions(+), 16 deletions(-) commit 620cc5d80882ef6f7decfd26af8a6c7c1ddf80d1 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:50:06 2011 -0400 API of Retrieve._try_to_validate_privkey(), trying to remove reader.server src/allmydata/mutable/retrieve.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) commit 92f43f856f4a8b36c207d1b190ed8699b5a4ecb4 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:48:08 2011 -0400 API of Retrieve._validate_block(), trying to remove reader.server src/allmydata/mutable/retrieve.py | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) commit 572d5070761861a2190349d1ed8d85dbc25698a5 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:36:58 2011 -0400 API of Retrieve._mark_bad_share(), trying to remove reader.server src/allmydata/mutable/retrieve.py | 21 +++++++++------------ 1 files changed, 9 insertions(+), 12 deletions(-) commit a793ff00c0de1e2eec7b46288fdf388c7a2bec89 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:06:13 2011 -0400 remove now-unused get_rref_for_serverid() src/allmydata/mutable/servermap.py | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) commit 1b9827cc9366bf90b93297fdd6832f2ad0480ce7 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:03:09 2011 -0400 Retrieve: stop adding .serverid attributes to readers src/allmydata/mutable/retrieve.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) commit 5d4e9d491b19e49d2e443a1dfff2c672842c36ef Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:03:34 2011 -0400 return value of Retrieve(verify=True) src/allmydata/mutable/checker.py | 11 ++++++----- src/allmydata/mutable/retrieve.py | 3 +-- 2 files changed, 7 insertions(+), 7 deletions(-) commit e9ab7978c384e1f677cb7779dc449b1044face82 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:54:23 2011 -0400 Retrieve._bad_shares (but not return value, used by Verifier) src/allmydata/mutable/retrieve.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) commit 2d91926de233ec5c881f30e36b4a30ad92ab42a9 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:51:23 2011 -0400 Publish: stop adding .serverid attributes to writers src/allmydata/mutable/publish.py | 9 ++------- 1 files changed, 2 insertions(+), 7 deletions(-) commit 47c7a0105dec7cbf4f7e0a3ce800bbb85b15df4a Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:56:33 2011 -0400 API of get_write_enabler() src/allmydata/mutable/filenode.py | 7 ++++--- src/allmydata/mutable/publish.py | 4 ++-- src/allmydata/test/no_network.py | 3 +++ 3 files changed, 9 insertions(+), 5 deletions(-) commit 9196a5c6590fdbfd660325ea8358b345887d3db0 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:46:24 2011 -0400 API of get_(renewal|cancel)_secret() src/allmydata/mutable/filenode.py | 14 ++++++++------ src/allmydata/mutable/publish.py | 8 ++++---- src/allmydata/mutable/servermap.py | 5 ++--- 3 files changed, 14 insertions(+), 13 deletions(-) commit de7c1552f8c163eff5b6d820b5fb3b21c1b47cb5 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:41:52 2011 -0400 API of CorruptShareError. Also comment out some related+unused test_web.py code src/allmydata/mutable/common.py | 13 +++++-------- src/allmydata/mutable/retrieve.py | 10 +++++----- src/allmydata/mutable/servermap.py | 8 +++----- src/allmydata/test/common.py | 13 ++++++++----- 4 files changed, 21 insertions(+), 23 deletions(-) commit 2c1c314046b620c16f1e66d030c150d768b7d01e Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 12:01:46 2011 -0400 API of ServerMap.mark_bad_share() src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/retrieve.py | 6 +++--- src/allmydata/mutable/servermap.py | 6 ++---- src/allmydata/test/test_mutable.py | 3 +-- 4 files changed, 7 insertions(+), 10 deletions(-) commit 1bed349030779fd0c378ae4e821384f953c6f6ff Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:11:17 2011 -0400 API+name of ServerMap.shares_on_server() : only for tests, so debug_ prefix src/allmydata/mutable/servermap.py | 7 ++----- src/allmydata/test/test_mutable.py | 6 +++--- 2 files changed, 5 insertions(+), 8 deletions(-) commit 2d32e448677d6b818692e801045d4115b29abf21 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:07:10 2011 -0400 API of ServerMap.all_servers_for_version() src/allmydata/mutable/servermap.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit 48f3204d1889c3e7179578125c4bdef515af3d6a Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:04:50 2011 -0400 internals of ServerMap methods that use make_versionmap(), remove temp copy src/allmydata/mutable/servermap.py | 28 +++++++++---------------- 1 files changed, 10 insertions(+), 18 deletions(-) commit 5c3da77b6c777a145bd5ddfaa4db849dc9495548 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 11:01:28 2011 -0400 API of ServerMap.make_versionmap() src/allmydata/mutable/checker.py | 4 ++-- src/allmydata/mutable/retrieve.py | 5 ++--- src/allmydata/mutable/servermap.py | 4 ++-- src/allmydata/test/test_mutable.py | 7 ++++--- 4 files changed, 10 insertions(+), 10 deletions(-) commit b6882ece49afb4c507d118af2db346fa329209dc Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 10:53:38 2011 -0400 make a copy of ServerMap.make_versionmap() (_make_versionmap2) for internal use src/allmydata/mutable/servermap.py | 18 +++++++++++++----- 1 files changed, 13 insertions(+), 5 deletions(-) commit 963f8e63faf32b950eb1b8103cd2ff16fe8f0151 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:45:58 2011 -0400 API of RetrieveStatus.add_problem() src/allmydata/mutable/retrieve.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 4976d29ffae565a048851601c29013bbae2976d8 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:45:05 2011 -0400 API of RetrieveStatus.add_fetch_timing() src/allmydata/mutable/retrieve.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit d057d3bbba72663ee148a8b916bc2d52be2e3982 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:44:04 2011 -0400 API of Retrieve.notify_server_corruption() src/allmydata/mutable/retrieve.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit 8a2a81e46671c860610e0e96d6add1a57551f22d Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:42:32 2011 -0400 remove unused _outstanding_queries src/allmydata/mutable/retrieve.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) commit 56d12cc9968d03ccd53764455c671122c4f391d1 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:40:57 2011 -0400 change Retrieve.remaining_sharemap src/allmydata/mutable/retrieve.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit 4f0b7af4821f43290bfc70f2b1fc30149ad81281 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 10:40:18 2011 -0400 accessor for PublishStatus._problems src/allmydata/mutable/publish.py | 4 +++- src/allmydata/web/status.py | 2 +- 2 files changed, 4 insertions(+), 2 deletions(-) commit 627087cf66d0b8cc519f4d551a967a7bd9b6a741 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 10:36:39 2011 -0400 accessor for RetrieveStatus._problems src/allmydata/mutable/retrieve.py | 8 ++++++-- src/allmydata/web/status.py | 2 +- 2 files changed, 7 insertions(+), 3 deletions(-) commit ca7dea81f03801b1c7353fc00ecba689268109cf Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:35:32 2011 -0400 add .server to "reader", so we can get at it later src/allmydata/mutable/retrieve.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 6ef516e24908ec195af084a7550d1921a5e983b0 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:32:32 2011 -0400 temporarily give Retrieve a _storage_broker, so it can map serverids to servers src/allmydata/mutable/checker.py | 3 ++- src/allmydata/mutable/filenode.py | 6 ++++-- src/allmydata/mutable/retrieve.py | 5 +++-- src/allmydata/test/test_mutable.py | 4 ++-- 4 files changed, 11 insertions(+), 7 deletions(-) commit afe08e4dd3f4ff9ff7e8a2a8d28b181e3625bcc9 Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:21:51 2011 -0400 mutable/retrieve.py: s/peer/server/ src/allmydata/mutable/retrieve.py | 82 +++++++++++++------------- src/allmydata/test/test_mutable.py | 6 +- 2 files changed, 44 insertions(+), 44 deletions(-) commit 910afcb5d7f274880f68dd6cdb5b05f2bbc29adc Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:16:01 2011 -0400 web.status.PublishStatusPage: add comment, I think .problems isn't exercised src/allmydata/web/status.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) commit 311466dd8c931bbba40d590ade867704282e7f1a Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:48:16 2011 -0400 API of PublishStatus.add_per_server_time() src/allmydata/mutable/publish.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 2df5faa1b6cbfbaded520d2320305a62fe961118 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:46:37 2011 -0400 more simplifications src/allmydata/mutable/publish.py | 4 +--- 1 files changed, 1 insertions(+), 3 deletions(-) commit 6ac4544a3da385f2aad9392f906b90192f4f919a Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:44:08 2011 -0400 API of ServerMap.version_on_server() src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 4 ++-- src/allmydata/test/test_mutable.py | 5 ++--- 3 files changed, 5 insertions(+), 6 deletions(-) commit 3e187e322511072e4683329df6b2c6c733a66dba Author: Brian Warner <warner@lothar.com> Date: Tue Oct 4 00:16:32 2011 -0400 API of ServerMap.make_sharemap() src/allmydata/mutable/servermap.py | 4 ++-- src/allmydata/test/test_mutable.py | 7 ++++--- src/allmydata/web/status.py | 4 ++-- 3 files changed, 8 insertions(+), 7 deletions(-) commit 318feed8437bdd8d4943c6569d38f7b54b6313cc Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:36:19 2011 -0400 small cleanups src/allmydata/mutable/publish.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit bd459ed5714e1db5a7163935c54b7b0b56db8349 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:33:39 2011 -0400 API of ServerMap.add_new_share() src/allmydata/mutable/publish.py | 4 ++-- src/allmydata/mutable/servermap.py | 6 ++---- 2 files changed, 4 insertions(+), 6 deletions(-) commit f2804fb6ed11d80088e0da8ed48e6c2922f2ffef Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:30:26 2011 -0400 API of ServerMap.get_bad_shares() src/allmydata/mutable/publish.py | 3 +-- src/allmydata/mutable/servermap.py | 9 ++++----- 2 files changed, 5 insertions(+), 7 deletions(-) commit 965074a47b3ce1431cb46d9a233840afcf9105f5 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:26:58 2011 -0400 more small cleanups src/allmydata/mutable/publish.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit 38020da34f034f8889947dd3dc05e087ffff7106 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:18:47 2011 -0400 change Publish.bad_share_checkstrings src/allmydata/mutable/publish.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit 5efebcbd2ee0c2f299ea86f7591d856c0f265304 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:16:31 2011 -0400 change internals of Publish.update_goal() src/allmydata/mutable/publish.py | 8 +++----- 1 files changed, 3 insertions(+), 5 deletions(-) commit e91b55ff4c2a69165b71f2c7b217ac319ff4c527 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:11:42 2011 -0400 get rid of Publish.connections src/allmydata/mutable/publish.py | 27 +++++---------------------- 1 files changed, 5 insertions(+), 22 deletions(-) commit 64e9a53b3229ebe2f9ebf7ed502d539311d0e037 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:05:32 2011 -0400 change Publish.bad_servers src/allmydata/mutable/publish.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) commit b85a934bef315a06bcfe00c9c12a3627fed2b918 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:03:07 2011 -0400 Publish.bad_servers: fix bug, this should be a set of serverids, not writers src/allmydata/mutable/publish.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) commit 605ea15ec15ed671513819003ccd211cdb9761e0 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 23:00:21 2011 -0400 change .placed src/allmydata/mutable/publish.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit f7aba37b1b345d5b6d5cb16e3b3f6f3c1afb658e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 22:59:22 2011 -0400 temporarily stash IServer as .server on the "writer" object src/allmydata/mutable/publish.py | 2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) commit f9b551d788e7db1f187fce5ab98ab5d5fe4e1c36 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 22:48:18 2011 -0400 change Publish.goal and API of log_goal() to use IServer, not serverid src/allmydata/mutable/publish.py | 48 ++++++++++++++-------------- 1 files changed, 24 insertions(+), 24 deletions(-) commit 75f20616558e4900b8b1f685dd99aa838de6d452 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:27:02 2011 -0400 API of ServerMap.get_known_shares() src/allmydata/mutable/publish.py | 16 ++++++++++------ src/allmydata/mutable/servermap.py | 7 ++----- 2 files changed, 12 insertions(+), 11 deletions(-) commit 1c38c9d37bb08221b4418762234b1a62397b3b4b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:20:29 2011 -0400 Publish.full_serverlist src/allmydata/mutable/publish.py | 10 +++++----- 1 files changed, 5 insertions(+), 5 deletions(-) commit b6cbd215a04b9cde31a7d92a97a7f048622b16f1 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:12:31 2011 -0400 API of ServerMap.all_servers() src/allmydata/mutable/servermap.py | 19 ++++++------------- 1 files changed, 6 insertions(+), 13 deletions(-) commit e63cd0315fae65357b1727ec6d5ff3c6e0d27c98 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:10:18 2011 -0400 remove ServerMap.connections, set_rref_for_serverid() src/allmydata/mutable/servermap.py | 11 +---------- 1 files changed, 1 insertions(+), 10 deletions(-) commit 4df52db2f80eb12eefa5d57103c24893cde89553 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:04:06 2011 -0400 API of ServerMap.mark_server_reachable() src/allmydata/mutable/servermap.py | 7 ++----- 1 files changed, 2 insertions(+), 5 deletions(-) commit 69c715bde77944dc25181b3dbbeb042c816f9a1b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:03:21 2011 -0400 API of ServerMap.mark_server_unreachable() src/allmydata/mutable/servermap.py | 9 +++------ 1 files changed, 3 insertions(+), 6 deletions(-) commit 3d784d60eec1c508858e3a617e4411ffbcc3c1fa Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:02:03 2011 -0400 API of status.set_privkey_from() src/allmydata/mutable/servermap.py | 7 +++---- 1 files changed, 3 insertions(+), 4 deletions(-) commit 544ed3ea29bed7e66da7fd29ca3f6f076f27a9e6 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:01:15 2011 -0400 API of status.add_per_server_time() src/allmydata/mutable/servermap.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) commit fffe5008b6320bd1e04c3c68389a2bf2ee383fa8 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:59:02 2011 -0400 remove unused .versionmap src/allmydata/mutable/servermap.py | 7 ------- 1 files changed, 0 insertions(+), 7 deletions(-) commit 2816562e090d2294179db3588dafcca18de1bc2b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:57:51 2011 -0400 remove serverid from all log messages. Also one unused lambda. src/allmydata/mutable/servermap.py | 30 +++++++++++++------------- 1 files changed, 15 insertions(+), 15 deletions(-) commit 28fa6b1a2738fa98c1f1dbd3d0e01ae98912d11f Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:54:30 2011 -0400 removed unused _readers src/allmydata/mutable/servermap.py | 3 --- 1 files changed, 0 insertions(+), 3 deletions(-) commit a8e4ed3d645ab592d1add6a1e69b6d1ebfb77817 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:54:16 2011 -0400 remove unused _sharemap src/allmydata/mutable/servermap.py | 1 - 1 files changed, 0 insertions(+), 1 deletions(-) commit 3f072e55cf1d0700f9fffe23f8f3a475725df588 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:49:03 2011 -0400 _must_query src/allmydata/mutable/servermap.py | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) commit c599a059b8df3f5785e4bf89fb6ecc6d8dcd708b Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:48:05 2011 -0400 _queries_outstanding src/allmydata/mutable/servermap.py | 16 +++++++--------- 1 files changed, 7 insertions(+), 9 deletions(-) commit 7743759f98ac2c07926b2fdbd80bf52dfab33085 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:46:17 2011 -0400 _empty_servers src/allmydata/mutable/servermap.py | 5 ++--- 1 files changed, 2 insertions(+), 3 deletions(-) commit 6bb1825916828a713a32cdf7f7411fa3ea2e1e5d Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:45:39 2011 -0400 _good_servers src/allmydata/mutable/servermap.py | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) commit 1768fab1b51d8dd93ecabbaaabfadfa20cf6c3d4 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:44:59 2011 -0400 _bad_servers src/allmydata/mutable/servermap.py | 14 +++++++------- 1 files changed, 7 insertions(+), 7 deletions(-) commit dccbaef30f0ba714c746bf6d4a1a803c36e17b65 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:41:54 2011 -0400 API of _try_to_set_pubkey() src/allmydata/mutable/servermap.py | 7 ++++--- 1 files changed, 4 insertions(+), 3 deletions(-) commit 0481ea70042ba3575f15eac7fd0780f8ece580cc Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:35:02 2011 -0400 API of notify_server_corruption() src/allmydata/mutable/servermap.py | 6 +++--- 1 files changed, 3 insertions(+), 3 deletions(-) commit bea9cba18fb3b9c11bb22f18356a263ecec7351e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:34:09 2011 -0400 API of _got_signature_one_share() src/allmydata/mutable/servermap.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) commit 1520123583cf78650706e114b15bb5b0ac1f4a14 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:32:33 2011 -0400 API of _try_to_validate_privkey() src/allmydata/mutable/servermap.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) commit 938852c9c8519c7a078f58a9b1f4dd8ec8b6715e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:31:48 2011 -0400 API and internals of _add_lease_failed() src/allmydata/mutable/servermap.py | 8 ++++---- 1 files changed, 4 insertions(+), 4 deletions(-) commit 3843dba367e3c19e176a622ab853cb51d2472ddf Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:30:37 2011 -0400 API of _privkey_query_failed() src/allmydata/mutable/servermap.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit 2219a710e1633cd57d0ca0786490de87b3e19ba7 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:29:43 2011 -0400 fix bug in call to _privkey_query_failed, unrelated to refactoring src/allmydata/mutable/servermap.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) commit ae615bec7d0d1b269710b6902797b12f9592ad62 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:27:17 2011 -0400 API of _got_corrupt_share() src/allmydata/mutable/servermap.py | 17 +++++++++-------- 1 files changed, 9 insertions(+), 8 deletions(-) commit cb51c95a6f4e077278157a77dab060c8c1ad7a81 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:23:16 2011 -0400 API of _got_results() src/allmydata/mutable/servermap.py | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) commit bac9154fe0af18f226999a58ffc2362d8cf4b802 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:19:19 2011 -0400 API of _query_failed() src/allmydata/mutable/servermap.py | 5 +++-- 1 files changed, 3 insertions(+), 2 deletions(-) commit fdc29a8ca95d4b5c503e5382b9e5d4d02141ba12 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:17:20 2011 -0400 API of _do_read() src/allmydata/mutable/servermap.py | 6 ++++-- 1 files changed, 4 insertions(+), 2 deletions(-) commit e7e9e338f28d004aa4d423d11c65f1e271ac7322 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:20:21 2011 -0400 API of _do_query() src/allmydata/mutable/servermap.py | 15 +++++++-------- 1 files changed, 7 insertions(+), 8 deletions(-) commit 330625b9dac4cdbe72a11464a893065b9aeed453 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 14:43:05 2011 -0400 next step: first batch of updates to ServermapUpdater updates: most method-local variables in update() API of _build_initial_querylist() API of _send_initial_requests() .full_serverlist .extra_servers src/allmydata/mutable/servermap.py | 39 ++++++++++++++------------ 1 files changed, 21 insertions(+), 18 deletions(-) commit 4aadc584fa7dcb2daa86b048c81dee0049ba26d9 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 15:07:00 2011 -0400 internal change: index _bad_shares with IServer src/allmydata/mutable/servermap.py | 20 ++++++++++---------- 1 files changed, 10 insertions(+), 10 deletions(-) commit 16d4e6fa82a9907dbdc92094213387c6a4164e41 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:20:47 2011 +0100 internal change: index _known_shares with IServer instead of serverid callers are unchanged src/allmydata/mutable/servermap.py | 42 +++++++++++++++---------- 1 files changed, 25 insertions(+), 17 deletions(-) commit ceeb5f4938cc814a0c75d1b8f4018aed965c2176 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:11:43 2011 +0100 accessors and name cleanup for servermap.Servermap.last_update_mode/time src/allmydata/mutable/filenode.py | 6 +++--- src/allmydata/mutable/publish.py | 4 ++-- src/allmydata/mutable/servermap.py | 17 +++++++++++------ 3 files changed, 16 insertions(+), 11 deletions(-) commit 8d3cbda82661c0a7e5c3d3b65cf7a5d5ab7e32c0 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:11:14 2011 +0100 accessors and name cleanup for servermap.Servermap.problems src/allmydata/mutable/servermap.py | 21 +++++++++++++-------- src/allmydata/test/test_mutable.py | 6 +++--- 2 files changed, 16 insertions(+), 11 deletions(-) commit 348f57988f79389db0aab7672e6eaa9a6d8e3219 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:10:41 2011 +0100 accessors and name cleanup for servermap.Servermap.bad_shares src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 30 ++++++++++++++----------- 2 files changed, 18 insertions(+), 14 deletions(-) commit 520c9368134673cdf76c653c5e1bb91c2ab5d51e Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:10:05 2011 +0100 accessors and name cleanup for servermap.Servermap.servermap . src/allmydata/mutable/publish.py | 14 +++++---- src/allmydata/mutable/servermap.py | 38 ++++++++++++++----------- 2 files changed, 29 insertions(+), 23 deletions(-) commit b8b8dc38287a91dbdf494426ac801d9381ce5841 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:08:02 2011 +0100 fix reachable_servers src/allmydata/mutable/checker.py | 3 ++- src/allmydata/mutable/publish.py | 4 +++- src/allmydata/mutable/servermap.py | 12 ++++++++++-- 3 files changed, 15 insertions(+), 4 deletions(-) commit cb0cfd1adfefad357c187aaaf690c3df68b622bc Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:06:03 2011 +0100 fix Servermap.unreachable_servers src/allmydata/mutable/servermap.py | 11 ++++++++--- 1 files changed, 8 insertions(+), 3 deletions(-) commit 2d9ea79b94bd4db674d40386fda90825785ac495 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 18:03:48 2011 +0100 give ServerMap a StorageFarmBroker, temporary this makes it possible for the ServerMap to accept bare serverids and still build data structures with IServers src/allmydata/mutable/checker.py | 2 +- src/allmydata/mutable/filenode.py | 2 +- src/allmydata/mutable/publish.py | 2 +- src/allmydata/mutable/servermap.py | 5 +++-- src/allmydata/test/test_mutable.py | 8 ++++---- 5 files changed, 10 insertions(+), 9 deletions(-) commit 718d1aeff6fded893f65397806d22ece928b0dd4 Author: Brian Warner <warner@lothar.com> Date: Mon Oct 3 13:43:30 2011 -0400 add StorageFarmBroker.get_server_for_id(), temporary helper This will go away once we're passing IServers everywhere. src/allmydata/storage_client.py | 2 ++ src/allmydata/test/no_network.py | 13 +++++++++++++ 2 files changed, 15 insertions(+), 0 deletions(-) commit ece20231d7fda0d503704842a4aa068dfbc2e54e Author: Brian Warner <warner@lothar.com> Date: Sun Oct 2 01:11:50 2011 +0100 add proper accessors for Servermap.connections, to make refactoring easier src/allmydata/mutable/publish.py | 6 +++--- src/allmydata/mutable/retrieve.py | 10 +++++----- src/allmydata/mutable/servermap.py | 17 +++++++++++------ 3 files changed, 19 insertions(+), 14 deletions(-) commit 3b943d6bf302ff702668081a612fc4fe2604cf9c Author: Brian Warner <warner@lothar.com> Date: Fri Sep 23 10:34:30 2011 -0700 mutable/servermap.py and neighbors: s/peer/server/ src/allmydata/mutable/checker.py | 22 +- src/allmydata/mutable/publish.py | 204 +++++++------- src/allmydata/mutable/servermap.py | 402 +++++++++++++------------- src/allmydata/test/test_mutable.py | 18 +- 4 files changed, 323 insertions(+), 323 deletions(-)	2011-10-31 21:03:19 -07:00
Brian Warner	05dbb1cf86	update project-home URLs: /trac/tahoe/ was replaced by /trac/tahoe-lafs/	2011-10-30 12:14:28 -07:00
Zooko O'Whielacronx	d9c1064d42	undo the effects of a patch I hadn't intended to commit, named "debugprint the values of blocks and hashes thereof; make the test data and the seg size small in order to make the debugprints easy to look at" ... rolling back: Thu Sep 29 23:46:28 MDT 2011 zooko@zooko.com * debugprint the values of blocks and hashes thereof; make the test data and the seg size small in order to make the debugprints easy to look at M ./src/allmydata/mutable/publish.py -1 +2 M ./src/allmydata/mutable/retrieve.py +3 M ./src/allmydata/test/test_mutable.py -2 +2	2011-10-28 22:03:49 +00:00
Zooko O'Whielacronx	416701e404	debugprint the values of blocks and hashes thereof; make the test data and the seg size small in order to make the debugprints easy to look at	2011-09-30 05:46:28 +00:00
Brian Warner	633641174a	mutable/retrieve: don't write() after we've been pauseProducer'ed ... This fixes a test failure found against current Twisted trunk in test_mutable.Filenode.test_retrieve_producer_mdmf (when it uses PausingAndStoppingConsumer). There must be some sort of race: I could make it fail against Twisted-11.0 if I just increased the 0.5s delay in test_download.PausingAndStoppingConsumer to about 0.6s, and could make Twisted-trunk pass by reducing it to about 0.3s . I fixed the test (as opposed to the bug) by replacing the delay with a simple reliable eventually(), and adding extra asserts to fail the test if the consumer's write() method is called while the producer is supposed to be paused The bug itself was that mutable.retrieve.Retrieve wasn't checking the "stopped" flag after resuming from a pause, and thus delivered one segment to a consumer that wasn't expecting it. I split out stopped-flag-checking to separate function, which is now called immediately after _check_for_paused(). I also cleaned up some Deferred usage and whitespace.	2011-10-16 17:24:00 -07:00
Brian Warner	fff237be9a	remove interpreter shbang lines from non-executables ... thanks to Greg Troxel for the catch	2011-10-14 10:23:01 -07:00
Brian Warner	d28041fbe4	CLI: don't deprecate --mutable, small docs fixes. refs #1561 ... Also don't accept 'tahoe mkdir --format=chk'.	2011-10-13 21:00:02 -07:00
Brian Warner	57ee56533f	add --format= to 'tahoe put'/'mkdir', remove --mutable-type. Closes #1561	2011-10-13 20:15:00 -07:00
Brian Warner	3eb60ff986	web/filenode.py: rely on Request.notifyFinish. Closes #1366. ... This is safe now that tahoe depends upon Twisted>=10.1, since notifyFinish first appeared in Twisted-9.0	2011-10-13 13:12:19 -07:00
david-sarah	3dc491758d	util/happinessutil.py: suppress a warning from check-miscaptures. (It is not a bug because the capturing function is only used by a 'map' in the same iteration.) refs #1556	2011-10-08 22:21:06 -07:00
david-sarah	22da015dd1	Fix some more potential bugs in test code exposed by check-miscaptures.py. refs #1556	2011-10-06 20:38:47 -07:00
david-sarah	a31a701e42	Fix some potential bugs (in non-test code) exposed by check-miscaptures.py. refs #1556	2011-10-06 20:24:44 -07:00
david-sarah	bdfa7b3772	Fix some potential bugs in test code exposed by check-miscaptures.py. refs #1556	2011-10-06 19:34:43 -07:00
Brian Warner	11ce612589	oops, missed a test failure	2011-10-13 09:37:13 -07:00
Brian Warner	210761cd49	misc mutable-type fixes: ... * fix tahoe.cfg control of default mutable type * tolerate arbitrary case in [client]mutable.format value * small docs improvements * use get_mutable_type() as a format-is-mutable predicate * tighten up error message	2011-10-13 09:32:29 -07:00
Brian Warner	01b00dc7dc	webapi: use all-caps "SDMF"/"MDMF" acronyms in t=json response ... docs: upcase examples of t=json output and format= input	2011-10-13 09:31:43 -07:00
Brian Warner	dad354b275	webapi: handle format=, remove mutable-type= ... * fix CLI commands (put, mkdir) to send format=, not mutable-type= * fix tests * test_cli: fix tests that observe t=json output, don't ignore failures in 'tahoe put' * fix handling of version= to make it easier to use the default * interpret ?mutable=true&format=MDMF as MDMF, not SDMF	2011-10-13 09:29:51 -07:00
david-sarah	2acc0419b9	Tests for ref #1547	2011-10-01 20:53:16 -07:00
david-sarah	978fd83856	Change the file upload forms on directory and welcome pages to use a 3-way radio button to select immutable, SDMF, or MDMF. Add '(experimental)' to the label for creating an MDMF directory. Also improve the spacing of form elements. refs #1547	2011-10-01 20:45:03 -07:00
david-sarah	ac780a4148	test_web.py: minor cleanups, mainly to make the first argument to shouldFail tests consistent	2011-10-01 21:03:32 -07:00
david-sarah	38d819ef44	Tests for ref #1552	2011-10-01 21:00:36 -07:00
david-sarah	5ae446fbd2	test/common.py: in shouldFail and shouldHTTPError, when the raised exception does not include the expected substring (or, for shouldHTTPError, when the status code is wrong), mention which test that happened in.	2011-10-10 17:22:27 -07:00
david-sarah	cf9bf2ea10	interfaces.py: remove get_extension_params and set_extension_params methods from IMutableFileURI. refs #393, #1526	2011-10-10 12:48:42 -07:00
david-sarah	de00b277cc	interfaces.py: fix a typo in the name of IMutableSlotWriter.put_encprivkey. refs #393	2011-10-10 12:46:42 -07:00
david-sarah	bc0d9b682e	immutable/literal.py: add pauseProducing method to LiteralProducer. refs #1537	2011-10-03 12:52:39 -07:00
david-sarah	bc50edc86e	no_network.py: Clean up whitespace around code changed by previous patch.	2011-10-03 18:04:07 -07:00
david-sarah	ee25dcd2fc	no_network.py: Fix potential bugs in some tests due to capture of slots in for loops.	2011-10-03 18:02:31 -07:00
Brian Warner	0716c496c8	MDMF: remove extension fields from caps, tolerate arbitrary ones. Fixes #1526 ... The filecaps used to be produced with hints for 'k' and segsize, but they weren't actually used, and doing so had the potential to limit how we change those filecaps in the future. Also the parsing code had some problems dealing with other numbers of extensions. Removing the existing fields and making the parser tolerate (and ignore) extra ones makes MDMF more future-proof.	2011-10-02 00:35:53 +01:00
david-sarah	5ba0529b87	test/test_runner.py: BinTahoe.test_path has rare nondeterministic failures; this patch probably fixes a problem where the actual cause of failure is masked by a string conversion error.	2011-09-27 15:53:36 -07:00
david-sarah	f72cba50ba	mutable/publish.py: fix an unused import. refs #1542	2011-09-24 22:22:06 -07:00
david-sarah	c88adf0ac0	mutable/layout.py: make unpack_sdmf_checkstring and unpack_mdmf_checkstring more similar, and change an assert to give a more useful message if it fails. refs #1540	2011-09-24 19:36:51 -07:00
kevan	70d27cfa7c	mutable/publish: handle unknown mutable share formats when handling errors	2011-09-24 17:43:05 -07:00
kevan	e8afe84813	mutable/layout: break unpack_checkstring into unpack_mdmf_checkstring and unpack_sdmf_checkstring, add distinguisher function for checkstrings	2011-09-24 17:41:34 -07:00
kevan	2187f27ad0	test/test_mutable: reenable mdmf publish surprise test	2011-09-24 16:54:15 -07:00
kevan	a911e15783	mutable/publish: use unpack_mdmf_checkstring and unpack_sdmf_checkstring instead of unpack_checkstring. fixes #1540	2011-09-24 16:51:37 -07:00
david-sarah	1fa5c729b7	mutable/publish.py: copy the self.writers dict before iterating over it, since we remove elements from it during the iteration. refs #393	2011-09-24 14:12:08 -07:00
david-sarah	f94eb86fc9	mutable/publish.py: simplify by refactoring self.outstanding to self.num_outstanding. refs #393	2011-09-24 13:50:04 -07:00
david-sarah	4af626a798	test_mutable.py: update SkipTest message for test_publish_surprise_mdmf to reference the right ticket number. refs #1540.	2011-09-23 14:16:22 -07:00
Brian Warner	f61bdbffd7	control.py: unbreak speed-test: overwrite() wants a MutableData, not str ... Really, all the upload/modify APIs should take a string or a filehandle, and internally wrap it as needed. Callers should not need to be aware of Uploadable() or MutableData() classes.	2011-09-23 00:37:48 -07:00
david-sarah	5d3d0dc336	test_mutable.py: skip test_publish_surprise_mdmf, which is causing an error. refs #1534, #393	2011-09-20 11:33:19 -07:00
kevan	d93b2965c3	test/test_mutable: write publish surprise test for MDMF, rename existing test_publish_surprise to clarify that it is for SDMF	2011-09-17 17:36:57 -07:00
kevan	19cf58f2aa	test/test_mutable: refactor publish surprise test into common test fixture, rewrite test_publish_surprise to use test fixture	2011-09-17 17:35:33 -07:00
kevan	8c7406508a	mutable/publish: add errback immediately after write, don't consume errors from other parts of the publisher	2011-09-17 16:47:08 -07:00
david-sarah	87a936564e	uri.py: fix two interface violations in verifier URI classes. refs #1474	2011-09-19 20:01:56 -07:00
Brian Warner	8e69b94588	Make platform-detection code tolerate linux-3.0, patch by zooko. ... Otherwise address-autodetection can't find ifconfig. refs #1536	2011-09-15 13:26:20 -07:00
david-sarah	3f756e9429	test_web.py: fix a bug in _count_leases that was causing us to check only the lease count of one share file, not of all share files as intended.	2011-09-15 11:51:26 -07:00
Zooko O'Whielacronx	2025e42ec2	tests: bump up the timeout in this test that fails on FreeStorm's CentOS in order to see if it is just very slow	2011-09-12 19:42:55 -07:00
david-sarah	c10099f982	interfaces: document that the 'fills-holes-with-zero-bytes' key should be used to detect whether a storage server has that behavior. refs #1528	2011-09-12 17:28:43 -07:00
Zooko O'Whielacronx	32f80625c9	storage: more paranoid handling of bounds and palimpsests in mutable share files ... * storage server ignores requests to extend shares by sending a new_length * storage server fills exposed holes (created by sending a write vector whose offset begins after the end of the current data) with 0 to avoid "palimpsest" exposure of previous contents * storage server zeroes out lease info at the old location when moving it to a new location ref. #1528	2011-09-12 15:26:55 -07:00
Zooko O'Whielacronx	942c5e5162	storage: test that the storage server ignores requests to extend shares by sending a new_length, and that the storage server fills exposed holes with 0 to avoid "palimpsest" exposure of previous contents ... ref. #1528	2011-09-12 15:25:54 -07:00
Zooko O'Whielacronx	20e2910c61	immutable: prevent clients from reading past the end of share data, which would allow them to learn the cancellation secret ... Declare explicitly that we prevent this problem in the server's version dict. fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them)	2011-09-12 15:24:58 -07:00
Zooko O'Whielacronx	5476f67dc1	storage: remove the storage server's "remote_cancel_lease" function ... We're removing this function because it is currently unused, because it is dangerous, and because the bug described in #1528 leaks the cancellation secret, which allows anyone who knows a file's storage index to abuse this function to delete shares of that file. fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them)	2011-09-12 15:23:31 -07:00
Zooko O'Whielacronx	65de17245d	storage: test that the storage server does *not* have a "remote_cancel_lease" function ... We're removing this function because it is currently unused, because it is dangerous, and because the bug described in #1528 leaks the cancellation secret, which allows anyone who knows a file's storage index to abuse this function to delete shares of that file. ref. #1528	2011-09-12 15:23:24 -07:00
Zooko O'Whielacronx	cffc987804	immutable: test whether the server allows clients to read past the end of share data, which would allow them to learn the cancellation secret ... Also test whether the server explicitly declares that it prevents this problem. ref #1528	2011-09-12 15:12:01 -07:00
Brian Warner	bd709c4833	Retrieve._activate_enough_peers: rewrite Verify logic	2011-09-09 11:11:50 -07:00
Brian Warner	a15ce96846	Retrieve: implement/test stopProducing	2011-09-09 11:11:50 -07:00
Brian Warner	748e419a9b	move DownloadStopped from download.common to interfaces	2011-09-09 11:11:50 -07:00
Brian Warner	425152c34e	retrieve.py: remove vestigal self._validated_readers	2011-09-09 11:11:50 -07:00
Brian Warner	df07060f93	Retrieve: rewrite flow-control: use a top-level loop() to catch all errors ... This ought to close the potential for dropped errors and hanging downloads. Verify needs to be examined, I may have broken it, although all tests pass.	2011-09-09 11:11:50 -07:00
Brian Warner	53bbc1d06a	Retrieve: merge _validate_active_prefixes into _add_active_peers	2011-09-09 11:11:50 -07:00
Brian Warner	2edfb1a334	Retrieve: remove the initial prefix-is-still-good check ... This check needs to be done with each fetch from the storage server, to detect when someone has changed the share (i.e. our servermap goes stale). Doing it just once at the beginning of retrieve isn't enough: a write might occur after the first segment but before the second, etc. _try_to_validate_prefix() was not removed: it will be used by the future check-with-each-fetch code. test_mutable.Roundtrip.test_corrupt_all_seqnum_late was disabled, since it fails until this check is brought back. (the corruption it applies only touches the prefix, not the block data, so the check-less retrieve actually tolerates it). Don't forget to re-enable it once the check is brought back.	2011-09-09 11:11:50 -07:00
Brian Warner	2b4f2b7fa3	MDMFSlotReadProxy: remove the queue ... This is a neat trick to reduce Foolscap overhead, but the need for an explicit flush() complicates the Retrieve path and makes it prone to lost-progress bugs. Also change test_mutable.FakeStorageServer to tolerate multiple reads of the same share in a row, a limitation exposed by turning off the queue.	2011-09-09 11:11:50 -07:00
Brian Warner	1597aafea1	rearrange Retrieve: first step, shouldn't change order of execution	2011-09-09 11:11:49 -07:00
david-sarah	6b97e6f407	CLI: test_cli.py -- remove an unnecessary call in test_mkdir_mutable_type. refs #1527	2011-09-06 11:37:30 -07:00
david-sarah	1e59930fda	CLI: improve test for 'tahoe mkdir --mutable-type='. refs #1527	2011-09-06 11:30:20 -07:00
david-sarah	23f46b758e	CLI: make the --mutable-type option value for 'tahoe put' and 'tahoe mkdir' case-insensitive, and change --help for these commands accordingly. fixes #1527	2011-09-04 19:09:22 -07:00
Kevan Carstensen	64996a913d	cli: make --mutable-type imply --mutable in 'tahoe put'	2011-09-03 12:09:20 -07:00
david-sarah	1b7b3bd0d4	SFTP: add a comment about a subtle interaction between OverwriteableFileConsumer and GeneralSFTPFile, and test the case it is commenting on.	2011-09-03 15:23:04 -07:00
Brian Warner	01b5124d0a	improve the storage/mutable.py asserts even more	2011-09-01 09:05:43 -07:00
wilcoxjg	d2e5de3b69	storage/mutable.py: special characters in struct.foo arguments indicate standard as opposed to native sizes, we should be using these characters in these asserts	2011-09-01 01:41:44 -07:00
Brian Warner	7191b0b32e	test_mutable.Version: consolidate some tests, reduce runtime from 19s to 15s	2011-08-30 22:04:51 -07:00
Kevan Carstensen	32e30c9023	mutable/retrieve: handle the case where self._read_length is 0. ... Note that the downloader will still fetch a segment for a zero-length read, which is wasteful. Fixing that isn't specifically required to fix #1512, but it should probably be fixed before 1.9.	2011-08-30 14:01:41 -07:00
Brian Warner	980eb778c1	test_mutable.Update: only upload the files needed for each test. refs #1500 ... This first step shaves 15% off the runtime: from 139s to 119s on my laptop. It also fixes a couple of places where a Deferred was being dropped, which would cause two tests to run in parallel and also confuse error reporting.	2011-08-29 00:27:17 -07:00
Brian Warner	fd676a5846	Let Uploader retain History instead of passing it into upload(). Fixes #1079. ... This consistently records all immutable uploads in the Recent Uploads And Downloads page, regardless of code path. Previously, certain webapi upload operations (like PUT /uri/$DIRCAP/newchildname) failed to pass the History object and were left out.	2011-08-28 23:32:46 -07:00
Brian Warner	9756146d61	Fix mutable publish/retrieve timing status displays. Fixes #1505. ... publish: * encrypt and encode times are cumulative, not just current-segment retrieve: * same for decrypt and decode times * update "current status" to include segment number * set status to Finished/Failed when download is complete * set progress to 1.0 when complete More improvements to consider: * progress is currently 0% or 100%: should calculate how many segments are involved (remembering retrieve can be less than the whole file) and set it to a fraction * "fetch" time is fuzzy: what we want is to know how much of the delay is not our own fault, but since we do decode/decrypt work while waiting for more shares, it's not straightforward	2011-08-28 16:22:21 -07:00
Brian Warner	d575ccba28	Teach 'tahoe debug catalog-shares about MDMF. Closes #1507.	2011-08-28 01:09:31 -07:00
Brian Warner	97b601f75a	debug.py: remove some dead comments	2011-08-28 00:45:56 -07:00
Brian Warner	9f827ffa7b	hush pyflakes	2011-08-28 00:42:54 -07:00
Brian Warner	0f1d674d76	MutableFileNode.set_downloader_hints: never depend upon order of dict.values() ... The old code was calculating the "extension parameters" (a list) from the downloader hints (a dictionary) with hints.values(), which is not stable, and would result in corrupted filecaps (with the 'k' and 'segsize' hints occasionally swapped). The new code always uses [k,segsize].	2011-08-28 00:41:03 -07:00
Brian Warner	4b0078dc49	layout.py: fix MDMF share layout documentation	2011-08-28 00:39:21 -07:00
Brian Warner	a3d6fd4992	teach 'tahoe debug dump-share' about MDMF and offsets. refs #1507	2011-08-28 00:38:34 -07:00
Brian Warner	101ca8dfe4	test_mutable.Version.test_debug: use splitlines() to fix buildslaves ... Any slave running in a directory with spaces in the name was miscounting shares, causing the test to fail.	2011-08-27 23:47:28 -07:00
Brian Warner	7d48f902fc	test_mutable.Version: exercise 'tahoe debug find-shares' on MDMF. refs #1507 ... Also changes NoNetworkGrid to put shares in storage/shares/ .	2011-08-27 17:55:42 -07:00
Brian Warner	80bdc0f5b7	test_mutable.py: oops, missed a .todo	2011-08-27 17:21:18 -07:00
Brian Warner	e0eaeff7f5	test_mutable: merge davidsarah's patch with my Version refactorings	2011-08-27 16:57:07 -07:00
david-sarah	3c92b832f2	Make the immutable/read-only constraint checking for MDMF URIs identical to that for SSK URIs. refs #393	2011-08-22 18:27:20 -07:00
david-sarah	88989a4ea2	Additional tests for MDMF URIs and for zero-length files. refs #393	2011-08-22 18:15:32 -07:00
david-sarah	ac7b8400d4	Additional tests for zero-length partial reads and updates to mutable versions. refs #393	2011-08-21 18:41:11 -07:00
Brian Warner	03ba8065ce	test_mutable.Version: factor out some expensive uploads, save 25% runtime	2011-08-27 16:27:37 -07:00
Brian Warner	370e6f271e	SDMF: update filenode with correct k/N after Retrieve. Fixes #1510. ... Without this, we get a regression when modifying a mutable file that was created with more shares (larger N) than our current tahoe.cfg . The modification attempt creates new versions of the (0,1,..,newN-1) shares, but leaves the old versions of the (newN,..,oldN-1) shares alone (and throws a assertion error in SDMFSlotWriteProxy.finish_publishing in the process). The mixed versions that result (some shares with e.g. N=10, some with N=20, such that both versions are recoverable) cause problems for the Publish code, even before MDMF landed. Might be related to refs #1390 and refs #1042.	2011-08-27 15:50:31 -07:00
Brian Warner	b8c90d24fc	layout.py: annotate assertion to figure out 'tahoe backup' failure	2011-08-27 12:52:53 -07:00
Brian Warner	482a7dd3f1	Add 'tahoe debug dump-cap' support for MDMF, DIR2-CHK, DIR2-MDMF. refs #1507. ... This also adds tests for all those cases, and fixes an omission in uri.py that broke parsing of DIR2-MDMF-Verifier and DIR2-CHK-Verifier.	2011-08-27 12:50:48 -07:00
Brian Warner	522868d5e9	MDMF: more writable/writeable consistentifications	2011-08-27 12:06:02 -07:00
Brian Warner	48544a251d	MDMF: s/Writable/Writeable/g, for consistency with existing SDMF code	2011-08-27 11:33:57 -07:00
Brian Warner	349093fa2e	test_mutable.Update: increase timeout from 120s to 400s, slaves are failing	2011-08-25 16:01:40 -07:00
Zooko O'Whielacronx	5bf5404ba1	tests: fix check_memory test ... fixes #1503	2011-08-25 13:11:16 -07:00
Brian Warner	9e613ee9b3	test_mutable.Update: remove .timeout overrides, otherwise tests ERROR	2011-08-24 19:24:55 -07:00
Brian Warner	445e9eaf37	blacklist.py: add read() method too, for completeness	2011-08-24 19:19:02 -07:00
david-sarah	3d7a32647c	Implementation, tests and docs for blacklists. This version allows listing directories containing a blacklisted child. Inclusion of blacklist.py fixed. fixes #1425	2011-08-24 08:59:28 -07:00
david-sarah	505ebeb353	mutable/layout.py: fix unused import. refs #393	2011-08-16 15:50:43 -07:00
david-sarah	4b62b53cd6	mutable/retrieve.py: cosmetics and remove a stale comment. refs #393	2011-08-16 14:46:12 -07:00
Kevan Carstensen	62c72881bb	mutable/filenode.py: don't fetch more segments than necesasry to update the file	2011-08-13 14:00:05 -07:00
Kevan Carstensen	b49986f801	test/test_mutable: test for incorrect div_ceil equations	2011-08-13 11:39:36 -07:00
Kevan Carstensen	3423b7d6f7	mutable/retrieve.py: use floor division to calculate segment boundaries, don't fetch more segments than necessary	2011-08-13 11:38:33 -07:00
Kevan Carstensen	a1a334e1a0	mdmf: clean up boolean expressions, correct typos, remove self._paused, and don't unconditionally initialize block hash trees, asll as suggested by davidsarahs' review comments	2011-08-13 11:37:10 -07:00
Brian Warner	fa6a768bc1	now that tests pass with full-size keys, return test-keys to normal (522bit)	2011-08-11 10:54:18 -07:00
Brian Warner	eec428ba5f	fix SHARE_HASH_CHAIN_SIZE computation	2011-08-11 10:53:50 -07:00
david-sarah	dd23f265bf	More idiomatic resolution of the conflict between ticket393-MDMF-2 and trunk. refs #393	2011-08-10 13:29:42 -07:00
david-sarah	c55c44e70a	Replace the hard-coded 522-bit RSA key size used for tests with a TEST_RSA_KEY_SIZE constant defined in test/common.py (part 2). refs #393	2011-08-10 13:23:10 -07:00
david-sarah	fbe0e42bbf	Replace the hard-coded 522-bit RSA key size used for tests with a TEST_RSA_KEY_SIZE constant defined in test/common.py (part 1). refs #393	2011-08-10 13:22:43 -07:00
Zooko O'Whielacronx	95a7551586	merge some minor conflicts in test code from the 393-2 branch and trunk	2011-08-10 10:21:39 -07:00
david-sarah	b60506b0ba	test_web.py: fix a test failure dependent on whether simplejson.loads returns a unicode or str object.	2011-08-08 14:39:25 -07:00
Kevan Carstensen	c413a8fae1	immutable/filenode: fix pyflakes warnings	2011-08-06 17:45:14 -07:00
Kevan Carstensen	47eab03c6c	test: fix assorted tests broken by MDMF changes	2011-08-06 17:44:59 -07:00
Kevan Carstensen	a7fb55f986	uri: add MDMF and MDMF directory caps, add extension hint support	2011-08-06 17:44:36 -07:00
Kevan Carstensen	52e9512b49	test/test_mutable: tests for MDMF ... These are their own patch because they cut across a lot of the changes I've made in implementing MDMF in such a way as to make it difficult to split them up into the other patches.	2011-08-06 17:44:14 -07:00
Kevan Carstensen	a9cada2e03	webapi changes for MDMF ... - Learn how to create MDMF files and directories through the mutable-type argument. - Operate with the interface changes associated with MDMF and #993. - Learn how to do partial updates of mutable files.	2011-08-06 17:43:48 -07:00
Kevan Carstensen	bb10d685ed	mutable/servermap: Rework the servermap to work with MDMF mutable files	2011-08-06 17:42:59 -07:00
Kevan Carstensen	cde17ec041	dirnode: teach dirnode to make MDMF directories	2011-08-06 17:42:24 -07:00
david-sarah	50c692d1c5	Fix some test failures caused by #393 patch.	2011-08-01 20:28:10 -07:00
Kevan Carstensen	4d09ef3d83	cli: teach CLI how to create MDMF mutable files ... Specifically, 'tahoe mkdir' and 'tahoe put' now take a --mutable-type argument.	2011-08-01 19:16:13 -07:00
Kevan Carstensen	a87fc8ffab	frontends/sftpd: Resolve incompatibilities between SFTP frontend and MDMF changes	2011-08-01 19:12:07 -07:00
Kevan Carstensen	b1b77d3b89	mutable/layout: Define MDMF share format, write tools for working with MDMF share format ... The changes in layout.py are mostly concerned with the MDMF share format. In particular, we define read and write proxy objects used by retrieval, publishing, and other code to write and read the MDMF share format. We create equivalent proxies for SDMF objects so that these objects can be suitably general.	2011-08-01 19:11:20 -07:00
Kevan Carstensen	edf9858fb8	immutable/filenode: implement unified filenode interface	2011-08-01 19:09:05 -07:00
Kevan Carstensen	3e996e0c94	immutable/literal.py: Implement interface changes in literal nodes.	2011-08-01 19:08:14 -07:00
Kevan Carstensen	8ae564d8ad	test/common: Alter common test code to work with MDMF. ... This mostly has to do with making the test code implement the new unified filenode interfaces.	2011-08-01 18:56:43 -07:00
Kevan Carstensen	f80a7fdf18	mutable: train checker and repairer to work with MDMF mutable files	2011-08-01 18:51:40 -07:00
Kevan Carstensen	893a9028f0	nodemaker: teach nodemaker about MDMF caps	2011-08-01 18:49:26 -07:00
Kevan Carstensen	d3f98498c0	client: teach client how to create and work with MDMF files	2011-08-01 18:48:11 -07:00
Kevan Carstensen	e4f2b27305	mutable/filenode: Modify mutable filenodes for use with MDMF ... In particular: - Break MutableFileNode and MutableFileVersion into distinct classes. - Implement the interface modifications made for MDMF. - Be aware of MDMF caps. - Learn how to create and work with MDMF files.	2011-08-01 18:45:01 -07:00
Kevan Carstensen	4c48e2e1b6	nodemaker: teach nodemaker how to create MDMF mutable files	2011-08-01 18:42:58 -07:00
Kevan Carstensen	126d1ad010	interfaces: change interfaces to work with MDMF ... A lot of this work concerns #993, in that it unifies (to an extent) the interfaces of mutable and immutable files.	2011-08-01 18:41:19 -07:00
Kevan Carstensen	1576c35d38	mutable/publish: teach the publisher how to publish MDMF mutable files ... Like the downloader, the publisher needs some substantial changes to handle multiple segment mutable files.	2011-08-01 18:39:31 -07:00
Kevan Carstensen	ac3b2647dd	mutable/retrieve: rework the mutable downloader to handle multiple-segment files ... The downloader needs substantial reworking to handle multiple segment mutable files, which it needs to handle for MDMF.	2011-08-01 18:35:24 -07:00
david-sarah	c41bf14029	Cleanups for suppression of UserWarnings. refs #1435	2011-08-17 21:07:49 -07:00
Zooko O'Whielacronx	f627af3b9f	suppress warning emitted by newer zope.interface with Nevow 0.10 ... refs #1435	2011-08-17 13:31:34 -07:00
david-sarah	a98a4ec13f	_auto_deps.py: change the requirement for zope.interface to <= 3.6.2, >= 3.6.6. fixes #1435	2011-08-14 19:53:47 -07:00
david-sarah	a48aee11b3	allmydata/__init__.py, test_version.py: make version parsing understand '<=', with test. refs #1435	2011-08-14 20:51:53 -07:00
Brian Warner	5aad81f84d	replace tabs with spaces in the #1441 'tahoe debug' synopsis	2011-08-11 10:37:04 -07:00
david-sarah	0dc917af08	Correct the information printed by '/usr/bin/tahoe debug --help' on Debian/Ubuntu. fixes #1441	2011-07-24 09:25:30 -07:00
david-sarah	6c2670641e	Add test for webopen. fixes #1149	2011-07-24 14:16:59 -07:00
david-sarah	ab9eb12f70	test_client.py: relax a check in test_create_drop_uploader so that it should pass on Python 2.4.x. refs #1429	2011-08-09 22:25:04 -07:00
david-sarah	0dc56daaf9	test/common_util.py: correct fix to mkdir_nonascii. refs #1472	2011-08-09 22:19:06 -07:00
david-sarah	2deba3319c	test/common_util.py: fix a typo. refs #1472	2011-08-09 21:42:35 -07:00
david-sarah	0104dbab92	test_client.py, test_drop_upload.py: fix pyflakes warnings.	2011-08-09 20:45:05 -07:00
david-sarah	db22fdc20d	Factor out methods dealing with non-ASCII directories and filenames from test_drop_upload.py into common_util.py. refs #1429, #1472	2011-08-09 20:15:58 -07:00
david-sarah	c102056ac1	test_client.py: add a test that the drop-uploader is initialized correctly by client.py. Also give the DropUploader service a name, which is necessary for the test. refs #1429	2011-08-09 20:05:38 -07:00
david-sarah	10ee22f50e	drop-upload: rename 'start' method to 'startService', which is what you're supposed to use to start a Service. refs #1429	2011-08-09 20:03:45 -07:00
david-sarah	f157b73367	test_drop_upload.py: add comment explaining why we don't use FilePath.setContent. refs #1429	2011-08-09 19:59:42 -07:00