Brian Warner
d10f260f36
retrieve.py: unconditionally check share-hash-tree. Fixes #1654 .
...
Add Kevan's unit test, update known_issues.rst
2012-01-12 13:35:53 -08:00
david-sarah
7f8bbcc155
Use a private/drop_upload_dircap file instead of the [drop_upload]upload.dircap option in tahoe.cfg. Fail if the upload.dircap option is used, or options are missing. Also updates tests and docs. fixes #1593
2011-11-20 23:24:26 +00:00
Brian Warner
7989fe21cc
test_mutable: don't use 75 shares (slow), now that the bug is fixed
...
I missed this part of Kevan's fix-1628.darcs.2.patch .
2011-12-28 14:38:19 -08:00
Kevan Carstensen
e29323f68f
mutable publish: track multiple servers-per-share. Fixes some of #1628 .
...
The remaining work is to write additional tests.
src/allmydata/test/no_network.py:
This supports tests in which servers leave the grid only to return with
their shares intact at a later time.
src/allmydata/test/test_mutable.py:
The UCWEs in the incident reports associated with #1628 all seem to be
associated with shares that the servermap knows about, but which aren't
accounted for during the publish process for whatever reason. Specifically,
it looks like the publisher is only capable of keeping track of a single
storage server for a given share. This makes the repair process worse than
it was pre-MDMF at updating all of the shares of a particular file to the
newest version, and can also cause spurious UCWEs. This test simulates such
a layout and fails if an UCWE is thrown. We need to write another test to
ensure that all copies of a share are updated to the latest version (or
alter this test to do that), so that the test suite doesn't pass unless both
regressions are fixed.
We want the publisher to follow the existing share placement when uploading
a new version of a mutable file, and we don't want this test to pass unless
it does.
src/allmydata/mutable/publish.py:
Before this commit, the publisher only kept track of a single writer for
each share. This is insufficient to handle updates in which a single share
may live on multiple servers. In the best case, an update will only update
one of the existing shares instead of all of them. In some cases, the update
will encounter the existing shares when publishing some other share,
interpret it as a sign of an uncoordinated update, and fail. Keeping track
of all of the writers helps ensure that all existing shares are updated, and
helps avoid spurious uncoordinated write errors.
2011-12-27 21:33:58 -08:00
david-sarah
531f562fcf
Tests for ref #1592 .
2011-12-17 04:31:30 +00:00
david-sarah
4618571389
test_web.py cleanup: use failUnlessIn/failIfIn in preference to 'in' operator.
2011-12-17 04:27:10 +00:00
Brian Warner
990f294eea
remove remaining uses of nevow's "formless" module
...
We're slowly moving away from Nevow, and marcusw's previous patch removed
uses of the formless CSS file, so now we can stop testing that nevow can find
that file, and remove the lingering unused "import formless" call.
2011-11-17 14:54:23 -08:00
Marcus Wanner
0ed6260a3c
1585-webui.darcs.patch
2011-11-17 21:49:23 +00:00
Brian Warner
7da112ccef
Rewrite download-status-timeline visualizer ('viz') with d3.js
...
* use d3.js v2.4.6
* add a "toggle misc events" button, to get hash/bitmap-checking details
* only draw data that's on screen, for speed
* add fragment-arg to fetch timeline data.json from somewhere else
2011-10-31 23:18:21 -07:00
Brian Warner
5bf1ffbc87
IServer refactoring: pass IServer instances around, instead of peerids
...
refs #1363
This collapses 88 small incremental changes (each of which passes all tests)
into one big patch. The development process for the long path started with
adding some temporary scaffolding, changing one method at a time, then
removing the scaffolding. The individual pieces are as follows, in reverse
chronological order (the first patch is at the end of this comment):
commit 9bbe4174fd0d98a6cf47a8ef96e85d9ef34b2f9a
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 16:05:00 2011 -0400
immutable/downloader/status.py: correct comment
src/allmydata/immutable/downloader/status.py | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
commit 72146a7c7c91eac2f7c3ceb801eb7a1721376889
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 15:46:20 2011 -0400
remove temporary ServerMap._storage_broker
src/allmydata/mutable/checker.py | 2 +-
src/allmydata/mutable/filenode.py | 2 +-
src/allmydata/mutable/publish.py | 2 +-
src/allmydata/mutable/servermap.py | 5 ++---
src/allmydata/test/test_mutable.py | 8 ++++----
5 files changed, 9 insertions(+), 10 deletions(-)
commit d703096b41632c47d76414b12672e076a422ff5c
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 15:37:05 2011 -0400
remove temporary storage_broker.get_server_for_id()
src/allmydata/storage_client.py | 3 ---
src/allmydata/test/no_network.py | 13 -------------
2 files changed, 0 insertions(+), 16 deletions(-)
commit 620cc5d80882ef6f7decfd26af8a6c7c1ddf80d1
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 12:50:06 2011 -0400
API of Retrieve._try_to_validate_privkey(), trying to remove reader.server
src/allmydata/mutable/retrieve.py | 10 +++++-----
1 files changed, 5 insertions(+), 5 deletions(-)
commit 92f43f856f4a8b36c207d1b190ed8699b5a4ecb4
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 12:48:08 2011 -0400
API of Retrieve._validate_block(), trying to remove reader.server
src/allmydata/mutable/retrieve.py | 14 +++++++-------
1 files changed, 7 insertions(+), 7 deletions(-)
commit 572d5070761861a2190349d1ed8d85dbc25698a5
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 12:36:58 2011 -0400
API of Retrieve._mark_bad_share(), trying to remove reader.server
src/allmydata/mutable/retrieve.py | 21 +++++++++------------
1 files changed, 9 insertions(+), 12 deletions(-)
commit a793ff00c0de1e2eec7b46288fdf388c7a2bec89
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 12:06:13 2011 -0400
remove now-unused get_rref_for_serverid()
src/allmydata/mutable/servermap.py | 3 ---
1 files changed, 0 insertions(+), 3 deletions(-)
commit 1b9827cc9366bf90b93297fdd6832f2ad0480ce7
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 12:03:09 2011 -0400
Retrieve: stop adding .serverid attributes to readers
src/allmydata/mutable/retrieve.py | 1 -
1 files changed, 0 insertions(+), 1 deletions(-)
commit 5d4e9d491b19e49d2e443a1dfff2c672842c36ef
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 12:03:34 2011 -0400
return value of Retrieve(verify=True)
src/allmydata/mutable/checker.py | 11 ++++++-----
src/allmydata/mutable/retrieve.py | 3 +--
2 files changed, 7 insertions(+), 7 deletions(-)
commit e9ab7978c384e1f677cb7779dc449b1044face82
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:54:23 2011 -0400
Retrieve._bad_shares (but not return value, used by Verifier)
src/allmydata/mutable/retrieve.py | 7 ++++---
1 files changed, 4 insertions(+), 3 deletions(-)
commit 2d91926de233ec5c881f30e36b4a30ad92ab42a9
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:51:23 2011 -0400
Publish: stop adding .serverid attributes to writers
src/allmydata/mutable/publish.py | 9 ++-------
1 files changed, 2 insertions(+), 7 deletions(-)
commit 47c7a0105dec7cbf4f7e0a3ce800bbb85b15df4a
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:56:33 2011 -0400
API of get_write_enabler()
src/allmydata/mutable/filenode.py | 7 ++++---
src/allmydata/mutable/publish.py | 4 ++--
src/allmydata/test/no_network.py | 3 +++
3 files changed, 9 insertions(+), 5 deletions(-)
commit 9196a5c6590fdbfd660325ea8358b345887d3db0
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:46:24 2011 -0400
API of get_(renewal|cancel)_secret()
src/allmydata/mutable/filenode.py | 14 ++++++++------
src/allmydata/mutable/publish.py | 8 ++++----
src/allmydata/mutable/servermap.py | 5 ++---
3 files changed, 14 insertions(+), 13 deletions(-)
commit de7c1552f8c163eff5b6d820b5fb3b21c1b47cb5
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:41:52 2011 -0400
API of CorruptShareError. Also comment out some related+unused test_web.py code
src/allmydata/mutable/common.py | 13 +++++--------
src/allmydata/mutable/retrieve.py | 10 +++++-----
src/allmydata/mutable/servermap.py | 8 +++-----
src/allmydata/test/common.py | 13 ++++++++-----
4 files changed, 21 insertions(+), 23 deletions(-)
commit 2c1c314046b620c16f1e66d030c150d768b7d01e
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 12:01:46 2011 -0400
API of ServerMap.mark_bad_share()
src/allmydata/mutable/publish.py | 2 +-
src/allmydata/mutable/retrieve.py | 6 +++---
src/allmydata/mutable/servermap.py | 6 ++----
src/allmydata/test/test_mutable.py | 3 +--
4 files changed, 7 insertions(+), 10 deletions(-)
commit 1bed349030779fd0c378ae4e821384f953c6f6ff
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:11:17 2011 -0400
API+name of ServerMap.shares_on_server() : only for tests, so debug_ prefix
src/allmydata/mutable/servermap.py | 7 ++-----
src/allmydata/test/test_mutable.py | 6 +++---
2 files changed, 5 insertions(+), 8 deletions(-)
commit 2d32e448677d6b818692e801045d4115b29abf21
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:07:10 2011 -0400
API of ServerMap.all_servers_for_version()
src/allmydata/mutable/servermap.py | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
commit 48f3204d1889c3e7179578125c4bdef515af3d6a
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:04:50 2011 -0400
internals of ServerMap methods that use make_versionmap(), remove temp copy
src/allmydata/mutable/servermap.py | 28 +++++++++----------------
1 files changed, 10 insertions(+), 18 deletions(-)
commit 5c3da77b6c777a145bd5ddfaa4db849dc9495548
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:01:28 2011 -0400
API of ServerMap.make_versionmap()
src/allmydata/mutable/checker.py | 4 ++--
src/allmydata/mutable/retrieve.py | 5 ++---
src/allmydata/mutable/servermap.py | 4 ++--
src/allmydata/test/test_mutable.py | 7 ++++---
4 files changed, 10 insertions(+), 10 deletions(-)
commit b6882ece49afb4c507d118af2db346fa329209dc
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 10:53:38 2011 -0400
make a copy of ServerMap.make_versionmap() (_make_versionmap2) for internal use
src/allmydata/mutable/servermap.py | 18 +++++++++++++-----
1 files changed, 13 insertions(+), 5 deletions(-)
commit 963f8e63faf32b950eb1b8103cd2ff16fe8f0151
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:45:58 2011 -0400
API of RetrieveStatus.add_problem()
src/allmydata/mutable/retrieve.py | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
commit 4976d29ffae565a048851601c29013bbae2976d8
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:45:05 2011 -0400
API of RetrieveStatus.add_fetch_timing()
src/allmydata/mutable/retrieve.py | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
commit d057d3bbba72663ee148a8b916bc2d52be2e3982
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:44:04 2011 -0400
API of Retrieve.notify_server_corruption()
src/allmydata/mutable/retrieve.py | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
commit 8a2a81e46671c860610e0e96d6add1a57551f22d
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:42:32 2011 -0400
remove unused _outstanding_queries
src/allmydata/mutable/retrieve.py | 1 -
1 files changed, 0 insertions(+), 1 deletions(-)
commit 56d12cc9968d03ccd53764455c671122c4f391d1
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:40:57 2011 -0400
change Retrieve.remaining_sharemap
src/allmydata/mutable/retrieve.py | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
commit 4f0b7af4821f43290bfc70f2b1fc30149ad81281
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 10:40:18 2011 -0400
accessor for PublishStatus._problems
src/allmydata/mutable/publish.py | 4 +++-
src/allmydata/web/status.py | 2 +-
2 files changed, 4 insertions(+), 2 deletions(-)
commit 627087cf66d0b8cc519f4d551a967a7bd9b6a741
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 10:36:39 2011 -0400
accessor for RetrieveStatus._problems
src/allmydata/mutable/retrieve.py | 8 ++++++--
src/allmydata/web/status.py | 2 +-
2 files changed, 7 insertions(+), 3 deletions(-)
commit ca7dea81f03801b1c7353fc00ecba689268109cf
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:35:32 2011 -0400
add .server to "reader", so we can get at it later
src/allmydata/mutable/retrieve.py | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
commit 6ef516e24908ec195af084a7550d1921a5e983b0
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:32:32 2011 -0400
temporarily give Retrieve a _storage_broker, so it can map serverids to servers
src/allmydata/mutable/checker.py | 3 ++-
src/allmydata/mutable/filenode.py | 6 ++++--
src/allmydata/mutable/retrieve.py | 5 +++--
src/allmydata/test/test_mutable.py | 4 ++--
4 files changed, 11 insertions(+), 7 deletions(-)
commit afe08e4dd3f4ff9ff7e8a2a8d28b181e3625bcc9
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:21:51 2011 -0400
mutable/retrieve.py: s/peer/server/
src/allmydata/mutable/retrieve.py | 82 +++++++++++++-------------
src/allmydata/test/test_mutable.py | 6 +-
2 files changed, 44 insertions(+), 44 deletions(-)
commit 910afcb5d7f274880f68dd6cdb5b05f2bbc29adc
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:16:01 2011 -0400
web.status.PublishStatusPage: add comment, I think .problems isn't exercised
src/allmydata/web/status.py | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
commit 311466dd8c931bbba40d590ade867704282e7f1a
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:48:16 2011 -0400
API of PublishStatus.add_per_server_time()
src/allmydata/mutable/publish.py | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
commit 2df5faa1b6cbfbaded520d2320305a62fe961118
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:46:37 2011 -0400
more simplifications
src/allmydata/mutable/publish.py | 4 +---
1 files changed, 1 insertions(+), 3 deletions(-)
commit 6ac4544a3da385f2aad9392f906b90192f4f919a
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:44:08 2011 -0400
API of ServerMap.version_on_server()
src/allmydata/mutable/publish.py | 2 +-
src/allmydata/mutable/servermap.py | 4 ++--
src/allmydata/test/test_mutable.py | 5 ++---
3 files changed, 5 insertions(+), 6 deletions(-)
commit 3e187e322511072e4683329df6b2c6c733a66dba
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:16:32 2011 -0400
API of ServerMap.make_sharemap()
src/allmydata/mutable/servermap.py | 4 ++--
src/allmydata/test/test_mutable.py | 7 ++++---
src/allmydata/web/status.py | 4 ++--
3 files changed, 8 insertions(+), 7 deletions(-)
commit 318feed8437bdd8d4943c6569d38f7b54b6313cc
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:36:19 2011 -0400
small cleanups
src/allmydata/mutable/publish.py | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
commit bd459ed5714e1db5a7163935c54b7b0b56db8349
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:33:39 2011 -0400
API of ServerMap.add_new_share()
src/allmydata/mutable/publish.py | 4 ++--
src/allmydata/mutable/servermap.py | 6 ++----
2 files changed, 4 insertions(+), 6 deletions(-)
commit f2804fb6ed11d80088e0da8ed48e6c2922f2ffef
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:30:26 2011 -0400
API of ServerMap.get_bad_shares()
src/allmydata/mutable/publish.py | 3 +--
src/allmydata/mutable/servermap.py | 9 ++++-----
2 files changed, 5 insertions(+), 7 deletions(-)
commit 965074a47b3ce1431cb46d9a233840afcf9105f5
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:26:58 2011 -0400
more small cleanups
src/allmydata/mutable/publish.py | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
commit 38020da34f034f8889947dd3dc05e087ffff7106
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:18:47 2011 -0400
change Publish.bad_share_checkstrings
src/allmydata/mutable/publish.py | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
commit 5efebcbd2ee0c2f299ea86f7591d856c0f265304
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:16:31 2011 -0400
change internals of Publish.update_goal()
src/allmydata/mutable/publish.py | 8 +++-----
1 files changed, 3 insertions(+), 5 deletions(-)
commit e91b55ff4c2a69165b71f2c7b217ac319ff4c527
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:11:42 2011 -0400
get rid of Publish.connections
src/allmydata/mutable/publish.py | 27 +++++----------------------
1 files changed, 5 insertions(+), 22 deletions(-)
commit 64e9a53b3229ebe2f9ebf7ed502d539311d0e037
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:05:32 2011 -0400
change Publish.bad_servers
src/allmydata/mutable/publish.py | 10 +++++-----
1 files changed, 5 insertions(+), 5 deletions(-)
commit b85a934bef315a06bcfe00c9c12a3627fed2b918
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:03:07 2011 -0400
Publish.bad_servers: fix bug, this should be a set of serverids, not writers
src/allmydata/mutable/publish.py | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
commit 605ea15ec15ed671513819003ccd211cdb9761e0
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:00:21 2011 -0400
change .placed
src/allmydata/mutable/publish.py | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
commit f7aba37b1b345d5b6d5cb16e3b3f6f3c1afb658e
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 22:59:22 2011 -0400
temporarily stash IServer as .server on the "writer" object
src/allmydata/mutable/publish.py | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
commit f9b551d788e7db1f187fce5ab98ab5d5fe4e1c36
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 22:48:18 2011 -0400
change Publish.goal and API of log_goal() to use IServer, not serverid
src/allmydata/mutable/publish.py | 48 ++++++++++++++--------------
1 files changed, 24 insertions(+), 24 deletions(-)
commit 75f20616558e4900b8b1f685dd99aa838de6d452
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:27:02 2011 -0400
API of ServerMap.get_known_shares()
src/allmydata/mutable/publish.py | 16 ++++++++++------
src/allmydata/mutable/servermap.py | 7 ++-----
2 files changed, 12 insertions(+), 11 deletions(-)
commit 1c38c9d37bb08221b4418762234b1a62397b3b4b
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:20:29 2011 -0400
Publish.full_serverlist
src/allmydata/mutable/publish.py | 10 +++++-----
1 files changed, 5 insertions(+), 5 deletions(-)
commit b6cbd215a04b9cde31a7d92a97a7f048622b16f1
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:12:31 2011 -0400
API of ServerMap.all_servers()
src/allmydata/mutable/servermap.py | 19 ++++++-------------
1 files changed, 6 insertions(+), 13 deletions(-)
commit e63cd0315fae65357b1727ec6d5ff3c6e0d27c98
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:10:18 2011 -0400
remove ServerMap.connections, set_rref_for_serverid()
src/allmydata/mutable/servermap.py | 11 +----------
1 files changed, 1 insertions(+), 10 deletions(-)
commit 4df52db2f80eb12eefa5d57103c24893cde89553
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:04:06 2011 -0400
API of ServerMap.mark_server_reachable()
src/allmydata/mutable/servermap.py | 7 ++-----
1 files changed, 2 insertions(+), 5 deletions(-)
commit 69c715bde77944dc25181b3dbbeb042c816f9a1b
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:03:21 2011 -0400
API of ServerMap.mark_server_unreachable()
src/allmydata/mutable/servermap.py | 9 +++------
1 files changed, 3 insertions(+), 6 deletions(-)
commit 3d784d60eec1c508858e3a617e4411ffbcc3c1fa
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:02:03 2011 -0400
API of status.set_privkey_from()
src/allmydata/mutable/servermap.py | 7 +++----
1 files changed, 3 insertions(+), 4 deletions(-)
commit 544ed3ea29bed7e66da7fd29ca3f6f076f27a9e6
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:01:15 2011 -0400
API of status.add_per_server_time()
src/allmydata/mutable/servermap.py | 7 ++++---
1 files changed, 4 insertions(+), 3 deletions(-)
commit fffe5008b6320bd1e04c3c68389a2bf2ee383fa8
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:59:02 2011 -0400
remove unused .versionmap
src/allmydata/mutable/servermap.py | 7 -------
1 files changed, 0 insertions(+), 7 deletions(-)
commit 2816562e090d2294179db3588dafcca18de1bc2b
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:57:51 2011 -0400
remove serverid from all log messages. Also one unused lambda.
src/allmydata/mutable/servermap.py | 30 +++++++++++++-------------
1 files changed, 15 insertions(+), 15 deletions(-)
commit 28fa6b1a2738fa98c1f1dbd3d0e01ae98912d11f
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:54:30 2011 -0400
removed unused _readers
src/allmydata/mutable/servermap.py | 3 ---
1 files changed, 0 insertions(+), 3 deletions(-)
commit a8e4ed3d645ab592d1add6a1e69b6d1ebfb77817
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:54:16 2011 -0400
remove unused _sharemap
src/allmydata/mutable/servermap.py | 1 -
1 files changed, 0 insertions(+), 1 deletions(-)
commit 3f072e55cf1d0700f9fffe23f8f3a475725df588
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:49:03 2011 -0400
_must_query
src/allmydata/mutable/servermap.py | 8 ++++----
1 files changed, 4 insertions(+), 4 deletions(-)
commit c599a059b8df3f5785e4bf89fb6ecc6d8dcd708b
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:48:05 2011 -0400
_queries_outstanding
src/allmydata/mutable/servermap.py | 16 +++++++---------
1 files changed, 7 insertions(+), 9 deletions(-)
commit 7743759f98ac2c07926b2fdbd80bf52dfab33085
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:46:17 2011 -0400
_empty_servers
src/allmydata/mutable/servermap.py | 5 ++---
1 files changed, 2 insertions(+), 3 deletions(-)
commit 6bb1825916828a713a32cdf7f7411fa3ea2e1e5d
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:45:39 2011 -0400
_good_servers
src/allmydata/mutable/servermap.py | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
commit 1768fab1b51d8dd93ecabbaaabfadfa20cf6c3d4
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:44:59 2011 -0400
_bad_servers
src/allmydata/mutable/servermap.py | 14 +++++++-------
1 files changed, 7 insertions(+), 7 deletions(-)
commit dccbaef30f0ba714c746bf6d4a1a803c36e17b65
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:41:54 2011 -0400
API of _try_to_set_pubkey()
src/allmydata/mutable/servermap.py | 7 ++++---
1 files changed, 4 insertions(+), 3 deletions(-)
commit 0481ea70042ba3575f15eac7fd0780f8ece580cc
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:35:02 2011 -0400
API of notify_server_corruption()
src/allmydata/mutable/servermap.py | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
commit bea9cba18fb3b9c11bb22f18356a263ecec7351e
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:34:09 2011 -0400
API of _got_signature_one_share()
src/allmydata/mutable/servermap.py | 9 +++++----
1 files changed, 5 insertions(+), 4 deletions(-)
commit 1520123583cf78650706e114b15bb5b0ac1f4a14
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:32:33 2011 -0400
API of _try_to_validate_privkey()
src/allmydata/mutable/servermap.py | 9 +++++----
1 files changed, 5 insertions(+), 4 deletions(-)
commit 938852c9c8519c7a078f58a9b1f4dd8ec8b6715e
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:31:48 2011 -0400
API and internals of _add_lease_failed()
src/allmydata/mutable/servermap.py | 8 ++++----
1 files changed, 4 insertions(+), 4 deletions(-)
commit 3843dba367e3c19e176a622ab853cb51d2472ddf
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:30:37 2011 -0400
API of _privkey_query_failed()
src/allmydata/mutable/servermap.py | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
commit 2219a710e1633cd57d0ca0786490de87b3e19ba7
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:29:43 2011 -0400
fix bug in call to _privkey_query_failed, unrelated to refactoring
src/allmydata/mutable/servermap.py | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
commit ae615bec7d0d1b269710b6902797b12f9592ad62
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:27:17 2011 -0400
API of _got_corrupt_share()
src/allmydata/mutable/servermap.py | 17 +++++++++--------
1 files changed, 9 insertions(+), 8 deletions(-)
commit cb51c95a6f4e077278157a77dab060c8c1ad7a81
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:23:16 2011 -0400
API of _got_results()
src/allmydata/mutable/servermap.py | 9 +++++----
1 files changed, 5 insertions(+), 4 deletions(-)
commit bac9154fe0af18f226999a58ffc2362d8cf4b802
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:19:19 2011 -0400
API of _query_failed()
src/allmydata/mutable/servermap.py | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
commit fdc29a8ca95d4b5c503e5382b9e5d4d02141ba12
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:17:20 2011 -0400
API of _do_read()
src/allmydata/mutable/servermap.py | 6 ++++--
1 files changed, 4 insertions(+), 2 deletions(-)
commit e7e9e338f28d004aa4d423d11c65f1e271ac7322
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:20:21 2011 -0400
API of _do_query()
src/allmydata/mutable/servermap.py | 15 +++++++--------
1 files changed, 7 insertions(+), 8 deletions(-)
commit 330625b9dac4cdbe72a11464a893065b9aeed453
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:43:05 2011 -0400
next step: first batch of updates to ServermapUpdater
updates:
most method-local variables in update()
API of _build_initial_querylist()
API of _send_initial_requests()
.full_serverlist
.extra_servers
src/allmydata/mutable/servermap.py | 39 ++++++++++++++------------
1 files changed, 21 insertions(+), 18 deletions(-)
commit 4aadc584fa7dcb2daa86b048c81dee0049ba26d9
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:07:00 2011 -0400
internal change: index _bad_shares with IServer
src/allmydata/mutable/servermap.py | 20 ++++++++++----------
1 files changed, 10 insertions(+), 10 deletions(-)
commit 16d4e6fa82a9907dbdc92094213387c6a4164e41
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:20:47 2011 +0100
internal change: index _known_shares with IServer instead of serverid
callers are unchanged
src/allmydata/mutable/servermap.py | 42 +++++++++++++++----------
1 files changed, 25 insertions(+), 17 deletions(-)
commit ceeb5f4938cc814a0c75d1b8f4018aed965c2176
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:11:43 2011 +0100
accessors and name cleanup for servermap.Servermap.last_update_mode/time
src/allmydata/mutable/filenode.py | 6 +++---
src/allmydata/mutable/publish.py | 4 ++--
src/allmydata/mutable/servermap.py | 17 +++++++++++------
3 files changed, 16 insertions(+), 11 deletions(-)
commit 8d3cbda82661c0a7e5c3d3b65cf7a5d5ab7e32c0
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:11:14 2011 +0100
accessors and name cleanup for servermap.Servermap.problems
src/allmydata/mutable/servermap.py | 21 +++++++++++++--------
src/allmydata/test/test_mutable.py | 6 +++---
2 files changed, 16 insertions(+), 11 deletions(-)
commit 348f57988f79389db0aab7672e6eaa9a6d8e3219
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:10:41 2011 +0100
accessors and name cleanup for servermap.Servermap.bad_shares
src/allmydata/mutable/publish.py | 2 +-
src/allmydata/mutable/servermap.py | 30 ++++++++++++++-----------
2 files changed, 18 insertions(+), 14 deletions(-)
commit 520c9368134673cdf76c653c5e1bb91c2ab5d51e
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:10:05 2011 +0100
accessors and name cleanup for servermap.Servermap.servermap .
src/allmydata/mutable/publish.py | 14 +++++----
src/allmydata/mutable/servermap.py | 38 ++++++++++++++-----------
2 files changed, 29 insertions(+), 23 deletions(-)
commit b8b8dc38287a91dbdf494426ac801d9381ce5841
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:08:02 2011 +0100
fix reachable_servers
src/allmydata/mutable/checker.py | 3 ++-
src/allmydata/mutable/publish.py | 4 +++-
src/allmydata/mutable/servermap.py | 12 ++++++++++--
3 files changed, 15 insertions(+), 4 deletions(-)
commit cb0cfd1adfefad357c187aaaf690c3df68b622bc
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:06:03 2011 +0100
fix Servermap.unreachable_servers
src/allmydata/mutable/servermap.py | 11 ++++++++---
1 files changed, 8 insertions(+), 3 deletions(-)
commit 2d9ea79b94bd4db674d40386fda90825785ac495
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:03:48 2011 +0100
give ServerMap a StorageFarmBroker, temporary
this makes it possible for the ServerMap to accept bare serverids and still
build data structures with IServers
src/allmydata/mutable/checker.py | 2 +-
src/allmydata/mutable/filenode.py | 2 +-
src/allmydata/mutable/publish.py | 2 +-
src/allmydata/mutable/servermap.py | 5 +++--
src/allmydata/test/test_mutable.py | 8 ++++----
5 files changed, 10 insertions(+), 9 deletions(-)
commit 718d1aeff6fded893f65397806d22ece928b0dd4
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 13:43:30 2011 -0400
add StorageFarmBroker.get_server_for_id(), temporary helper
This will go away once we're passing IServers everywhere.
src/allmydata/storage_client.py | 2 ++
src/allmydata/test/no_network.py | 13 +++++++++++++
2 files changed, 15 insertions(+), 0 deletions(-)
commit ece20231d7fda0d503704842a4aa068dfbc2e54e
Author: Brian Warner <warner@lothar.com>
Date: Sun Oct 2 01:11:50 2011 +0100
add proper accessors for Servermap.connections, to make refactoring easier
src/allmydata/mutable/publish.py | 6 +++---
src/allmydata/mutable/retrieve.py | 10 +++++-----
src/allmydata/mutable/servermap.py | 17 +++++++++++------
3 files changed, 19 insertions(+), 14 deletions(-)
commit 3b943d6bf302ff702668081a612fc4fe2604cf9c
Author: Brian Warner <warner@lothar.com>
Date: Fri Sep 23 10:34:30 2011 -0700
mutable/servermap.py and neighbors: s/peer/server/
src/allmydata/mutable/checker.py | 22 +-
src/allmydata/mutable/publish.py | 204 +++++++-------
src/allmydata/mutable/servermap.py | 402 +++++++++++++-------------
src/allmydata/test/test_mutable.py | 18 +-
4 files changed, 323 insertions(+), 323 deletions(-)
IServer refactoring: pass IServer instances around, instead of peerids
refs #1363
This collapses 88 small incremental changes (each of which passes all tests)
into one big patch. The development process for the long path started with
adding some temporary scaffolding, changing one method at a time, then
removing the scaffolding. The individual pieces are as follows, in reverse
chronological order (the first patch is at the end of this comment):
commit 9bbe4174fd0d98a6cf47a8ef96e85d9ef34b2f9a
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 16:05:00 2011 -0400
immutable/downloader/status.py: correct comment
src/allmydata/immutable/downloader/status.py | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
commit 72146a7c7c91eac2f7c3ceb801eb7a1721376889
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 15:46:20 2011 -0400
remove temporary ServerMap._storage_broker
src/allmydata/mutable/checker.py | 2 +-
src/allmydata/mutable/filenode.py | 2 +-
src/allmydata/mutable/publish.py | 2 +-
src/allmydata/mutable/servermap.py | 5 ++---
src/allmydata/test/test_mutable.py | 8 ++++----
5 files changed, 9 insertions(+), 10 deletions(-)
commit d703096b41632c47d76414b12672e076a422ff5c
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 15:37:05 2011 -0400
remove temporary storage_broker.get_server_for_id()
src/allmydata/storage_client.py | 3 ---
src/allmydata/test/no_network.py | 13 -------------
2 files changed, 0 insertions(+), 16 deletions(-)
commit 620cc5d80882ef6f7decfd26af8a6c7c1ddf80d1
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 12:50:06 2011 -0400
API of Retrieve._try_to_validate_privkey(), trying to remove reader.server
src/allmydata/mutable/retrieve.py | 10 +++++-----
1 files changed, 5 insertions(+), 5 deletions(-)
commit 92f43f856f4a8b36c207d1b190ed8699b5a4ecb4
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 12:48:08 2011 -0400
API of Retrieve._validate_block(), trying to remove reader.server
src/allmydata/mutable/retrieve.py | 14 +++++++-------
1 files changed, 7 insertions(+), 7 deletions(-)
commit 572d5070761861a2190349d1ed8d85dbc25698a5
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 12:36:58 2011 -0400
API of Retrieve._mark_bad_share(), trying to remove reader.server
src/allmydata/mutable/retrieve.py | 21 +++++++++------------
1 files changed, 9 insertions(+), 12 deletions(-)
commit a793ff00c0de1e2eec7b46288fdf388c7a2bec89
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 12:06:13 2011 -0400
remove now-unused get_rref_for_serverid()
src/allmydata/mutable/servermap.py | 3 ---
1 files changed, 0 insertions(+), 3 deletions(-)
commit 1b9827cc9366bf90b93297fdd6832f2ad0480ce7
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 12:03:09 2011 -0400
Retrieve: stop adding .serverid attributes to readers
src/allmydata/mutable/retrieve.py | 1 -
1 files changed, 0 insertions(+), 1 deletions(-)
commit 5d4e9d491b19e49d2e443a1dfff2c672842c36ef
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 12:03:34 2011 -0400
return value of Retrieve(verify=True)
src/allmydata/mutable/checker.py | 11 ++++++-----
src/allmydata/mutable/retrieve.py | 3 +--
2 files changed, 7 insertions(+), 7 deletions(-)
commit e9ab7978c384e1f677cb7779dc449b1044face82
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:54:23 2011 -0400
Retrieve._bad_shares (but not return value, used by Verifier)
src/allmydata/mutable/retrieve.py | 7 ++++---
1 files changed, 4 insertions(+), 3 deletions(-)
commit 2d91926de233ec5c881f30e36b4a30ad92ab42a9
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:51:23 2011 -0400
Publish: stop adding .serverid attributes to writers
src/allmydata/mutable/publish.py | 9 ++-------
1 files changed, 2 insertions(+), 7 deletions(-)
commit 47c7a0105dec7cbf4f7e0a3ce800bbb85b15df4a
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:56:33 2011 -0400
API of get_write_enabler()
src/allmydata/mutable/filenode.py | 7 ++++---
src/allmydata/mutable/publish.py | 4 ++--
src/allmydata/test/no_network.py | 3 +++
3 files changed, 9 insertions(+), 5 deletions(-)
commit 9196a5c6590fdbfd660325ea8358b345887d3db0
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:46:24 2011 -0400
API of get_(renewal|cancel)_secret()
src/allmydata/mutable/filenode.py | 14 ++++++++------
src/allmydata/mutable/publish.py | 8 ++++----
src/allmydata/mutable/servermap.py | 5 ++---
3 files changed, 14 insertions(+), 13 deletions(-)
commit de7c1552f8c163eff5b6d820b5fb3b21c1b47cb5
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:41:52 2011 -0400
API of CorruptShareError. Also comment out some related+unused test_web.py code
src/allmydata/mutable/common.py | 13 +++++--------
src/allmydata/mutable/retrieve.py | 10 +++++-----
src/allmydata/mutable/servermap.py | 8 +++-----
src/allmydata/test/common.py | 13 ++++++++-----
4 files changed, 21 insertions(+), 23 deletions(-)
commit 2c1c314046b620c16f1e66d030c150d768b7d01e
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 12:01:46 2011 -0400
API of ServerMap.mark_bad_share()
src/allmydata/mutable/publish.py | 2 +-
src/allmydata/mutable/retrieve.py | 6 +++---
src/allmydata/mutable/servermap.py | 6 ++----
src/allmydata/test/test_mutable.py | 3 +--
4 files changed, 7 insertions(+), 10 deletions(-)
commit 1bed349030779fd0c378ae4e821384f953c6f6ff
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:11:17 2011 -0400
API+name of ServerMap.shares_on_server() : only for tests, so debug_ prefix
src/allmydata/mutable/servermap.py | 7 ++-----
src/allmydata/test/test_mutable.py | 6 +++---
2 files changed, 5 insertions(+), 8 deletions(-)
commit 2d32e448677d6b818692e801045d4115b29abf21
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:07:10 2011 -0400
API of ServerMap.all_servers_for_version()
src/allmydata/mutable/servermap.py | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
commit 48f3204d1889c3e7179578125c4bdef515af3d6a
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:04:50 2011 -0400
internals of ServerMap methods that use make_versionmap(), remove temp copy
src/allmydata/mutable/servermap.py | 28 +++++++++----------------
1 files changed, 10 insertions(+), 18 deletions(-)
commit 5c3da77b6c777a145bd5ddfaa4db849dc9495548
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 11:01:28 2011 -0400
API of ServerMap.make_versionmap()
src/allmydata/mutable/checker.py | 4 ++--
src/allmydata/mutable/retrieve.py | 5 ++---
src/allmydata/mutable/servermap.py | 4 ++--
src/allmydata/test/test_mutable.py | 7 ++++---
4 files changed, 10 insertions(+), 10 deletions(-)
commit b6882ece49afb4c507d118af2db346fa329209dc
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 10:53:38 2011 -0400
make a copy of ServerMap.make_versionmap() (_make_versionmap2) for internal use
src/allmydata/mutable/servermap.py | 18 +++++++++++++-----
1 files changed, 13 insertions(+), 5 deletions(-)
commit 963f8e63faf32b950eb1b8103cd2ff16fe8f0151
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:45:58 2011 -0400
API of RetrieveStatus.add_problem()
src/allmydata/mutable/retrieve.py | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
commit 4976d29ffae565a048851601c29013bbae2976d8
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:45:05 2011 -0400
API of RetrieveStatus.add_fetch_timing()
src/allmydata/mutable/retrieve.py | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
commit d057d3bbba72663ee148a8b916bc2d52be2e3982
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:44:04 2011 -0400
API of Retrieve.notify_server_corruption()
src/allmydata/mutable/retrieve.py | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
commit 8a2a81e46671c860610e0e96d6add1a57551f22d
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:42:32 2011 -0400
remove unused _outstanding_queries
src/allmydata/mutable/retrieve.py | 1 -
1 files changed, 0 insertions(+), 1 deletions(-)
commit 56d12cc9968d03ccd53764455c671122c4f391d1
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:40:57 2011 -0400
change Retrieve.remaining_sharemap
src/allmydata/mutable/retrieve.py | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
commit 4f0b7af4821f43290bfc70f2b1fc30149ad81281
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 10:40:18 2011 -0400
accessor for PublishStatus._problems
src/allmydata/mutable/publish.py | 4 +++-
src/allmydata/web/status.py | 2 +-
2 files changed, 4 insertions(+), 2 deletions(-)
commit 627087cf66d0b8cc519f4d551a967a7bd9b6a741
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 10:36:39 2011 -0400
accessor for RetrieveStatus._problems
src/allmydata/mutable/retrieve.py | 8 ++++++--
src/allmydata/web/status.py | 2 +-
2 files changed, 7 insertions(+), 3 deletions(-)
commit ca7dea81f03801b1c7353fc00ecba689268109cf
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:35:32 2011 -0400
add .server to "reader", so we can get at it later
src/allmydata/mutable/retrieve.py | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
commit 6ef516e24908ec195af084a7550d1921a5e983b0
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:32:32 2011 -0400
temporarily give Retrieve a _storage_broker, so it can map serverids to servers
src/allmydata/mutable/checker.py | 3 ++-
src/allmydata/mutable/filenode.py | 6 ++++--
src/allmydata/mutable/retrieve.py | 5 +++--
src/allmydata/test/test_mutable.py | 4 ++--
4 files changed, 11 insertions(+), 7 deletions(-)
commit afe08e4dd3f4ff9ff7e8a2a8d28b181e3625bcc9
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:21:51 2011 -0400
mutable/retrieve.py: s/peer/server/
src/allmydata/mutable/retrieve.py | 82 +++++++++++++-------------
src/allmydata/test/test_mutable.py | 6 +-
2 files changed, 44 insertions(+), 44 deletions(-)
commit 910afcb5d7f274880f68dd6cdb5b05f2bbc29adc
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:16:01 2011 -0400
web.status.PublishStatusPage: add comment, I think .problems isn't exercised
src/allmydata/web/status.py | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
commit 311466dd8c931bbba40d590ade867704282e7f1a
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:48:16 2011 -0400
API of PublishStatus.add_per_server_time()
src/allmydata/mutable/publish.py | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
commit 2df5faa1b6cbfbaded520d2320305a62fe961118
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:46:37 2011 -0400
more simplifications
src/allmydata/mutable/publish.py | 4 +---
1 files changed, 1 insertions(+), 3 deletions(-)
commit 6ac4544a3da385f2aad9392f906b90192f4f919a
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:44:08 2011 -0400
API of ServerMap.version_on_server()
src/allmydata/mutable/publish.py | 2 +-
src/allmydata/mutable/servermap.py | 4 ++--
src/allmydata/test/test_mutable.py | 5 ++---
3 files changed, 5 insertions(+), 6 deletions(-)
commit 3e187e322511072e4683329df6b2c6c733a66dba
Author: Brian Warner <warner@lothar.com>
Date: Tue Oct 4 00:16:32 2011 -0400
API of ServerMap.make_sharemap()
src/allmydata/mutable/servermap.py | 4 ++--
src/allmydata/test/test_mutable.py | 7 ++++---
src/allmydata/web/status.py | 4 ++--
3 files changed, 8 insertions(+), 7 deletions(-)
commit 318feed8437bdd8d4943c6569d38f7b54b6313cc
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:36:19 2011 -0400
small cleanups
src/allmydata/mutable/publish.py | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
commit bd459ed5714e1db5a7163935c54b7b0b56db8349
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:33:39 2011 -0400
API of ServerMap.add_new_share()
src/allmydata/mutable/publish.py | 4 ++--
src/allmydata/mutable/servermap.py | 6 ++----
2 files changed, 4 insertions(+), 6 deletions(-)
commit f2804fb6ed11d80088e0da8ed48e6c2922f2ffef
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:30:26 2011 -0400
API of ServerMap.get_bad_shares()
src/allmydata/mutable/publish.py | 3 +--
src/allmydata/mutable/servermap.py | 9 ++++-----
2 files changed, 5 insertions(+), 7 deletions(-)
commit 965074a47b3ce1431cb46d9a233840afcf9105f5
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:26:58 2011 -0400
more small cleanups
src/allmydata/mutable/publish.py | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
commit 38020da34f034f8889947dd3dc05e087ffff7106
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:18:47 2011 -0400
change Publish.bad_share_checkstrings
src/allmydata/mutable/publish.py | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
commit 5efebcbd2ee0c2f299ea86f7591d856c0f265304
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:16:31 2011 -0400
change internals of Publish.update_goal()
src/allmydata/mutable/publish.py | 8 +++-----
1 files changed, 3 insertions(+), 5 deletions(-)
commit e91b55ff4c2a69165b71f2c7b217ac319ff4c527
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:11:42 2011 -0400
get rid of Publish.connections
src/allmydata/mutable/publish.py | 27 +++++----------------------
1 files changed, 5 insertions(+), 22 deletions(-)
commit 64e9a53b3229ebe2f9ebf7ed502d539311d0e037
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:05:32 2011 -0400
change Publish.bad_servers
src/allmydata/mutable/publish.py | 10 +++++-----
1 files changed, 5 insertions(+), 5 deletions(-)
commit b85a934bef315a06bcfe00c9c12a3627fed2b918
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:03:07 2011 -0400
Publish.bad_servers: fix bug, this should be a set of serverids, not writers
src/allmydata/mutable/publish.py | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
commit 605ea15ec15ed671513819003ccd211cdb9761e0
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 23:00:21 2011 -0400
change .placed
src/allmydata/mutable/publish.py | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
commit f7aba37b1b345d5b6d5cb16e3b3f6f3c1afb658e
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 22:59:22 2011 -0400
temporarily stash IServer as .server on the "writer" object
src/allmydata/mutable/publish.py | 2 ++
1 files changed, 2 insertions(+), 0 deletions(-)
commit f9b551d788e7db1f187fce5ab98ab5d5fe4e1c36
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 22:48:18 2011 -0400
change Publish.goal and API of log_goal() to use IServer, not serverid
src/allmydata/mutable/publish.py | 48 ++++++++++++++--------------
1 files changed, 24 insertions(+), 24 deletions(-)
commit 75f20616558e4900b8b1f685dd99aa838de6d452
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:27:02 2011 -0400
API of ServerMap.get_known_shares()
src/allmydata/mutable/publish.py | 16 ++++++++++------
src/allmydata/mutable/servermap.py | 7 ++-----
2 files changed, 12 insertions(+), 11 deletions(-)
commit 1c38c9d37bb08221b4418762234b1a62397b3b4b
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:20:29 2011 -0400
Publish.full_serverlist
src/allmydata/mutable/publish.py | 10 +++++-----
1 files changed, 5 insertions(+), 5 deletions(-)
commit b6cbd215a04b9cde31a7d92a97a7f048622b16f1
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:12:31 2011 -0400
API of ServerMap.all_servers()
src/allmydata/mutable/servermap.py | 19 ++++++-------------
1 files changed, 6 insertions(+), 13 deletions(-)
commit e63cd0315fae65357b1727ec6d5ff3c6e0d27c98
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:10:18 2011 -0400
remove ServerMap.connections, set_rref_for_serverid()
src/allmydata/mutable/servermap.py | 11 +----------
1 files changed, 1 insertions(+), 10 deletions(-)
commit 4df52db2f80eb12eefa5d57103c24893cde89553
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:04:06 2011 -0400
API of ServerMap.mark_server_reachable()
src/allmydata/mutable/servermap.py | 7 ++-----
1 files changed, 2 insertions(+), 5 deletions(-)
commit 69c715bde77944dc25181b3dbbeb042c816f9a1b
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:03:21 2011 -0400
API of ServerMap.mark_server_unreachable()
src/allmydata/mutable/servermap.py | 9 +++------
1 files changed, 3 insertions(+), 6 deletions(-)
commit 3d784d60eec1c508858e3a617e4411ffbcc3c1fa
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:02:03 2011 -0400
API of status.set_privkey_from()
src/allmydata/mutable/servermap.py | 7 +++----
1 files changed, 3 insertions(+), 4 deletions(-)
commit 544ed3ea29bed7e66da7fd29ca3f6f076f27a9e6
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:01:15 2011 -0400
API of status.add_per_server_time()
src/allmydata/mutable/servermap.py | 7 ++++---
1 files changed, 4 insertions(+), 3 deletions(-)
commit fffe5008b6320bd1e04c3c68389a2bf2ee383fa8
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:59:02 2011 -0400
remove unused .versionmap
src/allmydata/mutable/servermap.py | 7 -------
1 files changed, 0 insertions(+), 7 deletions(-)
commit 2816562e090d2294179db3588dafcca18de1bc2b
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:57:51 2011 -0400
remove serverid from all log messages. Also one unused lambda.
src/allmydata/mutable/servermap.py | 30 +++++++++++++-------------
1 files changed, 15 insertions(+), 15 deletions(-)
commit 28fa6b1a2738fa98c1f1dbd3d0e01ae98912d11f
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:54:30 2011 -0400
removed unused _readers
src/allmydata/mutable/servermap.py | 3 ---
1 files changed, 0 insertions(+), 3 deletions(-)
commit a8e4ed3d645ab592d1add6a1e69b6d1ebfb77817
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:54:16 2011 -0400
remove unused _sharemap
src/allmydata/mutable/servermap.py | 1 -
1 files changed, 0 insertions(+), 1 deletions(-)
commit 3f072e55cf1d0700f9fffe23f8f3a475725df588
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:49:03 2011 -0400
_must_query
src/allmydata/mutable/servermap.py | 8 ++++----
1 files changed, 4 insertions(+), 4 deletions(-)
commit c599a059b8df3f5785e4bf89fb6ecc6d8dcd708b
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:48:05 2011 -0400
_queries_outstanding
src/allmydata/mutable/servermap.py | 16 +++++++---------
1 files changed, 7 insertions(+), 9 deletions(-)
commit 7743759f98ac2c07926b2fdbd80bf52dfab33085
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:46:17 2011 -0400
_empty_servers
src/allmydata/mutable/servermap.py | 5 ++---
1 files changed, 2 insertions(+), 3 deletions(-)
commit 6bb1825916828a713a32cdf7f7411fa3ea2e1e5d
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:45:39 2011 -0400
_good_servers
src/allmydata/mutable/servermap.py | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
commit 1768fab1b51d8dd93ecabbaaabfadfa20cf6c3d4
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:44:59 2011 -0400
_bad_servers
src/allmydata/mutable/servermap.py | 14 +++++++-------
1 files changed, 7 insertions(+), 7 deletions(-)
commit dccbaef30f0ba714c746bf6d4a1a803c36e17b65
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:41:54 2011 -0400
API of _try_to_set_pubkey()
src/allmydata/mutable/servermap.py | 7 ++++---
1 files changed, 4 insertions(+), 3 deletions(-)
commit 0481ea70042ba3575f15eac7fd0780f8ece580cc
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:35:02 2011 -0400
API of notify_server_corruption()
src/allmydata/mutable/servermap.py | 6 +++---
1 files changed, 3 insertions(+), 3 deletions(-)
commit bea9cba18fb3b9c11bb22f18356a263ecec7351e
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:34:09 2011 -0400
API of _got_signature_one_share()
src/allmydata/mutable/servermap.py | 9 +++++----
1 files changed, 5 insertions(+), 4 deletions(-)
commit 1520123583cf78650706e114b15bb5b0ac1f4a14
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:32:33 2011 -0400
API of _try_to_validate_privkey()
src/allmydata/mutable/servermap.py | 9 +++++----
1 files changed, 5 insertions(+), 4 deletions(-)
commit 938852c9c8519c7a078f58a9b1f4dd8ec8b6715e
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:31:48 2011 -0400
API and internals of _add_lease_failed()
src/allmydata/mutable/servermap.py | 8 ++++----
1 files changed, 4 insertions(+), 4 deletions(-)
commit 3843dba367e3c19e176a622ab853cb51d2472ddf
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:30:37 2011 -0400
API of _privkey_query_failed()
src/allmydata/mutable/servermap.py | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
commit 2219a710e1633cd57d0ca0786490de87b3e19ba7
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:29:43 2011 -0400
fix bug in call to _privkey_query_failed, unrelated to refactoring
src/allmydata/mutable/servermap.py | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
commit ae615bec7d0d1b269710b6902797b12f9592ad62
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:27:17 2011 -0400
API of _got_corrupt_share()
src/allmydata/mutable/servermap.py | 17 +++++++++--------
1 files changed, 9 insertions(+), 8 deletions(-)
commit cb51c95a6f4e077278157a77dab060c8c1ad7a81
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:23:16 2011 -0400
API of _got_results()
src/allmydata/mutable/servermap.py | 9 +++++----
1 files changed, 5 insertions(+), 4 deletions(-)
commit bac9154fe0af18f226999a58ffc2362d8cf4b802
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:19:19 2011 -0400
API of _query_failed()
src/allmydata/mutable/servermap.py | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
commit fdc29a8ca95d4b5c503e5382b9e5d4d02141ba12
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:17:20 2011 -0400
API of _do_read()
src/allmydata/mutable/servermap.py | 6 ++++--
1 files changed, 4 insertions(+), 2 deletions(-)
commit e7e9e338f28d004aa4d423d11c65f1e271ac7322
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:20:21 2011 -0400
API of _do_query()
src/allmydata/mutable/servermap.py | 15 +++++++--------
1 files changed, 7 insertions(+), 8 deletions(-)
commit 330625b9dac4cdbe72a11464a893065b9aeed453
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 14:43:05 2011 -0400
next step: first batch of updates to ServermapUpdater
updates:
most method-local variables in update()
API of _build_initial_querylist()
API of _send_initial_requests()
.full_serverlist
.extra_servers
src/allmydata/mutable/servermap.py | 39 ++++++++++++++------------
1 files changed, 21 insertions(+), 18 deletions(-)
commit 4aadc584fa7dcb2daa86b048c81dee0049ba26d9
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 15:07:00 2011 -0400
internal change: index _bad_shares with IServer
src/allmydata/mutable/servermap.py | 20 ++++++++++----------
1 files changed, 10 insertions(+), 10 deletions(-)
commit 16d4e6fa82a9907dbdc92094213387c6a4164e41
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:20:47 2011 +0100
internal change: index _known_shares with IServer instead of serverid
callers are unchanged
src/allmydata/mutable/servermap.py | 42 +++++++++++++++----------
1 files changed, 25 insertions(+), 17 deletions(-)
commit ceeb5f4938cc814a0c75d1b8f4018aed965c2176
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:11:43 2011 +0100
accessors and name cleanup for servermap.Servermap.last_update_mode/time
src/allmydata/mutable/filenode.py | 6 +++---
src/allmydata/mutable/publish.py | 4 ++--
src/allmydata/mutable/servermap.py | 17 +++++++++++------
3 files changed, 16 insertions(+), 11 deletions(-)
commit 8d3cbda82661c0a7e5c3d3b65cf7a5d5ab7e32c0
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:11:14 2011 +0100
accessors and name cleanup for servermap.Servermap.problems
src/allmydata/mutable/servermap.py | 21 +++++++++++++--------
src/allmydata/test/test_mutable.py | 6 +++---
2 files changed, 16 insertions(+), 11 deletions(-)
commit 348f57988f79389db0aab7672e6eaa9a6d8e3219
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:10:41 2011 +0100
accessors and name cleanup for servermap.Servermap.bad_shares
src/allmydata/mutable/publish.py | 2 +-
src/allmydata/mutable/servermap.py | 30 ++++++++++++++-----------
2 files changed, 18 insertions(+), 14 deletions(-)
commit 520c9368134673cdf76c653c5e1bb91c2ab5d51e
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:10:05 2011 +0100
accessors and name cleanup for servermap.Servermap.servermap .
src/allmydata/mutable/publish.py | 14 +++++----
src/allmydata/mutable/servermap.py | 38 ++++++++++++++-----------
2 files changed, 29 insertions(+), 23 deletions(-)
commit b8b8dc38287a91dbdf494426ac801d9381ce5841
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:08:02 2011 +0100
fix reachable_servers
src/allmydata/mutable/checker.py | 3 ++-
src/allmydata/mutable/publish.py | 4 +++-
src/allmydata/mutable/servermap.py | 12 ++++++++++--
3 files changed, 15 insertions(+), 4 deletions(-)
commit cb0cfd1adfefad357c187aaaf690c3df68b622bc
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:06:03 2011 +0100
fix Servermap.unreachable_servers
src/allmydata/mutable/servermap.py | 11 ++++++++---
1 files changed, 8 insertions(+), 3 deletions(-)
commit 2d9ea79b94bd4db674d40386fda90825785ac495
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 18:03:48 2011 +0100
give ServerMap a StorageFarmBroker, temporary
this makes it possible for the ServerMap to accept bare serverids and still
build data structures with IServers
src/allmydata/mutable/checker.py | 2 +-
src/allmydata/mutable/filenode.py | 2 +-
src/allmydata/mutable/publish.py | 2 +-
src/allmydata/mutable/servermap.py | 5 +++--
src/allmydata/test/test_mutable.py | 8 ++++----
5 files changed, 10 insertions(+), 9 deletions(-)
commit 718d1aeff6fded893f65397806d22ece928b0dd4
Author: Brian Warner <warner@lothar.com>
Date: Mon Oct 3 13:43:30 2011 -0400
add StorageFarmBroker.get_server_for_id(), temporary helper
This will go away once we're passing IServers everywhere.
src/allmydata/storage_client.py | 2 ++
src/allmydata/test/no_network.py | 13 +++++++++++++
2 files changed, 15 insertions(+), 0 deletions(-)
commit ece20231d7fda0d503704842a4aa068dfbc2e54e
Author: Brian Warner <warner@lothar.com>
Date: Sun Oct 2 01:11:50 2011 +0100
add proper accessors for Servermap.connections, to make refactoring easier
src/allmydata/mutable/publish.py | 6 +++---
src/allmydata/mutable/retrieve.py | 10 +++++-----
src/allmydata/mutable/servermap.py | 17 +++++++++++------
3 files changed, 19 insertions(+), 14 deletions(-)
commit 3b943d6bf302ff702668081a612fc4fe2604cf9c
Author: Brian Warner <warner@lothar.com>
Date: Fri Sep 23 10:34:30 2011 -0700
mutable/servermap.py and neighbors: s/peer/server/
src/allmydata/mutable/checker.py | 22 +-
src/allmydata/mutable/publish.py | 204 +++++++-------
src/allmydata/mutable/servermap.py | 402 +++++++++++++-------------
src/allmydata/test/test_mutable.py | 18 +-
4 files changed, 323 insertions(+), 323 deletions(-)
2011-10-31 21:03:19 -07:00
Brian Warner
05dbb1cf86
update project-home URLs: /trac/tahoe/ was replaced by /trac/tahoe-lafs/
2011-10-30 12:14:28 -07:00
Zooko O'Whielacronx
d9c1064d42
undo the effects of a patch I hadn't intended to commit, named "debugprint the values of blocks and hashes thereof; make the test data and the seg size small in order to make the debugprints easy to look at"
...
rolling back:
Thu Sep 29 23:46:28 MDT 2011 zooko@zooko.com
* debugprint the values of blocks and hashes thereof; make the test data and the seg size small in order to make the debugprints easy to look at
M ./src/allmydata/mutable/publish.py -1 +2
M ./src/allmydata/mutable/retrieve.py +3
M ./src/allmydata/test/test_mutable.py -2 +2
2011-10-28 22:03:49 +00:00
Zooko O'Whielacronx
416701e404
debugprint the values of blocks and hashes thereof; make the test data and the seg size small in order to make the debugprints easy to look at
2011-09-30 05:46:28 +00:00
Brian Warner
633641174a
mutable/retrieve: don't write() after we've been pauseProducer'ed
...
This fixes a test failure found against current Twisted trunk in
test_mutable.Filenode.test_retrieve_producer_mdmf (when it uses
PausingAndStoppingConsumer). There must be some sort of race: I could
make it fail against Twisted-11.0 if I just increased the 0.5s delay in
test_download.PausingAndStoppingConsumer to about 0.6s, and could make
Twisted-trunk pass by reducing it to about 0.3s .
I fixed the test (as opposed to the bug) by replacing the delay with a
simple reliable eventually(), and adding extra asserts to fail the test
if the consumer's write() method is called while the producer is
supposed to be paused
The bug itself was that mutable.retrieve.Retrieve wasn't checking the
"stopped" flag after resuming from a pause, and thus delivered one
segment to a consumer that wasn't expecting it. I split out
stopped-flag-checking to separate function, which is now called
immediately after _check_for_paused(). I also cleaned up some Deferred
usage and whitespace.
2011-10-16 17:24:00 -07:00
Brian Warner
fff237be9a
remove interpreter shbang lines from non-executables
...
thanks to Greg Troxel for the catch
2011-10-14 10:23:01 -07:00
Brian Warner
d28041fbe4
CLI: don't deprecate --mutable, small docs fixes. refs #1561
...
Also don't accept 'tahoe mkdir --format=chk'.
2011-10-13 21:00:02 -07:00
Brian Warner
57ee56533f
add --format= to 'tahoe put'/'mkdir', remove --mutable-type. Closes #1561
2011-10-13 20:15:00 -07:00
david-sarah
22da015dd1
Fix some more potential bugs in test code exposed by check-miscaptures.py. refs #1556
2011-10-06 20:38:47 -07:00
david-sarah
bdfa7b3772
Fix some potential bugs in test code exposed by check-miscaptures.py. refs #1556
2011-10-06 19:34:43 -07:00
Brian Warner
11ce612589
oops, missed a test failure
2011-10-13 09:37:13 -07:00
Brian Warner
210761cd49
misc mutable-type fixes:
...
* fix tahoe.cfg control of default mutable type
* tolerate arbitrary case in [client]mutable.format value
* small docs improvements
* use get_mutable_type() as a format-is-mutable predicate
* tighten up error message
2011-10-13 09:32:29 -07:00
Brian Warner
01b00dc7dc
webapi: use all-caps "SDMF"/"MDMF" acronyms in t=json response
...
docs: upcase examples of t=json output and format= input
2011-10-13 09:31:43 -07:00
Brian Warner
dad354b275
webapi: handle format=, remove mutable-type=
...
* fix CLI commands (put, mkdir) to send format=, not mutable-type=
* fix tests
* test_cli: fix tests that observe t=json output, don't ignore failures in
'tahoe put'
* fix handling of version= to make it easier to use the default
* interpret ?mutable=true&format=MDMF as MDMF, not SDMF
2011-10-13 09:29:51 -07:00
david-sarah
2acc0419b9
Tests for ref #1547
2011-10-01 20:53:16 -07:00
david-sarah
ac780a4148
test_web.py: minor cleanups, mainly to make the first argument to shouldFail tests consistent
2011-10-01 21:03:32 -07:00
david-sarah
38d819ef44
Tests for ref #1552
2011-10-01 21:00:36 -07:00
david-sarah
5ae446fbd2
test/common.py: in shouldFail and shouldHTTPError, when the raised exception does not include the expected substring (or, for shouldHTTPError, when the status code is wrong), mention which test that happened in.
2011-10-10 17:22:27 -07:00
david-sarah
bc50edc86e
no_network.py: Clean up whitespace around code changed by previous patch.
2011-10-03 18:04:07 -07:00
david-sarah
ee25dcd2fc
no_network.py: Fix potential bugs in some tests due to capture of slots in for loops.
2011-10-03 18:02:31 -07:00
Brian Warner
0716c496c8
MDMF: remove extension fields from caps, tolerate arbitrary ones. Fixes #1526
...
The filecaps used to be produced with hints for 'k' and segsize, but they
weren't actually used, and doing so had the potential to limit how we change
those filecaps in the future. Also the parsing code had some problems dealing
with other numbers of extensions. Removing the existing fields and making the
parser tolerate (and ignore) extra ones makes MDMF more future-proof.
2011-10-02 00:35:53 +01:00
david-sarah
5ba0529b87
test/test_runner.py: BinTahoe.test_path has rare nondeterministic failures; this patch probably fixes a problem where the actual cause of failure is masked by a string conversion error.
2011-09-27 15:53:36 -07:00
kevan
2187f27ad0
test/test_mutable: reenable mdmf publish surprise test
2011-09-24 16:54:15 -07:00
david-sarah
4af626a798
test_mutable.py: update SkipTest message for test_publish_surprise_mdmf to reference the right ticket number. refs #1540 .
2011-09-23 14:16:22 -07:00
david-sarah
5d3d0dc336
test_mutable.py: skip test_publish_surprise_mdmf, which is causing an error. refs #1534 , #393
2011-09-20 11:33:19 -07:00
kevan
d93b2965c3
test/test_mutable: write publish surprise test for MDMF, rename existing test_publish_surprise to clarify that it is for SDMF
2011-09-17 17:36:57 -07:00
kevan
19cf58f2aa
test/test_mutable: refactor publish surprise test into common test fixture, rewrite test_publish_surprise to use test fixture
2011-09-17 17:35:33 -07:00
Brian Warner
8e69b94588
Make platform-detection code tolerate linux-3.0, patch by zooko.
...
Otherwise address-autodetection can't find ifconfig. refs #1536
2011-09-15 13:26:20 -07:00
david-sarah
3f756e9429
test_web.py: fix a bug in _count_leases that was causing us to check only the lease count of one share file, not of all share files as intended.
2011-09-15 11:51:26 -07:00
Zooko O'Whielacronx
2025e42ec2
tests: bump up the timeout in this test that fails on FreeStorm's CentOS in order to see if it is just very slow
2011-09-12 19:42:55 -07:00
Zooko O'Whielacronx
942c5e5162
storage: test that the storage server ignores requests to extend shares by sending a new_length, and that the storage server fills exposed holes with 0 to avoid "palimpsest" exposure of previous contents
...
ref. #1528
2011-09-12 15:25:54 -07:00
Zooko O'Whielacronx
65de17245d
storage: test that the storage server does *not* have a "remote_cancel_lease" function
...
We're removing this function because it is currently unused, because it is dangerous, and because the bug described in #1528 leaks the cancellation secret, which allows anyone who knows a file's storage index to abuse this function to delete shares of that file.
ref. #1528
2011-09-12 15:23:24 -07:00
Zooko O'Whielacronx
cffc987804
immutable: test whether the server allows clients to read past the end of share data, which would allow them to learn the cancellation secret
...
Also test whether the server explicitly declares that it prevents this problem.
ref #1528
2011-09-12 15:12:01 -07:00
Brian Warner
a15ce96846
Retrieve: implement/test stopProducing
2011-09-09 11:11:50 -07:00
Brian Warner
748e419a9b
move DownloadStopped from download.common to interfaces
2011-09-09 11:11:50 -07:00
Brian Warner
2edfb1a334
Retrieve: remove the initial prefix-is-still-good check
...
This check needs to be done with each fetch from the storage server, to
detect when someone has changed the share (i.e. our servermap goes stale).
Doing it just once at the beginning of retrieve isn't enough: a write might
occur after the first segment but before the second, etc.
_try_to_validate_prefix() was not removed: it will be used by the future
check-with-each-fetch code.
test_mutable.Roundtrip.test_corrupt_all_seqnum_late was disabled, since it
fails until this check is brought back. (the corruption it applies only
touches the prefix, not the block data, so the check-less retrieve actually
tolerates it). Don't forget to re-enable it once the check is brought back.
2011-09-09 11:11:50 -07:00
Brian Warner
2b4f2b7fa3
MDMFSlotReadProxy: remove the queue
...
This is a neat trick to reduce Foolscap overhead, but the need for an
explicit flush() complicates the Retrieve path and makes it prone to
lost-progress bugs.
Also change test_mutable.FakeStorageServer to tolerate multiple reads of the
same share in a row, a limitation exposed by turning off the queue.
2011-09-09 11:11:50 -07:00
david-sarah
6b97e6f407
CLI: test_cli.py -- remove an unnecessary call in test_mkdir_mutable_type. refs #1527
2011-09-06 11:37:30 -07:00
david-sarah
1e59930fda
CLI: improve test for 'tahoe mkdir --mutable-type='. refs #1527
2011-09-06 11:30:20 -07:00
david-sarah
23f46b758e
CLI: make the --mutable-type option value for 'tahoe put' and 'tahoe mkdir' case-insensitive, and change --help for these commands accordingly. fixes #1527
2011-09-04 19:09:22 -07:00
Kevan Carstensen
64996a913d
cli: make --mutable-type imply --mutable in 'tahoe put'
2011-09-03 12:09:20 -07:00
david-sarah
1b7b3bd0d4
SFTP: add a comment about a subtle interaction between OverwriteableFileConsumer and GeneralSFTPFile, and test the case it is commenting on.
2011-09-03 15:23:04 -07:00
Brian Warner
7191b0b32e
test_mutable.Version: consolidate some tests, reduce runtime from 19s to 15s
2011-08-30 22:04:51 -07:00
Kevan Carstensen
32e30c9023
mutable/retrieve: handle the case where self._read_length is 0.
...
Note that the downloader will still fetch a segment for a zero-length
read, which is wasteful. Fixing that isn't specifically required to fix
#1512 , but it should probably be fixed before 1.9.
2011-08-30 14:01:41 -07:00
Brian Warner
980eb778c1
test_mutable.Update: only upload the files needed for each test. refs #1500
...
This first step shaves 15% off the runtime: from 139s to 119s on my laptop.
It also fixes a couple of places where a Deferred was being dropped, which
would cause two tests to run in parallel and also confuse error reporting.
2011-08-29 00:27:17 -07:00
Brian Warner
fd676a5846
Let Uploader retain History instead of passing it into upload(). Fixes #1079 .
...
This consistently records all immutable uploads in the Recent Uploads And
Downloads page, regardless of code path. Previously, certain webapi upload
operations (like PUT /uri/$DIRCAP/newchildname) failed to pass the History
object and were left out.
2011-08-28 23:32:46 -07:00
Brian Warner
d575ccba28
Teach 'tahoe debug catalog-shares about MDMF. Closes #1507 .
2011-08-28 01:09:31 -07:00
Brian Warner
a3d6fd4992
teach 'tahoe debug dump-share' about MDMF and offsets. refs #1507
2011-08-28 00:38:34 -07:00
Brian Warner
101ca8dfe4
test_mutable.Version.test_debug: use splitlines() to fix buildslaves
...
Any slave running in a directory with spaces in the name was miscounting
shares, causing the test to fail.
2011-08-27 23:47:28 -07:00
Brian Warner
7d48f902fc
test_mutable.Version: exercise 'tahoe debug find-shares' on MDMF. refs #1507
...
Also changes NoNetworkGrid to put shares in storage/shares/ .
2011-08-27 17:55:42 -07:00
Brian Warner
80bdc0f5b7
test_mutable.py: oops, missed a .todo
2011-08-27 17:21:18 -07:00
Brian Warner
e0eaeff7f5
test_mutable: merge davidsarah's patch with my Version refactorings
2011-08-27 16:57:07 -07:00
david-sarah
88989a4ea2
Additional tests for MDMF URIs and for zero-length files. refs #393
2011-08-22 18:15:32 -07:00
david-sarah
ac7b8400d4
Additional tests for zero-length partial reads and updates to mutable versions. refs #393
2011-08-21 18:41:11 -07:00
Brian Warner
03ba8065ce
test_mutable.Version: factor out some expensive uploads, save 25% runtime
2011-08-27 16:27:37 -07:00
Brian Warner
370e6f271e
SDMF: update filenode with correct k/N after Retrieve. Fixes #1510 .
...
Without this, we get a regression when modifying a mutable file that was
created with more shares (larger N) than our current tahoe.cfg . The
modification attempt creates new versions of the (0,1,..,newN-1) shares, but
leaves the old versions of the (newN,..,oldN-1) shares alone (and throws a
assertion error in SDMFSlotWriteProxy.finish_publishing in the process).
The mixed versions that result (some shares with e.g. N=10, some with N=20,
such that both versions are recoverable) cause problems for the Publish code,
even before MDMF landed. Might be related to refs #1390 and refs #1042 .
2011-08-27 15:50:31 -07:00
Brian Warner
482a7dd3f1
Add 'tahoe debug dump-cap' support for MDMF, DIR2-CHK, DIR2-MDMF. refs #1507 .
...
This also adds tests for all those cases, and fixes an omission in uri.py
that broke parsing of DIR2-MDMF-Verifier and DIR2-CHK-Verifier.
2011-08-27 12:50:48 -07:00
Brian Warner
522868d5e9
MDMF: more writable/writeable consistentifications
2011-08-27 12:06:02 -07:00
Brian Warner
48544a251d
MDMF: s/Writable/Writeable/g, for consistency with existing SDMF code
2011-08-27 11:33:57 -07:00
Brian Warner
349093fa2e
test_mutable.Update: increase timeout from 120s to 400s, slaves are failing
2011-08-25 16:01:40 -07:00
Zooko O'Whielacronx
5bf5404ba1
tests: fix check_memory test
...
fixes #1503
2011-08-25 13:11:16 -07:00
Brian Warner
9e613ee9b3
test_mutable.Update: remove .timeout overrides, otherwise tests ERROR
2011-08-24 19:24:55 -07:00
david-sarah
3d7a32647c
Implementation, tests and docs for blacklists. This version allows listing directories containing a blacklisted child. Inclusion of blacklist.py fixed. fixes #1425
2011-08-24 08:59:28 -07:00
Kevan Carstensen
b49986f801
test/test_mutable: test for incorrect div_ceil equations
2011-08-13 11:39:36 -07:00
Brian Warner
fa6a768bc1
now that tests pass with full-size keys, return test-keys to normal (522bit)
2011-08-11 10:54:18 -07:00
david-sarah
dd23f265bf
More idiomatic resolution of the conflict between ticket393-MDMF-2 and trunk. refs #393
2011-08-10 13:29:42 -07:00
david-sarah
c55c44e70a
Replace the hard-coded 522-bit RSA key size used for tests with a TEST_RSA_KEY_SIZE constant defined in test/common.py (part 2). refs #393
2011-08-10 13:23:10 -07:00
david-sarah
fbe0e42bbf
Replace the hard-coded 522-bit RSA key size used for tests with a TEST_RSA_KEY_SIZE constant defined in test/common.py (part 1). refs #393
2011-08-10 13:22:43 -07:00
Zooko O'Whielacronx
95a7551586
merge some minor conflicts in test code from the 393-2 branch and trunk
2011-08-10 10:21:39 -07:00
david-sarah
b60506b0ba
test_web.py: fix a test failure dependent on whether simplejson.loads returns a unicode or str object.
2011-08-08 14:39:25 -07:00
Kevan Carstensen
47eab03c6c
test: fix assorted tests broken by MDMF changes
2011-08-06 17:44:59 -07:00
Kevan Carstensen
a7fb55f986
uri: add MDMF and MDMF directory caps, add extension hint support
2011-08-06 17:44:36 -07:00
Kevan Carstensen
52e9512b49
test/test_mutable: tests for MDMF
...
These are their own patch because they cut across a lot of the changes
I've made in implementing MDMF in such a way as to make it difficult to
split them up into the other patches.
2011-08-06 17:44:14 -07:00
Kevan Carstensen
a9cada2e03
webapi changes for MDMF
...
- Learn how to create MDMF files and directories through the
mutable-type argument.
- Operate with the interface changes associated with MDMF and #993 .
- Learn how to do partial updates of mutable files.
2011-08-06 17:43:48 -07:00
Kevan Carstensen
cde17ec041
dirnode: teach dirnode to make MDMF directories
2011-08-06 17:42:24 -07:00
david-sarah
50c692d1c5
Fix some test failures caused by #393 patch.
2011-08-01 20:28:10 -07:00
Kevan Carstensen
4d09ef3d83
cli: teach CLI how to create MDMF mutable files
...
Specifically, 'tahoe mkdir' and 'tahoe put' now take a --mutable-type
argument.
2011-08-01 19:16:13 -07:00
Kevan Carstensen
a87fc8ffab
frontends/sftpd: Resolve incompatibilities between SFTP frontend and MDMF changes
2011-08-01 19:12:07 -07:00
Kevan Carstensen
b1b77d3b89
mutable/layout: Define MDMF share format, write tools for working with MDMF share format
...
The changes in layout.py are mostly concerned with the MDMF share
format. In particular, we define read and write proxy objects used by
retrieval, publishing, and other code to write and read the MDMF share
format. We create equivalent proxies for SDMF objects so that these
objects can be suitably general.
2011-08-01 19:11:20 -07:00
Kevan Carstensen
edf9858fb8
immutable/filenode: implement unified filenode interface
2011-08-01 19:09:05 -07:00
Kevan Carstensen
3e996e0c94
immutable/literal.py: Implement interface changes in literal nodes.
2011-08-01 19:08:14 -07:00
Kevan Carstensen
8ae564d8ad
test/common: Alter common test code to work with MDMF.
...
This mostly has to do with making the test code implement the new
unified filenode interfaces.
2011-08-01 18:56:43 -07:00
david-sarah
a48aee11b3
allmydata/__init__.py, test_version.py: make version parsing understand '<=', with test. refs #1435
2011-08-14 20:51:53 -07:00
david-sarah
6c2670641e
Add test for webopen. fixes #1149
2011-07-24 14:16:59 -07:00
david-sarah
ab9eb12f70
test_client.py: relax a check in test_create_drop_uploader so that it should pass on Python 2.4.x. refs #1429
2011-08-09 22:25:04 -07:00
david-sarah
0dc56daaf9
test/common_util.py: correct fix to mkdir_nonascii. refs #1472
2011-08-09 22:19:06 -07:00
david-sarah
2deba3319c
test/common_util.py: fix a typo. refs #1472
2011-08-09 21:42:35 -07:00
david-sarah
0104dbab92
test_client.py, test_drop_upload.py: fix pyflakes warnings.
2011-08-09 20:45:05 -07:00
david-sarah
db22fdc20d
Factor out methods dealing with non-ASCII directories and filenames from test_drop_upload.py into common_util.py. refs #1429 , #1472
2011-08-09 20:15:58 -07:00
david-sarah
c102056ac1
test_client.py: add a test that the drop-uploader is initialized correctly by client.py. Also give the DropUploader service a name, which is necessary for the test. refs #1429
2011-08-09 20:05:38 -07:00
david-sarah
10ee22f50e
drop-upload: rename 'start' method to 'startService', which is what you're supposed to use to start a Service. refs #1429
2011-08-09 20:03:45 -07:00
david-sarah
f157b73367
test_drop_upload.py: add comment explaining why we don't use FilePath.setContent. refs #1429
2011-08-09 19:59:42 -07:00
david-sarah
369e30b1df
test_drop_upload.py: fix some grammatical and spelling nits. refs #1429
2011-08-09 15:12:31 -07:00
Zooko O'Whielacronx
5633375d26
drop-upload test for non-existent local dir separately from test for non-directory local dir
...
A candidate patch for #1429 has a bug when it is using FilePath.is_dir() to detect whether the configured local dir exists and is a directory. FilePath.is_dir() raises exception, instead of returning False, if the thing doesn't exist. This test is to make sure that DropUploader.__init__ raise different exceptions for those two cases.
refs #1429
2011-08-09 15:01:15 -07:00
Zooko O'Whielacronx
720bc2433b
drop-upload: unit tests for the configuration options being named "cap" instead of "uri"
...
This is a subset of a patch that David-Sarah attached to #1429 . This is just the unit-tests part of that patch, and uses darcs record instead of hunks to change the names.
refs #1429
2011-08-09 14:59:13 -07:00
david-sarah
4c592f1505
test_storage.py: test that we are using the filesystem of storage/shares/, rather than storage/, to calculate remaining space, and that the HTML status output reflects the values returned by fileutil.get_disk_stats. This version works with older versions of the mock library. refs #1384
2011-08-09 12:07:22 -07:00
david-sarah
70f03bd615
Work around ref #1472 by having test_drop_upload delete the non-ASCII directories it creates.
2011-08-08 18:23:34 -07:00
david-sarah
bbed522033
Remove all trailing whitespace from .py files.
2011-08-08 17:11:17 -07:00
david-sarah
08af9cea50
test_drop_upload.py: fix unused imports. refs #1429
2011-08-08 16:54:22 -07:00
david-sarah
32a7717205
Drop-upload frontend, rerecorded for 1.9 beta (and correcting a minor mistake). Includes some fixes for Windows but not the Windows inotify implementation. fixes #1429
2011-08-08 16:40:49 -07:00
david-sarah
b9121e045f
test_runner.py: fix a race condition in the test when NODE_URL_FILE is written before PORTNUM_FILE. refs #1469
2011-08-06 16:18:42 -07:00
david-sarah
743cb898e7
test_runner.py: cleanups of HOTLINE_FILE writing and removal.
2011-08-06 16:16:52 -07:00
david-sarah
e3c600d047
test_runner.py: remove an unused constant.
2011-08-06 15:14:16 -07:00
david-sarah
531758df19
test_runner.py: test that client and introducer nodes record their port number and use that port on the next restart. This tests for a regression caused by ref #1385 .
2011-08-06 15:06:35 -07:00
david-sarah
b9eb0235ea
test_runner.py: fix a bug in CreateNode.do_create introduced in changeset [5114] when the tahoe.cfg file has been written with CRLF line endings. refs #1385
2011-08-03 17:30:32 -07:00
david-sarah
b6cfbbeb23
test_client.py: repair Basic.test_error_on_old_config_files. refs #1385
2011-08-03 16:50:36 -07:00
david-sarah
1c77c5f5dc
test_checker.py: increase timeout for TooParallel.test_immutable again. The ARM buildslave took 38 seconds, so 40 seconds is too close to the edge; make it 80.
2011-08-03 14:40:42 -07:00
david-sarah
521754b506
test_runner.py: fix RunNode.test_introducer to not rely on the mtime of introducer.furl to detect when the node has restarted. Instead we detect when node.url has been written. refs #1385
2011-08-03 11:09:17 -07:00
david-sarah
ed3dbe4985
test_checker.py: increase timeout for TooParallel.test_immutable (was consistently failing on ARM buildslave).
2011-08-03 09:32:13 -07:00
david-sarah
e74387f4f1
test_runner.py: modify RunNode.test_introducer to test that starting an introducer works when the introducer.furl file already exists. refs #1385
2011-08-02 18:27:04 -07:00
david-sarah
9f8d34e63a
test_repairer: add a deterministic test of share data corruption that always flips the bits of the last byte of the share data. refs #1395
2011-08-02 10:58:41 -07:00
Zooko O'Whielacronx
f426e82287
verifier: serialize the fetching of blocks within a share so that we don't use too much RAM
...
Shares are still verified in parallel, but within a share, don't request a
block until the previous block has been verified and the memory we used to hold
it has been freed up.
Patch originally due to Brian. This version has a mockery-patchery-style test
which is "low tech" (it implements the patching inline in the test code instead
of using an extension of the mock.patch() function from the mock library) and
which unpatches in case of exception.
fixes #1395
2011-08-01 23:37:03 -07:00
Brian Warner
3668cb3d06
remove nodeid from WriteBucketProxy classes and customers
...
refs #1363
2011-08-01 15:43:17 -07:00
Brian Warner
550d67f51f
remove get_serverid() from ReadBucketProxy and customers, including Checker
...
and debug.py dump-share commands
refs #1363
2011-08-01 15:43:07 -07:00
Zooko O'Whielacronx
e5c4e83f4c
reject old-style (pre-Tahoe-LAFS-v1.3) configuration files
...
Check for the existence of any of them and if any are found raise exception which will abort the startup of the node.
This is a backwards-incompatible change for anyone who is still using old-style configuration files.
fixes #1385
2011-08-01 16:24:23 -07:00
Zooko O'Whielacronx
420e3aa293
whitespace-cleanup
2011-07-24 18:55:46 -07:00
Zooko O'Whielacronx
3505a8a31c
tests: use fileutil.write() instead of open() to ensure timely close even without CPython-style reference counting
...
Some of these already had an explicit close() but I went ahead and replaced them with fileutil.write() as well for the sake of uniformity.
2011-03-31 07:54:27 -07:00
david-sarah
06a5d0c1a3
cleanup: implement rm as a synonym for unlink rather than vice-versa. refs #776
2011-08-01 15:01:08 -07:00
david-sarah
07ecac1d83
WUI: change the label of the button to unlink a file from 'del' to 'unlink'. Also change some internal names to 'unlink', and allow 't=unlink' as a synonym for 't=delete' in the web-API interface. Incidentally, improve a test to check for the rename button as well as the unlink button. fixes #1104
2011-07-12 17:12:18 -07:00
Brian Warner
294dc5d0c7
fix typo introduced during rebasing of 'remove get_serverid from
...
DownloadStatus.add_dyhb_request and customers' patch, to fix test failure.
2011-08-01 13:03:41 -07:00
Zooko O'Whielacronx
6b2e798595
remove get_serverid from DownloadStatus.add_dyhb_request and customers
...
This patch is a rebase of a patch originally written by Brian. I didn't change any of the intent of Brian's patch, just ported it to current trunk.
refs #1363
2011-08-01 11:54:01 -07:00
Brian Warner
0605c77f08
test_immutable.Test: rewrite to use NoNetworkGrid, now takes 2.7s not 97s
...
remove now-unused ShareManglingMixin
refs #1363
2011-08-01 10:44:44 -07:00
Brian Warner
0f11d35f85
replace IServer.name() with get_name(), and get_longname()
...
This patch was originally written by Brian, but was re-recorded by Zooko to use
darcs replace instead of hunks for any file in which it would result in fewer
total hunks.
refs #1363
2011-08-01 10:44:28 -07:00
david-sarah
00fefeba49
test_cli.py: Test for error message when 'tahoe rm' is invoked without a path. refs #1292
2011-01-04 04:51:08 -07:00
david-sarah
16fd14a78a
test_cli.py: use to_str on fields loaded using simplejson.loads in new tests. refs #1304
2011-07-29 20:25:21 -07:00
Kevan Carstensen
448278e807
cli: make 'tahoe cp' overwrite mutable files in-place
2011-07-29 13:20:39 -07:00
david-sarah
a2699ea6f6
SFTP: write an error message to standard error for unrecognized shell commands. Change the existing message for shell sessions to be written to standard error, and refactor some duplicated code. Also change the lines of the error messages to end in CRLF, and take into account Kevan's review comments. fixes #1442 , #1446
2011-07-29 16:31:02 -07:00
david-sarah
b978378392
Fix the help synopses of CLI commands to include [options] in the right place. fixes #1359 , fixes #636
2011-07-24 15:54:40 -07:00
david-sarah
f9d218c673
encodingutil: argv and output encodings are always the same on all platforms. Lose the unnecessary generality of them being different. fixes #1120
2011-06-29 11:53:56 -07:00
david-sarah
5fcd3f8803
test_sftp.py cleanup: remove a redundant definition of failUnlessReallyEqual.
2011-07-16 11:18:13 -07:00
Zooko O'Whielacronx
4f8e3e5ae8
contrib: remove the contributed fuse modules and the entire contrib/ directory, which is now empty
...
Also remove a couple of vestigial references to figleaf, which is long gone.
fixes #1409 (remove contrib/fuse)
2011-07-12 08:32:29 -07:00
Brian Warner
d8358f2863
add more download-status data, fix tests
2011-06-29 15:25:55 -07:00
Brian Warner
fc5c2208fb
prepare for viz: improve DownloadStatus events
...
consolidate IDownloadStatusHandlingConsumer stuff into DownloadNode
2011-06-29 15:25:42 -07:00
Brian Warner
fb4dd43310
Rename test_package_initialization.py to (much shorter) test_import.py .
...
The former name was making my 'ls' listings hard to read, by forcing them
down to just two columns.
2011-06-11 12:02:34 -07:00
Zooko O'Whielacronx
be1ebde6e5
tests: fix tests to accomodate [20110611153758-92b7f-0ba5e4726fb6318dac28fb762a6512a003f4c430]
...
Apparently none of the two authors (stercor, terrell), three reviewers (warner, davidsarah, terrell), or one committer (me) actually ran the tests. This is presumably due to #20 .
fixes #1412
2011-06-11 09:37:41 -07:00
wilcoxjg
67ad0175cd
server.py: get_latencies now reports percentiles _only_ if there are sufficient observations for the interpretation of the percentile to be unambiguous.
...
interfaces.py: modified the return type of RIStatsProvider.get_stats to allow for None as a return value
NEWS.rst, stats.py: documentation of change to get_latencies
stats.rst: now documents percentile modification in get_latencies
test_storage.py: test_latencies now expects None in output categories that contain too few samples for the associated percentile to be unambiguously reported.
fixes #1392
2011-05-27 05:01:35 -07:00
david-sarah
a817163cc6
docs: convert NEWS to NEWS.rst and change all references to it.
2011-05-16 18:02:55 -07:00
david-sarah
a09d7300e0
tests: add test for including the ImportError message and traceback entry in the summary of errors from importing dependencies. refs #1389
2011-04-10 08:58:44 -07:00
david-sarah
f3246a9ca0
allmydata/__init__.py: Nicer reporting of unparseable version numbers in dependencies. fixes #1388
2011-04-01 13:27:50 -07:00
Brian Warner
9ae026d9f4
immutable/downloader/share.py: reduce get_serverid(), one left, update ext deps
...
test_download.py: create+check MyShare instances better, make sure they share
Server objects, now that finder.py cares
2011-02-26 19:11:50 -07:00
Brian Warner
2f63d9b522
immutable/upload.py: reduce use of get_serverid()
2011-02-26 19:11:38 -07:00
Brian Warner
9672b34880
add remaining get_* methods to storage_client.Server, NoNetworkServer, and
...
MockIServer stubs
2011-02-26 19:11:32 -07:00
Brian Warner
ee14f2ad67
happinessutil.py: finally rename merge_peers to merge_servers
2011-02-26 19:11:24 -07:00
Brian Warner
13d284b087
test_upload.py: factor out FakeServerTracker
2011-02-26 19:11:20 -07:00
Brian Warner
9fc55bad25
test_upload.py: server-vs-tracker cleanup
2011-02-26 19:11:15 -07:00
Brian Warner
ebfcb649f9
refactor: s/peer/server/ in immutable/upload, happinessutil.py, test_upload
...
No behavioral changes, just updating variable/method names and log messages.
The effects outside these three files should be minimal: some exception
messages changed (to say "server" instead of "peer"), and some internal class
names were changed. A few things still use "peer" to minimize external
changes, like UploadResults.timings["peer_selection"] and
happinessutil.merge_peers, which can be changed later.
2011-02-26 19:11:00 -07:00
Brian Warner
9b9ea3f739
storage_client.py: clean up test_add_server/test_add_descriptor, remove .test_servers
2011-02-26 19:10:56 -07:00
Brian Warner
cba3de04a4
test_client.py, upload.py:: remove KiB/MiB/etc constants, and other dead code
2011-02-26 19:10:51 -07:00
Zooko O'Whielacronx
578a117c5b
test: increase timeout on a network test because Francois's ARM machine hit that timeout
...
I'm skeptical that the test was proceeding correctly but ran out of time. It seems more likely that it had gotten hung. But if we raise the timeout to an even more extravagant number then we can be even more certain that the test was never going to finish.
2011-03-17 09:59:09 -07:00
david-sarah
787d12165a
Add unit tests for cross_check_pkg_resources_versus_import, and a regression test for ref #1355 . This requires a little refactoring to make it testable.
2011-02-20 17:58:17 -08:00
Brian Warner
ffd296fc5a
Refactor StorageFarmBroker handling of servers
...
Pass around IServer instance instead of (peerid, rref) tuple. Replace
"descriptor" with "server". Other replacements:
get_all_servers -> get_connected_servers/get_known_servers
get_servers_for_index -> get_servers_for_psi (now returns IServers)
This change still needs to be pushed further down: lots of code is now
getting the IServer and then distributing (peerid, rref) internally.
Instead, it ought to distribute the IServer internally and delay
extracting a serverid or rref until the last moment.
no_network.py was updated to retain parallelism.
2011-02-20 17:58:04 -08:00
david-sarah
ed67df866c
SFTP: report unknown sizes as "0" instead of "?", to satisfy some clients. fixes #1337
2011-01-27 22:22:49 -08:00
david-sarah
d318122cd2
Change tests for check_requirement to ensure that it accepts None in the comment field of a (version, location, comment) triple, and add a test that it raises ImportError when a module couldn't be imported. Also fix typo in a comment. refs #1339
2011-01-28 11:24:52 -08:00
Zooko O'Whielacronx
6f84f1b869
setup: adjust tests to use the new interface of check_requirement which has a 3-tuples instead of a 2-tuple
...
fixes #1339
2011-01-28 06:15:46 -08:00
david-sarah
f84f2aa987
tests: use verlib to compare versions, and get the versions of Twisted and Nevow from get_package_versions() instead of pkg_resources. refs #1287
2011-01-21 17:42:56 -08:00
david-sarah
8587c63bdc
Make bb-freeze (and probably other static packaging tools) work. This updates various places where we assumed that the tahoe process was executed via the Python interpreter. It also allows tests to recursively invoke the same tahoe.exe, rather than bin/tahoe. refs #585
2011-01-21 00:04:29 -08:00
david-sarah
29336a0916
Refactor _auto_deps.py and __init__.py, adding more robust checking of dependency versions, and not trusting pkg_resources to get the versions right. refs #1258 , #1287
2011-01-20 21:36:10 -08:00
david-sarah
6dd8b6f471
Eliminate dependencies on pywin32, even via Twisted. refs #1274
2011-01-19 20:32:38 -08:00
david-sarah
fd6cdc48ae
src/allmydata/test/test_runner.py: add test_import_from_repl, which checks that we are running the right code in a bin/tahoe subprocess. refs #1258
2011-01-19 00:21:45 -08:00
david-sarah
a9fc4668c0
docs/frontends/CLI.rst, src/allmydata/test/trialtest.py: add trailing newlines. refs #1296
2011-01-19 00:19:55 -08:00
david-sarah
8f0af33ba6
src/allmydata/test/test_cli.py: add test for 'tahoe debug trial' options help. refs #1296
2011-01-18 18:42:24 -08:00
david-sarah
1819c25c88
Add src/allmydata/test/trialtest.py needed by tests for 'tahoe debug trial'. refs #1296
2011-01-18 18:02:39 -08:00
david-sarah
0d6df9c9fc
Tests for 'tahoe debug trial' (rebased and fixed to work with Twisted 10.2). refs #1296
2011-01-18 17:38:59 -08:00
david-sarah
5a3acc05b7
CLI: tests for ref #1305 (v2, remove spurious extra arg to create-alias in test)
2011-01-13 20:03:27 -08:00
Brian Warner
b9fb74d2a0
Improve 'tahoe ln' help text. Patch by David-Sarah. Closes #1230 .
2011-01-17 00:14:21 -08:00
Brian Warner
09a2241471
Tolerate Twisted-10.2's endpoints, patch by David-Sarah. Closes #1286 .
...
The service generated by strports.service() changed in 10.2, and the ugly
private-attribute-reading hack we used to glean a kernel-allocated port
number (e.g. when using "tcp:0", especially during unit tests) broke, causing
Tahoe to be completely unusable with Twisted-10.2 . The new ugly
private-attribute-reading hack starts by figuring out what sort of service
was generated, then reads different attributes accordingly.
This also hushes a warning when using schemeless strports strings like "0" or
"3456", by quietly prepending a "tcp:" scheme, since 10.2 complains about
those. It also adds getURL() and getPortnum() accessors to the "webish"
service, rather than having unit tests dig through _url and _portnum and such
to find out what they are.
2011-01-16 23:47:51 -08:00
david-sarah
8a3cb60bf7
Test changes to take account of ref #1311 .
2011-01-16 22:05:40 -08:00
Brian Warner
45212d8000
Set "reserved_space=1G" in newly-created storage nodes. Closes #1208 .
2011-01-16 12:58:22 -08:00
david-sarah
e62a69e7ff
test_runner: avoid unnecessary use of non-ASCII.
2011-01-01 02:01:01 -08:00
david-sarah
5027753e23
test_storage.py: fix a pyflakes unused import warning.
2010-12-31 14:07:56 -08:00
david-sarah
0da10d4c37
test_storage.py: leave at least 512 MiB free when running test_large_share. refs #1195
2010-12-31 12:32:15 -08:00
Zooko O'Whielacronx
123a1a3f04
storage: use fileutil's version of get_disk_stats() and get_available_space(), use mockery/fakery in tests, enable large share test on platforms with sparse files and if > 4 GiB of disk space is currently available
2010-09-10 08:36:29 -08:00
Zooko O'Whielacronx
d5e71c2940
fileutil: copy in the get_disk_stats() and get_available_space() functions from storage/server.py
2010-09-10 08:35:20 -08:00
david-sarah
9ceb527787
Change another doc reference from .txt to .rst in a text string.
2010-11-28 09:45:07 -08:00
Brian Warner
69b42c6cb7
remove --multiple/-m option from all CLI commands: closes #1262
...
I personally used "tahoe start/restart -m ../MY-TESTNET/node*" all the time,
to spin up or update a local testgrid while iterating over new code. However,
with the recent switch from "subprocess.Popen(/bin/twistd)" to "import and
call twistd.run()" in scripts/startstop_node.py (yay fewer processes!),
"start -m" broke, and fixing it requires os.fork, which is unavailable on
windows (boo windows!). And I was probably the only one using -m. So in the
interests of uniformity among platforms and simpler code (yay negative code
days!), we're just removing -m from everything. I will start using a little
shell script or something to simulate the removed functionality.
This patch also cleans up CLI-function calling a bit: get the basedir from
the config dict (instead of sometimes from a separate argument), and always
return a numeric exit code.
2010-11-27 00:38:09 -08:00
Brian Warner
f36bda2780
Revert previous commit: there's an ugly corner-case on windows that fails tests.
...
Specifically, test_runner.CreateNode.test_client failed, because the
os.fork-is-present test decided that --multiple should not be allowed on
windows, even though --multiple works just fine for 'tahoe create-client'.
The only restriction on --multiple is for 'tahoe start' and 'tahoe restart'.
This needs a different approach, probably by cleaning up BasedirMixin. We
should only be withholding --multiple on windows for "start" and
"restart". (we should continue withholding --multiple on all platforms for
"run").
This reverts (git) commit f3adb037ae
2010-11-26 16:44:11 -08:00
Brian Warner
f3adb037ae
startstop_node.py: fix "tahoe start -m" by forking before non-final targets
...
* don't advertise -m flag on tahoe start/restart/run unless os.fork is
available (i.e. windows)
* test_runner.py: add test to exercise "start/stop/restart -m"
2010-11-26 14:45:38 -08:00
david-sarah
fdf6946669
test_cli.py: test that 'tahoe mv' reports errors from the DELETE operation. refs #1255
2010-11-11 17:46:53 -08:00
Zooko O'Whielacronx
93dc740f01
tests: bump up the timeout on test_dirnode.Dirnode from 240s to 480s since it apparently took longer than 240s just now on François's ARM buildslave
2010-11-15 01:21:19 -08:00
david-sarah
76371ae5a9
test_cli.py: fix a stale comment that incorrectly implied that test_cli runs CLI commands in subprocesses (it actually runs them using deferToThread).
2010-11-09 18:28:19 -08:00
david-sarah
01a5365051
test_system.py: test for 'tahoe mv' with the http_proxy and HTTP_PROXY variables set. refs #1253
2010-11-09 19:36:49 -08:00
david-sarah
edec9c9869
test_client.py: change overzealous use of failUnlessReallyEqual on integers back to failUnlessEqual.
2010-11-09 15:08:16 -08:00
david-sarah
56f4c1218b
test_runner.py: fix test failure in test_the_right_code after applying zooko's change to test-with-fake-pkg.py
2010-11-01 20:59:05 -07:00
david-sarah
6eaa7f2356
tahoe backup: perform tilde expansion in exclude-from filename (version 2). fixes #1241
2010-10-30 20:52:31 -07:00
david-sarah
10a5f23df9
test_runner.py: fix error in BinTahoe.test_version_no_noise introduced by last patch. refs #1235
2010-10-29 15:11:23 -07:00
david-sarah
d249589667
test_runner.py: also allow 'from pkg_resources import load_entry_point' as noise. refs #1235 .
2010-10-29 13:42:46 -07:00
david-sarah
8c95f88b4f
test_runner.py: if the only noise is 'UserWarning: Unbuilt egg for setuptools', skip instead of failing the no_noise tests. This version avoids 'any' to be compatible with Python < 2.5. refs #1235 .
2010-10-29 12:18:04 -07:00
david-sarah
4061258c85
make ResponseCache smarter to avoid memory leaks: don't record timestamps, use DataSpans to merge entries, and clear the cache when we see a new seqnum. refs #1045 , #1229
2010-10-26 21:33:02 -07:00
Brian Warner
c88eb097d9
scripts/tahoe_backup.py: oops, fix missing import, thanks pyflakes
...
test_cli.py: hush minor pyflakes complaint
2010-10-29 02:42:23 -07:00
Francois Deppierraz
14ee763c54
tahoe_cp.py: Don't call urllib.quote with an Unicode argument, fix #1224
...
tahoe_backup.py: Fix another (potential) occurrence of calling urllib.quote()
with an Unicode parameter
2010-10-29 01:45:20 -07:00
Brian Warner
c18953c169
fix #1223 , crash+inefficiency during repair due to read overrun
...
* repairer (really the uploader) reads beyond end of input file (Uploadable)
* new-downloader does not tolerate overreads
* uploader does lots of tiny reads (inefficient)
This fixes the last two. The uploader still does a single overread at the end
of the input file, but now that's ok so we can leave it in place. The
uploader now expects the Uploadable to behave like a normal disk
file (reading beyond EOF will return less data than was asked for), and now
the new-downloadable behaves that way.
2010-10-29 01:20:36 -07:00
david-sarah
59e80b2ec9
test_mutable.py: add test for ResponseCache memory leak. refs #1045 , #1129
2010-10-24 12:34:09 -07:00
david-sarah
daf2b8314a
test_encodingutil.py: test_argv_to_unicode modified the wrong encoding variable. fixes #1214
2010-10-22 20:58:10 -07:00
Kevan Carstensen
ec4f87a98c
immutable/repairer.py: don't use the default happiness setting when repairing
2010-09-27 13:01:02 -07:00
david-sarah
6c4ba62c5d
test_download.py: simplify and possibly fix test_lost_servers; refs #1197
2010-09-11 17:32:51 -07:00
david-sarah
b23a7fcc10
test_runner.py: fix spurious failures of test_path on case-insensitive filesystems, and restore the check that the version is not 'unknown'.
2010-09-11 15:19:13 -07:00
Zooko O'Whielacronx
fd17d63c95
test: make tests stop relying on pyutil version class accepting the string 'unknown' for its version, and make them forward-compatible with the future Python Rational Version Numbering standard
2010-09-10 08:41:35 -07:00
Zooko O'Whielacronx
56a3258ff7
immutable downloader: add a test specifically of whether the finder sometimes announces "no more shares ever" and then announces a newly found share
...
(The current code fails this test, ref #1191.)
2010-09-08 21:16:54 -07:00
Zooko O'Whielacronx
9d421cb022
tests: assign the storage servers to a fixed order which triggers a bug in new downloader every time this test is run (formerly this test would detect the bug in new-downloader only sporadically)
...
If you are investigating the bug in new-downloader, one way to investigate might be to change this ordering to a different fixed order (e.g. rotate by 4 instead of rotate by 5) and observe how the behavior of new-downloader differs in that case.
2010-09-03 21:15:15 -07:00
francois
485bfc0fd6
DownloadStatus: show active immutable downloads in Active Operations, Fix #1172
2010-09-02 03:17:28 -07:00
Zooko O'Whielacronx
cdc625e0f8
tests: bump up the allowed number of reads
...
Kyle's OpenBSD buildslave used 41 reads when doing this test. The fact that I'm blindly bumping this number up to match the observed behavior probably means this isn't a good criterion to be testing for anyway. But perhaps someone else (Brian) could investigate why that run on Kyle's OpenBSD box took four more reads than we expected, and whether the fact that it took 41 reads to do this operation is indicative of an actual problem.
2010-09-01 22:38:01 -07:00
Brian Warner
00e9e4e676
SegmentFetcher: use new diversity-seeking share-selection algorithm, and
...
deliver all shares at once instead of feeding them out one-at-a-time.
Also fix distribution of real-number-of-segments information: now all
CommonShares (not just the ones used for the first segment) get a
correctly-sized hashtree. Previously, the late ones might not, which would
make them crash and get dropped (causing the download to fail if the initial
set were insufficient, perhaps because one of their servers went away).
Update tests, add some TODO notes, improve variable names and comments.
Improve logging: add logparents, set more appropriate levels.
2010-08-31 18:37:02 -07:00
Brian Warner
c89a464510
Share: drop received data after each block finishes. Quick fix for the #1170 spans.py complexity bug.
2010-08-31 18:35:58 -07:00
francois
f026927f86
web: refactor rate computation, fixes #1166
2010-08-15 07:19:33 -07:00
Brian Warner
a3cf3331d2
DownloadStatus: put real numbers in progress/status rows, not placeholders.
...
Improve tests.
2010-08-09 23:06:03 -07:00
Brian Warner
dc1afc81bb
web download-status: tolerate DYHBs that haven't retired yet. Fixes #1160 .
...
Also add a better unit test for it.
2010-08-09 15:51:00 -07:00
david-sarah
fd9a2205de
test_runner.py: correct and simplify normalization of package directory for case-insensitive filesystems.
2010-08-08 11:50:05 -07:00
david-sarah
f88e4f0df6
test_runner.py: make test_path work for test-from-installdir.
2010-08-08 10:13:40 -07:00
david-sarah
dc53d4f5ef
test_runner.py: another try at calculating the rootdir correctly for test-from-egg and test-from-prefixdir.
2010-08-08 08:43:07 -07:00
david-sarah
39fa9a81d1
test_runner.py: calculate the location of bin/tahoe correctly for test-from-prefixdir (by copying code from misc/build_helpers/run_trial.py). Also fix the false-positive check for Unicode paths in test_the_right_code, which was causing skips that should have been failures.
2010-08-07 21:28:17 -07:00
david-sarah
a79ec1c6cd
test_util.py: use SHA-256 from pycryptopp instead of MD5 from hashlib (for uses in which any hash will do), since hashlib was only added to the stdlib in Python 2.5.
2010-08-05 22:00:51 -07:00
david-sarah
600cb95fe2
test_runner.py: increase timeout to cater for Francois' ARM buildslave.
2010-08-05 21:26:01 -07:00
david-sarah
8e90255349
test_util.py: remove use of 'a if p else b' syntax that requires Python 2.5.
2010-08-05 21:16:16 -07:00
Brian Warner
a0124e95ee
test_download.DownloadTest.test_simultaneous_goodguess: enable some disabled
...
checks that used to fail but work now.
2010-08-05 11:55:07 -07:00
Brian Warner
f6f9a97627
DownloadNode: fix lost-progress in fetch_failed, tolerate cancel when no segment-fetch is active. Fixes #1154 .
...
The lost-progress bug occurred when two simultanous read() calls fetched
different segments, and the first one failed (due to corruption, or the other
bugs in #1154 ): the second read() would never complete. While in this state,
cancelling the second read by having its consumer call stopProducing) would
trigger the cancel-intolerance bug. Finally, in downloader.node.Cancel,
prevent late cancels by adding an 'active' flag
2010-08-05 11:55:07 -07:00
Brian Warner
43c5032105
util/spans.py: __nonzero__ cannot return a long either. for #1154
2010-08-05 11:55:07 -07:00
david-sarah
cd8d41584e
test_storage.py: change skip note for test_large_share to say that Windows doesn't support sparse files. refs #569
2010-08-04 19:26:12 -07:00
Brian Warner
8844655705
One fix for bug #1154 : webapi GETs with a 'Range' header broke new-downloader.
...
The Range header causes n.read() to be called with an offset= of type 'long',
which eventually got used in a Spans/DataSpans object's __len__ method.
Apparently python doesn't permit __len__() to return longs, only ints.
Rewrote Spans/DataSpans to use s.len() instead of len(s) aka s.__len__() .
Added a test in test_download. Note that test_web didn't catch this because
it uses mock FileNodes for speed: it's probably time to rewrite that.
There is still an unresolved error-recovery problem in #1154 , so I'm not
closing the ticket quite yet.
2010-08-04 11:45:49 -07:00
Brian Warner
2bd8749849
test_download: minor cleanup
2010-08-04 10:55:55 -07:00
Brian Warner
2a05aa2d91
lazily create DownloadNode upon first read()/get_segment()
2010-08-04 00:28:08 -07:00
Brian Warner
abcd6e0e96
test_hung_server: update comments, remove dead "stage_4_d" code
2010-08-04 00:28:00 -07:00
Brian Warner
919938dd95
copy the rest of David-Sarah's changes to make my tree match 1.8.0beta
2010-08-04 00:27:52 -07:00
Brian Warner
20847dd876
ShareFinder: add 10s OVERDUE timer, send new requests to replace overdue ones
...
The fixed 10-second timer will eventually be replaced with a per-server
value, calculated based on observed response times.
test_hung_server.py: enhance to exercise DYHB=OVERDUE state. Split existing
mutable+immutable tests into two pieces for clarity. Reenabled several tests.
Deleted the now-obsolete "test_failover_during_stage_4".
2010-08-04 00:27:41 -07:00
Brian Warner
63b61ce7bd
Rewrite immutable downloader ( #798 ). This patch adds and updates unit tests.
2010-08-04 00:27:10 -07:00
Brian Warner
cbcb728e7e
Add a byte-spans utility class, like perl's Set::IntSpan for .newsrc files.
...
Also a data-spans class, which records a byte (instead of a bit) for each
index.
2010-08-04 00:26:00 -07:00
david-sarah
f10f9a8646
test_storage.py: avoid spurious test failure by accepting either 'Next crawl in 59 minutes' or 'Next crawl in 60 minutes'. fixes #1140
2010-08-03 03:20:58 -07:00
david-sarah
a7c474a098
CLI: further improve consistency of basedir options and add tests. addresses #118
2010-08-03 01:54:16 -07:00
david-sarah
51eeea900f
test_cli.py: make 'tahoe mkdir' tests slightly less dumb (check for 'URI:' in the output).
2010-08-03 01:47:20 -07:00
david-sarah
b9174b5fe7
test_cli.py: use u-escapes instead of UTF-8.
2010-08-03 01:35:38 -07:00
david-sarah
d81e8acf88
test_runner.py: add test_run_with_python_options, which checks that the Windows script changes haven't broken 'python <options> bin/tahoe'.
2010-08-01 23:25:58 -07:00
david-sarah
3ddb4a9bbf
test_runner.py: fix missing import of get_filesystem_encoding
2010-08-01 23:09:02 -07:00
david-sarah
3af6f19cb0
misc/build_helpers/run_trial.py: check that the root from which the module we are testing was loaded is the current directory. This version of the patch folds in later fixes to the logic for caculating the directories to compare, and improvements to error messages. addresses #1137
2010-08-01 21:55:35 -07:00
david-sarah
1fad717aaa
test_runner.py: change to code for locating the bin/tahoe script that was missed when rebasing the patch for #1074 .
2010-07-25 11:20:08 -07:00
david-sarah
37b07a545f
Changes to Tahoe needed to work with new zetuptoolz (that does not use .exe wrappers on Windows), and to support Unicode arguments and stdout/stderr -- v5
2010-07-25 01:32:16 -07:00
david-sarah
f952532f08
Basedir/node directory option improvements for ticket798 branch. addresses #188 , #706 , #715 , #772 , #890
2010-08-01 21:30:04 -07:00
david-sarah
9aae9208e3
test_runner.py: Fix error in message arguments to 'fail' calls.
2010-08-01 18:35:26 -07:00
david-sarah
2b33704d61
test_system: correct a failure in _test_runner caused by Unicode basedir patch on non-Unicode platforms.
2010-07-23 20:21:23 -07:00
david-sarah
f036dfaa4b
Fix test failures due to Unicode basedir patches.
2010-07-24 18:03:18 -07:00
david-sarah
28e6ad51a7
util.encodingutil: change quote_output to do less unnecessary escaping, and to use double-quotes more consistently when needed. This version avoids u-escaping for characters that are representable in the output encoding, when double quotes are used, and includes tests. fixes #1135
2010-07-23 00:53:14 -07:00
david-sarah
618db4867c
Replace uses of os.path.abspath with abspath_expanduser_unicode where necessary. This makes basedir paths consistently represented as Unicode.
2010-07-21 17:14:18 -07:00
david-sarah
11b18824c7
util.fileutil, test.test_util: add abspath_expanduser_unicode function, to work around < http://bugs.python.org/issue3426 >. util.encodingutil: add a convenience function argv_to_abspath.
2010-07-21 16:15:07 -07:00
jacob.lyles
e276050d5a
abbreviate time edge case python2.5 unit test
2010-07-29 14:06:38 -07:00
david-sarah
23c649c897
test_upload.py: rename test_problem_layout_ticket1124 to test_problem_layout_ticket_1124 -- fix .todo reference.
2010-07-29 08:29:27 -07:00
david-sarah
0ebe18cc73
test_upload.py: rename test_problem_layout_ticket1124 to test_problem_layout_ticket_1124 for consistency.
2010-07-29 07:22:50 -07:00
Zooko O'Whielacronx
28822dfb4e
tests: remove debug print
2010-08-01 23:33:39 -07:00
Zooko O'Whielacronx
fd874d6669
setup: fix a couple instances of hard-coded 'allmydata-tahoe' in the scripts, tighten the tests (as suggested by David-Sarah)
2010-08-01 09:42:07 -07:00
Zooko O'Whielacronx
44edecb36c
setup: replace hardcoded 'allmydata-tahoe' with allmydata.__appname__
2010-08-01 09:05:17 -07:00
Zooko O'Whielacronx
0f94923f22
immutable: use PrefixingLogMixin to organize logging in Tahoe2PeerSelector and add more detailed messages about peer
2010-07-19 01:20:00 -07:00
Zooko O'Whielacronx
2ebe2c2ff3
benchmarking: update bench_dirnode to be correct and use the shiniest new pyutil.benchutil features concerning what units you measure in
2010-07-18 21:49:48 -07:00
Zooko O'Whielacronx
9b4f412e49
trivial: rename and add in-line doc to clarify "used_peers" => "upload_servers"
2010-07-18 21:47:44 -07:00
david-sarah
bdc6fd2f8b
test_runner: test that 'tahoe --version' outputs no noise (e.g. DeprecationWarnings).
2010-07-19 18:13:45 -07:00
Zooko O'Whielacronx
c076c0b19b
immutable: add test case of #1128 , fix test case of #1118
2010-07-19 01:16:12 -07:00
Zooko O'Whielacronx
1dbfcf753d
immutable: extend the tests to check that the shares that got uploaded really do make a sufficiently Happy distribution
...
This patch also renames some instances of "find_shares()" to "find_all_shares()" and other instances to "find_uri_shares()" as appropriate -- the conflation between those names confused me at first when writing these tests.
2010-07-18 21:50:47 -07:00
Zooko O'Whielacronx
2e83f243c2
immutable: test for #1118
2010-07-18 15:15:37 -07:00
Zooko O'Whielacronx
0f46766a51
immutable: test for #1124
2010-07-18 15:29:07 -07:00
Zooko O'Whielacronx
8f201c75f0
trivial: fix unused import (sorry about that, pyflakes)
2010-07-18 14:51:33 -07:00
Zooko O'Whielacronx
16bb529339
tests, NEWS, CREDITS re: #1117
...
Give Brian and Kevan promotions, move release date in NEWS to the 18th, commit Brian's test for #1117 .
fixes #1117
2010-07-18 13:32:25 -07:00
Kevan Carstensen
054c4553b5
test/test_upload.py: test to see that aborted buckets are ignored by the storage server
2010-07-15 17:10:46 -07:00
Kevan Carstensen
677f3b1f63
test/test_storage.py: test for the new remote_abort semantics.
2010-07-15 16:21:48 -07:00
Kevan Carstensen
5becaf0309
test/test_upload.py: changes to test plumbing for #1117 tests
...
- Add a callRemoteOnly method to FakeBucketWriter.
- Change the abort method in FakeBucketWriter to not return a
RuntimeError.
2010-07-15 16:18:20 -07:00
david-sarah
638a3b2af4
test_encodingutil: correct an error in the previous patch to StdlibUnicode.test_open_representable.
2010-07-18 08:14:20 -07:00
david-sarah
74c41ebb8b
test_dirnode and test_web: don't use failUnlessReallyEqual in cases where the return type from simplejson.loads can vary between unicode and str. Use to_str when comparing URIs parsed from JSON.
2010-07-18 07:29:15 -07:00
david-sarah
9a4a864573
test_encodingutil: StdlibUnicode.test_open_representable no longer uses a mock.
2010-07-18 05:54:12 -07:00
Zooko O'Whielacronx
7cc98759bd
docs: add comment clarifying #1051
2010-07-17 22:32:50 -07:00
david-sarah
d346e0853d
Add tests of caps from the future that have non-ASCII characters in them (encoded as UTF-8). The changes to test_uri.py, test_client.py, and test_dirnode.py add tests of non-ASCII future caps in addition to the current tests. The changes to test_web.py just replace the tests of all-ASCII future caps with tests of non-ASCII future caps. We also change uses of failUnlessEqual to failUnlessReallyEqual, in order to catch cases where the type of a string is not as expected.
2010-07-11 13:02:52 -07:00
Zooko O'Whielacronx
3c4fe0706f
M-x whitespace-cleanup
2010-07-17 20:27:39 -07:00
Zooko O'Whielacronx
6a7023efbe
benchmarking: update bench_dirnode.py to reflect the new directory interfaces
2010-07-17 20:17:10 -07:00
david-sarah
bdb10553eb
test_encodingutil: fix test_open_representable, which is only valid when run on a platform for which we know an unrepresentable filename.
2010-07-17 20:03:33 -07:00
david-sarah
a8161c915a
Correct stringutils->encodingutil patch to be the newer version, rather than the old version that was committed in error.
2010-07-17 18:34:35 -07:00
david-sarah
0d79a4a7d1
test_cli.py: fix error that crept in when rebasing the patch for #1072 .
2010-07-17 17:01:23 -07:00
david-sarah
c59f9b162f
stringutils: add test for when sys.stdout has no encoding attribute ( fixes #1099 ).
2010-07-16 21:58:16 -07:00
david-sarah
8090785716
CLI: add 'tahoe unlink' as an alias to 'tahoe rm', for forward-compatibility.
2010-07-17 15:04:11 -07:00
Zooko O'Whielacronx
6e8477114e
minor code clean-up in dirnode.py
...
Impose micro-POLA by passing only the writekey instead of the whole node object to {{{_encrypt_rw_uri()}}}. Remove DummyImmutableFileNode in nodemaker.py, which is obviated by this. Add micro-optimization by precomputing the netstring of the empty string and branching on whether the writekey is present or not outside of {{{_encrypt_rw_uri()}}}. Add doc about writekey to docstring.
fixes #967
2010-07-13 23:02:55 -07:00
david-sarah
11077ea74d
Rename stringutils to encodingutil, and drop listdir_unicode and open_unicode (since the Python stdlib functions work fine with Unicode paths). Also move some utility functions to fileutil.
2010-07-11 17:30:15 -07:00
david-sarah
3b1b0147a8
test_storage.py: potential fix for failures when logging is enabled.
1970-07-12 21:05:46 -07:00
freestorm77
97579af73c
runnin_test_tahoe_css.dpatch
...
Runnin test for changes in tahoe.css file
2010-06-22 14:47:14 -07:00
freestorm77
0304918a77
runnin_test_directory_xhtml.dpatch
...
Runnin test for diretory.xhtml top banner
2010-06-22 13:14:03 -07:00
david-sarah
6b2f99fa9a
test_backupdb.py: skip test_unicode if we can't represent the test filenames.
2010-06-18 19:26:20 -07:00
david-sarah
a2caf6f4b2
test_web.py: correct a test that was missed in the change to not write ctime/mtime.
2010-06-18 19:17:18 -07:00
david-sarah
4712875193
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 16:01:19 -07:00
david-sarah
72e395d878
test_storage.py: print more information on test failures.
2010-06-16 20:46:23 -07:00
david-sarah
390fc78a9a
test_cli.py: fix test failure in CLI.test_listdir_unicode_good due to filenames returned from listdir_unicode no longer being normalized.
2010-06-17 21:51:10 -07:00
david-sarah
178401eb4e
tahoe backup: unicode tests.
2010-06-17 20:52:11 -07:00
david-sarah
a9fe3792de
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-17 17:02:49 -07:00
david-sarah
5ada31034b
test_stringutils.py: take account of the output of listdir_unicode no longer being normalized. Also use Unicode escapes, not UTF-8.
2010-06-16 20:44:09 -07:00
david-sarah
025aede9e4
test_dirnode.py: partial tests for normalization changes.
2010-06-16 20:40:25 -07:00
david-sarah
408b12a7e6
test_system.py: investigate failure in allmydata.test.test_system.SystemTest.test_upload_and_download_random_key due to bytes_sent not being an int
2010-06-15 17:16:48 -07:00
david-sarah
11e88c340a
SFTP: test that renaming onto a just-opened file fails.
2010-06-11 20:37:09 -07:00
david-sarah
b67f8b66c8
SFTP: further small improvements to test coverage. Also ensure that after a test failure, later tests don't fail spuriously due to the checks for heisenfile leaks.
2010-06-11 20:07:37 -07:00
david-sarah
b4e074d250
SFTP: further improve test coverage (paths containing '.', bad data for posix-rename extension, and error in test of openShell).
2010-06-11 14:31:42 -07:00
david-sarah
52f87904ed
SFTP: improve test coverage for no-write on mutable files, and check for heisenfile table leaks in all relevant tests. Delete test_memory_leak since it is now redundant.
2010-06-11 13:57:52 -07:00
david-sarah
546c3d2ed4
SFTP: add test for extension of file opened with FXF_APPEND.
2010-06-10 11:26:47 -07:00
Zooko O'Whielacronx
f19c240ca8
tests: bump up the timeout on this iputil test from 2s to 4s
2010-06-09 07:30:17 -07:00
david-sarah
3c44389440
SFTP: fix most significant memory leak described in #1045 (due to a file being added to all_heisenfiles under more than one direntry when renamed).
2010-06-09 01:00:03 -07:00
david-sarah
5786874d38
test_stringutils.py: Fix test failure on CentOS builder, possibly Python 2.4.3-related.
2010-06-08 23:50:56 -07:00
david-sarah
ffd1c5aca5
Fix for Unicode-related test failures on Zooko's OS X 10.6 machine.
2010-06-08 22:54:48 -07:00
david-sarah
32d9deace3
Back out Windows-specific Unicode argument support for v1.7.
2010-06-08 17:08:03 -07:00
david-sarah
195b4afa8f
SFTP: ignore permissions when opening a file (needed for sshfs interoperability).
2010-06-07 22:57:00 -07:00
david-sarah
a67e745b26
test_web.py: fix pyflakes warnings introduced by byterange patch.
2010-06-07 21:20:12 -07:00
Jeremy Fitzhardinge
63b28d707b
Improve HTTP/1.1 byterange handling
...
Fix parsing of a Range: header to support:
- multiple ranges (parsed, but not returned)
- suffix byte ranges ("-2139")
- correct handling of incorrectly formatted range headers
(correct behaviour is to ignore the header and return the full
file)
- return appropriate error for ranges outside the file
Multiple ranges are parsed, but only the first range is returned.
Returning multiple ranges requires using the multipart/byterange
content type.
2010-03-09 20:59:13 -07:00
Zooko O'Whielacronx
d13784b649
tests: bump up the timeout on these tests; MM's buildslave is sometimes extremely slow on tests, but it will complete them if given enough time. MM is working on making that buildslave more predictable in how long it takes to run tests.
2010-06-07 20:37:54 -07:00
david-sarah
3c883e6e44
test_cli.py: remove invalid 'test_listdir_unicode_bad' test.
2010-06-07 11:37:30 -07:00
david-sarah
71f8757bd1
check_memory.py: adapt to servers-of-happiness changes.
2010-06-07 18:35:28 -07:00
david-sarah
7092de1b6f
Remove the 'tahoe debug consolidate' subcommand.
2010-06-07 11:37:57 -07:00
Zooko O'Whielacronx
faade8b20a
tests: drastically increase timeout of this very time-consuming test in honor of François's ARM box
2010-06-07 04:59:29 -07:00
david-sarah
8b014372b1
test_stringutils.py: trivial error in exception message for skipped test.
2010-06-06 23:14:55 -07:00
david-sarah
529c9f673a
More Unicode test fixes.
2010-06-06 22:33:58 -07:00
david-sarah
65b6f4e3ce
Unicode fixes for platforms with non-native-Unicode filesystems.
2010-06-06 21:32:38 -07:00
david-sarah
80252debcd
Unicode fixes.
2010-06-06 18:02:15 -07:00
Zooko O'Whielacronx
5bcca5151e
unicode tests: fix missing import
2010-06-04 07:26:30 -07:00
Zooko O'Whielacronx
db8a6f3aa6
unicode: make test_cli test a non-ascii argument, and make the fallback term encoding be locale.getpreferredencoding()
2010-06-04 07:12:51 -07:00
Zooko O'Whielacronx
442008a690
unicode: always decode json manifest as utf-8 then encode for stdout
...
pyflakes pointed out that the exception handler fallback called an un-imported function, showing that the fallback wasn't being exercised.
I'm not 100% sure that this patch is right and would appreciate François or someone reviewing it.
2010-06-04 01:48:40 -07:00
Zooko O'Whielacronx
952f3bf3ef
fix flakes
2010-06-04 00:58:45 -07:00
Francois Deppierraz
0eb4d83937
test_stringutils.py: Skip test test_listdir_unicode_good if filesystem supports only ASCII filenames
2010-05-21 09:08:39 -07:00
Francois Deppierraz
01da7f06e6
test_stringutils.py: Skip test_listdir_unicode on mocked platform which cannot store non-ASCII filenames
2010-05-21 09:05:59 -07:00
Francois Deppierraz
58e9df9f5a
test_stringutils.py: Add a test class for OpenBSD 4.1 with LANG=C
2010-05-21 07:00:53 -07:00
Francois Deppierraz
1aeff62161
test_stringutils.py: Mock the open() call in test_open_unicode
...
This test ensure that open(a_unicode_string) is used on Unicode platforms
(Windows or MacOS X) and that open(a_correctly_encoded_bytestring) on other
platforms such as Unix.
2010-05-21 06:58:17 -07:00
Francois Deppierraz
eecaaca6b7
test_stringutils.py: Fix a trivial Python 2.4 syntax incompatibility
2010-05-21 02:33:45 -07:00
Francois Deppierraz
6c70fb7b3c
test_cli.py: Fix tests when sys.stdout.encoding=None and refactor this code into functions
2010-05-20 01:44:47 -07:00
Francois Deppierraz
496802420e
Fix handling of correctly encoded unicode filenames ( #534 )
...
Tahoe CLI commands working on local files, for instance 'tahoe cp' or 'tahoe
backup', have been improved to correctly handle filenames containing non-ASCII
characters.
In the case where Tahoe encounters a filename which cannot be decoded using the
system encoding, an error will be returned and the operation will fail. Under
Linux, this typically happens when the filesystem contains filenames encoded
with another encoding, for instance latin1, than the system locale, for
instance UTF-8. In such case, you'll need to fix your system with tools such
as 'convmv' before using Tahoe CLI.
All CLI commands have been improved to support non-ASCII parameters such as
filenames and aliases on all supported Operating Systems except Windows as of
now.
2010-05-19 17:43:56 -07:00
Francois Deppierraz
d0ed14e1bb
stringutils.py: Unicode helper functions + associated tests
...
This file contains a bunch of helper functions which converts
unicode string from and to argv, filenames and stdout.
2010-05-19 17:41:05 -07:00
david-sarah
c9984c40e9
SFTP: further improvements to test coverage.
2010-06-02 16:44:22 -07:00
david-sarah
2a791b0d05
SFTP: improve test coverage. Also make creating a directory fail when permissions are read-only (rather than ignoring the permissions).
2010-06-01 21:19:34 -07:00
david-sarah
29a06457d2
dirnode.py: fix a bug in the no-write change for Adder, and improve test coverage. Add a 'metadata' argument to create_subdirectory, with documentation. Also update some comments in test_dirnode.py made stale by the ctime/mtime change.
2010-06-01 20:26:41 -07:00
david-sarah
de95140b7b
SFTP: changes for #1063 ('no-write' field) including comment:1 (clearing owner write permission diminishes to a read cap). Includes documentation changes, but not tests for the new behaviour.
2010-05-31 22:11:39 -07:00
david-sarah
e867985539
SFTP: cater to clients that assume a file is created as soon as they have made an open request; also, fix some race conditions associated with closing a file at about the same time as renaming or removing it.
2010-05-28 21:52:53 -07:00
david-sarah
a143b1297b
SFTP: handle removing a file while it is open. Also some simplifications of the logout handling.
2010-05-25 11:42:10 -07:00
david-sarah
e81ce2bbd2
SFTP: a posix-rename response should actually return an FXP_STATUS reply, not an FXP_EXTENDED_REPLY as Twisted Conch assumes. Work around this by raising an SFTPError with code FX_OK.
2010-05-24 20:33:23 -07:00
david-sarah
8f86490d48
SFTP: fix problem with posix-rename code returning a Deferred for the renamed filenode, not for the result of the request (an empty string).
2010-05-24 19:02:09 -07:00
david-sarah
549dccd608
SFTP: name of the POSIX rename extension should be 'posix-rename@openssh.com', not 'extposix-rename@openssh.com'.
2010-05-23 19:11:56 -07:00
david-sarah
75357fd2d5
SFTP: fix pyflakes warnings; drop 'noisy' versions of eventually_callback and eventually_errback; robustify conversion of exception messages to UTF-8.
2010-05-23 07:09:05 -07:00
david-sarah
38964fb35a
SFTP: fixes and test cases for renaming of open files.
2010-05-22 20:25:49 -07:00
david-sarah
42ecd77f4e
SFTP: Increase test_sftp timeout to cater for francois' ARM buildslave.
2010-05-22 12:16:39 -07:00
david-sarah
5974773969
SFTP: Fix error in support for getAttrs on an open file, to index open files by directory entry rather than path. Extend that support to renaming open files. Also, implement the extposix-rename@openssh.org extension, and some other minor refactoring.
2010-05-21 20:58:36 -07:00
david-sarah
5545a678bf
SFTP tests: fix test_openDirectory_and_attrs that was failing in timezones west of UTC.
2010-05-20 11:10:27 -07:00
david-sarah
ce3872d10e
SFTP: allow getAttrs to succeed on a file that has been opened for creation but not yet uploaded or linked (part of #1050 ).
2010-05-19 20:56:13 -07:00
david-sarah
7f9e351bce
SFTP: improve logging so that results of requests are (usually) logged.
2010-05-19 17:36:52 -07:00
david-sarah
a62733ed08
SFTP: add tests for more combinations of open flags.
2010-05-18 22:39:33 -07:00
david-sarah
f0945526ce
SFTP: allow FXF_WRITE | FXF_TRUNC ( #1050 ).
2010-05-18 21:32:40 -07:00
david-sarah
819eaa74c1
SFTP: fixes related to reporting of permissions (needed for sshfs).
2010-05-17 22:45:21 -07:00
david-sarah
8d118e28f1
SFTP: change error code returned for ExistingChildError to FX_FAILURE (fixes gvfs with some picky programs such as gedit).
2010-05-17 17:42:05 -07:00
david-sarah
57699fd1eb
SFTP: fixed bugs that caused hangs during write ( #1037 ).
2010-05-16 21:42:28 -07:00
david-sarah
5f9c10901b
SFTP: work around a probable bug in twisted.conch.ssh.session:loseConnection(). Also some minor error handling cleanups.
2010-05-16 18:26:06 -07:00
david-sarah
6a2b0229f0
SFTP: Support statvfs extensions, avoid logging actual data, and decline shell sessions politely.
2010-05-16 08:43:47 -07:00
david-sarah
31994aaf74
SFTP: fix error in SFTPUserHandler arguments introduced by execCommand patch.
2010-05-15 18:40:45 -07:00
david-sarah
8028bf01bc
SFTP: implement execCommand to interoperate with clients that issue a 'df -P -k /' command. Also eliminate use of Zope adaptation.
2010-05-15 18:27:54 -07:00
david-sarah
14d0721246
Update SFTP implementation and tests: fix #1038 and switch to foolscap logging; also some code reorganization.
2010-05-13 21:31:13 -07:00
david-sarah
5fdd63d0ad
Tests for new SFTP implementation
2010-05-11 23:05:52 -07:00
david-sarah
4556702044
Fix test failures in test_web caused by changes to web page titles in #1062 . Also, change a 'target' field to '_blank' instead of 'blank' in welcome.xhtml.
2010-06-03 16:21:05 -07:00
david-sarah
4f58355a0e
dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.
2010-06-01 18:46:44 -07:00
david-sarah
53f7d2c7fe
dirnode.py: Fix #1034 (MetadataSetter does not enforce restriction on setting 'tahoe' subkeys), and expose the metadata updater for use by SFTP. Also, support diminishing a child cap to read-only if 'no-write' is set in the metadata.
2010-05-31 21:54:28 -07:00
david-sarah
6d91b1e00e
Trivial whitespace changes.
2010-05-27 12:41:14 -07:00
david-sarah
e706f043f0
Change shouldFail to avoid Unicode errors when converting Failure to str
2010-05-11 23:07:54 -07:00
david-sarah
90e9fe28af
Resolve conflict in patch to change imports to absolute.
2010-06-02 22:46:08 -07:00
Zooko O'Whielacronx
9e2da05837
tests: refactor test_web.py to have less duplication of literal caps-from-the-future
...
This is a prelude to a patch which will add tests of caps from the future which have non-ascii chars in them.
2010-05-18 22:51:46 -07:00
david-sarah
a80f19a084
Correct harmless indentation errors found by pylint
2010-02-25 23:21:51 -07:00
david-sarah
e76092e16c
Change relative imports to absolute
2010-02-26 01:14:33 -07:00
Zooko O'Whielacronx
d4f6d4092c
test: show stdout and stderr in case of non-zero exit code from "tahoe" command
2010-05-24 00:33:48 -07:00
Zooko O'Whielacronx
294c0a6279
tests: fix pyflakes warnings in bench_dirnode.py
2010-05-21 13:25:11 -07:00
Zooko O'Whielacronx
35efdf04ca
tests: don't let bench_dirnode.py do stuff and have side-effects at import time (unless __name__ == '__main__')
2010-05-21 05:20:52 -07:00
Zooko O'Whielacronx
a48979516e
tests: increase timeout to give François's ARM buildslave a chance to complete the tests
2010-05-20 06:45:26 -07:00
Kevan Carstensen
d0706d27ea
Hush pyflakes warnings
2010-05-15 11:43:44 -07:00
Kevan Carstensen
9179dd099a
Improve code coverage of the Tahoe2PeerSelector tests.
2010-05-14 20:29:13 -07:00
Kevan Carstensen
4e29060847
Change stray "shares_of_happiness" to "servers_of_happiness"
2009-11-16 15:24:59 -07:00
Kevan Carstensen
5683112a02
Revisions of the #778 tests, per reviewers' comments
...
- Fix comments and confusing naming.
- Add tests for the new error messages suggested by David-Sarah
and Zooko.
- Alter existing tests for new error messages.
- Make sure that the tests continue to work with the trunk.
- Add a test for a mutual disjointedness assertion that I added to
upload.servers_of_happiness.
- Fix the comments to correctly reflect read-onlyness
- Add a test for an edge case in should_add_server
- Add an assertion to make sure that share redistribution works as it
should
- Alter tests to work with revised servers_of_happiness semantics
- Remove tests for should_add_server, since that function no longer exists.
- Alter tests to know about merge_peers, and to use it before calling
servers_of_happiness.
- Add tests for merge_peers.
- Add Zooko's puzzles to the tests.
- Edit encoding tests to expect the new kind of failure message.
- Edit tests to expect error messages with the word "only" moved as far
to the right as possible.
- Extended and cleaned up some helper functions.
- Changed some tests to call more appropriate helper functions.
- Added a test for the failing redistribution algorithm
- Added a test for the progress message
- Added a test for the upper bound on readonly peer share discovery.
2010-05-13 18:25:42 -07:00
Kevan Carstensen
d360ee0d2e
Alter various unit tests to work with the new happy behavior
2010-01-07 12:13:25 -07:00
Kevan Carstensen
953c9555ca
Replace "UploadHappinessError" with "UploadUnhappinessError" in tests.
2009-12-04 22:34:53 -07:00
Kevan Carstensen
c0f3dbb919
Add tests for the behavior described in #834 .
2009-11-22 19:20:08 -07:00
Kevan Carstensen
c3b11dedea
Re-work 'test_upload.py' to be more readable; add more tests for #778
2009-11-16 13:23:34 -07:00
Kevan Carstensen
9590690450
Test Tahoe2PeerSelector to make sure that it recognizeses existing shares on readonly servers
2009-11-08 18:37:35 -07:00
Kevan Carstensen
324a72e4a3
Add more tests for comment:53 in ticket #778
2009-11-04 05:28:49 -07:00
Kevan Carstensen
697f1cfae8
Add a test for upload.shares_by_server
2009-11-04 05:13:24 -07:00
Kevan Carstensen
0d8e70bfe5
Minor tweak to an existing test -- make the first server read-write, instead of read-only
2009-11-03 21:42:32 -07:00
Kevan Carstensen
5ad29d4e8b
Alter tests to use the new form of set_shareholders
2009-11-03 21:36:02 -07:00
"Kevan Carstensen"
ee9690b357
Refactor some behavior into a mixin, and add tests for the behavior described in #778
2009-10-30 02:19:08 -07:00
Kevan Carstensen
362f204075
Alter NoNetworkGrid to allow the creation of readonly servers for testing purposes.
2009-10-17 18:30:13 -07:00
Zooko O'Whielacronx
6037df55b3
web: add a simple test that you can load directory.xhtml
2010-05-09 23:37:29 -07:00
Francois Deppierraz
dd95e8e71b
CLI: Support for https url in option --node-url
...
This patch modifies the regular expression used for verifying of '--node-url'
parameter. Support for accessing a Tahoe gateway over HTTPS was already
present, thanks to Python's urllib.
2010-04-30 11:56:09 -07:00
Brian Warner
f6f8256b35
"tahoe webopen": add --info flag, to get ?t=info
...
Also fix some trailing whitespace.
2010-04-24 16:30:03 -07:00
david-sarah
a2ed17f2a0
Additional test for DIR2-LIT directories in test_web.py, fixed version ( #948 )
2010-02-24 20:18:24 -08:00
david-sarah
e6aee33bb7
directories: add DIR2-LIT directories to test_deepcheck.py ( #948 )
2010-02-23 23:54:33 -08:00
david-sarah
187d837c1d
dirnode: add tests of literal dirnodes (current and fix for #948 )
2010-02-23 20:33:45 -08:00
david-sarah
fec9185f2a
Additional fix for abbrev_si, with test
2010-02-21 19:36:52 -08:00
david-sarah
d29ec184a6
Additions to test_web.py for #948
2010-02-21 18:53:52 -08:00
david-sarah
973f0afdd3
Change direct accesses to an_uri.storage_index to calls to .get_storage_index() ( fixes #948 )
2010-02-21 18:45:04 -08:00
david-sarah
9741b9655f
Test behaviour of 'tahoe ls' for unknown objects ( #837 )
2010-02-23 18:59:13 -08:00
Brian Warner
1de4d2c594
test_util.py: improve coverage of util.time_format
2010-02-23 23:46:37 -05:00
Kevan Carstensen
81ad52d6eb
Change OphandleTable to use a deterministic clock, so we can test it
...
To test the changes for #577 , we need a deterministic way to simulate
the passage of long periods of time. twisted.internet.task.Clock seems,
from my Googling, to be the way to go for this functionality. I changed
a few things so that OphandleTable would use twisted.internet.task.Clock
when testing:
* WebishServer.__init___ now takes an optional 'clock' parameter,
* which it passes to the root.Root instance it creates.
* root.Root.__init__ now takes an optional 'clock' parameter, which it
passes to the OphandleTable.__init__ method.
* OphandleTable.__init__ now takes an optional 'clock' parameter. If
it is provided, and it isn't None, its callLater method will be used
to schedule ophandle expirations (as opposed to using
reactor.callLater, which is what OphandleTable does normally).
* The WebMixin object in test_web.py now sets a self.clock parameter,
which is a twisted.internet.task.Clock that it feeds to the
WebishServer it creates.
Tests using the WebMixin can control the passage of time in
OphandleTable by accessing self.clock.
2010-02-20 13:07:13 -08:00
Kevan Carstensen
bcdc78bd44
Add tests for the ophandle expiration behavior in #577
2010-02-20 17:04:55 -08:00
david-sarah
85a50feeaa
More cleanups to test_cli using new utilities for reading and writing files.
2010-02-05 17:38:55 -08:00
david-sarah
c984a09fe7
Fix race conditions and missing callback in allmydata.test.test_cli.Cp.test_copy_using_filecap, add utilities for one-liner reading and writing of files, and fix cases in test_cli where files were not being closed after writing.
2010-02-05 17:37:27 -08:00
Kevan Carstensen
c00a62a2e7
Add tests for #939
2010-02-11 22:21:37 -08:00
Brian Warner
4040b1000b
web/storage.py: display total-seen on the last-complete-cycle line. For #940 .
2010-02-07 16:20:10 -08:00
Brian Warner
880f824103
code coverage: replace figleaf with coverage.py, should work on py2.6 now.
...
It still lacks the right HTML report (the builtin report is very pretty, but
lacks the "lines uncovered" numbers that I want), and the half-finished
delta-from-last-run measurements.
2010-02-03 08:54:21 -08:00
Zooko O'Whielacronx
3e4342ecb3
immutable: downloader accepts notifications of buckets even if those notifications arrive after he has begun downloading shares.
...
This can be useful if one of the ones that he has already begun downloading fails. See #287 for discussion. This fixes part of #287 which part was a regression caused by #928 , namely this fixes fail-over in case a share is corrupted (or the server returns an error or disconnects). This does not fix the related issue mentioned in #287 if a server hangs and doesn't reply to requests for blocks.
2010-01-31 22:16:10 -08:00
Zooko O'Whielacronx
e4e2599017
tests: don't require tahoe to run with no noise if we are using an old twisted that emits DeprecationWarnings
2010-01-31 21:23:23 -08:00
Zooko O'Whielacronx
7d32fafc44
tests: stop being surprised that Nevow no longer prints out warnings when it tries to find its static files
...
Unless we are using a sufficiently new version of Nevow, in which case if it prints out warnings then this is a hard test failure. :-)
2010-01-31 20:11:44 -08:00
david-sarah
6215ebd95e
cli: suppress DeprecationWarnings emitted from importing nevow and twisted. Fixes #859
2010-01-31 16:44:29 -08:00
david-sarah
ea3954372a
Improvements to test_unknownnode to cover invalid cap URIs with known prefixes
2010-01-29 22:39:08 -08:00
david-sarah
37a242e01a
Improvements to test_hung_server, and fix for status updates in download.py
2010-01-29 22:43:03 -08:00
Zooko O'Whielacronx
d62428c1e6
immutable: fix bug in tests, change line-endings to unix style, add comment
2010-01-29 10:42:37 -08:00
david-sarah
baa11a0ad4
New tests for #928
2010-01-29 04:38:45 -08:00
david-sarah
14280b009c
Eliminate 'foo if test else bar' syntax that isn't supported by Python 2.4
2010-01-28 19:52:10 -08:00
david-sarah
4560e021a9
Show -IMM and -RO suffixes for types of immutable and read-only unknown nodes in directory listings
2010-01-28 14:08:00 -08:00
david-sarah
1bb627405f
Fix inaccurate comment in test_mutant_dirnodes_are_omitted
2010-01-28 12:24:56 -08:00
david-sarah
b9eda4de6a
Address comments by Kevan on 833 and add test for stripping spaces
2010-01-27 15:06:42 -08:00
david-sarah
56c00cb381
Miscellaneous documentation, test, and code formatting tweaks.
2010-01-26 23:03:09 -08:00
david-sarah
6057bc02cc
Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.
2010-01-26 22:44:30 -08:00
Brian Warner
3880486f91
test_runner: cleanup, refactor common code into a non-executable method
...
Having both test_node() and test_client() (one of which calls the other) felt
confusing to me, so I changed it to have test_node(), test_client(), and a
common do_create() helper method.
2010-01-27 14:40:40 -08:00
Brian Warner
e769bbb6dd
tahoe backup: skip all symlinks, with warning. Fixes #850 , addresses #641 .
2010-01-27 14:35:17 -08:00
david-sarah
5c5a6fe610
Patch to accept t=set-children as well as t=set_children
2010-01-23 19:00:20 -08:00
francois
b03406af9d
tahoe_backup.py: display warnings on errors instead of stopping the whole backup. Fix #729 .
...
This patch displays a warning to the user in two cases:
1. When special files like symlinks, fifos, devices, etc. are found in the
local source.
2. If files or directories are not readables by the user running the 'tahoe
backup' command.
In verbose mode, the number of skipped files and directories is printed at the
end of the backup.
Exit status returned by 'tahoe backup':
- 0 everything went fine
- 1 the backup failed
- 2 files were skipped during the backup
2010-01-20 01:42:49 -08:00
david-sarah
174d356692
Message saying that we couldn't find bin/tahoe should say where we looked
2010-01-16 12:45:56 -08:00
david-sarah
87f1bae7fe
Add create-node CLI command, and make create-client equivalent to create-node --no-storage ( fixes #760 )
2010-01-15 21:20:55 -08:00
Brian Warner
de14791caf
Fix webapi t=mkdir with multpart/form-data, as on the Welcome page. Closes #919 .
2010-01-20 22:50:52 -08:00
Brian Warner
9ab7524f0d
test_dirnode.py: reduce scope of a Client instance, suggested by Kevan.
2010-01-14 22:27:13 -08:00
Brian Warner
01a7630334
test_provisioning: STAN is not always a list. Fix by David-Sarah Hopwood.
2010-01-14 17:46:32 -08:00
Brian Warner
e9223fcd0e
hush pyflakes-0.4.0 warnings: slightly less-trivial fixes. Closes #900 .
...
This includes one fix (in test_web) which was testing the wrong thing.
2010-01-14 14:17:19 -08:00
Brian Warner
731d15e56f
hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900 .
2010-01-14 14:15:29 -08:00
Brian Warner
874a979a8e
tahoe add-alias/create-alias: don't corrupt non-newline-terminated alias
...
file. Closes #741 .
2010-01-14 13:02:46 -08:00
Brian Warner
d3d1293d2f
change docs and --help to use "grid" instead of "virtual drive": closes #892 .
...
Thanks to David-Sarah Hopwood for the patch.
2010-01-14 12:11:19 -08:00
Brian Warner
d888bf3377
Clean up log.err calls, for one of the issues in #889 .
...
allmydata.util.log.err() either takes a Failure as the first positional
argument, or takes no positional arguments and must be invoked in an
exception handler. Fixed its signature to match both foolscap.logging.log.err
and twisted.python.log.err . Included a brief unit test.
2010-01-11 17:33:43 -08:00
Brian Warner
bacb6fe5aa
tidy up DeadReferenceError handling, ignore them in add_lease calls
...
Stop checking separately for ConnectionDone/ConnectionLost, since those have
been folded into DeadReferenceError since foolscap-0.3.1 . Write
rrefutil.trap_deadref() in terms of rrefutil.trap_and_discard() to improve
code coverage.
2010-01-11 16:07:23 -08:00
Brian Warner
1ed5bbdcb6
test_repairer: add (commented-out) test_each_byte, to see exactly what the
...
Verifier misses
The results (described in #819 ) match our expectations: it misses corruption
in unused share fields and in most container fields (which are only visible
to the storage server, not the client). 1265 bytes of a 2753 byte
share (hosting a 56-byte file with an artifically small segment size) are
unused, mostly in the unused tail of the overallocated UEB space (765 bytes),
and the allocated-but-unwritten plaintext_hash_tree (480 bytes).
2010-01-10 12:35:52 -08:00
Zooko O'Whielacronx
e7a9c000bf
repairer: fix some wrong offsets in the randomized verifier tests, debugged by Brian
...
fixes #819
2010-01-10 12:37:21 -08:00
Brian Warner
1ff49a15bb
test_repairer: fix colliding basedir names, which caused test inconsistencies
2010-01-10 00:46:19 -08:00
Zooko O'Whielacronx
876c4a153b
repairer: add deterministic test for #819 , mark as TODO
2010-01-09 17:36:19 -08:00
Brian Warner
ba0690c9d7
mutable repair: return successful=False when numshares<k (thus repair fails),
...
instead of weird errors. Closes #874 and #786 .
Previously, if the file had 0 shares, this would raise TypeError as it tried
to call download_version(None). If the file had some shares but fewer than
'k', it would incorrectly raise MustForceRepairError.
Added get_successful() to the IRepairResults API, to give repair() a place to
report non-code-bug problems like this.
2009-12-29 15:37:46 -08:00
Brian Warner
794e32738f
checker: don't let failures in add-lease affect checker results. Closes #875 .
...
Mutable servermap updates and the immutable checker, when run with
add_lease=True, send both the do-you-have-block and add-lease commands in
parallel, to avoid an extra round trip time. Many older servers have problems
with add-lease and raise various exceptions, which don't generally matter.
The client-side code was catching+ignoring some of them, but unrecognized
exceptions were passed through to the DYHB code, concealing the DYHB results
from the checker, making it think the server had no shares.
The fix is to separate the code paths. Both commands are sent at the same
time, but the errback path from add-lease is handled separately. Known
exceptions are ignored, the others (both unknown-remote and all-local) are
logged (log.WEIRD, which will trigger an Incident), but neither will affect
the DYHB results.
The add-lease message is sent first, and we know that the server handles them
synchronously. So when the checker is done, we can be sure that all the
add-lease messages have been retired. This makes life easier for unit tests.
2009-12-29 15:01:08 -08:00
Brian Warner
a50fdca6ab
test_cli: verify fix for "tahoe get" not creating empty file on error ( #121 )
2009-12-27 18:54:44 -05:00
Brian Warner
00d0ca3902
addendum to "Fix 'tahoe ls' on files ( #771 )"
...
tahoe_ls.py: tolerate missing metadata
web/filenode.py: minor cleanups
test_cli.py: test 'tahoe ls FILECAP'
2009-12-27 18:21:49 -05:00
Brian Warner
a8a768ef9d
Fix 'tahoe ls' on files ( #771 ). Patch adapted from Kevan Carstensen.
...
web/filenode.py: also serve edge metadata when using t=json on a
DIRCAP/childname object.
tahoe_ls.py: list file objects as if we were listing one-entry directories.
Show edge metadata if we have it, which will be true when doing
'tahoe ls DIRCAP/filename' and false when doing 'tahoe ls
FILECAP'
2009-12-27 17:54:43 -05:00
Brian Warner
499add09e6
webapi: don't accept zero-length childnames during traversal. Closes #358 , #676 .
...
This forbids operations that would implicitly create a directory with a
zero-length (empty string) name, like what you'd get if you did "tahoe put
local /oops/blah" (#358 ) or "POST /uri/CAP//?t=mkdir" (#676 ). The error
message is fairly friendly too.
Also added code to "tahoe put" to catch this error beforehand and suggest the
correct syntax (i.e. without the leading slash).
2009-12-27 15:10:43 -05:00
Brian Warner
2e0a61a953
CLI: send 'Accept:' header to ask for text/plain tracebacks. Closes #646 .
...
The webapi has been looking for an Accept header since 1.4.0, but it treats a
missing header as equal to */* (to honor RFC2616). This change finally
modifies our CLI tools to ask for "text/plain, application/octet-stream",
which seems roughly correct (we either want a plain-text traceback or error
message, or an uninterpreted chunk of binary data to save to disk). Some day
we'll figure out how JSON fits into this scheme.
2009-12-27 14:58:28 -05:00
Brian Warner
a4a6c02ef8
more #859 : avoid deprecation warning for unit tests too, hush pyflakes
...
* factor maybe-import-sha logic into util.hashutil
2009-12-14 16:01:47 -08:00
Brian Warner
aa3caf812d
mutable/servermap.py: stop reaching into private MutableFileNode attributes
2009-12-08 09:26:08 -08:00
Brian Warner
a980ea337b
remove MutableFileNode.download(), prefer download_best_version() instead
2009-12-01 17:54:38 -05:00
Brian Warner
96834da0a2
Simplify immutable download API: use just filenode.read(consumer, offset, size)
...
* remove Downloader.download_to_data/download_to_filename/download_to_filehandle
* remove download.Data/FileName/FileHandle targets
* remove filenode.download/download_to_data/download_to_filename methods
* leave Downloader.download (the whole Downloader will go away eventually)
* add util.consumer.MemoryConsumer/download_to_data, for convenience
(this is mostly used by unit tests, but it gets used by enough non-test
code to warrant putting it in allmydata.util)
* update tests
* removes about 180 lines of code. Yay negative code days!
Overall plan is to rewrite immutable/download.py and leave filenode.read() as
the sole read-side API.
2009-12-01 17:53:30 -05:00
Brian Warner
74974b27fe
Implement more coherent behavior when copying with dircaps/filecaps ( closes #761 ). Patch by Kevan Carstensen.
2009-11-30 13:10:09 -08:00
david-sarah
ef002c935a
storage server: detect disk space usage on Windows too ( fixes #637 )
2009-11-20 21:56:44 -08:00
Brian Warner
9901a08fd3
Add dirnodes to backupdb and "tahoe backup", closes #606 .
...
* backups now share dirnodes with any previous backup, in any location,
so renames and moves are handled very efficiently
* "tahoe backup" no longer bothers reading the previous snapshot
* if you switch grids, you should delete ~/.tahoe/private/backupdb.sqlite,
to force new uploads of all files and directories
2009-11-26 15:42:57 -08:00
Brian Warner
366a309795
webapi: fix t=check for DIR2-LIT (i.e. empty immutable directories)
2009-11-26 15:27:31 -08:00
Brian Warner
41bcc9f39e
PipelineError: fix str() on python2.4 . Closes #842 .
2009-11-24 13:25:12 -08:00
Brian Warner
d918ab407e
test_uri.py: s/NewDirnode/Dirnode/ , now that they aren't "new" anymore
2009-11-19 23:55:53 -08:00
Brian Warner
0cf320c2ab
interface name cleanups: IFileNode, IImmutableFileNode, IMutableFileNode
...
The proper hierarchy is:
IFilesystemNode
+IFileNode
++IMutableFileNode
++IImmutableFileNode
+IDirectoryNode
Also expand test_client.py (NodeMaker) to hit all IFilesystemNode types.
2009-11-19 23:52:55 -08:00
Brian Warner
d2badbea78
class name cleanups: s/FileNode/ImmutableFileNode/
...
also fix test/bench_dirnode.py for recent dirnode changes
2009-11-19 23:22:39 -08:00
Brian Warner
6e7fb1006d
Use DIR-IMM and t=mkdir-immutable for "tahoe backup", for #828
2009-11-18 11:28:13 -08:00
Brian Warner
e046744f40
make get_size/get_current_size consistent for all IFilesystemNode classes
...
* stop caching most_recent_size in dirnode, rely upon backing filenode for it
* start caching most_recent_size in MutableFileNode
* return None when you don't know, not "?"
* only render None as "?" in the web "more info" page
* add get_size/get_current_size to UnknownNode
2009-11-18 11:16:24 -08:00
Brian Warner
512fe3ad62
ImmutableDirectoryURIVerifier: fix verifycap handling
2009-11-18 08:42:38 -08:00
Brian Warner
f85690697a
Add t=mkdir-immutable to the webapi. Closes #607 .
...
* change t=mkdir-with-children to not use multipart/form encoding. Instead,
the request body is all JSON. t=mkdir-immutable uses this format too.
* make nodemaker.create_immutable_dirnode() get convergence from SecretHolder,
but let callers override it
* raise NotDeepImmutableError instead of using assert()
* add mutable= argument to DirectoryNode.create_subdirectory(), default True
2009-11-17 23:09:00 -08:00
Brian Warner
d2426ea3bd
move convergence secret into SecretHolder, next to lease secret
2009-11-17 17:54:44 -08:00
Brian Warner
5fe713fc52
nodemaker: implement immutable directories (internal interface), for #607
...
* nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT
* client.create_immutable_dirnode() is used to create them
* no webapi yet
2009-11-11 16:22:33 -08:00
Brian Warner
cc422f8dc0
stop using IURI()/etc as an adapter
2009-11-11 14:45:42 -08:00
Brian Warner
131e05b155
clean up uri-vs-cap terminology, emphasize cap instances instead of URI strings
...
* "cap" means a python instance which encapsulates a filecap/dircap (uri.py)
* "uri" means a string with a "URI:" prefix
* FileNode instances are created with (and retain) a cap instance, and
generate uri strings on demand
* .get_cap/get_readcap/get_verifycap/get_repaircap return cap instances
* .get_uri/get_readonly_uri return uri strings
* add filenode.download_to_filename() for control.py, should find a better way
* use MutableFileNode.init_from_cap, not .init_from_uri
* directory URI instances: use get_filenode_cap, not get_filenode_uri
* update/cleanup bench_dirnode.py to match, add Makefile target to run it
2009-11-11 14:26:19 -08:00
Brian Warner
f47672d12a
add parser for immutable directory caps: DIR2-CHK, DIR2-LIT, DIR2-CHK-Verifier
2009-11-04 10:13:51 -08:00
Zooko O'Whielacronx
9733201c0b
wui: s/TahoeLAFS/Tahoe-LAFS/
2009-10-28 18:50:50 -08:00
Zooko O'Whielacronx
1be4d45779
tests: bump up the timeout on test_repairer to see if 120 seconds was too short for François's ARM box to do the test even when it was doing it right.
2009-10-27 13:48:00 -08:00
Brian Warner
2695af91a7
dirnode.pack_children(): add deep_immutable= argument
...
This will be used by DIR2:CHK to enforce the deep-immutability requirement.
2009-10-26 09:28:09 -07:00
Brian Warner
768c76aa5f
webapi: use t=mkdir-with-children instead of a children= arg to t=mkdir .
...
This is safer: in the earlier API, an old webapi server would silently ignore
the initial children, and clients trying to set them would have to fetch the
newly-created directory to discover the incompatibility. In the new API,
clients using t=mkdir-with-children against an old webapi server will get a
clear error.
2009-10-25 18:13:21 -07:00
Brian Warner
1273b5c233
nodemaker.create_new_mutable_directory: pack_children() in initial_contents=
...
instead of creating an empty file and then adding the children later.
This should speed up mkdir(initial_children) considerably, removing two
roundtrips and an entire read-modify-write cycle, probably bringing it down
to a single roundtrip. A quick test (against the volunteergrid) suggests a
30% speedup.
test_dirnode: add new tests to enforce the restrictions that interfaces.py
claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts
2009-10-19 17:51:18 -07:00
Brian Warner
59ee02e82f
test_dirnode.py: add tests of initial_children= args to client.create_dirnode
...
and nodemaker.create_new_mutable_directory
2009-10-17 12:41:59 -07:00
Brian Warner
b4ec86c95a
update many dirnode interfaces to accept dict-of-nodes instead of dict-of-caps
...
interfaces.py: define INodeMaker, document argument values, change
create_new_mutable_directory() to take dict-of-nodes. Change
dirnode.set_nodes() and dirnode.create_subdirectory() too.
nodemaker.py: use INodeMaker, update create_new_mutable_directory()
client.py: have create_dirnode() delegate initial_children= to nodemaker
dirnode.py (Adder): take dict-of-nodes instead of list-of-nodes, which
updates set_nodes() and create_subdirectory()
web/common.py (convert_initial_children_json): create dict-of-nodes
web/directory.py: same
web/unlinked.py: same
test_dirnode.py: update tests to match
2009-10-17 12:28:29 -07:00
Brian Warner
ea373de042
move dirnode.CachingDict to dictutil.AuxValueDict, generalize method names,
...
improve tests. Let dirnode _pack_children accept either dict or AuxValueDict.
2009-10-17 11:00:05 -07:00
Brian Warner
e63f59f50c
test/common.py: update FakeMutableFileNode to new contents= callable scheme
2009-10-12 22:21:54 -07:00
Brian Warner
c2520e4ec7
client.create_mutable_file(contents=) now accepts a callable, which is
...
invoked with the new MutableFileNode and is supposed to return the initial
contents. This can be used by e.g. a new dirnode which needs the filenode's
writekey to encrypt its initial children.
create_mutable_file() still accepts a bytestring too, or None for an empty
file.
2009-10-12 20:12:32 -07:00
Brian Warner
b30041c5ec
webapi: t=mkdir now accepts initial children, using the same JSON that t=json
...
emits.
client.create_dirnode(initial_children=) now works.
2009-10-12 19:34:44 -07:00
Brian Warner
cf65cc2ae3
replace dirnode.create_empty_directory() with create_subdirectory(), which
...
takes an initial_children= argument
2009-10-12 19:15:20 -07:00
Brian Warner
d079eb45f6
dirnode.set_children: change return value: fire with self instead of None
2009-10-12 18:50:26 -07:00
Brian Warner
f871c3bb3d
dirnode.set_nodes: change return value: fire with self instead of None
2009-10-12 18:45:46 -07:00
Brian Warner
304aadd4f7
dirnode.set_children: take a dict, not a list
2009-10-12 17:24:40 -07:00
Brian Warner
e2ffc3dc03
dirnode.set_uri/set_children: change signature to take writecap+readcap
...
instead of a single cap. The webapi t=set_children call benefits too.
2009-10-12 16:51:26 -07:00
Brian Warner
3ee740628a
replace Client.create_empty_dirnode() with create_dirnode(), in anticipation
...
of adding initial_children= argument.
Includes stubbed-out initial_children= support.
2009-10-12 15:45:06 -07:00
Brian Warner
576c47d61e
test_web.py: use a less-fake client, making test harness smaller
2009-10-12 15:28:08 -07:00
Brian Warner
f4aa418086
Verifier: check the full cryptext-hash tree on each share. Removed .todos
...
from the last few test_repairer tests that were waiting on this.
2009-10-05 15:18:49 -07:00
Brian Warner
504c767d03
Verifier: check the full block-hash-tree on each share
...
Removed the .todo from two test_repairer tests that check this. The only
remaining .todos are on the three crypttext-hash-tree tests.
2009-10-05 14:48:44 -07:00
Brian Warner
e8f56af5a7
Verifier: check the full share-hash chain on each share
...
Removed the .todo from two test_repairer tests that check this.
2009-10-05 14:34:43 -07:00
Brian Warner
bbd00de495
test_repairer: rename Verifier test cases to be more precise and less verbose
2009-10-05 13:11:15 -07:00
Brian Warner
9dba4ec641
test/common.py: wrap docstrings to 80cols so I can read them more easily
2009-10-05 13:01:43 -07:00
Zooko O'Whielacronx
aaaa633f18
setup: remove a convenience copy of figleaf, to ease inclusion into Ubuntu Karmic Koala
...
We need to carefully document the licence of figleaf in order to get Tahoe-LAFS into Ubuntu Karmic Koala. However, figleaf isn't really a part of Tahoe-LAFS per se -- this is just a "convenience copy" of a development tool. The quickest way to make Tahoe-LAFS acceptable for Karmic then, is to remove figleaf from the Tahoe-LAFS tarball itself. People who want to run figleaf on Tahoe-LAFS (as everyone should want) can install figleaf themselves. I haven't tested this -- there may be incompatibilities between upstream figleaf and the copy that we had here...
2009-09-23 22:32:15 -07:00
Zooko O'Whielacronx
603e08c4a9
doc: licensing cleanups
...
Use nice utf-8 © instead of "(c)". Remove licensing statements on utility modules that have been assigned to allmydata.com by their original authors. (Nattraverso was not assigned to allmydata.com -- it was LGPL'ed -- but I checked and src/allmydata/util/iputil.py was completely rewritten and doesn't contain any line of code from nattraverso.) Add notes to misc/debian/copyright about licensing on files that aren't just allmydata.com-licensed.
2009-09-20 10:16:31 -07:00
Brian Warner
14cb5ca436
test/check_speed.py: fix shbang line
2009-08-17 17:59:48 -07:00
Brian Warner
5283d4c19e
de-Service-ify Helper, pass in storage_broker and secret_holder directly.
...
This makes it more obvious that the Helper currently generates leases with
the Helper's own secrets, rather than getting values from the client, which
is arguably a bug that will likely be resolved with the Accounting project.
2009-08-15 13:17:37 -07:00
Brian Warner
4a4a4f9520
immutable.Downloader: pass StorageBroker to constructor, stop being a Service
...
child of the client, access with client.downloader instead of
client.getServiceNamed("downloader"). The single "Downloader" instance is
scheduled for demolition anyways, to be replaced by individual
filenode.download calls.
2009-08-15 12:25:43 -07:00
Zooko O'Whielacronx
22c962bbc2
tests: double the timeout on test_runner.RunNode.test_introducer since feisty hit a timeout
...
I'm not sure if this is an actual timing issue (feisty is running on an overloaded VM if I recall correctly), or it there is a deeper bug.
2009-08-15 09:05:12 -07:00
Brian Warner
0d5dc51617
Overhaul IFilesystemNode handling, to simplify tests and use POLA internally.
...
* stop using IURI as an adapter
* pass cap strings around instead of URI instances
* move filenode/dirnode creation duties from Client to new NodeMaker class
* move other Client duties to KeyGenerator, SecretHolder, History classes
* stop passing Client reference to dirnode/filenode constructors
- pass less-powerful references instead, like StorageBroker or Uploader
* always create DirectoryNodes by wrapping a filenode (mutable for now)
* remove some specialized mock classes from unit tests
Detailed list of changes (done one at a time, then merged together)
always pass a string to create_node_from_uri(), not an IURI instance
always pass a string to IFilesystemNode constructors, not an IURI instance
stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri()
client.py: move SecretHolder code out to a separate class
test_web.py: hush pyflakes
client.py: move NodeMaker functionality out into a separate object
LiteralFileNode: stop storing a Client reference
immutable Checker: remove Client reference, it only needs a SecretHolder
immutable Upload: remove Client reference, leave SecretHolder and StorageBroker
immutable Repairer: replace Client reference with StorageBroker and SecretHolder
immutable FileNode: remove Client reference
mutable.Publish: stop passing Client
mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference
MutableChecker: reference StorageBroker and History directly, not through Client
mutable.FileNode: removed unused indirection to checker classes
mutable.FileNode: remove Client reference
client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker
move create_mutable_file() into NodeMaker
test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests.
test_mutable.py: clean up basedir names
client.py: move create_empty_dirnode() into NodeMaker
dirnode.py: get rid of DirectoryNode.create
remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match
stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker
remove Client from NodeMaker
move helper status into History, pass History to web.Status instead of Client
test_mutable.py: fix minor typo
2009-08-15 04:28:46 -07:00
Zooko O'Whielacronx
2a63fc9159
docs: update relnotes.txt, relnotes-short.txt, and others documentation bits for v1.5.0 release!
2009-08-01 19:57:10 -07:00
Brian Warner
0d0e0ba2fb
test_runner.py: tolerate symlinks in the source tree directory's ancestor path
2009-07-30 10:17:24 -07:00
Brian Warner
bf1e61c8f3
Touch up #705 changes:
...
webapi.txt: clarify replace=only-files argument, mention replace= on POST t=uri
test_cli.py: insert whitespace between logical operations
web.common.parse_replace_arg: make it case-insensitive, to match the docs
2009-07-20 11:38:03 -04:00
kevan
0d8b1e29fa
Add tests for tahoe mv behavior
2009-07-19 20:46:09 -07:00
kevan
4331326b21
Add tests for tahoe mv help text
2009-07-19 20:45:53 -07:00
kevan
8eb7ddab6b
Add tests for parse_replace_arg()
2009-07-19 20:47:05 -07:00
kevan
40360a7a9c
Add tests for new PUT behavior
2009-07-19 20:46:32 -07:00
kevan
18a80d99b1
Alter Adder + Adder tests to look for 'only-files' instead of 'only_files'
2009-07-19 20:43:18 -07:00
Zooko O'Whielacronx
c6fce3a1c6
directories: minor change: fix warning message from the benchmarking script
2009-07-19 17:21:14 -07:00
kevan
ca4de9ee97
Add unit tests for the Adder in dirnode.py
2009-07-18 12:50:49 -07:00
Brian Warner
531cc7899f
rename NewDirectoryNode to DirectoryNode, NewDirectoryURI to DirectoryURI
2009-07-17 17:15:49 -05:00
Brian Warner
1192b61dfe
upload: fix #758 recursion-loop in peer-selection when servers report errors.
...
The bug was in the code that handles a third-or-later pass, and was
previously untested.
2009-07-17 00:07:09 -05:00
Brian Warner
efa608a135
'tahoe stats': tolerate empty directories. Closes #693 .
2009-07-15 00:51:09 -07:00
Brian Warner
d8ba8c2eb5
Allow tests to pass with -OO by turning some AssertionErrors (the ones that
...
we actually exercise during tests) into more specific exceptions, so they
don't get optimized away. The best rule to follow is probably this: if an
exception is worth testing, then it's part of the API, and AssertionError
should never be part of the API. Closes #749 .
2009-07-14 23:45:10 -07:00
Zooko O'Whielacronx
10ad3ad7d6
wui: fix test for welcome page
2009-07-14 20:34:29 -07:00
kevan
44c652fbbe
Modify bench_dirnode.py to use CachingDict.
2009-07-05 15:31:42 -07:00
kevan
325793457d
Add tests for CachingDict, _pack_contents, _unpack_contents
2009-07-03 20:43:28 -07:00
Zooko O'Whielacronx
efafcfb91a
directories: keep track of your position as you decode netstring after netstring from an input buffer instead of copying the trailing part
...
This makes decoding linear in the number of netstrings instead of O(N^2).
2009-07-04 19:51:09 -07:00
Zooko O'Whielacronx
4206a2c1c7
directories: in bench_dirnode.py, use a real CacheDirectoryManager instead of a fake one (because CacheDirectoryManager is a significant user of CPU and/or time)
2009-07-06 20:41:19 -07:00
Zooko O'Whielacronx
0df5b446eb
directories: make the profiling behavior of bench_dirnode.py accessible by adding '--profile' to the cmdline
2009-07-06 20:30:35 -07:00
Zooko O'Whielacronx
e414c73877
directories: update the directory benchmarks to exercise the unpack-and-repack functionality, and add optional profiling
2009-07-05 09:29:53 -07:00
Zooko O'Whielacronx
859d7ba8b0
test/benchmark: benchmark the time to pack and unpack dirnodes
...
See also the prof_benchmarks() function in this file which will run the benchmarks under profiling.
2009-07-04 15:43:00 -07:00
Brian Warner
ef1b6ae8e3
Tolerate unknown URI types in directory structures. Part of #683 .
...
The idea is that future versions of Tahoe will add new URI types that this
version won't recognize, but might store them in directories that we *can*
read. We should handle these "objects from the future" as best we can.
Previous releases of Tahoe would just explode. With this change, we'll
continue to be able to work with everything else in the directory.
The code change is to wrap anything we don't recognize as an UnknownNode
instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how
to render these (mostly by leaving fields blank), deep-check knows to skip
over them, deep-stats counts them in "count-unknown". You can rename and
delete these things, but you can't add new ones (because we wouldn't know how
to generate a readcap to put into the dirnode's rocap slot, and because this
lets us catch typos better).
2009-07-02 18:07:49 -07:00
Brian Warner
4a46e91192
test_client.py: minor refactoring of BASECONFIG usage
2009-07-02 17:54:50 -07:00
Brian Warner
6237aeabd7
create_node_from_uri: take both writecap+readcap, move logic out of dirnode.py
2009-07-02 15:25:37 -07:00
Brian Warner
b640baf500
cli: webopen: when called with no arguments, open the Welcome page
2009-07-01 13:05:48 -07:00
Brian Warner
e330abc3c3
mutable repairer: skip repair of readcaps instead of throwing an exception.
...
This should improve the behavior of #625 a bit: at least all the files will
get repaired.
2009-06-30 18:13:43 -07:00
Brian Warner
52fa421430
use 522-bit RSA keys in all unit tests (except one)
...
This reduces the total test time on my laptop from 400s to 283s.
* src/allmydata/test/test_system.py (SystemTest.test_mutable._test_debug):
Remove assertion about container_size/data_size, this changes with keysize
and was too variable anyways.
* src/allmydata/mutable/filenode.py (MutableFileNode.create): add keysize=
* src/allmydata/dirnode.py (NewDirectoryNode.create): same
* src/allmydata/client.py (Client.DEFAULT_MUTABLE_KEYSIZE): add default,
this overrides the one in MutableFileNode
2009-06-29 15:31:24 -07:00
Brian Warner
c6ae255847
remove trailing whitespace
2009-06-29 13:03:58 -07:00
Brian Warner
2947da457e
hush pyflakes
2009-06-24 19:18:09 -07:00
Brian Warner
bd6ecc9f44
Split out NoSharesError, stop adding attributes to NotEnoughSharesError, change humanize_failure to include the original exception string, update tests, behave better if humanize_failure fails.
2009-06-24 19:17:07 -07:00
Brian Warner
837733641f
remove introducer/old.py, will create something similar when the RIIntroducer changes
2009-06-24 12:40:38 -07:00
Brian Warner
f573bb2929
check_grid.py: print stderr when a subcommand fails
2009-06-22 19:28:33 -07:00
Brian Warner
8df15e9f30
big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging
2009-06-22 19:10:47 -07:00
Brian Warner
699510c8f1
PollMixin: snoop trial's error observer to halt the test early if an error is seen. This turns a lot of timeouts into fast failures.
2009-06-22 19:07:31 -07:00
Brian Warner
711c09bc5d
clean up storage_broker interface: should fix #732
2009-06-21 16:51:19 -07:00
Brian Warner
f14004eeb3
hush pyflakes with recent FileTooLarge removal
2009-06-21 16:17:57 -07:00
kevan
efcc45951d
change max filesize limit tests
...
Instead of testing to see that the previous SDMF filesize limit was being
obeyed, we now test to make sure that we can insert files larger than that
limit.
2009-06-20 14:28:22 -07:00
Brian Warner
2c5aa12151
test_util: add known-answer tests for hashutil tags
2009-06-17 21:57:09 -07:00
Zooko O'Whielacronx
6a93caf7a3
tests: bump up a timeout that expired on Zandr's box
2009-06-13 12:57:03 -07:00
Zooko O'Whielacronx
45928315f6
util: Brian's horrible hack to figure out how much localtime and utctime differ. Now we'll see if it works on Windows.
2009-06-12 13:45:56 -07:00
Zooko O'Whielacronx
4a30c5899c
util: oops, time.tzset() doesn't work on Windows -- hopefully the new "London" unit test passes on Windows when we skip tzset() on platforms that don't have it
2009-06-11 17:09:20 -07:00
Zooko O'Whielacronx
8978cb0738
util: fix time_format.iso_utc_time_to_seconds() so that it works even in London
2009-06-11 15:11:29 -07:00
Zooko O'Whielacronx
0ed65fba34
tests: significantly increase timeouts that triggered on Zandr's ARM box
2009-06-10 09:10:43 -07:00
Zooko O'Whielacronx
69a3b23b34
test: multiple by 10 or so all timeouts that Zandr's ARM box just overran
2009-06-10 05:56:39 -07:00
Zooko O'Whielacronx
bdfb80834f
tests: bump up timeout on a test that timed out on draco
2009-06-09 21:46:28 -07:00
Zooko O'Whielacronx
974295f963
tests: raise the timeout for test_cli since Zandr's ARM machine totally burst through the old one
2009-06-09 14:05:09 -07:00
Brian Warner
2e35648901
test_cli.Backup: increase timeout massively, it takes 1200s on zandr's ARM linkstation
2009-06-08 22:28:01 -07:00
Zooko O'Whielacronx
3cfc00f5cf
tests: double the timeouts on some tests which time-out on Francois's box
2009-06-08 19:17:53 -07:00
Zooko O'Whielacronx
0bc116b3a2
tests: bump up timeouts so that the tests can finish before timeout on Francois's little arm box
2009-06-08 15:55:57 -07:00
Zooko O'Whielacronx
c3e9d36d97
tests: increase timeouts on some other tests that timed-out on Francois's arm box
2009-06-05 07:34:37 -07:00
Zooko O'Whielacronx
63c8f0c424
tests: bump up the timeout on a bunch of tests that took longer than the default timeout (120s) on François Lenny-armv5tel
2009-06-04 20:14:44 -07:00
Zooko O'Whielacronx
8c18ac38a9
backup: remove the --no-backupdb command, the handling of "can't import sqlite", and the related tests, and change an error message to more correctly indicate failure to load the database from disk rather than failure to import sqlite module
...
Fixes #728 .
2009-06-04 10:31:31 -07:00
Brian Warner
35b3f7f426
more refactoring: move get_all_serverids() and get_nickname_for_serverid() from Client to storage_broker
2009-06-01 20:07:50 -07:00
Brian Warner
b1290633b8
more storage_broker refactoring: downloader gets a broker instead of a client,
...
use Client.get_storage_broker() accessor instead of direct attribute access.
2009-06-01 19:25:11 -07:00
Brian Warner
3a1196701f
test_runner.py: remove test_client_no_noise: the issue in question is
...
ticketed in http://divmod.org/trac/ticket/2830 and doesn't need a Tahoe-side
change, plus this test fails on win32 for unrelated reasons (and test_client
is the place to think about the win32 issue).
2009-06-01 15:50:07 -07:00
Brian Warner
c516361fd2
start to factor server-connection-management into a distinct 'StorageServerFarmBroker' object, separate from the client and the introducer. This is the starting point for #467 : static server selection
2009-06-01 14:06:04 -07:00
Kevin Reid
23441389a5
Modify markup of Tahoe web pages to be more amenable to styling; some minor changes of wording.
2009-05-26 16:25:45 -07:00
Kevin Reid
a71c914e50
Tweak wording in directory page: not-read-only is "modifiable", mention creating a directory _in this directory_.
2009-05-26 16:24:14 -07:00
Brian Warner
1863aee0aa
switch to using RemoteException instead of 'wrapped' RemoteReferences. Should fix #653 , the rref-EQ problem
2009-05-21 17:46:32 -07:00
Brian Warner
c9803d5217
switch all foolscap imports to use foolscap.api or foolscap.logging
2009-05-21 17:38:23 -07:00
Brian Warner
b542d5aca6
test_runner.py: fix minor typo
2009-05-19 20:36:20 -07:00
Brian Warner
9f0bc04d32
test_introducer.py: add a test for the python2.4.0/2.4.1 bug in base64.b32decode
2009-05-18 20:41:01 -07:00
Brian Warner
e76c6b606f
util/pipeline.py: new utility class to manage size-limited work pipelines, for #392
2009-05-18 16:43:26 -07:00
Zooko O'Whielacronx
718537ab8c
tests: mark test_runner as coded in utf-8 instead of ascii
2009-05-07 15:31:51 -07:00
Zooko O'Whielacronx
04ff9a3f97
tests: raise timeout on test_runner.RunNode.test_introducer from 120s to 240s, since it hit the 120s time-out on François Lenny-armv5tel
2009-05-07 14:50:12 -07:00
Zooko O'Whielacronx
6350634ee3
tests: remove obsolete test (it tests functionality that is long gone) which occasionally ERRORs now that we have more metadata (since [20090411225205-92b7f-7adfb89cb4db4ac7d28427934dea3d2c108f6476])
2009-04-12 19:36:21 -07:00
Zooko O'Whielacronx
6b9899ade1
trivial: remove trailing whitespace and unused import
2009-04-11 19:17:42 -07:00
Zooko O'Whielacronx
9729753692
dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API
...
Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628 .
2009-04-11 15:52:05 -07:00
Brian Warner
922b3034be
web: make sure that PUT /uri?mutable=false really means immutable, fixes #675
2009-04-07 19:13:40 -07:00
Zooko O'Whielacronx
37c6e77764
wui: edit some of the human-readable parts of the wui such as button labels
...
(The word "parent" suggests that you can go up a directory hierarchy -- perhaps that word is vestigial.)
2009-04-07 11:54:59 -07:00
Zooko O'Whielacronx
8b1cd154da
wui: fix bug in which empty directory is marked as "unreadable", add test, remove exclamation point
2009-04-07 11:28:34 -07:00
Zooko O'Whielacronx
de8475389b
hashtree: fix tests of depth_of
2009-04-03 16:56:14 -07:00
Zooko O'Whielacronx
a07e746d9c
trivial: whitespace
2009-04-03 16:33:02 -07:00
Zooko O'Whielacronx
b12a7f9ee8
leases, time_format: modify time stamping in lease description
...
* emit lease expiry date in ISO-8601'ish format as well as Brian's format
* rename iso_utc_time_to_localseconds() to iso_utc_time_to_seconds()
* add iso_utc_date()
* simplify the body of iso_utc_time_to_seconds()
2009-04-03 15:59:04 -07:00
Brian Warner
466014f66f
hashtree: fix O(N**2) behavior, to improve fatal alacrity problems in a 10GB file ( #670 ). Also improve docstring.
2009-03-31 13:21:27 -07:00
Brian Warner
bd93430c53
expirer: include crawler progress in the JSON status output
2009-03-24 13:51:37 -07:00
Brian Warner
01e2032669
hashutil: add constant-time comparison function, to avoid timing attacks when python's short-circuiting data-dependent == operator is used to, say, check a write-enabler
2009-03-22 20:20:55 -07:00
Brian Warner
6599eae6f9
WUI: fix display of empty directories, it threw an exception before
2009-03-20 16:58:09 -07:00
Brian Warner
3b65607926
expirer: tolerate empty buckets, refactor bucketsize-counting code a bit, don't increment -mutable/-immutable counters unless we actually know the sharetype
2009-03-20 12:18:16 -07:00
Brian Warner
8645738c77
storage: improve wording of status message
2009-03-19 11:48:37 -07:00
Brian Warner
66fe8ca24c
parse_date: insist that it returns an int
2009-03-19 11:44:43 -07:00
Brian Warner
186b6a8c01
storage status: report expiration-cutoff-date like 19-Mar-2009 (as opposed to the tahoe.cfg input format of 2009-03-19), for redundancy: someone who gets the month and day switched will have a better chance to spot the problem in the storage-status output if it's in a different format
2009-03-19 11:07:56 -07:00
Brian Warner
f0071c2571
expirer: clean up constructor args, add tahoe.cfg controls, use cutoff_date instead of date_cutoff
2009-03-18 18:00:09 -07:00
Brian Warner
babcf632da
util/time_format: new routine to parse dates like 2009-03-18, switch expirer to use it. I'd prefer to use 18-Mar-2009, but it is surprisingly non-trivial to build a parser that will take UTC dates instead of local dates
2009-03-18 17:58:14 -07:00
Brian Warner
8eaee28550
expirer: change setup, config options, in preparation for adding tahoe.cfg controls
2009-03-18 17:21:38 -07:00
Brian Warner
fffab0d724
expirer: track mutable-vs-immutable sharecounts and sizes, report them on the web status page for comparison
2009-03-18 13:25:04 -07:00
Brian Warner
406fdba61f
add utility function to parse durations, for lease-expiration config
2009-03-17 00:01:17 -07:00
Brian Warner
24ab5ec26f
expirer: add mode to expire only-mutable or only-immutable shares
2009-03-16 23:51:18 -07:00
Brian Warner
c7254c5f1d
GC: add date-cutoff -based expiration, add proposed docs
2009-03-16 22:10:41 -07:00
Brian Warner
2b525a42d3
test_deepcheck: remove the 10s timeout: our dapper buildslave requires 30s, and the reduced timeout was only there because this tests fails by timeout rather than explicitly
2009-03-15 17:51:34 -07:00
Brian Warner
f0b0ad1c8f
tahoe cp -r: add --caps-only flag, to write filecaps into local files instead of actual file contents. Used only for debugging and as a quick tree-comparison tool.
2009-03-15 16:19:58 -07:00
Brian Warner
6e57576f2e
dirnode deep_traverse: insert a turn break (fireEventually) at least once every 100 files, otherwise a CHK followed by more than 158 LITs can overflow the stack, sort of like #237 .
2009-03-13 16:31:35 -07:00
Brian Warner
ddc9a7ae74
consolidator: re-use more directories, add total directories seen-vs-used counts
2009-03-12 20:48:01 -07:00
Brian Warner
1c24707f19
consolidator: add more verbose traversal of directories
2009-03-12 16:29:00 -07:00
Brian Warner
760688a224
consolidate: create multiple numbered backups of the original Archives directory, not just the first time
2009-03-12 16:04:27 -07:00
Brian Warner
c4c6a62954
add 'tahoe debug consolidate' command, to merge directories created by repeated 'tahoe cp -r' or the allmydata win32 backup tool, into the form that would have been created by 'tahoe backup'.
2009-03-12 13:56:06 -07:00
Brian Warner
1ccd426a34
expirer: fix prediction math, thanks to Zandr for the catch
2009-03-09 13:42:17 -07:00
Brian Warner
a68ad06254
storage.expirer: exercise the last missing line of webstatus code
2009-03-08 20:38:28 -07:00
Brian Warner
6d6049430c
expirer: tolerate corrupt shares, add them to the state and history for future examination
2009-03-08 20:08:40 -07:00
Brian Warner
1a98521c3d
storage/immutable: raise a specific error upon seeing a bad version number, instead of using assert. Also wrap to 80cols.
2009-03-08 20:07:32 -07:00
Brian Warner
02b40ec499
storage: add test coverage for BucketReader.__repr__
2009-03-08 20:05:27 -07:00
Brian Warner
6d7319c588
storage/mutable: raise a specific error upon seeing bad magic, instead of using assert
2009-03-08 19:02:01 -07:00
Brian Warner
5675b4e7e0
expirer: make web display a bit more consistent
2009-03-07 16:14:42 -07:00
Brian Warner
2c2c3c2295
test_cli.py: assert that 'ls' on an unrecoverable file now gives a better error message
2009-03-07 05:08:15 -07:00
Brian Warner
67d7600781
test_web: test behavior of broken-dirnode GET, both html and json
2009-03-07 04:57:07 -07:00
Brian Warner
9c7fafd5af
test_cli: use explicit (and stable) testdirs, instead of using self.mktemp
2009-03-07 03:04:28 -07:00
Brian Warner
091cfd04ca
test_storage: solaris doesn't appear to give the same block count as other platforms, so don't assert as much about 'diskbytes' recovered
2009-03-07 02:45:18 -07:00
Brian Warner
0dee2a6036
storage: add a lease-checker-and-expirer crawler, plus web status page.
...
This walks slowly through all shares, examining their leases, deciding which
are still valid and which have expired. Once enabled, it will then remove the
expired leases, and delete shares which no longer have any valid leases. Note
that there is not yet a tahoe.cfg option to enable lease-deletion: the
current code is read-only. A subsequent patch will add a tahoe.cfg knob to
control this, as well as docs. Some other minor items included in this patch:
tahoe debug dump-share has a new --leases-only flag
storage sharefile/leaseinfo code is cleaned up
storage web status page (/storage) has more info, more tests coverage
space-left measurement on OS-X should be more accurate (it was off by 2048x)
(use stat .f_frsize instead of f_bsize)
2009-03-06 22:45:17 -07:00
Brian Warner
c5ecb7b3ba
test_cli: validate non-HTML error response of 'tahoe get' on an unrecoverable file
2009-03-03 22:11:46 -07:00
Brian Warner
f42e3bb107
web: full patch for HTML-vs-plaintext traceback renderings, improve test coverage of exception rendering
2009-03-03 21:56:30 -07:00
Brian Warner
53f50fcc78
test/common: rearrange shouldHTTPError
2009-03-03 19:40:59 -07:00
Brian Warner
67571eb033
add more information to NotEnoughSharesError, split out new exceptions for no-servers and no-source-of-ueb-hash
2009-03-03 19:37:15 -07:00
Brian Warner
f15c0be5af
test_web: move shouldHTTPError into common.WebErrorMixin
2009-03-03 16:56:20 -07:00
Brian Warner
559c369574
test_web: clean up shouldHTTPError methods
2009-03-03 16:46:04 -07:00
Brian Warner
8c3013c4f7
rrefutil: add check_remote utility function
2009-02-27 00:59:57 -07:00
Brian Warner
1b3e635936
rrefutil: add trap_remote utility and friends
2009-02-27 00:55:24 -07:00
Brian Warner
8251572e01
web: improve layout of storage status with a table
2009-02-26 19:58:38 -07:00
Brian Warner
112dc35563
crawler: add ETA to get_progress()
2009-02-26 19:42:48 -07:00
Brian Warner
63b19e567c
web: fix the ERROR: line to work the same in python2.4 and 2.5
2009-02-25 01:46:21 -07:00
Brian Warner
7d4901197a
test_cli/test_web: fix spurious test failure on solaris (maybe python2.4?) due to variations in the way that exceptions are stringified
2009-02-25 00:01:28 -07:00
Brian Warner
a3c1fe35d9
CLI: modify 'tahoe manifest' and 'tahoe deep-check' to report ERROR: properly. For #590 .
2009-02-24 23:44:15 -07:00
Brian Warner
fd4ceb6a87
webapi: modify streaming deep-manifest/deep-checker to emit an ERROR: line if they encounter an unrecoverable+untraversable directory. For #590 .
2009-02-24 23:13:35 -07:00
Brian Warner
7f8708cb7c
scripts/common: fix alias handling on windows again, emit slightly nicer error message in response to an unknown alias
2009-02-24 22:21:36 -07:00
Brian Warner
51e3baeefc
test_deepcheck: switch deep-check tests to use no-network too. This cuts the runtime down by about 50%
2009-02-24 21:04:57 -07:00
Brian Warner
6cf4096156
test_deepcheck: convert MutableChecker to no-network GridTest
2009-02-24 20:00:10 -07:00
Brian Warner
2e7f64d392
tests/no_network: move GET into the GridTestMixin class
2009-02-24 18:33:00 -07:00
Brian Warner
6f981e4eab
common_web.py: oops, add .fields to the other FakeRequest
2009-02-24 18:04:59 -07:00
Brian Warner
376eab0c04
test_cli: exercise the recent tolerate-'c:\dir\file.txt' fix in scripts/common, recorded in a separate match to make it easier to merge the fix to prod
2009-02-24 17:56:20 -07:00
Brian Warner
8d8250e44a
test/common_web.py: add a .fields attribute to our FakeRequest, since we support versions of Nevow that are old enough to not do it themselves
2009-02-24 17:20:50 -07:00
Brian Warner
2810de32b1
test_web: add (disabled) test to see what happens when deep-check encounters an unrecoverable directory. We still need code changes to improve this behavior.
2009-02-24 15:40:17 -07:00
Brian Warner
7cfbb9c832
test_repairer: change to use faster no_network.GridTestMixin, split Verifier tests into separate cases, refactor judgement funcs into shared methods
2009-02-23 22:15:06 -07:00
Brian Warner
2be729b1e4
test_repairer: change Repairer to use much-faster no_network.GridTestMixin. As a side-effect, fix what I think was a bug: some of the assert-minimal-effort-expended checks were mixing write counts and allocate counts
2009-02-23 17:42:27 -07:00
Brian Warner
cc3c1ae8df
test/no_network.py: add a basic stats provider
2009-02-23 17:39:37 -07:00
Brian Warner
f35c659b2f
tests: stop using setUpClass/tearDownClass, since they've been deprecated in Twisted-8.2.0
2009-02-23 14:43:12 -07:00
Brian Warner
bc91689f8e
test_checker: improve test coverage for checker results
2009-02-23 14:19:43 -07:00
Alberto Berti
d9649052d4
Fixed tests again so they will pass on windows.
2009-02-22 18:35:02 -07:00
Brian Warner
c4f8005867
remove more RuntimeError from unit tests, for #639
2009-02-22 17:28:55 -07:00
Brian Warner
5c3d7d8e73
stop using RuntimeError in unit tests, for #639
2009-02-22 17:27:22 -07:00
Alberto Berti
b945c275c2
Added tests for the cse when listdir is an iterator
2009-02-22 16:43:56 -07:00
Alberto Berti
cd0bd97954
Fixed tests so that they pass also on buildbots.
2009-02-22 16:43:11 -07:00
Alberto Berti
b2f8fe04c6
Use failUnlessEqual instead of failUnless(a == b)
2009-02-22 16:42:14 -07:00
Alberto Berti
5d629d0050
Added tests for the --exclude* options of backup command.
2009-02-22 10:51:06 -07:00
Alberto Berti
ff6a77cc6b
Added tests for the fixed alias related command's synopsis
2009-02-22 10:37:32 -07:00
Brian Warner
c85eccbf64
test_web.py: increase test coverage of web.status.plural()
2009-02-21 18:01:16 -07:00
Brian Warner
77f3b83d68
crawler: fix performance problems: only save state once per timeslice (not after every bucket), don't start the crawler until 5 minutes after node startup
2009-02-21 14:56:49 -07:00
Brian Warner
a04d3b8fe8
test_system: oops, don't assume that all files in storage/ are in a deep storage/shares/prefix/si/shnum path, since now the crawler pickle has a short path
2009-02-21 00:17:10 -07:00
Brian Warner
3c6471c717
test_storage: startService the server, as is now the standard practice
2009-02-20 21:57:55 -07:00
Brian Warner
1077826357
BucketCountingCrawler: rename status and state keys to use 'bucket' instead of 'share', because the former is more accurate
2009-02-20 21:46:06 -07:00
Brian Warner
b3cd4952bd
storage: add bucket-counting share crawler, add its output (number of files+directories maintained by a storage server) and status to the webapi /storage page
2009-02-20 21:04:08 -07:00
Brian Warner
73e05bf967
crawler: add get_progress, clean up get_state
2009-02-20 18:27:43 -07:00
Brian Warner
2e45619844
web/storage: make sure we can handle platforms without os.statvfs too
2009-02-20 16:03:53 -07:00
Brian Warner
c6a061e600
crawler: provide for one-shot crawlers, which stop after their first full cycle, for share-upgraders and database-populaters
2009-02-20 15:19:11 -07:00
Brian Warner
00677ff9a5
web: add Storage status page, improve tests
2009-02-20 14:29:26 -07:00
Brian Warner
c6c30ac5d4
webapi: pass client through constructor arguments, remove IClient, should make it easier to test web renderers in isolation
2009-02-20 12:15:54 -07:00
Brian Warner
e357b8a679
test/no_network: do startService on the storage servers, make it easier to customize the storage servers
2009-02-19 20:22:54 -07:00
Brian Warner
ef4ff21ae7
crawler: modify API to support upcoming bucket-counting crawler
2009-02-19 19:31:42 -07:00
Brian Warner
9bc08158c6
test_backupdb: make the not-a-database file larger, since the older sqlite-2.3.2 on OS-X is easily fooled
2009-02-19 18:04:09 -07:00
Brian Warner
2f751332e8
test_crawler: hush pyflakes
2009-02-19 14:23:40 -07:00
Brian Warner
ff9811211d
test_crawler: disable the percentage-of-cpu-used test, since it is too unreliable on our slow buildslaves. But leave the code in place for developers to run by hand.
2009-02-19 14:16:54 -07:00
Brian Warner
790a10d1b2
reliability.py: fix the numpy conversion, it was completely broken. Thanks to Terrell Russell for the help.
2009-02-19 13:55:15 -07:00
Brian Warner
268cd538e2
reliability: switch to NumPy, since Numeric is deprecated
2009-02-19 01:44:35 -07:00
Brian Warner
f10aa59e5b
test_crawler: don't require >=1 cycle on cygwin
2009-02-19 00:58:18 -07:00
Brian Warner
b949ea4f32
crawler: fix problems on windows and our slow cygwin slave
2009-02-18 22:24:31 -07:00
Brian Warner
193889f793
#633 : first version of a rate-limited interruptable share-crawler
2009-02-18 21:46:33 -07:00
Brian Warner
4646451de6
change StorageServer to take nodeid in the constructor, instead of assigning it later, since it's cleaner and because the original problem (Tubs not being ready until later) went away
2009-02-18 16:23:01 -07:00
Brian Warner
b398a69f14
test_system: split off checker tests to test_deepcheck.py, this file is too big
2009-02-18 15:42:34 -07:00
Brian Warner
ef53da2b12
break storage.py into smaller pieces in storage/*.py . No behavioral changes.
2009-02-18 14:46:55 -07:00
Brian Warner
bce4a5385b
add --add-lease to 'tahoe check', 'tahoe deep-check', and webapi.
2009-02-17 19:32:43 -07:00
Brian Warner
e9563ebc02
change RIStorageServer.remote_add_lease to exit silently in case of no-such-bucket, instead of raising IndexError, because that makes the upcoming --add-lease feature faster and less noisy
2009-02-17 19:30:53 -07:00
Brian Warner
fde2289e7b
CLI #590 : convert 'tahoe deep-check' to streaming form, improve display, add tests
2009-02-17 17:15:11 -07:00
Brian Warner
d52eadfc58
test_backupdb: improve error messages if the test fails
2009-02-17 11:08:38 -07:00
Brian Warner
476a5c8fac
webapi #590 : add streaming deep-check. Still need a CLI tool to use it.
2009-02-16 23:35:53 -07:00
Brian Warner
8579e25059
test_web.Grid: change the CHECK() function to make it easier to test t= values with hyphens in them
2009-02-16 23:00:34 -07:00
Brian Warner
522641bb32
test_web: improve checker-results coverage with a no-network -based test, enhance no-network harness to assist, fix some bugs in web/check_results.py that were exposed
2009-02-16 22:12:42 -07:00
Brian Warner
f2f8ede11f
test/no_network: update comments with setup timing: no_network takes 50ms, SystemTestMixin takes 2s (on my laptop)
2009-02-16 18:06:43 -07:00
Brian Warner
040cb39613
test_upload: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary
2009-02-16 17:44:57 -07:00
Brian Warner
911abcc34b
test_download: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary
2009-02-16 17:36:58 -07:00
Brian Warner
e9427ea777
test_dirnode.py: convert Deleter to new no-network gridtest
2009-02-16 17:23:48 -07:00
Brian Warner
267d91f797
test_cli.py: modify to use the new 'no-network' gridtest instead of SystemTestMixin, which speeds it up from 73s to 43s on my system
2009-02-16 17:20:05 -07:00
Brian Warner
191fd8556a
tests: fix no_network framework to work with upload/download and checker
2009-02-16 17:19:47 -07:00
Brian Warner
770f976a85
test_cli: add test coverage for help strings
2009-02-16 15:08:33 -07:00
Brian Warner
ebe0f2ac08
test/no_network: new test harness, like system-test but doesn't use the network so it's faster
2009-02-16 14:58:44 -07:00
Brian Warner
df90dd8e73
test_cli: improve test coverage slightly
2009-02-15 21:04:51 -07:00
Brian Warner
3235b9630b
test_util: get almost full test coverage of dictutil, starting with the original pyutil tests as a base. The remaining three uncovered lines involve funny cases of ValueOrderedDict that I can't figure out how to get at
2009-02-15 20:32:10 -07:00
Brian Warner
cc27b218f5
provisioning/reliability: add tests, hush pyflakes, remove dead code, fix web links
2009-02-15 16:24:51 -07:00
Brian Warner
e1380b132b
util/statistics: add tests, fix mean_repair_cost
2009-02-15 16:23:26 -07:00
Brian Warner
812f23585d
test_repairer: hush pyflakes
2009-02-15 16:23:10 -07:00
Shawn Willden
21832280da
Statistics module
...
Added a statistics module for calculating various facets of
share survival statistics.
2009-01-13 20:12:35 -07:00
Brian Warner
60b06b8d32
test_dirnode: #625 run deep-check on a readonly dirnode too
2009-02-13 14:53:37 -07:00
Zooko O'Whielacronx
2d45d192d1
immutable repairer: add tests of how it handles short reads
2009-02-12 16:48:53 -07:00
Zooko O'Whielacronx
8303f90da0
nodeadmin: node stops itself if a hotline file hasn't been touched in 60 seconds now, instead of in 40 seconds
...
A test failed on draco (MacPPC) because it took 49 seconds to get around to running the test, and the node had already stopped itself when the hotline file was 40 seconds old.
2009-02-12 15:21:31 -07:00
Zooko O'Whielacronx
76d7cc4404
immutable repairer: errback any pending readers of DownUpConnectorwhen it runs out of bytes, and test that fact
2009-02-11 20:11:29 -07:00
Zooko O'Whielacronx
7eb260a9cf
versioning: include an "appname" in the application version string in the versioning protocol, and make that appname be controlled by setup.py
...
It is currently hardcoded in setup.py to be 'allmydata-tahoe'. Ticket #556 is to make it configurable by a runtime command-line argument to setup.py: "--appname=foo", but I suddenly wondered if we really wanted that and at the same time realized that we don't need that for tahoe-1.3.0 release, so this patch just hardcodes it in setup.py.
setup.py inspects a file named 'src/allmydata/_appname.py' and assert that it contains the string "__appname__ = 'allmydata-tahoe'", and creates it if it isn't already present. src/allmydata/__init__.py import _appname and reads __appname__ from it. The rest of the Python code imports allmydata and inspects "allmydata.__appname__", although actually every use it uses "allmydata.__full_version__" instead, where "allmydata.__full_version__" is created in src/allmydata/__init__.py to be:
__full_version__ = __appname + '-' + str(__version__).
All the code that emits an "application version string" when describing what version of a protocol it supports (introducer server, storage server, upload helper), or when describing itself in general (introducer client), usese allmydata.__full_version__.
This fixes ticket #556 at least well enough for tahoe-1.3.0 release.
2009-02-11 17:18:16 -07:00
Brian Warner
5f288842d5
test_repairer: disable repair-from-corruption tests until other things are improved well enough to make it useful
2009-02-11 15:01:59 -07:00
Zooko O'Whielacronx
079c1851a9
trivial: whitespace
2009-02-11 11:19:35 -07:00
Brian Warner
7d907f35aa
storage: make add-lease work, change default ownernum=1 since 0 is reserved to mean 'no lease here'
2009-02-10 23:39:38 -07:00
Brian Warner
13a3ef5ec1
#620 : storage: allow mutable shares to be deleted, with a writev where new_length=0
2009-02-10 23:37:56 -07:00
Brian Warner
6a5f28f47d
test_cli.Backup: insert some stalls to make sure two successive backups get distinct timestamps, avoiding intermittent failures
2009-02-10 20:37:09 -07:00
Brian Warner
f7a263eb0b
#619 : make 'tahoe backup' complain and refuse to run if sqlite is unavailable and --no-backupdb is not passed
2009-02-10 18:49:10 -07:00
Brian Warner
3b1c8ceb86
tests: increase the default timeout for SystemTestMixin -based tests to 300 seconds, since our slower buildslaves sometimes take longer than the default 120s
2009-02-10 14:44:12 -07:00
Zooko O'Whielacronx
ef1bfdd2bf
immutable: repairer: add a simple test to exercise the "leftover" code path, fix the bug (and rename the variable "leftover" to "extra")
2009-02-10 12:12:45 -07:00
Zooko O'Whielacronx
75e4e67ed7
immutable: tighten preconditions -- you can write empty strings or read zero bytes, and add the first simple unit test of DownUpConnector
2009-02-10 00:56:47 -07:00
Zooko O'Whielacronx
d436c6235d
immutable: tests: the real WRITE_LEEWAY is 35 (it was a mistake to move it from 10 to 35 earlier -- I had seen a failure in which it took 35 times as many writes as I thought were optimal, but I misread and thought it took only 20 times as many)
2009-02-09 23:53:48 -07:00
Zooko O'Whielacronx
65c12b24b8
immutable: tests: sigh, raise, again the limit of how many extra writes you can do and still pass this test
...
Obviously requiring the code under test to perform within some limit isn't very meaningful if we raise the limit whenever the test goes outside of it.
But I still don't want to remove the test code which measures how many writes (and, elsewhere, how many reads) a client does in order to fulfill these duties.
Let this number -- now 20 -- stand as an approximation of the inefficiency of our code divided by my mental model of how many operations are actually optimal for these duties.
2009-02-09 20:09:31 -07:00
Zooko O'Whielacronx
5738d94ccd
immutable: tests: assert that verifier gives a clean bill of health after corruption and repair (the previous patch mistakenly did this only after deletion and repair), and also test whether deleting seven other shares and then downloading works. Also count the number of shares stored in the local filesystem.
2009-02-09 20:08:41 -07:00
Zooko O'Whielacronx
bbf48f4a88
immutable: test: add a test after attempting to repair from corruption: does a full verify run give the file a clean bill of health? If not, the you haven't successfully repaired it.
...
This will make the repairer tests more consistent -- less accidentally passing due to getting lucky.
2009-02-09 19:01:49 -07:00
Zooko O'Whielacronx
4091f73f34
immutable: tests: put shares back to their pristine condition in between each test of corrupting-and-repairing them
...
This is important, because if the repairer doesn't completely repair all kinds of corruption (as the current one doesn't), then the successive tests get messed up by assuming that the shares were uncorrupted when the test first set about to corrupt them.
2009-02-09 18:29:56 -07:00
Brian Warner
b197c5bb8c
test_runner: skip all spawnProcess-using tests on cygwin, since spawnProcess just hangs forever
2009-02-09 02:34:00 -07:00
Brian Warner
19902a0f16
test_runner.py: revert the CreateNode section to using runner() inline, rather than spawning a process, to get more precise coverage
2009-02-09 02:26:17 -07:00
Brian Warner
89683a3906
test_upload: add test of maximum-immutable-share-size, to complete the last item of #538
2009-02-08 19:41:27 -07:00
Brian Warner
d8e00c37b5
test_cli.Backup: capture stderr when sqlite is unavailable
2009-02-07 15:14:40 -07:00
Brian Warner
213ef3d64b
test_repairer: wrap comments to 80cols, my laptop does not have a wide screen. No functional changes.
2009-02-07 14:06:26 -07:00
Brian Warner
50886ea494
test/common.py: in share-layout-reading code, use '>L' consistently, since '>l' doesn't specify the signedness and the windows tests appear to be failing with an endianness-like problem (version==0x01000000). Also use binary mode when editing sharefiles
2009-02-07 13:38:17 -07:00
Brian Warner
1fd8f023ab
test_cli: increase timeout on test_backup, since our dapper buildslave is really slow
2009-02-06 02:17:53 -07:00
Brian Warner
9933aa0d49
test_backupdb.py: reset the check-timers after one step, otherwise a slow host can false-fail
2009-02-06 01:33:05 -07:00
Brian Warner
814fec5495
test_cli.backup: oops, fix test to work even when sqlite is unavailable
2009-02-05 22:10:42 -07:00
Brian Warner
ca32db5b39
#598 : add cli+backupdb tests, improve user display, update docs, move docs out of proposed/
2009-02-05 22:07:01 -07:00
Brian Warner
4be2e11d11
add sqlite-based backupdb, for #598 and others (including 'tahoe cp'). Not enabled yet.
2009-02-05 18:17:56 -07:00
Brian Warner
4e572a5f24
webapi: add verifycap (spelled 'verify_url') to the t=json output on files and directories. Closes #559 .
2009-02-03 19:22:48 -07:00
Brian Warner
cfce8b5eab
#598 : first cut of 'tahoe backup' command: no backupdb, but yes shared-unchanged-directories and Archives/TIMESTAMP and Latest/
2009-02-02 21:09:02 -07:00
Zooko O'Whielacronx
e08e8d3f41
storage: disable test_large_share again: my linux laptop has less than 4 GiB free
2009-01-30 22:16:49 -07:00
Zooko O'Whielacronx
f0682cf43e
setup: more verbose assertion failure in test_runner
2009-01-29 10:49:06 -07:00
Zooko O'Whielacronx
204629be43
storage: enable the test of a share larger than 2 GiB; this will probably be too expensive on Mac OS X, but I think it won't be on all the other platforms ; I will probably set it to SkipTest if the sys.platform is Mac after seeing the results of this buildbot run
2009-01-28 16:33:12 -07:00
Zooko O'Whielacronx
8d7fe2adb3
trivial: remove unused imports noticed by pyflakes
2009-01-27 15:11:48 -07:00
Zooko O'Whielacronx
64d12a9402
setup: fix test_runner to invoke bin/tahoe.exe instead of bin/tahoe if on Windows (this is what happens when a user invokes bin/tahoe on Windows)
2009-01-27 14:37:17 -07:00
Zooko O'Whielacronx
1ae08acc8f
setup: fix test_runner to assert that lines which are output to stderr must end with a punctuation mark (apparently re.search("x$", "x\r\n") does not match. :-()
2009-01-27 14:35:05 -07:00
Zooko O'Whielacronx
3cd4722c9b
setup: fix test_runner.RunNode.test_baddir -- it was left incomplete and broken by a recent patch
2009-01-27 14:32:45 -07:00
Zooko O'Whielacronx
688bb7a453
setup: go ahead and check for noise in test_client_no_noise
2009-01-26 17:46:16 -07:00
Zooko O'Whielacronx
8d2a3e78ca
setup: add a test for a warning message from importing nevow, marked as TODO, comment-out the assertion of no-noise inside other test_runner tests
2009-01-26 17:30:46 -07:00
Zooko O'Whielacronx
3e4450cf3f
setup: find a "bin/tahoe" executable to test based on allmydata.__file__ instead of based on the CWD
...
This means that the tests still work if you are executing them from a CWD other than the src dir -- *if* the "bin/tahoe" is found at os.path.dirname(os.path.dirname(os.path.dirname(allmydata.__file__))).
If no file is found at that location, then just skip the tests of executing the "tahoe" executable, because we don't want to accidentally run those tests against an executable from a different version of tahoe.
2009-01-23 18:34:37 -07:00
Brian Warner
1de4110b35
#509 : remove non-streaming 'tahoe manifest' CLI form
2009-01-23 17:00:02 -07:00
Brian Warner
6a04103cc2
#509 CLI: add 'tahoe manifest --stream'
2009-01-23 16:33:21 -07:00
Brian Warner
5a0e3f3a40
#509 : test_system.py: add test for streamed-manifest
2009-01-23 16:32:47 -07:00
Brian Warner
9ade197c3b
test_system: rearrange DeepCheckWebGood to make it easier to add CLI tests
2009-01-23 16:13:06 -07:00
Brian Warner
26260374e9
#590 : add webish t=stream-manifest
2009-01-22 22:01:36 -07:00
Brian Warner
72adeccf2d
dirnode: add get_repair_cap()
2009-01-22 21:44:49 -07:00
Brian Warner
d8b3505cf5
filenode: add get_repair_cap(), which uses the read-write filecap for immutable files, and the verifycap for immutable files
2009-01-22 21:38:36 -07:00
Zooko O'Whielacronx
b77c89ac60
setup: add test that the tests are testing the right source code
...
This is a test of #145 , and I think that now the code passes this test.
2009-01-22 15:52:40 -07:00
Zooko O'Whielacronx
967102e1b9
trivial: removed unused imports noticed by pyflakes
2009-01-22 15:52:13 -07:00
Zooko O'Whielacronx
210da5cd9a
setup: change test_runner to invoke "bin/tahoe" in a subprocess instead of executing runner.runner()
...
This is necessary because loading allmydata code now depends on PYTHONPATH manipulation which is done in the "bin/tahoe" script. Unfortunately it makes test_runner slower since it launches and waits for many subprocesses.
2009-01-22 15:38:18 -07:00
Zooko O'Whielacronx
7d91dc5efe
rollback the feature of making "ambient upload authority" configurable
...
This reverses some, but not all, of the changes that were committed in the following set of patches.
rolling back:
Sun Jan 18 09:54:30 MST 2009 toby.murray
* add 'web.ambient_upload_authority' as a paramater to tahoe.cfg
M ./src/allmydata/client.py -1 +3
M ./src/allmydata/test/common.py -7 +9
A ./src/allmydata/test/test_ambient_upload_authority.py
M ./src/allmydata/web/root.py +12
M ./src/allmydata/webish.py -1 +4
Sun Jan 18 09:56:08 MST 2009 zooko@zooko.com
* trivial: whitespace
I ran emacs's "M-x whitespace-cleanup" on the files that Toby's recent patch had touched that had trailing whitespace on some lines.
M ./src/allmydata/test/test_ambient_upload_authority.py -9 +8
M ./src/allmydata/web/root.py -2 +1
M ./src/allmydata/webish.py -2 +1
Mon Jan 19 14:16:19 MST 2009 zooko@zooko.com
* trivial: remove unused import noticed by pyflakes
M ./src/allmydata/test/test_ambient_upload_authority.py -1
Mon Jan 19 21:38:35 MST 2009 toby.murray
* doc: describe web.ambient_upload_authority
M ./docs/configuration.txt +14
M ./docs/frontends/webapi.txt +11
Mon Jan 19 21:38:57 MST 2009 zooko@zooko.com
* doc: add Toby Murray to the CREDITS
M ./CREDITS +4
2009-01-20 20:47:35 -07:00
Zooko O'Whielacronx
6bbbb375e1
setup: fix test_system to require tahoe under its package==distribution name "allmydata-tahoe" instead of its module name "allmydata"
2009-01-20 12:38:09 -07:00
Zooko O'Whielacronx
845b21a5ad
trivial: remove unused import noticed by pyflakes
2009-01-19 15:16:19 -07:00
Zooko O'Whielacronx
4771e1c9d5
trivial: whitespace
...
Ran "M-x whitespace-cleanup" on files that Toby's recent patch touched, even though they didn't have trailing whitespace.
2009-01-18 10:58:15 -07:00
Zooko O'Whielacronx
ce7c448bc0
trivial: whitespace
...
I ran emacs's "M-x whitespace-cleanup" on the files that Toby's recent patch had touched that had trailing whitespace on some lines.
2009-01-18 10:56:08 -07:00
toby.murray
66f83c7356
add 'web.ambient_upload_authority' as a paramater to tahoe.cfg
2009-01-18 10:54:30 -07:00
Zooko O'Whielacronx
f9ee60a88e
cli: tests: skip symlink test if there is no os.symlink
2009-01-14 18:10:10 -07:00
Brian Warner
2fe099a0b3
mutable: move recent operation history management code (MutableWatcher) into history.py, have History provide stats
2009-01-14 17:36:20 -07:00
Brian Warner
cc50e2f4aa
upload: use WriteBucketProxy_v2 when uploading a large file (with shares larger than 4GiB). This finally closes #346 . I think we can now handle immutable files up to 48EiB.
2009-01-12 20:14:42 -07:00
Brian Warner
8f2ec16b2c
test_repairer.py: hush pyflakes: remove duplicate/shadowed function name, by using the earlier definition (which is identical)
2009-01-12 15:45:09 -07:00
Brian Warner
fe362c0021
hush pyflakes by removing unused imports
2009-01-12 15:41:20 -07:00
Zooko O'Whielacronx
25063688b4
immutable repairer
...
This implements an immutable repairer by marrying a CiphertextDownloader to a CHKUploader. It extends the IDownloadTarget interface so that the downloader can provide some metadata that the uploader requires.
The processing is incremental -- it uploads the first segments before it finishes downloading the whole file. This is necessary so that you can repair large files without running out of RAM or using a temporary file on the repairer.
It requires only a verifycap, not a readcap. That is: it doesn't need or use the decryption key, only the integrity check codes.
There are several tests marked TODO and several instances of XXX in the source code. I intend to open tickets to document further improvements to functionality and testing, but the current version is probably good enough for Tahoe-1.3.0.
2009-01-12 11:00:22 -07:00
Zooko O'Whielacronx
8dd3b3185f
immutable: separate tests of immutable upload/download from tests of immutable checking/repair
2009-01-10 15:07:39 -07:00
Zooko O'Whielacronx
ef60e85ec6
naming: finish renaming "CheckerResults" to "CheckResults"
2009-01-09 18:00:52 -07:00
Brian Warner
45fc3819a2
test_helper: hush pyflakes by avoiding use of 'uri' as a variable, since it shadows an import of the same name
2009-01-08 20:59:41 -07:00
