Commit Graph

91 Commits

Author SHA1 Message Date
Zooko O'Whielacronx
3c4fe0706f M-x whitespace-cleanup 2010-07-17 20:27:39 -07:00
david-sarah
90e9fe28af Resolve conflict in patch to change imports to absolute. 2010-06-02 22:46:08 -07:00
david-sarah
e76092e16c Change relative imports to absolute 2010-02-26 01:14:33 -07:00
Kevan Carstensen
d0706d27ea Hush pyflakes warnings 2010-05-15 11:43:44 -07:00
Kevan Carstensen
9179dd099a Improve code coverage of the Tahoe2PeerSelector tests. 2010-05-14 20:29:13 -07:00
Kevan Carstensen
5683112a02 Revisions of the #778 tests, per reviewers' comments
- Fix comments and confusing naming.
- Add tests for the new error messages suggested by David-Sarah
  and Zooko.
- Alter existing tests for new error messages.
- Make sure that the tests continue to work with the trunk.
- Add a test for a mutual disjointedness assertion that I added to
  upload.servers_of_happiness.
- Fix the comments to correctly reflect read-onlyness
- Add a test for an edge case in should_add_server
- Add an assertion to make sure that share redistribution works as it 
  should
- Alter tests to work with revised servers_of_happiness semantics
- Remove tests for should_add_server, since that function no longer exists.
- Alter tests to know about merge_peers, and to use it before calling 
  servers_of_happiness.
- Add tests for merge_peers.
- Add Zooko's puzzles to the tests.
- Edit encoding tests to expect the new kind of failure message.
- Edit tests to expect error messages with the word "only" moved as far
  to the right as possible.
- Extended and cleaned up some helper functions.
- Changed some tests to call more appropriate helper functions.
- Added a test for the failing redistribution algorithm
- Added a test for the progress message
- Added a test for the upper bound on readonly peer share discovery.
2010-05-13 18:25:42 -07:00
Kevan Carstensen
d360ee0d2e Alter various unit tests to work with the new happy behavior 2010-01-07 12:13:25 -07:00
Kevan Carstensen
953c9555ca Replace "UploadHappinessError" with "UploadUnhappinessError" in tests. 2009-12-04 22:34:53 -07:00
Kevan Carstensen
c0f3dbb919 Add tests for the behavior described in #834. 2009-11-22 19:20:08 -07:00
Kevan Carstensen
c3b11dedea Re-work 'test_upload.py' to be more readable; add more tests for #778 2009-11-16 13:23:34 -07:00
Kevan Carstensen
9590690450 Test Tahoe2PeerSelector to make sure that it recognizeses existing shares on readonly servers 2009-11-08 18:37:35 -07:00
Kevan Carstensen
324a72e4a3 Add more tests for comment:53 in ticket #778 2009-11-04 05:28:49 -07:00
Kevan Carstensen
697f1cfae8 Add a test for upload.shares_by_server 2009-11-04 05:13:24 -07:00
Kevan Carstensen
0d8e70bfe5 Minor tweak to an existing test -- make the first server read-write, instead of read-only 2009-11-03 21:42:32 -07:00
Kevan Carstensen
5ad29d4e8b Alter tests to use the new form of set_shareholders 2009-11-03 21:36:02 -07:00
"Kevan Carstensen"
ee9690b357 Refactor some behavior into a mixin, and add tests for the behavior described in #778 2009-10-30 02:19:08 -07:00
david-sarah
973f0afdd3 Change direct accesses to an_uri.storage_index to calls to .get_storage_index() (fixes #948) 2010-02-21 18:45:04 -08:00
Brian Warner
d888bf3377 Clean up log.err calls, for one of the issues in #889.
allmydata.util.log.err() either takes a Failure as the first positional
argument, or takes no positional arguments and must be invoked in an
exception handler. Fixed its signature to match both foolscap.logging.log.err
and twisted.python.log.err . Included a brief unit test.
2010-01-11 17:33:43 -08:00
Brian Warner
d2426ea3bd move convergence secret into SecretHolder, next to lease secret 2009-11-17 17:54:44 -08:00
Brian Warner
cc422f8dc0 stop using IURI()/etc as an adapter 2009-11-11 14:45:42 -08:00
Brian Warner
0d5dc51617 Overhaul IFilesystemNode handling, to simplify tests and use POLA internally.
* stop using IURI as an adapter
* pass cap strings around instead of URI instances
* move filenode/dirnode creation duties from Client to new NodeMaker class
* move other Client duties to KeyGenerator, SecretHolder, History classes
* stop passing Client reference to dirnode/filenode constructors
  - pass less-powerful references instead, like StorageBroker or Uploader
* always create DirectoryNodes by wrapping a filenode (mutable for now)
* remove some specialized mock classes from unit tests

Detailed list of changes (done one at a time, then merged together)

always pass a string to create_node_from_uri(), not an IURI instance
always pass a string to IFilesystemNode constructors, not an IURI instance
stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri()
client.py: move SecretHolder code out to a separate class
test_web.py: hush pyflakes
client.py: move NodeMaker functionality out into a separate object
LiteralFileNode: stop storing a Client reference
immutable Checker: remove Client reference, it only needs a SecretHolder
immutable Upload: remove Client reference, leave SecretHolder and StorageBroker
immutable Repairer: replace Client reference with StorageBroker and SecretHolder
immutable FileNode: remove Client reference
mutable.Publish: stop passing Client
mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference
MutableChecker: reference StorageBroker and History directly, not through Client
mutable.FileNode: removed unused indirection to checker classes
mutable.FileNode: remove Client reference
client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker
move create_mutable_file() into NodeMaker
test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests.
test_mutable.py: clean up basedir names
client.py: move create_empty_dirnode() into NodeMaker
dirnode.py: get rid of DirectoryNode.create
remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match
stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker
remove Client from NodeMaker
move helper status into History, pass History to web.Status instead of Client
test_mutable.py: fix minor typo
2009-08-15 04:28:46 -07:00
Brian Warner
1192b61dfe upload: fix #758 recursion-loop in peer-selection when servers report errors.
The bug was in the code that handles a third-or-later pass, and was
previously untested.
2009-07-17 00:07:09 -05:00
Brian Warner
2947da457e hush pyflakes 2009-06-24 19:18:09 -07:00
Brian Warner
bd6ecc9f44 Split out NoSharesError, stop adding attributes to NotEnoughSharesError, change humanize_failure to include the original exception string, update tests, behave better if humanize_failure fails. 2009-06-24 19:17:07 -07:00
Brian Warner
8df15e9f30 big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging 2009-06-22 19:10:47 -07:00
Zooko O'Whielacronx
0ed65fba34 tests: significantly increase timeouts that triggered on Zandr's ARM box 2009-06-10 09:10:43 -07:00
Zooko O'Whielacronx
3cfc00f5cf tests: double the timeouts on some tests which time-out on Francois's box 2009-06-08 19:17:53 -07:00
Zooko O'Whielacronx
0bc116b3a2 tests: bump up timeouts so that the tests can finish before timeout on Francois's little arm box 2009-06-08 15:55:57 -07:00
Brian Warner
b1290633b8 more storage_broker refactoring: downloader gets a broker instead of a client,
use Client.get_storage_broker() accessor instead of direct attribute access.
2009-06-01 19:25:11 -07:00
Brian Warner
c516361fd2 start to factor server-connection-management into a distinct 'StorageServerFarmBroker' object, separate from the client and the introducer. This is the starting point for #467: static server selection 2009-06-01 14:06:04 -07:00
Brian Warner
c9803d5217 switch all foolscap imports to use foolscap.api or foolscap.logging 2009-05-21 17:38:23 -07:00
Brian Warner
5c3d7d8e73 stop using RuntimeError in unit tests, for #639 2009-02-22 17:27:22 -07:00
Brian Warner
040cb39613 test_upload: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary 2009-02-16 17:44:57 -07:00
Zooko O'Whielacronx
7eb260a9cf versioning: include an "appname" in the application version string in the versioning protocol, and make that appname be controlled by setup.py
It is currently hardcoded in setup.py to be 'allmydata-tahoe'.  Ticket #556 is to make it configurable by a runtime command-line argument to setup.py: "--appname=foo", but I suddenly wondered if we really wanted that and at the same time realized that we don't need that for tahoe-1.3.0 release, so this patch just hardcodes it in setup.py.
setup.py inspects a file named 'src/allmydata/_appname.py' and assert that it contains the string "__appname__ = 'allmydata-tahoe'", and creates it if it isn't already present.  src/allmydata/__init__.py import _appname and reads __appname__ from it.  The rest of the Python code imports allmydata and inspects "allmydata.__appname__", although actually every use it uses "allmydata.__full_version__" instead, where "allmydata.__full_version__" is created in src/allmydata/__init__.py to be:

__full_version__ = __appname + '-' + str(__version__).

All the code that emits an "application version string" when describing what version of a protocol it supports (introducer server, storage server, upload helper), or when describing itself in general (introducer client), usese allmydata.__full_version__.

This fixes ticket #556 at least well enough for tahoe-1.3.0 release.
2009-02-11 17:18:16 -07:00
Brian Warner
89683a3906 test_upload: add test of maximum-immutable-share-size, to complete the last item of #538 2009-02-08 19:41:27 -07:00
Brian Warner
cc50e2f4aa upload: use WriteBucketProxy_v2 when uploading a large file (with shares larger than 4GiB). This finally closes #346. I think we can now handle immutable files up to 48EiB. 2009-01-12 20:14:42 -07:00
Brian Warner
0fab511be5 upload: don't use servers which can't support the share size we need. This ought to avoid #439 problems. Some day we'll have a storage server which advertises support for a larger share size. No tests yet. 2008-11-21 20:28:12 -07:00
Brian Warner
cae54ab118 tahoe.cfg: add controls for k and N (and shares-of-happiness) 2008-11-18 00:29:44 -07:00
Brian Warner
b73c380cdb move testutil into test/common_util.py, since it doesn't count as 'code under test' for our pyflakes numbers 2008-10-28 21:28:31 -07:00
Brian Warner
914655c52b interfaces.py: promote immutable.encode.NotEnoughSharesError.. it isn't just for immutable files any more 2008-10-27 13:34:49 -07:00
Brian Warner
7394607141 move encode/upload/download/checker.py into a new immutable/ directory. No behavior changes expected. 2008-07-16 13:14:39 -07:00
Brian Warner
5289064dcf move FileTooLargeError out to a common location 2008-06-03 00:01:15 -07:00
Brian Warner
8c37b8e3af Don't allow uploads of large files (about 12GiB or larger), since they're doomed to be corrupted. Closes #439 2008-06-02 16:57:01 -07:00
Brian Warner
1b4b4cbd4a mutable WIP: rename NotEnoughPeersError to NotEnoughSharesError 2008-04-15 16:08:32 -07:00
Zooko O'Whielacronx
fc3bd0c987 use added secret to protect convergent encryption
Now upload or encode methods take a required argument named "convergence" which can be either None, indicating no convergent encryption at all, or a string, which is the "added secret" to be mixed in to the content hash key.  If you want traditional convergent encryption behavior, set the added secret to be the empty string.

This patch also renames "content hash key" to "convergent encryption" in a argument names and variable names.  (A different and larger renaming is needed in order to clarify that Tahoe supports immutable files which are not encrypted content-hash-key a.k.a. convergent encryption.)

This patch also changes a few unit tests to use non-convergent encryption, because it doesn't matter for what they are testing and non-convergent encryption is slightly faster.
2008-03-24 09:46:06 -07:00
Brian Warner
96ca6c2033 test_upload.py: hush pyflakes 2008-02-06 20:04:31 -07:00
Brian Warner
f78c80d7aa test_upload.py: add test to exercise CHK hashing variations 2008-02-06 20:03:35 -07:00
Brian Warner
66f33ee504 upload: return an UploadResults instance (with .uri) instead of just a URI 2008-02-05 21:01:38 -07:00
Brian Warner
d98fde952c introducer: remove remaining bits of 'push-to-myself' flags. The uploading/downloading node is no longer special. 2008-02-05 14:16:01 -07:00
Brian Warner
daecca6589 big introducer refactoring: separate publish+subscribe. Addresses #271. 2008-02-05 13:05:13 -07:00