Commit Graph

27 Commits

Author SHA1 Message Date
Brian Warner
1192b61dfe upload: fix #758 recursion-loop in peer-selection when servers report errors.
The bug was in the code that handles a third-or-later pass, and was
previously untested.
2009-07-17 00:07:09 -05:00
Brian Warner
bd6ecc9f44 Split out NoSharesError, stop adding attributes to NotEnoughSharesError, change humanize_failure to include the original exception string, update tests, behave better if humanize_failure fails. 2009-06-24 19:17:07 -07:00
Brian Warner
711c09bc5d clean up storage_broker interface: should fix #732 2009-06-21 16:51:19 -07:00
Brian Warner
b1290633b8 more storage_broker refactoring: downloader gets a broker instead of a client,
use Client.get_storage_broker() accessor instead of direct attribute access.
2009-06-01 19:25:11 -07:00
Brian Warner
4177a3616b remove plaintext-hashing code from the helper interface, to close #722
and deny the Helper the ability to mount a partial-information-guessing
attack. This will probably break compatibility between new clients and very
old (pre-1.0) helpers.
2009-06-01 15:49:16 -07:00
Brian Warner
c516361fd2 start to factor server-connection-management into a distinct 'StorageServerFarmBroker' object, separate from the client and the introducer. This is the starting point for #467: static server selection 2009-06-01 14:06:04 -07:00
Brian Warner
1863aee0aa switch to using RemoteException instead of 'wrapped' RemoteReferences. Should fix #653, the rref-EQ problem 2009-05-21 17:46:32 -07:00
Brian Warner
c9803d5217 switch all foolscap imports to use foolscap.api or foolscap.logging 2009-05-21 17:38:23 -07:00
Brian Warner
67571eb033 add more information to NotEnoughSharesError, split out new exceptions for no-servers and no-source-of-ueb-hash 2009-03-03 19:37:15 -07:00
Brian Warner
ef53da2b12 break storage.py into smaller pieces in storage/*.py . No behavioral changes. 2009-02-18 14:46:55 -07:00
Brian Warner
a9a3b509df upload: add a think-of-the-compatibility note to UploadResults 2009-02-09 14:50:04 -07:00
Brian Warner
a5ab6c060d helper #609: uploading client should ignore old helper's UploadResults, which were in a different format 2009-02-09 14:45:43 -07:00
Brian Warner
10268a4f7f upload: move upload history into History object 2009-01-14 16:41:06 -07:00
Brian Warner
cc50e2f4aa upload: use WriteBucketProxy_v2 when uploading a large file (with shares larger than 4GiB). This finally closes #346. I think we can now handle immutable files up to 48EiB. 2009-01-12 20:14:42 -07:00
Zooko O'Whielacronx
25063688b4 immutable repairer
This implements an immutable repairer by marrying a CiphertextDownloader to a CHKUploader.  It extends the IDownloadTarget interface so that the downloader can provide some metadata that the uploader requires.
The processing is incremental -- it uploads the first segments before it finishes downloading the whole file.  This is necessary so that you can repair large files without running out of RAM or using a temporary file on the repairer.
It requires only a verifycap, not a readcap.  That is: it doesn't need or use the decryption key, only the integrity check codes.
There are several tests marked TODO and several instances of XXX in the source code.  I intend to open tickets to document further improvements to functionality and testing, but the current version is probably good enough for Tahoe-1.3.0.
2009-01-12 11:00:22 -07:00
Zooko O'Whielacronx
b496eba072 trivial: minor changes to in-line comments -- mark plaintext-hash-tree as obsolete 2009-01-10 14:56:01 -07:00
Zooko O'Whielacronx
6e3396fb88 immutable: redefine the "sharemap" member of the upload results to be a map from shnum to set of serverids
It used to be a map from shnum to a string saying "placed this share on XYZ server".  The new definition is more in keeping with the "sharemap" object that results from immutable file checking and repair, and it is more useful to the repairer, which is a consumer of immutable upload results.
2009-01-10 11:46:23 -07:00
Zooko O'Whielacronx
d5a6eed407 trivial: fix redefinition of name "log" in imports (pyflakes) 2009-01-06 22:08:29 -07:00
Zooko O'Whielacronx
c85f75bb08 immutable: refactor uploader to do just encoding-and-uploading, not encryption
This makes Uploader take an EncryptedUploadable object instead of an Uploadable object.  I also changed it to return a verify cap instead of a tuple of the bits of data that one finds in a verify cap.
This will facilitate hooking together an Uploader and a Downloader to make a Repairer.
Also move offloaded.py into src/allmydata/immutable/.
2009-01-06 21:48:22 -07:00
Zooko O'Whielacronx
81add135dc trivial: whitespace and docstring tidyups 2009-01-06 21:41:04 -07:00
Brian Warner
3e25efc010 upload: when using a Helper, insist that it provide protocols/helper/v1 . Related to #538. 2008-11-21 20:29:32 -07:00
Brian Warner
0fab511be5 upload: don't use servers which can't support the share size we need. This ought to avoid #439 problems. Some day we'll have a storage server which advertises support for a larger share size. No tests yet. 2008-11-21 20:28:12 -07:00
Brian Warner
bf06492a90 #538: fetch version and attach to the rref. Make IntroducerClient demand v1 support. 2008-11-21 20:07:27 -07:00
Brian Warner
914655c52b interfaces.py: promote immutable.encode.NotEnoughSharesError.. it isn't just for immutable files any more 2008-10-27 13:34:49 -07:00
Brian Warner
288d55825c storage: split WriteBucketProxy and ReadBucketProxy out into immutable/layout.py . No behavioral changes. 2008-10-09 17:08:00 -07:00
Brian Warner
9c505e49c2 stop using 'as' as an identifier: as with 'with', 'as' has become a reserved word in python 2.6 2008-10-02 17:27:49 -07:00
Brian Warner
7394607141 move encode/upload/download/checker.py into a new immutable/ directory. No behavior changes expected. 2008-07-16 13:14:39 -07:00