ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2024-12-28 08:48:53 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
5,493 Commits 185 Branches 90 Tags 89 MiB
8aa690b64e
Commit Graph

3138 Commits

Author SHA1 Message Date
kevan
a911e15783 mutable/publish: use unpack_mdmf_checkstring and unpack_sdmf_checkstring instead of unpack_checkstring. fixes #1540 2011-09-24 16:51:37 -07:00
david-sarah
1fa5c729b7 mutable/publish.py: copy the self.writers dict before iterating over it, since we remove elements from it during the iteration. refs #393 2011-09-24 14:12:08 -07:00
david-sarah
f94eb86fc9 mutable/publish.py: simplify by refactoring self.outstanding to self.num_outstanding. refs #393 2011-09-24 13:50:04 -07:00
david-sarah
4af626a798 test_mutable.py: update SkipTest message for test_publish_surprise_mdmf to reference the right ticket number. refs #1540. 2011-09-23 14:16:22 -07:00
Brian Warner
f61bdbffd7 control.py: unbreak speed-test: overwrite() wants a MutableData, not str 
Really, all the upload/modify APIs should take a string or a filehandle, and
internally wrap it as needed. Callers should not need to be aware of
Uploadable() or MutableData() classes.
 2011-09-23 00:37:48 -07:00
david-sarah
5d3d0dc336 test_mutable.py: skip test_publish_surprise_mdmf, which is causing an error. refs #1534, #393 2011-09-20 11:33:19 -07:00
kevan
d93b2965c3 test/test_mutable: write publish surprise test for MDMF, rename existing test_publish_surprise to clarify that it is for SDMF 2011-09-17 17:36:57 -07:00
kevan
19cf58f2aa test/test_mutable: refactor publish surprise test into common test fixture, rewrite test_publish_surprise to use test fixture 2011-09-17 17:35:33 -07:00
kevan
8c7406508a mutable/publish: add errback immediately after write, don't consume errors from other parts of the publisher 2011-09-17 16:47:08 -07:00
david-sarah
87a936564e uri.py: fix two interface violations in verifier URI classes. refs #1474 2011-09-19 20:01:56 -07:00
Brian Warner
8e69b94588 Make platform-detection code tolerate linux-3.0, patch by zooko. 
Otherwise address-autodetection can't find ifconfig. refs #1536
 2011-09-15 13:26:20 -07:00
david-sarah
3f756e9429 test_web.py: fix a bug in _count_leases that was causing us to check only the lease count of one share file, not of all share files as intended. 2011-09-15 11:51:26 -07:00
Zooko O'Whielacronx
2025e42ec2 tests: bump up the timeout in this test that fails on FreeStorm's CentOS in order to see if it is just very slow 2011-09-12 19:42:55 -07:00
david-sarah
c10099f982 interfaces: document that the 'fills-holes-with-zero-bytes' key should be used to detect whether a storage server has that behavior. refs #1528 2011-09-12 17:28:43 -07:00
Zooko O'Whielacronx
32f80625c9 storage: more paranoid handling of bounds and palimpsests in mutable share files 
* storage server ignores requests to extend shares by sending a new_length
* storage server fills exposed holes (created by sending a write vector whose offset begins after the end of the current data) with 0 to avoid "palimpsest" exposure of previous contents
* storage server zeroes out lease info at the old location when moving it to a new location
ref. #1528
 2011-09-12 15:26:55 -07:00
Zooko O'Whielacronx
942c5e5162 storage: test that the storage server ignores requests to extend shares by sending a new_length, and that the storage server fills exposed holes with 0 to avoid "palimpsest" exposure of previous contents 
ref. #1528
 2011-09-12 15:25:54 -07:00
Zooko O'Whielacronx
20e2910c61 immutable: prevent clients from reading past the end of share data, which would allow them to learn the cancellation secret 
Declare explicitly that we prevent this problem in the server's version dict.
fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them)
 2011-09-12 15:24:58 -07:00
Zooko O'Whielacronx
5476f67dc1 storage: remove the storage server's "remote_cancel_lease" function 
We're removing this function because it is currently unused, because it is dangerous, and because the bug described in #1528 leaks the cancellation secret, which allows anyone who knows a file's storage index to abuse this function to delete shares of that file.
fixes #1528 (there are two patches that are each a sufficient fix to #1528 and this is one of them)
 2011-09-12 15:23:31 -07:00
Zooko O'Whielacronx
65de17245d storage: test that the storage server does *not* have a "remote_cancel_lease" function 
We're removing this function because it is currently unused, because it is dangerous, and because the bug described in #1528 leaks the cancellation secret, which allows anyone who knows a file's storage index to abuse this function to delete shares of that file.
ref. #1528
 2011-09-12 15:23:24 -07:00
Zooko O'Whielacronx
cffc987804 immutable: test whether the server allows clients to read past the end of share data, which would allow them to learn the cancellation secret 
Also test whether the server explicitly declares that it prevents this problem.
ref #1528
 2011-09-12 15:12:01 -07:00
Brian Warner
bd709c4833 Retrieve._activate_enough_peers: rewrite Verify logic 2011-09-09 11:11:50 -07:00
Brian Warner
a15ce96846 Retrieve: implement/test stopProducing 2011-09-09 11:11:50 -07:00
Brian Warner
748e419a9b move DownloadStopped from download.common to interfaces 2011-09-09 11:11:50 -07:00
Brian Warner
425152c34e retrieve.py: remove vestigal self._validated_readers 2011-09-09 11:11:50 -07:00
Brian Warner
df07060f93 Retrieve: rewrite flow-control: use a top-level loop() to catch all errors 
This ought to close the potential for dropped errors and hanging downloads.
Verify needs to be examined, I may have broken it, although all tests pass.
 2011-09-09 11:11:50 -07:00
Brian Warner
53bbc1d06a Retrieve: merge _validate_active_prefixes into _add_active_peers 2011-09-09 11:11:50 -07:00
Brian Warner
2edfb1a334 Retrieve: remove the initial prefix-is-still-good check 
This check needs to be done with each fetch from the storage server, to
detect when someone has changed the share (i.e. our servermap goes stale).
Doing it just once at the beginning of retrieve isn't enough: a write might
occur after the first segment but before the second, etc.

_try_to_validate_prefix() was not removed: it will be used by the future
check-with-each-fetch code.

test_mutable.Roundtrip.test_corrupt_all_seqnum_late was disabled, since it
fails until this check is brought back. (the corruption it applies only
touches the prefix, not the block data, so the check-less retrieve actually
tolerates it). Don't forget to re-enable it once the check is brought back.
 2011-09-09 11:11:50 -07:00
Brian Warner
2b4f2b7fa3 MDMFSlotReadProxy: remove the queue 
This is a neat trick to reduce Foolscap overhead, but the need for an
explicit flush() complicates the Retrieve path and makes it prone to
lost-progress bugs.

Also change test_mutable.FakeStorageServer to tolerate multiple reads of the
same share in a row, a limitation exposed by turning off the queue.
 2011-09-09 11:11:50 -07:00
Brian Warner
1597aafea1 rearrange Retrieve: first step, shouldn't change order of execution 2011-09-09 11:11:49 -07:00
david-sarah
6b97e6f407 CLI: test_cli.py -- remove an unnecessary call in test_mkdir_mutable_type. refs #1527 2011-09-06 11:37:30 -07:00
david-sarah
1e59930fda CLI: improve test for 'tahoe mkdir --mutable-type='. refs #1527 2011-09-06 11:30:20 -07:00
david-sarah
23f46b758e CLI: make the --mutable-type option value for 'tahoe put' and 'tahoe mkdir' case-insensitive, and change --help for these commands accordingly. fixes #1527 2011-09-04 19:09:22 -07:00
Kevan Carstensen
64996a913d cli: make --mutable-type imply --mutable in 'tahoe put' 2011-09-03 12:09:20 -07:00
david-sarah
1b7b3bd0d4 SFTP: add a comment about a subtle interaction between OverwriteableFileConsumer and GeneralSFTPFile, and test the case it is commenting on. 2011-09-03 15:23:04 -07:00
Brian Warner
01b5124d0a improve the storage/mutable.py asserts even more 2011-09-01 09:05:43 -07:00
wilcoxjg
d2e5de3b69 storage/mutable.py: special characters in struct.foo arguments indicate standard as opposed to native sizes, we should be using these characters in these asserts 2011-09-01 01:41:44 -07:00
Brian Warner
7191b0b32e test_mutable.Version: consolidate some tests, reduce runtime from 19s to 15s 2011-08-30 22:04:51 -07:00
Kevan Carstensen
32e30c9023 mutable/retrieve: handle the case where self._read_length is 0. 
Note that the downloader will still fetch a segment for a zero-length
read, which is wasteful. Fixing that isn't specifically required to fix
#1512, but it should probably be fixed before 1.9.
 2011-08-30 14:01:41 -07:00
Brian Warner
980eb778c1 test_mutable.Update: only upload the files needed for each test. refs #1500 
This first step shaves 15% off the runtime: from 139s to 119s on my laptop.
It also fixes a couple of places where a Deferred was being dropped, which
would cause two tests to run in parallel and also confuse error reporting.
 2011-08-29 00:27:17 -07:00
Brian Warner
fd676a5846 Let Uploader retain History instead of passing it into upload(). Fixes #1079. 
This consistently records all immutable uploads in the Recent Uploads And
Downloads page, regardless of code path. Previously, certain webapi upload
operations (like PUT /uri/$DIRCAP/newchildname) failed to pass the History
object and were left out.
 2011-08-28 23:32:46 -07:00
Brian Warner
9756146d61 Fix mutable publish/retrieve timing status displays. Fixes #1505. 
publish:
* encrypt and encode times are cumulative, not just current-segment

retrieve:
* same for decrypt and decode times
* update "current status" to include segment number
* set status to Finished/Failed when download is complete
* set progress to 1.0 when complete

More improvements to consider:
* progress is currently 0% or 100%: should calculate how many segments are
  involved (remembering retrieve can be less than the whole file) and set it
  to a fraction
* "fetch" time is fuzzy: what we want is to know how much of the delay is not
  our own fault, but since we do decode/decrypt work while waiting for more
  shares, it's not straightforward
 2011-08-28 16:22:21 -07:00
Brian Warner
d575ccba28 Teach 'tahoe debug catalog-shares about MDMF. Closes #1507. 2011-08-28 01:09:31 -07:00
Brian Warner
97b601f75a debug.py: remove some dead comments 2011-08-28 00:45:56 -07:00
Brian Warner
9f827ffa7b hush pyflakes 2011-08-28 00:42:54 -07:00
Brian Warner
0f1d674d76 MutableFileNode.set_downloader_hints: never depend upon order of dict.values() 
The old code was calculating the "extension parameters" (a list) from the
downloader hints (a dictionary) with hints.values(), which is not stable, and
would result in corrupted filecaps (with the 'k' and 'segsize' hints
occasionally swapped). The new code always uses [k,segsize].
 2011-08-28 00:41:03 -07:00
Brian Warner
4b0078dc49 layout.py: fix MDMF share layout documentation 2011-08-28 00:39:21 -07:00
Brian Warner
a3d6fd4992 teach 'tahoe debug dump-share' about MDMF and offsets. refs #1507 2011-08-28 00:38:34 -07:00
Brian Warner
101ca8dfe4 test_mutable.Version.test_debug: use splitlines() to fix buildslaves 
Any slave running in a directory with spaces in the name was miscounting
shares, causing the test to fail.
 2011-08-27 23:47:28 -07:00
Brian Warner
7d48f902fc test_mutable.Version: exercise 'tahoe debug find-shares' on MDMF. refs #1507 
Also changes NoNetworkGrid to put shares in storage/shares/ .
 2011-08-27 17:55:42 -07:00
Brian Warner
80bdc0f5b7 test_mutable.py: oops, missed a .todo 2011-08-27 17:21:18 -07:00
Brian Warner
e0eaeff7f5 test_mutable: merge davidsarah's patch with my Version refactorings 2011-08-27 16:57:07 -07:00
david-sarah
3c92b832f2 Make the immutable/read-only constraint checking for MDMF URIs identical to that for SSK URIs. refs #393 2011-08-22 18:27:20 -07:00
david-sarah
88989a4ea2 Additional tests for MDMF URIs and for zero-length files. refs #393 2011-08-22 18:15:32 -07:00
david-sarah
ac7b8400d4 Additional tests for zero-length partial reads and updates to mutable versions. refs #393 2011-08-21 18:41:11 -07:00
Brian Warner
03ba8065ce test_mutable.Version: factor out some expensive uploads, save 25% runtime 2011-08-27 16:27:37 -07:00
Brian Warner
370e6f271e SDMF: update filenode with correct k/N after Retrieve. Fixes #1510. 
Without this, we get a regression when modifying a mutable file that was
created with more shares (larger N) than our current tahoe.cfg . The
modification attempt creates new versions of the (0,1,..,newN-1) shares, but
leaves the old versions of the (newN,..,oldN-1) shares alone (and throws a
assertion error in SDMFSlotWriteProxy.finish_publishing in the process).

The mixed versions that result (some shares with e.g. N=10, some with N=20,
such that both versions are recoverable) cause problems for the Publish code,
even before MDMF landed. Might be related to refs #1390 and refs #1042.
 2011-08-27 15:50:31 -07:00
Brian Warner
b8c90d24fc layout.py: annotate assertion to figure out 'tahoe backup' failure 2011-08-27 12:52:53 -07:00
Brian Warner
482a7dd3f1 Add 'tahoe debug dump-cap' support for MDMF, DIR2-CHK, DIR2-MDMF. refs #1507. 
This also adds tests for all those cases, and fixes an omission in uri.py
that broke parsing of DIR2-MDMF-Verifier and DIR2-CHK-Verifier.
 2011-08-27 12:50:48 -07:00
Brian Warner
522868d5e9 MDMF: more writable/writeable consistentifications 2011-08-27 12:06:02 -07:00
Brian Warner
48544a251d MDMF: s/Writable/Writeable/g, for consistency with existing SDMF code 2011-08-27 11:33:57 -07:00
Brian Warner
349093fa2e test_mutable.Update: increase timeout from 120s to 400s, slaves are failing 2011-08-25 16:01:40 -07:00
Zooko O'Whielacronx
5bf5404ba1 tests: fix check_memory test 
fixes #1503
 2011-08-25 13:11:16 -07:00
Brian Warner
9e613ee9b3 test_mutable.Update: remove .timeout overrides, otherwise tests ERROR 2011-08-24 19:24:55 -07:00
Brian Warner
445e9eaf37 blacklist.py: add read() method too, for completeness 2011-08-24 19:19:02 -07:00
david-sarah
3d7a32647c Implementation, tests and docs for blacklists. This version allows listing directories containing a blacklisted child. Inclusion of blacklist.py fixed. fixes #1425 2011-08-24 08:59:28 -07:00
david-sarah
505ebeb353 mutable/layout.py: fix unused import. refs #393 2011-08-16 15:50:43 -07:00
david-sarah
4b62b53cd6 mutable/retrieve.py: cosmetics and remove a stale comment. refs #393 2011-08-16 14:46:12 -07:00
Kevan Carstensen
62c72881bb mutable/filenode.py: don't fetch more segments than necesasry to update the file 2011-08-13 14:00:05 -07:00
Kevan Carstensen
b49986f801 test/test_mutable: test for incorrect div_ceil equations 2011-08-13 11:39:36 -07:00
Kevan Carstensen
3423b7d6f7 mutable/retrieve.py: use floor division to calculate segment boundaries, don't fetch more segments than necessary 2011-08-13 11:38:33 -07:00
Kevan Carstensen
a1a334e1a0 mdmf: clean up boolean expressions, correct typos, remove self._paused, and don't unconditionally initialize block hash trees, asll as suggested by davidsarahs' review comments 2011-08-13 11:37:10 -07:00
Brian Warner
fa6a768bc1 now that tests pass with full-size keys, return test-keys to normal (522bit) 2011-08-11 10:54:18 -07:00
Brian Warner
eec428ba5f fix SHARE_HASH_CHAIN_SIZE computation 2011-08-11 10:53:50 -07:00
david-sarah
dd23f265bf More idiomatic resolution of the conflict between ticket393-MDMF-2 and trunk. refs #393 2011-08-10 13:29:42 -07:00
david-sarah
c55c44e70a Replace the hard-coded 522-bit RSA key size used for tests with a TEST_RSA_KEY_SIZE constant defined in test/common.py (part 2). refs #393 2011-08-10 13:23:10 -07:00
david-sarah
fbe0e42bbf Replace the hard-coded 522-bit RSA key size used for tests with a TEST_RSA_KEY_SIZE constant defined in test/common.py (part 1). refs #393 2011-08-10 13:22:43 -07:00
Zooko O'Whielacronx
95a7551586 merge some minor conflicts in test code from the 393-2 branch and trunk 2011-08-10 10:21:39 -07:00
david-sarah
b60506b0ba test_web.py: fix a test failure dependent on whether simplejson.loads returns a unicode or str object. 2011-08-08 14:39:25 -07:00
Kevan Carstensen
c413a8fae1 immutable/filenode: fix pyflakes warnings 2011-08-06 17:45:14 -07:00
Kevan Carstensen
47eab03c6c test: fix assorted tests broken by MDMF changes 2011-08-06 17:44:59 -07:00
Kevan Carstensen
a7fb55f986 uri: add MDMF and MDMF directory caps, add extension hint support 2011-08-06 17:44:36 -07:00
Kevan Carstensen
52e9512b49 test/test_mutable: tests for MDMF 
These are their own patch because they cut across a lot of the changes
I've made in implementing MDMF in such a way as to make it difficult to
split them up into the other patches.
 2011-08-06 17:44:14 -07:00
Kevan Carstensen
a9cada2e03 webapi changes for MDMF 
- Learn how to create MDMF files and directories through the
      mutable-type argument.
    - Operate with the interface changes associated with MDMF and #993.
    - Learn how to do partial updates of mutable files.
 2011-08-06 17:43:48 -07:00
Kevan Carstensen
bb10d685ed mutable/servermap: Rework the servermap to work with MDMF mutable files 2011-08-06 17:42:59 -07:00
Kevan Carstensen
cde17ec041 dirnode: teach dirnode to make MDMF directories 2011-08-06 17:42:24 -07:00
david-sarah
50c692d1c5 Fix some test failures caused by #393 patch. 2011-08-01 20:28:10 -07:00
Kevan Carstensen
4d09ef3d83 cli: teach CLI how to create MDMF mutable files 
Specifically, 'tahoe mkdir' and 'tahoe put' now take a --mutable-type
argument.
 2011-08-01 19:16:13 -07:00
Kevan Carstensen
a87fc8ffab frontends/sftpd: Resolve incompatibilities between SFTP frontend and MDMF changes 2011-08-01 19:12:07 -07:00
Kevan Carstensen
b1b77d3b89 mutable/layout: Define MDMF share format, write tools for working with MDMF share format 
The changes in layout.py are mostly concerned with the MDMF share
format. In particular, we define read and write proxy objects used by
retrieval, publishing, and other code to write and read the MDMF share
format. We create equivalent proxies for SDMF objects so that these
objects can be suitably general.
 2011-08-01 19:11:20 -07:00
Kevan Carstensen
edf9858fb8 immutable/filenode: implement unified filenode interface 2011-08-01 19:09:05 -07:00
Kevan Carstensen
3e996e0c94 immutable/literal.py: Implement interface changes in literal nodes. 2011-08-01 19:08:14 -07:00
Kevan Carstensen
8ae564d8ad test/common: Alter common test code to work with MDMF. 
This mostly has to do with making the test code implement the new
unified filenode interfaces.
 2011-08-01 18:56:43 -07:00
Kevan Carstensen
f80a7fdf18 mutable: train checker and repairer to work with MDMF mutable files 2011-08-01 18:51:40 -07:00
Kevan Carstensen
893a9028f0 nodemaker: teach nodemaker about MDMF caps 2011-08-01 18:49:26 -07:00
Kevan Carstensen
d3f98498c0 client: teach client how to create and work with MDMF files 2011-08-01 18:48:11 -07:00
Kevan Carstensen
e4f2b27305 mutable/filenode: Modify mutable filenodes for use with MDMF 
In particular:
    - Break MutableFileNode and MutableFileVersion into distinct classes.
    - Implement the interface modifications made for MDMF.
    - Be aware of MDMF caps.
    - Learn how to create and work with MDMF files.
 2011-08-01 18:45:01 -07:00
Kevan Carstensen
4c48e2e1b6 nodemaker: teach nodemaker how to create MDMF mutable files 2011-08-01 18:42:58 -07:00
Kevan Carstensen
126d1ad010 interfaces: change interfaces to work with MDMF 
A lot of this work concerns #993, in that it unifies (to an extent) the
interfaces of mutable and immutable files.
 2011-08-01 18:41:19 -07:00
Kevan Carstensen
1576c35d38 mutable/publish: teach the publisher how to publish MDMF mutable files 
Like the downloader, the publisher needs some substantial changes to handle multiple segment mutable files.
 2011-08-01 18:39:31 -07:00
Kevan Carstensen
ac3b2647dd mutable/retrieve: rework the mutable downloader to handle multiple-segment files 
The downloader needs substantial reworking to handle multiple segment
mutable files, which it needs to handle for MDMF.
 2011-08-01 18:35:24 -07:00
david-sarah
c41bf14029 Cleanups for suppression of UserWarnings. refs #1435 2011-08-17 21:07:49 -07:00
Zooko O'Whielacronx
f627af3b9f suppress warning emitted by newer zope.interface with Nevow 0.10 
refs #1435
 2011-08-17 13:31:34 -07:00
david-sarah
a98a4ec13f _auto_deps.py: change the requirement for zope.interface to <= 3.6.2, >= 3.6.6. fixes #1435 2011-08-14 19:53:47 -07:00
david-sarah
a48aee11b3 allmydata/__init__.py, test_version.py: make version parsing understand '<=', with test. refs #1435 2011-08-14 20:51:53 -07:00
Brian Warner
5aad81f84d replace tabs with spaces in the #1441 'tahoe debug' synopsis 2011-08-11 10:37:04 -07:00
david-sarah
0dc917af08 Correct the information printed by '/usr/bin/tahoe debug --help' on Debian/Ubuntu. fixes #1441 2011-07-24 09:25:30 -07:00
david-sarah
6c2670641e Add test for webopen. fixes #1149 2011-07-24 14:16:59 -07:00
david-sarah
ab9eb12f70 test_client.py: relax a check in test_create_drop_uploader so that it should pass on Python 2.4.x. refs #1429 2011-08-09 22:25:04 -07:00
david-sarah
0dc56daaf9 test/common_util.py: correct fix to mkdir_nonascii. refs #1472 2011-08-09 22:19:06 -07:00
david-sarah
2deba3319c test/common_util.py: fix a typo. refs #1472 2011-08-09 21:42:35 -07:00
david-sarah
0104dbab92 test_client.py, test_drop_upload.py: fix pyflakes warnings. 2011-08-09 20:45:05 -07:00
david-sarah
db22fdc20d Factor out methods dealing with non-ASCII directories and filenames from test_drop_upload.py into common_util.py. refs #1429, #1472 2011-08-09 20:15:58 -07:00
david-sarah
c102056ac1 test_client.py: add a test that the drop-uploader is initialized correctly by client.py. Also give the DropUploader service a name, which is necessary for the test. refs #1429 2011-08-09 20:05:38 -07:00
david-sarah
10ee22f50e drop-upload: rename 'start' method to 'startService', which is what you're supposed to use to start a Service. refs #1429 2011-08-09 20:03:45 -07:00
david-sarah
f157b73367 test_drop_upload.py: add comment explaining why we don't use FilePath.setContent. refs #1429 2011-08-09 19:59:42 -07:00
david-sarah
369e30b1df test_drop_upload.py: fix some grammatical and spelling nits. refs #1429 2011-08-09 15:12:31 -07:00
Zooko O'Whielacronx
612abca271 drop-upload: report the configured local directory being absent differently from it being a file 
refs #1429
 2011-08-09 15:09:30 -07:00
Zooko O'Whielacronx
b7683d9b83 drop-upload: rename the 'upload.uri' parameter to 'upload.dircap', and a couple of cleanups to error messages. refs #1429 
I rerecorded this patch, originally by David-Sarah, to use "darcs replace" instead of editing to do the renames. This uncovered one missed rename in Client.init_drop_uploader. (Which also means that code isn't exercised by the current unit tests.)
refs #1429
 2011-08-09 15:05:08 -07:00
Zooko O'Whielacronx
5633375d26 drop-upload test for non-existent local dir separately from test for non-directory local dir 
A candidate patch for #1429 has a bug when it is using FilePath.is_dir() to detect whether the configured local dir exists and is a directory. FilePath.is_dir() raises exception, instead of returning False, if the thing doesn't exist. This test is to make sure that DropUploader.__init__ raise different exceptions for those two cases.
refs #1429
 2011-08-09 15:01:15 -07:00
Zooko O'Whielacronx
720bc2433b drop-upload: unit tests for the configuration options being named "cap" instead of "uri" 
This is a subset of a patch that David-Sarah attached to #1429. This is just the unit-tests part of that patch, and uses darcs record instead of hunks to change the names.
refs #1429
 2011-08-09 14:59:13 -07:00
david-sarah
c2972e22cb src/allmydata/storage/server.py: use the filesystem of storage/shares/, rather than storage/, to calculate remaining space. fixes #1384 2011-07-18 19:27:52 -07:00
david-sarah
4c592f1505 test_storage.py: test that we are using the filesystem of storage/shares/, rather than storage/, to calculate remaining space, and that the HTML status output reflects the values returned by fileutil.get_disk_stats. This version works with older versions of the mock library. refs #1384 2011-08-09 12:07:22 -07:00
david-sarah
70f03bd615 Work around ref #1472 by having test_drop_upload delete the non-ASCII directories it creates. 2011-08-08 18:23:34 -07:00
david-sarah
bbed522033 Remove all trailing whitespace from .py files. 2011-08-08 17:11:17 -07:00
david-sarah
08af9cea50 test_drop_upload.py: fix unused imports. refs #1429 2011-08-08 16:54:22 -07:00
david-sarah
32a7717205 Drop-upload frontend, rerecorded for 1.9 beta (and correcting a minor mistake). Includes some fixes for Windows but not the Windows inotify implementation. fixes #1429 2011-08-08 16:40:49 -07:00
david-sarah
05684b9251 node.py: ensure that client and introducer nodes record their port number and use that port on the next restart, fixing a regression caused by #1385. fixes #1469. 2011-08-06 15:19:34 -07:00
david-sarah
b9121e045f test_runner.py: fix a race condition in the test when NODE_URL_FILE is written before PORTNUM_FILE. refs #1469 2011-08-06 16:18:42 -07:00
david-sarah
743cb898e7 test_runner.py: cleanups of HOTLINE_FILE writing and removal. 2011-08-06 16:16:52 -07:00
david-sarah
e3c600d047 test_runner.py: remove an unused constant. 2011-08-06 15:14:16 -07:00
david-sarah
1967233f49 node.py: fix the error path for a missing config option so that it works for a Unicode base directory. 2011-08-06 15:10:07 -07:00
david-sarah
531758df19 test_runner.py: test that client and introducer nodes record their port number and use that port on the next restart. This tests for a regression caused by ref #1385. 2011-08-06 15:06:35 -07:00
david-sarah
b9eb0235ea test_runner.py: fix a bug in CreateNode.do_create introduced in changeset [5114] when the tahoe.cfg file has been written with CRLF line endings. refs #1385 2011-08-03 17:30:32 -07:00
david-sarah
b6cfbbeb23 test_client.py: repair Basic.test_error_on_old_config_files. refs #1385 2011-08-03 16:50:36 -07:00
david-sarah
1c77c5f5dc test_checker.py: increase timeout for TooParallel.test_immutable again. The ARM buildslave took 38 seconds, so 40 seconds is too close to the edge; make it 80. 2011-08-03 14:40:42 -07:00
david-sarah
521754b506 test_runner.py: fix RunNode.test_introducer to not rely on the mtime of introducer.furl to detect when the node has restarted. Instead we detect when node.url has been written. refs #1385 2011-08-03 11:09:17 -07:00
david-sarah
80300ea7a3 Further improve error message about old config files. refs #1385 2011-08-03 10:45:46 -07:00
david-sarah
f45bfeb3df Slightly improve error message about old config files (avoid unnecessary Unicode escaping). refs #1385 2011-08-03 09:38:48 -07:00
david-sarah
ed3dbe4985 test_checker.py: increase timeout for TooParallel.test_immutable (was consistently failing on ARM buildslave). 2011-08-03 09:32:13 -07:00
david-sarah
2d16a16ee3 Fix the bug that prevents an introducer from starting when introducer.furl already exists. Also remove some dead code that used to read old config files, and rename 'warn_about_old_config_files' to reflect that it's not a warning. refs #1385 2011-08-02 18:32:12 -07:00
david-sarah
e74387f4f1 test_runner.py: modify RunNode.test_introducer to test that starting an introducer works when the introducer.furl file already exists. refs #1385 2011-08-02 18:27:04 -07:00
david-sarah
c7f65ee8ad verifier: correct a bug introduced in changeset [5106] that caused us to only verify the first block of a file. refs #1395 2011-08-02 10:24:37 -07:00
david-sarah
9f8d34e63a test_repairer: add a deterministic test of share data corruption that always flips the bits of the last byte of the share data. refs #1395 2011-08-02 10:58:41 -07:00
Zooko O'Whielacronx
f426e82287 verifier: serialize the fetching of blocks within a share so that we don't use too much RAM 
Shares are still verified in parallel, but within a share, don't request a
block until the previous block has been verified and the memory we used to hold
it has been freed up.

Patch originally due to Brian. This version has a mockery-patchery-style test
which is "low tech" (it implements the patching inline in the test code instead
of using an extension of the mock.patch() function from the mock library) and
which unpatches in case of exception.

fixes #1395
 2011-08-01 23:37:03 -07:00
Brian Warner
3668cb3d06 remove nodeid from WriteBucketProxy classes and customers 
refs #1363
 2011-08-01 15:43:17 -07:00
Brian Warner
550d67f51f remove get_serverid() from ReadBucketProxy and customers, including Checker 
and debug.py dump-share commands
refs #1363
 2011-08-01 15:43:07 -07:00
Zooko O'Whielacronx
e5c4e83f4c reject old-style (pre-Tahoe-LAFS-v1.3) configuration files 
Check for the existence of any of them and if any are found raise exception which will abort the startup of the node.
This is a backwards-incompatible change for anyone who is still using old-style configuration files.
fixes #1385
 2011-08-01 16:24:23 -07:00
Zooko O'Whielacronx
420e3aa293 whitespace-cleanup 2011-07-24 18:55:46 -07:00
Zooko O'Whielacronx
3505a8a31c tests: use fileutil.write() instead of open() to ensure timely close even without CPython-style reference counting 
Some of these already had an explicit close() but I went ahead and replaced them with fileutil.write() as well for the sake of uniformity.
 2011-03-31 07:54:27 -07:00
david-sarah
2da3f69f25 Address Kevan's comment in #776 about Options classes missed when adding 'self.command_name'. refs #776, #1359 2011-08-01 15:13:17 -07:00