ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2024-12-28 08:48:53 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
5,493 Commits 185 Branches 90 Tags 89 MiB
8aa690b64e
Commit Graph

3138 Commits

Author SHA1 Message Date
david-sarah
c41bf14029 Cleanups for suppression of UserWarnings. refs #1435 2011-08-17 21:07:49 -07:00
Zooko O'Whielacronx
f627af3b9f suppress warning emitted by newer zope.interface with Nevow 0.10 
refs #1435
 2011-08-17 13:31:34 -07:00
david-sarah
a98a4ec13f _auto_deps.py: change the requirement for zope.interface to <= 3.6.2, >= 3.6.6. fixes #1435 2011-08-14 19:53:47 -07:00
david-sarah
a48aee11b3 allmydata/__init__.py, test_version.py: make version parsing understand '<=', with test. refs #1435 2011-08-14 20:51:53 -07:00
Brian Warner
5aad81f84d replace tabs with spaces in the #1441 'tahoe debug' synopsis 2011-08-11 10:37:04 -07:00
david-sarah
0dc917af08 Correct the information printed by '/usr/bin/tahoe debug --help' on Debian/Ubuntu. fixes #1441 2011-07-24 09:25:30 -07:00
david-sarah
6c2670641e Add test for webopen. fixes #1149 2011-07-24 14:16:59 -07:00
david-sarah
ab9eb12f70 test_client.py: relax a check in test_create_drop_uploader so that it should pass on Python 2.4.x. refs #1429 2011-08-09 22:25:04 -07:00
david-sarah
0dc56daaf9 test/common_util.py: correct fix to mkdir_nonascii. refs #1472 2011-08-09 22:19:06 -07:00
david-sarah
2deba3319c test/common_util.py: fix a typo. refs #1472 2011-08-09 21:42:35 -07:00
david-sarah
0104dbab92 test_client.py, test_drop_upload.py: fix pyflakes warnings. 2011-08-09 20:45:05 -07:00
david-sarah
db22fdc20d Factor out methods dealing with non-ASCII directories and filenames from test_drop_upload.py into common_util.py. refs #1429, #1472 2011-08-09 20:15:58 -07:00
david-sarah
c102056ac1 test_client.py: add a test that the drop-uploader is initialized correctly by client.py. Also give the DropUploader service a name, which is necessary for the test. refs #1429 2011-08-09 20:05:38 -07:00
david-sarah
10ee22f50e drop-upload: rename 'start' method to 'startService', which is what you're supposed to use to start a Service. refs #1429 2011-08-09 20:03:45 -07:00
david-sarah
f157b73367 test_drop_upload.py: add comment explaining why we don't use FilePath.setContent. refs #1429 2011-08-09 19:59:42 -07:00
david-sarah
369e30b1df test_drop_upload.py: fix some grammatical and spelling nits. refs #1429 2011-08-09 15:12:31 -07:00
Zooko O'Whielacronx
612abca271 drop-upload: report the configured local directory being absent differently from it being a file 
refs #1429
 2011-08-09 15:09:30 -07:00
Zooko O'Whielacronx
b7683d9b83 drop-upload: rename the 'upload.uri' parameter to 'upload.dircap', and a couple of cleanups to error messages. refs #1429 
I rerecorded this patch, originally by David-Sarah, to use "darcs replace" instead of editing to do the renames. This uncovered one missed rename in Client.init_drop_uploader. (Which also means that code isn't exercised by the current unit tests.)
refs #1429
 2011-08-09 15:05:08 -07:00
Zooko O'Whielacronx
5633375d26 drop-upload test for non-existent local dir separately from test for non-directory local dir 
A candidate patch for #1429 has a bug when it is using FilePath.is_dir() to detect whether the configured local dir exists and is a directory. FilePath.is_dir() raises exception, instead of returning False, if the thing doesn't exist. This test is to make sure that DropUploader.__init__ raise different exceptions for those two cases.
refs #1429
 2011-08-09 15:01:15 -07:00
Zooko O'Whielacronx
720bc2433b drop-upload: unit tests for the configuration options being named "cap" instead of "uri" 
This is a subset of a patch that David-Sarah attached to #1429. This is just the unit-tests part of that patch, and uses darcs record instead of hunks to change the names.
refs #1429
 2011-08-09 14:59:13 -07:00
david-sarah
c2972e22cb src/allmydata/storage/server.py: use the filesystem of storage/shares/, rather than storage/, to calculate remaining space. fixes #1384 2011-07-18 19:27:52 -07:00
david-sarah
4c592f1505 test_storage.py: test that we are using the filesystem of storage/shares/, rather than storage/, to calculate remaining space, and that the HTML status output reflects the values returned by fileutil.get_disk_stats. This version works with older versions of the mock library. refs #1384 2011-08-09 12:07:22 -07:00
david-sarah
70f03bd615 Work around ref #1472 by having test_drop_upload delete the non-ASCII directories it creates. 2011-08-08 18:23:34 -07:00
david-sarah
bbed522033 Remove all trailing whitespace from .py files. 2011-08-08 17:11:17 -07:00
david-sarah
08af9cea50 test_drop_upload.py: fix unused imports. refs #1429 2011-08-08 16:54:22 -07:00
david-sarah
32a7717205 Drop-upload frontend, rerecorded for 1.9 beta (and correcting a minor mistake). Includes some fixes for Windows but not the Windows inotify implementation. fixes #1429 2011-08-08 16:40:49 -07:00
david-sarah
05684b9251 node.py: ensure that client and introducer nodes record their port number and use that port on the next restart, fixing a regression caused by #1385. fixes #1469. 2011-08-06 15:19:34 -07:00
david-sarah
b9121e045f test_runner.py: fix a race condition in the test when NODE_URL_FILE is written before PORTNUM_FILE. refs #1469 2011-08-06 16:18:42 -07:00
david-sarah
743cb898e7 test_runner.py: cleanups of HOTLINE_FILE writing and removal. 2011-08-06 16:16:52 -07:00
david-sarah
e3c600d047 test_runner.py: remove an unused constant. 2011-08-06 15:14:16 -07:00
david-sarah
1967233f49 node.py: fix the error path for a missing config option so that it works for a Unicode base directory. 2011-08-06 15:10:07 -07:00
david-sarah
531758df19 test_runner.py: test that client and introducer nodes record their port number and use that port on the next restart. This tests for a regression caused by ref #1385. 2011-08-06 15:06:35 -07:00
david-sarah
b9eb0235ea test_runner.py: fix a bug in CreateNode.do_create introduced in changeset [5114] when the tahoe.cfg file has been written with CRLF line endings. refs #1385 2011-08-03 17:30:32 -07:00
david-sarah
b6cfbbeb23 test_client.py: repair Basic.test_error_on_old_config_files. refs #1385 2011-08-03 16:50:36 -07:00
david-sarah
1c77c5f5dc test_checker.py: increase timeout for TooParallel.test_immutable again. The ARM buildslave took 38 seconds, so 40 seconds is too close to the edge; make it 80. 2011-08-03 14:40:42 -07:00
david-sarah
521754b506 test_runner.py: fix RunNode.test_introducer to not rely on the mtime of introducer.furl to detect when the node has restarted. Instead we detect when node.url has been written. refs #1385 2011-08-03 11:09:17 -07:00
david-sarah
80300ea7a3 Further improve error message about old config files. refs #1385 2011-08-03 10:45:46 -07:00
david-sarah
f45bfeb3df Slightly improve error message about old config files (avoid unnecessary Unicode escaping). refs #1385 2011-08-03 09:38:48 -07:00
david-sarah
ed3dbe4985 test_checker.py: increase timeout for TooParallel.test_immutable (was consistently failing on ARM buildslave). 2011-08-03 09:32:13 -07:00
david-sarah
2d16a16ee3 Fix the bug that prevents an introducer from starting when introducer.furl already exists. Also remove some dead code that used to read old config files, and rename 'warn_about_old_config_files' to reflect that it's not a warning. refs #1385 2011-08-02 18:32:12 -07:00
david-sarah
e74387f4f1 test_runner.py: modify RunNode.test_introducer to test that starting an introducer works when the introducer.furl file already exists. refs #1385 2011-08-02 18:27:04 -07:00
david-sarah
c7f65ee8ad verifier: correct a bug introduced in changeset [5106] that caused us to only verify the first block of a file. refs #1395 2011-08-02 10:24:37 -07:00
david-sarah
9f8d34e63a test_repairer: add a deterministic test of share data corruption that always flips the bits of the last byte of the share data. refs #1395 2011-08-02 10:58:41 -07:00
Zooko O'Whielacronx
f426e82287 verifier: serialize the fetching of blocks within a share so that we don't use too much RAM 
Shares are still verified in parallel, but within a share, don't request a
block until the previous block has been verified and the memory we used to hold
it has been freed up.

Patch originally due to Brian. This version has a mockery-patchery-style test
which is "low tech" (it implements the patching inline in the test code instead
of using an extension of the mock.patch() function from the mock library) and
which unpatches in case of exception.

fixes #1395
 2011-08-01 23:37:03 -07:00
Brian Warner
3668cb3d06 remove nodeid from WriteBucketProxy classes and customers 
refs #1363
 2011-08-01 15:43:17 -07:00
Brian Warner
550d67f51f remove get_serverid() from ReadBucketProxy and customers, including Checker 
and debug.py dump-share commands
refs #1363
 2011-08-01 15:43:07 -07:00
Zooko O'Whielacronx
e5c4e83f4c reject old-style (pre-Tahoe-LAFS-v1.3) configuration files 
Check for the existence of any of them and if any are found raise exception which will abort the startup of the node.
This is a backwards-incompatible change for anyone who is still using old-style configuration files.
fixes #1385
 2011-08-01 16:24:23 -07:00
Zooko O'Whielacronx
420e3aa293 whitespace-cleanup 2011-07-24 18:55:46 -07:00
Zooko O'Whielacronx
3505a8a31c tests: use fileutil.write() instead of open() to ensure timely close even without CPython-style reference counting 
Some of these already had an explicit close() but I went ahead and replaced them with fileutil.write() as well for the sake of uniformity.
 2011-03-31 07:54:27 -07:00
david-sarah
2da3f69f25 Address Kevan's comment in #776 about Options classes missed when adding 'self.command_name'. refs #776, #1359 2011-08-01 15:13:17 -07:00
david-sarah
06a5d0c1a3 cleanup: implement rm as a synonym for unlink rather than vice-versa. refs #776 2011-08-01 15:01:08 -07:00
david-sarah
07ecac1d83 WUI: change the label of the button to unlink a file from 'del' to 'unlink'. Also change some internal names to 'unlink', and allow 't=unlink' as a synonym for 't=delete' in the web-API interface. Incidentally, improve a test to check for the rename button as well as the unlink button. fixes #1104 2011-07-12 17:12:18 -07:00
david-sarah
23b3a2f45a src/allmydata/web/filenode.py: delete a stale comment that was made incorrect by changeset [3133]. 2011-08-01 13:30:09 -07:00
Brian Warner
294dc5d0c7 fix typo introduced during rebasing of 'remove get_serverid from 
DownloadStatus.add_dyhb_request and customers' patch, to fix test failure.
 2011-08-01 13:03:41 -07:00
Zooko O'Whielacronx
6b2e798595 remove get_serverid from DownloadStatus.add_dyhb_request and customers 
This patch is a rebase of a patch originally written by Brian. I didn't change any of the intent of Brian's patch, just ported it to current trunk.
refs #1363
 2011-08-01 11:54:01 -07:00
Zooko O'Whielacronx
dc66875479 remove get_serverid from DownloadStatus.add_block_request and customers 
This is a rebase of a patch Brian originally wrote. I haven't changed the intent of that patch, just ported it to trunk.
refs #1363
 2011-08-01 11:53:44 -07:00
Brian Warner
feca907499 apply zooko's advice: storage_client get_known_servers() returns a frozenset, caller sorts 
refs #1363
 2011-08-01 10:44:52 -07:00
Brian Warner
0605c77f08 test_immutable.Test: rewrite to use NoNetworkGrid, now takes 2.7s not 97s 
remove now-unused ShareManglingMixin
refs #1363
 2011-08-01 10:44:44 -07:00
Brian Warner
b07af5e1a2 DownloadStatus.add_known_share wants to be used by Finder, web.status 
refs #1363
 2011-08-01 10:44:36 -07:00
Brian Warner
0f11d35f85 replace IServer.name() with get_name(), and get_longname() 
This patch was originally written by Brian, but was re-recorded by Zooko to use
darcs replace instead of hunks for any file in which it would result in fewer
total hunks.
refs #1363
 2011-08-01 10:44:28 -07:00
Zooko O'Whielacronx
880758340f upload.py: apply David-Sarah's advice rename (un)contacted(2) trackers to first_pass/second_pass/next_pass 
This patch was written by Brian but was re-recorded by Zooko (with David-Sarah looking on) to use darcs replace instead of editing to rename the three variables to their new names.
refs #1363
 2011-08-01 10:41:43 -07:00
Brian Warner
c9def76977 Coalesce multiple Share.loop() calls, make downloads faster. Closes #1268. 2011-08-01 08:18:34 -07:00
david-sarah
095efbf7a5 src/allmydata/_auto_deps.py: 'i686' is another way of spelling x86. 2011-07-31 20:40:35 -07:00
david-sarah
52963f4a76 tahoe_rm.py: better error message when there is no path. refs #1292 2011-01-22 00:42:12 -07:00
david-sarah
00fefeba49 test_cli.py: Test for error message when 'tahoe rm' is invoked without a path. refs #1292 2011-01-04 04:51:08 -07:00
david-sarah
a85a462a7a src/allmydata/__init__.py: suppress a spurious warning from 'bin/tahoe --version[-and-path]' about twisted-web and twisted-core packages. 2011-07-31 17:52:09 -07:00
david-sarah
16fd14a78a test_cli.py: use to_str on fields loaded using simplejson.loads in new tests. refs #1304 2011-07-29 20:25:21 -07:00
Kevan Carstensen
448278e807 cli: make 'tahoe cp' overwrite mutable files in-place 2011-07-29 13:20:39 -07:00
david-sarah
a2699ea6f6 SFTP: write an error message to standard error for unrecognized shell commands. Change the existing message for shell sessions to be written to standard error, and refactor some duplicated code. Also change the lines of the error messages to end in CRLF, and take into account Kevan's review comments. fixes #1442, #1446 2011-07-29 16:31:02 -07:00
david-sarah
b21a4f6b3f src/allmydata/scripts/cli.py: fix pyflakes warning. 2011-07-27 19:14:02 -07:00
david-sarah
b978378392 Fix the help synopses of CLI commands to include [options] in the right place. fixes #1359, fixes #636 2011-07-24 15:54:40 -07:00
david-sarah
f9d218c673 encodingutil: argv and output encodings are always the same on all platforms. Lose the unnecessary generality of them being different. fixes #1120 2011-06-29 11:53:56 -07:00
david-sarah
2ebaa916df Update the dependency on zope.interface to fix an incompatiblity between Nevow and zope.interface 3.6.4. fixes #1435 2011-07-21 16:49:41 -07:00
david-sarah
0f83b76190 frontends/ftpd.py: remove the check for IWriteFile.close since we're now guaranteed to be using Twisted >= 10.1 which has it. 2011-07-21 17:03:20 -07:00
david-sarah
8b40826774 Update the dependency on Twisted to >= 10.1. This allows us to simplify some documentation: it's no longer necessary to install pywin32 on Windows, or apply a patch to Twisted in order to use the FTP frontend. fixes #1274, #1438. refs #1429 2011-07-21 16:36:58 -07:00
Zooko O'Whielacronx
89c11d63f1 setup: remove the dependency on foolscap's "secure_connections" extra, add a dependency on pyOpenSSL 
fixes #1383
 2011-07-17 04:42:26 -07:00
david-sarah
5fcd3f8803 test_sftp.py cleanup: remove a redundant definition of failUnlessReallyEqual. 2011-07-16 11:18:13 -07:00
Zooko O'Whielacronx
4f8e3e5ae8 contrib: remove the contributed fuse modules and the entire contrib/ directory, which is now empty 
Also remove a couple of vestigial references to figleaf, which is long gone.
fixes #1409 (remove contrib/fuse)
 2011-07-12 08:32:29 -07:00
Brian Warner
0f79973401 add Protovis.js-based download-status timeline visualization 
provide status overlap info on the webapi t=json output, add decode/decrypt
rate tooltips, add zoomin/zoomout buttons
 2011-06-29 15:26:06 -07:00
Brian Warner
d8358f2863 add more download-status data, fix tests 2011-06-29 15:25:55 -07:00
Brian Warner
fc5c2208fb prepare for viz: improve DownloadStatus events 
consolidate IDownloadStatusHandlingConsumer stuff into DownloadNode
 2011-06-29 15:25:42 -07:00
Brian Warner
fb4dd43310 Rename test_package_initialization.py to (much shorter) test_import.py . 
The former name was making my 'ls' listings hard to read, by forcing them
down to just two columns.
 2011-06-11 12:02:34 -07:00
Zooko O'Whielacronx
be1ebde6e5 tests: fix tests to accomodate [20110611153758-92b7f-0ba5e4726fb6318dac28fb762a6512a003f4c430] 
Apparently none of the two authors (stercor, terrell), three reviewers (warner, davidsarah, terrell), or one committer (me) actually ran the tests. This is presumably due to #20.
fixes #1412
 2011-06-11 09:37:41 -07:00
Zooko O'Whielacronx
06bf0360d6 wui: right-align the size column in the WUI 
Thanks to Ted "stercor" Rolle Jr. and Terrell Russell.
fixes #1412
 2011-06-11 08:37:58 -07:00
Zooko O'Whielacronx
155d048d17 docs: three minor fixes 
CREDITS for arc for stats tweak
fix link to .zip file in quickstart.rst (thanks to ChosenOne for noticing)
English usage tweak
 2011-06-10 05:16:56 -07:00
wilcoxjg
67ad0175cd server.py: get_latencies now reports percentiles _only_ if there are sufficient observations for the interpretation of the percentile to be unambiguous. 
interfaces.py:  modified the return type of RIStatsProvider.get_stats to allow for None as a return value
NEWS.rst, stats.py: documentation of change to get_latencies
stats.rst: now documents percentile modification in get_latencies
test_storage.py:  test_latencies now expects None in output categories that contain too few samples for the associated percentile to be unambiguously reported.
fixes #1392
 2011-05-27 05:01:35 -07:00
david-sarah
a817163cc6 docs: convert NEWS to NEWS.rst and change all references to it. 2011-05-16 18:02:55 -07:00
david-sarah
49fd2e6e56 scripts/common.py: don't assume that the default alias is always 'tahoe' (it is, but the API of get_alias doesn't say so). refs #1342 2011-01-30 10:49:23 -07:00
Zooko O'Whielacronx
b30a269ec2 setup: don't catch all Exception from check_requirement(), but only PackagingError and ImportError 
I noticed this because I had accidentally inserted a bug which caused AssertionError to be raised from check_requirement().
 2011-01-28 08:20:06 -07:00
Zooko O'Whielacronx
299e8ad579 replace remaining .html docs with .rst docs 
Remove install.html (long since deprecated).
Also replace some obsolete references to install.html with references to quickstart.rst.
Fix some broken internal references within docs/historical/historical_known_issues.txt.
Thanks to Ravi Pinjala and Patrick McDonald.
refs #1227
 2011-05-10 12:16:50 -07:00
david-sarah
fffb3ae75b Fix a test failure in test_package_initialization on Python 2.4.x due to exceptions being stringified differently than in later versions of Python. refs #1389 2011-04-11 12:07:38 -07:00
david-sarah
a09d7300e0 tests: add test for including the ImportError message and traceback entry in the summary of errors from importing dependencies. refs #1389 2011-04-10 08:58:44 -07:00
david-sarah
449b7ad8c7 allmydata/__init__.py: preserve the message and last traceback entry (file, line number, function, and source line) of ImportErrors in the package versions string. fixes #1389 2011-04-10 08:57:05 -07:00
Zooko O'Whielacronx
4911510e0d remove unused variable detected by pyflakes 2011-04-07 10:22:31 -07:00
david-sarah
f3246a9ca0 allmydata/__init__.py: Nicer reporting of unparseable version numbers in dependencies. fixes #1388 2011-04-01 13:27:50 -07:00
Brian Warner
70f9f89c66 control.py: remove all uses of s.get_serverid() 2011-02-26 19:12:03 -07:00
Brian Warner
73a54f83d1 web: remove some uses of s.get_serverid(), not all 2011-02-26 19:11:59 -07:00
Brian Warner
3834192455 immutable/downloader/fetcher.py: remove all get_serverid() calls 2011-02-26 19:11:56 -07:00
Brian Warner
a4068dd1e0 immutable/downloader/fetcher.py: fix diversity bug in server-response handling 
When blocks terminate (either COMPLETE or CORRUPT/DEAD/BADSEGNUM), the
_shares_from_server dict was being popped incorrectly (using shnum as the
index instead of serverid). I'm still thinking through the consequences of
this bug. It was probably benign and really hard to detect. I think it would
cause us to incorrectly believe that we're pulling too many shares from a
server, and thus prefer a different server rather than asking for a second
share from the first server. The diversity code is intended to spread out the
number of shares simultaneously being requested from each server, but with
this bug, it might be spreading out the total number of shares requested at
all, not just simultaneously. (note that SegmentFetcher is scoped to a single
segment, so the effect doesn't last very long).
 2011-02-26 19:11:53 -07:00
Brian Warner
9ae026d9f4 immutable/downloader/share.py: reduce get_serverid(), one left, update ext deps 
test_download.py: create+check MyShare instances better, make sure they share
Server objects, now that finder.py cares
 2011-02-26 19:11:50 -07:00
Brian Warner
baf1506b9b immutable/downloader/finder.py: reduce use of get_serverid(), one left 2011-02-26 19:11:46 -07:00
Brian Warner
d9d55ad006 immutable/offloaded.py: reduce use of get_serverid() a bit more 2011-02-26 19:11:42 -07:00
Brian Warner
2f63d9b522 immutable/upload.py: reduce use of get_serverid() 2011-02-26 19:11:38 -07:00
Brian Warner
bdc5cfbdad immutable/checker.py: remove some uses of s.get_serverid(), not all 2011-02-26 19:11:34 -07:00
Brian Warner
9672b34880 add remaining get_* methods to storage_client.Server, NoNetworkServer, and 
MockIServer stubs
 2011-02-26 19:11:32 -07:00
Brian Warner
36fabb4f4a upload.py: rearrange _make_trackers a bit, no behavior changes 2011-02-26 19:11:28 -07:00
Brian Warner
ee14f2ad67 happinessutil.py: finally rename merge_peers to merge_servers 2011-02-26 19:11:24 -07:00
Brian Warner
13d284b087 test_upload.py: factor out FakeServerTracker 2011-02-26 19:11:20 -07:00
Brian Warner
9fc55bad25 test_upload.py: server-vs-tracker cleanup 2011-02-26 19:11:15 -07:00
Brian Warner
459b9d9e7c happinessutil.py: server-vs-tracker cleanup 2011-02-26 19:11:11 -07:00
Brian Warner
43488b025c upload.py: more tracker-vs-server cleanup 2011-02-26 19:11:07 -07:00
Brian Warner
0cf9e3b150 upload.py: fix var names to avoid confusion between 'trackers' and 'servers' 2011-02-26 19:11:03 -07:00
Brian Warner
ebfcb649f9 refactor: s/peer/server/ in immutable/upload, happinessutil.py, test_upload 
No behavioral changes, just updating variable/method names and log messages.
The effects outside these three files should be minimal: some exception
messages changed (to say "server" instead of "peer"), and some internal class
names were changed. A few things still use "peer" to minimize external
changes, like UploadResults.timings["peer_selection"] and
happinessutil.merge_peers, which can be changed later.
 2011-02-26 19:11:00 -07:00
Brian Warner
9b9ea3f739 storage_client.py: clean up test_add_server/test_add_descriptor, remove .test_servers 2011-02-26 19:10:56 -07:00
Brian Warner
cba3de04a4 test_client.py, upload.py:: remove KiB/MiB/etc constants, and other dead code 2011-02-26 19:10:51 -07:00
Zooko O'Whielacronx
578a117c5b test: increase timeout on a network test because Francois's ARM machine hit that timeout 
I'm skeptical that the test was proceeding correctly but ran out of time. It seems more likely that it had gotten hung. But if we raise the timeout to an even more extravagant number then we can be even more certain that the test was never going to finish.
 2011-03-17 09:59:09 -07:00
Brian Warner
44466fbb1b web/filenode.py: avoid calling req.finish() on closed HTTP connections. Closes #1366 2011-02-20 22:15:44 -08:00
david-sarah
787d12165a Add unit tests for cross_check_pkg_resources_versus_import, and a regression test for ref #1355. This requires a little refactoring to make it testable. 2011-02-20 17:58:17 -08:00
david-sarah
71c301ca34 allmydata/__init__.py: .name was used in place of the correct .__name__ when printing an exception. Also, robustify string formatting by using %r instead of %s in some places. fixes #1355. 2011-02-20 18:01:25 -08:00
Brian Warner
ffd296fc5a Refactor StorageFarmBroker handling of servers 
Pass around IServer instance instead of (peerid, rref) tuple. Replace
"descriptor" with "server". Other replacements:

 get_all_servers -> get_connected_servers/get_known_servers
 get_servers_for_index -> get_servers_for_psi (now returns IServers)

This change still needs to be pushed further down: lots of code is now
getting the IServer and then distributing (peerid, rref) internally.
Instead, it ought to distribute the IServer internally and delay
extracting a serverid or rref until the last moment.

no_network.py was updated to retain parallelism.
 2011-02-20 17:58:04 -08:00
david-sarah
ed67df866c SFTP: report unknown sizes as "0" instead of "?", to satisfy some clients. fixes #1337 2011-01-27 22:22:49 -08:00
david-sarah
d318122cd2 Change tests for check_requirement to ensure that it accepts None in the comment field of a (version, location, comment) triple, and add a test that it raises ImportError when a module couldn't be imported. Also fix typo in a comment. refs #1339 2011-01-28 11:24:52 -08:00
Zooko O'Whielacronx
6f84f1b869 setup: adjust tests to use the new interface of check_requirement which has a 3-tuples instead of a 2-tuple 
fixes #1339
 2011-01-28 06:15:46 -08:00
david-sarah
c71a19b9ed src/allmydata/__init__.py: fix #1339, give an indication in the --version[-and-path] output of when the imported setuptools is distribute, and use a separate element in _vers_and_locs_list tuples for information other than the package name and location. This also changes slightly how the sqlite version is reported. 2011-01-27 21:41:50 -08:00
david-sarah
74b6ea36c4 CLI: improve help synopsis for 'tahoe create-key-generator'. 2011-01-22 17:50:04 -08:00
david-sarah
cb02adb110 Don't put debugging information in 'My versions' section of the Welcome page. Also remove the extra blank line between version and command output when --version[-and-path] is used. refs #1306 2011-01-21 20:02:20 -08:00
david-sarah
f84f2aa987 tests: use verlib to compare versions, and get the versions of Twisted and Nevow from get_package_versions() instead of pkg_resources. refs #1287 2011-01-21 17:42:56 -08:00
david-sarah
39eede4217 src/allmydata/web/common.py: ensure that filenames in 'humanized failures' are quoted. 2011-01-21 17:40:06 -08:00
david-sarah
0c51ce3d89 CLI: improve help synopses for some commands. 2011-01-21 17:27:00 -08:00
david-sarah
c8ccb297fa src/allmydata/_auto_deps.py: setuptools is still an install requirement even in frozen builds, because .tac files import pkg_resources. refs #585 2011-01-21 17:24:42 -08:00
david-sarah
8587c63bdc Make bb-freeze (and probably other static packaging tools) work. This updates various places where we assumed that the tahoe process was executed via the Python interpreter. It also allows tests to recursively invoke the same tahoe.exe, rather than bin/tahoe. refs #585 2011-01-21 00:04:29 -08:00
david-sarah
28bd80812f Change --version and --version-and-path to not exit immediately, if a command is given. 2011-01-20 23:59:13 -08:00
david-sarah
f3b4f4181c src/allmydata/_auto_deps.py: 'pysqlite.dbapi2' module should have been 'pysqlite2.dbapi2'. refs #1306 2011-01-20 23:32:34 -08:00
david-sarah
b6c2c9591d src/allmydata/util/iputil.py: correct an error in the address-matching regexps introduced by the previous patch to iputil. refs #1274 2011-01-20 23:21:36 -08:00
david-sarah
b1f15a630a Add src/allmydata/util/verlib.py, which is a copy of verlib from https://bitbucket.org/tarek/distutilsversion/src/17df9a7d96ef . It is used to normalize and compare versions of dependencies. refs #1258 2011-01-20 22:04:49 -08:00
david-sarah
29336a0916 Refactor _auto_deps.py and __init__.py, adding more robust checking of dependency versions, and not trusting pkg_resources to get the versions right. refs #1258, #1287 2011-01-20 21:36:10 -08:00
david-sarah
d5138b3237 src/allmydata/util/iputil.py: loosen regexps and ensure that 'LANG=en_US.UTF-8' is set in the environment, to minimize problems with localized output of IP-address-finding tools. refs #1274 2011-01-20 00:48:27 -08:00
david-sarah
6dd8b6f471 Eliminate dependencies on pywin32, even via Twisted. refs #1274 2011-01-19 20:32:38 -08:00
david-sarah
fd6cdc48ae src/allmydata/test/test_runner.py: add test_import_from_repl, which checks that we are running the right code in a bin/tahoe subprocess. refs #1258 2011-01-19 00:21:45 -08:00
david-sarah
a9fc4668c0 docs/frontends/CLI.rst, src/allmydata/test/trialtest.py: add trailing newlines. refs #1296 2011-01-19 00:19:55 -08:00
david-sarah
d21f4071c3 Eliminate direct dependencies of Tahoe-LAFS on pywin32 (rebased to trunk). refs #1274 2011-01-18 23:59:11 -08:00
david-sarah
3798d9946e Add support to bin/tahoe for invoking a runner command prefixed with @, with the Tahoe libraries on the PYTHONPATH. This is documented in 'tahoe debug --help'. 2011-01-18 21:11:37 -08:00
david-sarah
74b1eec1d6 trivial: add comment in scripts/debug.py about trial option parsing. refs #1296 2011-01-18 22:08:08 -08:00
david-sarah
8d2983c493 Update foolscap requirement to >= 0.6.1. fixes #1329 2011-01-18 22:06:39 -08:00
david-sarah
93c1909847 bin/tahoe-script.template, src/windows/fixups.py: simplify the method of stripping initial arguments in sys.argv on Windows. This helps with bb-freeze and running tahoe via 'coverage'. Also includes some wording changes and minor refactoring of bin/tahoe-script.template. refs #585, #1303 2011-01-18 20:53:24 -08:00
david-sarah
8f0af33ba6 src/allmydata/test/test_cli.py: add test for 'tahoe debug trial' options help. refs #1296 2011-01-18 18:42:24 -08:00
david-sarah
1819c25c88 Add src/allmydata/test/trialtest.py needed by tests for 'tahoe debug trial'. refs #1296 2011-01-18 18:02:39 -08:00
david-sarah
0d6df9c9fc Tests for 'tahoe debug trial' (rebased and fixed to work with Twisted 10.2). refs #1296 2011-01-18 17:38:59 -08:00
david-sarah
420aadd95e Make 'mock' a run-time rather than setup-time dependency. This is necessary in order for 'tahoe debug trial' to work. refs #1296 2011-01-18 12:51:14 -08:00
david-sarah
7a887871b0 src/allmydata/scripts/debug.py: add 'tahoe debug trial' command (rebased for trunk). refs #1296 2011-01-18 12:46:59 -08:00
david-sarah
fc98aded8a src/allmydata/webish.py: clean-ups and correction to a comment. Also change an open and write to use fileutil.write. See ref #1286 comment 13. 2011-01-17 15:31:52 -08:00
Brian Warner
980fb0b2bb 'tahoe debug catalog-shares': sort SIs and shnums 
Without this, SIs or shnums could be emitted in random order, depending upon
what the filesystem happens to return.
 2011-01-17 01:59:32 -08:00
david-sarah
5a3acc05b7 CLI: tests for ref #1305 (v2, remove spurious extra arg to create-alias in test) 2011-01-13 20:03:27 -08:00
david-sarah
f10a2c0902 CLI: make 'tahoe create-alias' and 'tahoe add-alias' accept a trailing colon on the new alias name (v2, minor change not to rely on implicit Unicode conversion). Includes doc changes and news; tests in a separate patch. fixes #1305 2011-01-13 19:44:14 -08:00
Brian Warner
b9fb74d2a0 Improve 'tahoe ln' help text. Patch by David-Sarah. Closes #1230. 2011-01-17 00:14:21 -08:00
Brian Warner
09a2241471 Tolerate Twisted-10.2's endpoints, patch by David-Sarah. Closes #1286. 
The service generated by strports.service() changed in 10.2, and the ugly
private-attribute-reading hack we used to glean a kernel-allocated port
number (e.g. when using "tcp:0", especially during unit tests) broke, causing
Tahoe to be completely unusable with Twisted-10.2 . The new ugly
private-attribute-reading hack starts by figuring out what sort of service
was generated, then reads different attributes accordingly.

This also hushes a warning when using schemeless strports strings like "0" or
"3456", by quietly prepending a "tcp:" scheme, since 10.2 complains about
those. It also adds getURL() and getPortnum() accessors to the "webish"
service, rather than having unit tests dig through _url and _portnum and such
to find out what they are.
 2011-01-16 23:47:51 -08:00
david-sarah
8a3cb60bf7 Test changes to take account of ref #1311. 2011-01-16 22:05:40 -08:00
david-sarah
d3f24736ca create_node.py: add comments to default tahoe.cfg to clarify the meaning of each section. fixes #1311 2011-01-16 21:24:19 -08:00
david-sarah
e52d97a917 Undo the temporary hack to check the foolscap version. refs #1246 2011-01-16 21:20:42 -08:00
Brian Warner
45212d8000 Set "reserved_space=1G" in newly-created storage nodes. Closes #1208. 2011-01-16 12:58:22 -08:00
david-sarah
727b25f622 Temporary hack to investigate whether we are getting the right version of foolscap on trunk. refs #1258 2011-01-15 20:49:59 -08:00
david-sarah
e62a69e7ff test_runner: avoid unnecessary use of non-ASCII. 2011-01-01 02:01:01 -08:00
david-sarah
2c099ffa0d Move dependency imports from windows/depends.py (which has gone away) into src/allmydata/windows/tahoesvc.py. Also fix a pyflakes warning, and change the service display name from 'Allmydata Tahoe Node' to 'Tahoe-LAFS node'. refs #1282 2010-12-25 20:21:00 -08:00
david-sarah
3132c9b593 Remove unmaintained Windows GUI app, except for windows/tahoesvc.py which is moved to src/allmydata/windows. refs #1282 2010-12-25 20:02:37 -08:00
david-sarah
195ddbcf7c Remove unmaintained Mac GUI app. fixes #1282 2010-12-25 18:05:08 -08:00
david-sarah
399ee5aea0 Remove src/allmydata/util/find_exe.py which is no longer used. fixes #1150 2010-12-25 18:32:06 -08:00
freestorm77
4027a136f3 status_web_pages_review.darcs.patch 
I make some changes on status web pages

status.xhtml:
- Delete unused webform_css link
- Align tables on the left

tahoe-css:
- Do some minor changes on code synthax
- changes table.status-download-events style to look like other tables

status.py:
- Align table on the left
- Changes table header
- Add heading tags
- Modify google api graph: add image border, calculate height to feet data

signed-off-by: zooko@zooko.com
fixes #1219
 2011-01-01 19:42:14 -08:00
david-sarah
5027753e23 test_storage.py: fix a pyflakes unused import warning. 2010-12-31 14:07:56 -08:00
david-sarah
0da10d4c37 test_storage.py: leave at least 512 MiB free when running test_large_share. refs #1195 2010-12-31 12:32:15 -08:00
Zooko O'Whielacronx
123a1a3f04 storage: use fileutil's version of get_disk_stats() and get_available_space(), use mockery/fakery in tests, enable large share test on platforms with sparse files and if > 4 GiB of disk space is currently available 2010-09-10 08:36:29 -08:00
Zooko O'Whielacronx
d5e71c2940 fileutil: copy in the get_disk_stats() and get_available_space() functions from storage/server.py 2010-09-10 08:35:20 -08:00
david-sarah
c82b48f3d6 Update foolscap version requirement to 0.6.0, to address http://foolscap.lothar.com/trac/ticket/167 2010-12-30 22:00:39 -08:00
david-sarah
2d5f3f694f scripts/backupdb.py: more accurate comment about path field. 2010-12-12 09:03:20 -08:00
david-sarah
bb7c04ff9f scripts/cli.py: fix missing 'put' in usage example for 'tahoe put'. 2010-12-12 09:02:07 -08:00
david-sarah
ee4e0d8106 scripts/cli.py: remove the disclaimer in the help for 'tahoe cp' that it does not handle non-ASCII filenames well. (At least, we intend to handle them.) 2010-11-29 16:21:45 -08:00
david-sarah
9ceb527787 Change another doc reference from .txt to .rst in a text string. 2010-11-28 09:45:07 -08:00
david-sarah
d7bdf476b9 Change a doc reference from .txt to .rst in a text string. 2010-11-28 09:34:44 -08:00
Brian Warner
69b42c6cb7 remove --multiple/-m option from all CLI commands: closes #1262 
I personally used "tahoe start/restart -m ../MY-TESTNET/node*" all the time,
to spin up or update a local testgrid while iterating over new code. However,
with the recent switch from "subprocess.Popen(/bin/twistd)" to "import and
call twistd.run()" in scripts/startstop_node.py (yay fewer processes!),
"start -m" broke, and fixing it requires os.fork, which is unavailable on
windows (boo windows!). And I was probably the only one using -m. So in the
interests of uniformity among platforms and simpler code (yay negative code
days!), we're just removing -m from everything. I will start using a little
shell script or something to simulate the removed functionality.

This patch also cleans up CLI-function calling a bit: get the basedir from
the config dict (instead of sometimes from a separate argument), and always
return a numeric exit code.
 2010-11-27 00:38:09 -08:00
Brian Warner
f36bda2780 Revert previous commit: there's an ugly corner-case on windows that fails tests. 
Specifically, test_runner.CreateNode.test_client failed, because the
os.fork-is-present test decided that --multiple should not be allowed on
windows, even though --multiple works just fine for 'tahoe create-client'.
The only restriction on --multiple is for 'tahoe start' and 'tahoe restart'.

This needs a different approach, probably by cleaning up BasedirMixin. We
should only be withholding --multiple on windows for "start" and
"restart". (we should continue withholding --multiple on all platforms for
"run").

This reverts (git) commit f3adb037ae:
 "startstop_node.py: fix "tahoe start -m" by forking before non-final targets"
 2010-11-26 16:44:11 -08:00
Brian Warner
f3adb037ae startstop_node.py: fix "tahoe start -m" by forking before non-final targets 
* don't advertise -m flag on tahoe start/restart/run unless os.fork is
  available (i.e. windows)
* test_runner.py: add test to exercise "start/stop/restart -m"
 2010-11-26 14:45:38 -08:00
david-sarah
38a0d3f5fa tahoe_mv.py: when checking success of the DELETE operation, look at the status code from DELETE rather than from the previous PUT. fixes #1255 2010-11-09 17:09:16 -08:00
david-sarah
fdf6946669 test_cli.py: test that 'tahoe mv' reports errors from the DELETE operation. refs #1255 2010-11-11 17:46:53 -08:00
Zooko O'Whielacronx
93dc740f01 tests: bump up the timeout on test_dirnode.Dirnode from 240s to 480s since it apparently took longer than 240s just now on François's ARM buildslave 2010-11-15 01:21:19 -08:00
david-sarah
76371ae5a9 test_cli.py: fix a stale comment that incorrectly implied that test_cli runs CLI commands in subprocesses (it actually runs them using deferToThread). 2010-11-09 18:28:19 -08:00
david-sarah
cb777ad14f tahoe_mv.py: use do_http, not urllib.openurl, in order to avoid connecting to the webapi server via a proxy. refs #1253 2010-11-09 16:53:36 -08:00
david-sarah
01a5365051 test_system.py: test for 'tahoe mv' with the http_proxy and HTTP_PROXY variables set. refs #1253 2010-11-09 19:36:49 -08:00
david-sarah
edec9c9869 test_client.py: change overzealous use of failUnlessReallyEqual on integers back to failUnlessEqual. 2010-11-09 15:08:16 -08:00
david-sarah
56f4c1218b test_runner.py: fix test failure in test_the_right_code after applying zooko's change to test-with-fake-pkg.py 2010-11-01 20:59:05 -07:00
Zooko O'Whielacronx
a92912f513 setup: FreeStorm's WinXP-x86-py2.6 buildslave has informed us that there is yet a fourth way to spell "i386" in Python 2010-10-31 22:21:42 -07:00
david-sarah
b8905fc766 allmydata/__init__.py: move the call to require_auto_deps() to the top again, since the [4784] patch turned out not to be the cause of the failure on the 'mm netbsd5' buildslave. 2010-10-31 17:33:16 -07:00
david-sarah
63c010a9ca allmydata/__init__.py: put the _auto_deps.require_auto_deps() call back in exactly the same place it was prior to [4784]. 2010-10-31 10:46:49 -07:00
david-sarah
7e2ed2054c allmydata/__init__.py: suppress a DeprecationWarning for the sha module on importing pycrypto. 2010-10-31 09:06:36 -07:00
david-sarah
6e76653913 allmydata/__init__.py: call require_auto_deps() after importing nevow and twisted, reverting change in [4784]. Also fix a missing 'warnings.filters.pop()'. 2010-10-31 08:38:28 -07:00
david-sarah
2ab0b1aa2c allmydata.__init__.py: temporary hack to debug failure on midnightmagic's buildslave 2010-10-30 22:50:03 -07:00
david-sarah
6eaa7f2356 tahoe backup: perform tilde expansion in exclude-from filename (version 2). fixes #1241 2010-10-30 20:52:31 -07:00
Zooko O'Whielacronx
3b2d0d9609 setup: run require_auto_deps() before attempting to import any deps in __init__.py 
For one thing, this makes missing-dependency failures into DistributionNotFound errors instead of ImportErrors, which might be more useful to the user. For another thing, if someone is using distributions that were installed with --multi-version, then they might be not importable until after require_auto_deps() has been run. (The docs claim that this would be the case, but we don't have an example of this happening at this time.)
 2010-10-30 01:10:35 -07:00
david-sarah
1950d5a719 scripts/runner.py: fix unused import of allmydata. refs #1190 2010-10-29 17:31:49 -07:00
david-sarah
2a8f700026 scripts/runner.py: remove pkg_resources.require() calls. These are at best redundant because we have already called _auto_deps.require_auto_deps() (from allmydata.__init__) at that point, and they are causing failure of the test-from-prefixdir step on some buildslaves. refs #1190 2010-10-29 16:53:28 -07:00
david-sarah
10a5f23df9 test_runner.py: fix error in BinTahoe.test_version_no_noise introduced by last patch. refs #1235 2010-10-29 15:11:23 -07:00
david-sarah
d249589667 test_runner.py: also allow 'from pkg_resources import load_entry_point' as noise. refs #1235. 2010-10-29 13:42:46 -07:00
david-sarah
8c95f88b4f test_runner.py: if the only noise is 'UserWarning: Unbuilt egg for setuptools', skip instead of failing the no_noise tests. This version avoids 'any' to be compatible with Python < 2.5. refs #1235. 2010-10-29 12:18:04 -07:00
david-sarah
4061258c85 make ResponseCache smarter to avoid memory leaks: don't record timestamps, use DataSpans to merge entries, and clear the cache when we see a new seqnum. refs #1045, #1229 2010-10-26 21:33:02 -07:00
david-sarah
25d8103dde windows/fixups.py: limit length of string passed in a single call to WriteConsoleW. fixes #1232. 2010-10-26 19:16:36 -07:00
Brian Warner
c88eb097d9 scripts/tahoe_backup.py: oops, fix missing import, thanks pyflakes 
test_cli.py: hush minor pyflakes complaint
 2010-10-29 02:42:23 -07:00
Brian Warner
d61d41147c mutable/servermap.py: update comment. Closes #1231. 2010-10-29 02:14:24 -07:00
Francois Deppierraz
14ee763c54 tahoe_cp.py: Don't call urllib.quote with an Unicode argument, fix #1224 
tahoe_backup.py: Fix another (potential) occurrence of calling urllib.quote()
with an Unicode parameter
 2010-10-29 01:45:20 -07:00
Brian Warner
c18953c169 fix #1223, crash+inefficiency during repair due to read overrun 
* repairer (really the uploader) reads beyond end of input file (Uploadable)
* new-downloader does not tolerate overreads
* uploader does lots of tiny reads (inefficient)

This fixes the last two. The uploader still does a single overread at the end
of the input file, but now that's ok so we can leave it in place. The
uploader now expects the Uploadable to behave like a normal disk
file (reading beyond EOF will return less data than was asked for), and now
the new-downloadable behaves that way.
 2010-10-29 01:20:36 -07:00
david-sarah
270322ad47 startstop_node.py: pyflakes import fix. refs #1190 2010-10-27 18:48:05 -07:00
Brian Warner
ac3b26ecf2 'tahoe start': use import+call rather than find+spawn 
This removes the need to use a locally-built (dependency) bin/twistd, and
removes a big chunk of behavior differences between unix and windows. It
also happens to resolve the "client node probably started" uncertainty.
Might help with #1190, #602, and #71.
 2010-10-26 23:13:42 -07:00
david-sarah
59e80b2ec9 test_mutable.py: add test for ResponseCache memory leak. refs #1045, #1129 2010-10-24 12:34:09 -07:00
david-sarah
daf2b8314a test_encodingutil.py: test_argv_to_unicode modified the wrong encoding variable. fixes #1214 2010-10-22 20:58:10 -07:00
Zooko O'Whielacronx
0c2397523b doc: add explanation of the motivation for the surprising and awkward API to erasure coding 2010-10-14 23:02:02 -07:00
Zooko O'Whielacronx
98ffbfb31f setup: add --multi-version to the "setup.py develop" command-line 
fixes #530. I earlier tried this twice (see #530 for history) and then twice rolled it back due to some problems that arose. However, I didn't write down what the problems were in enough detail on the ticket that I can tell today whether those problems are still issues, so here goes the third attempt. (I did write down on the ticket that it would not create site.py or .pth files in the target directory with --multi-version mode, but I didn't explain why *that* was a problem.)
 2010-10-05 11:23:50 -07:00
Zooko O'Whielacronx
0c6bb0178b setup: use execfile to access _auto_deps.py in its proper location of src/allmydata/ instead of copying it into place when setup.py is executed 2010-09-05 22:57:14 -07:00
Zooko O'Whielacronx
e6a380241a trivial: M-x whitespace-cleanup 2010-09-03 07:47:12 -07:00
Zooko O'Whielacronx
cb83f2e41c minor: remove unused interface declaration, change allmydata.org to tahoe-lafs.org in email address, fix wording in relnotes.txt 2010-09-30 08:37:08 -07:00
Kevan Carstensen
ec4f87a98c immutable/repairer.py: don't use the default happiness setting when repairing 2010-09-27 13:01:02 -07:00
david-sarah
6c4ba62c5d test_download.py: simplify and possibly fix test_lost_servers; refs #1197 2010-09-11 17:32:51 -07:00
david-sarah
b23a7fcc10 test_runner.py: fix spurious failures of test_path on case-insensitive filesystems, and restore the check that the version is not 'unknown'. 2010-09-11 15:19:13 -07:00
Zooko O'Whielacronx
fd17d63c95 test: make tests stop relying on pyutil version class accepting the string 'unknown' for its version, and make them forward-compatible with the future Python Rational Version Numbering standard 2010-09-10 08:41:35 -07:00
Brian Warner
0475bd8e27 immutable download: have the finder inform its share consumer "no more shares" in a subsequent tick, thus avoiding accidentally telling it "no more shares" now and then telling it "here's another share" in a subsequent tick 
fixes #1191
Patch by Brian. This patch description was actually written by Zooko, but I forged Brian's name on the "author" field so that he would get credit for this patch in revision control history.
 2010-09-09 21:30:38 -07:00
Zooko O'Whielacronx
56a3258ff7 immutable downloader: add a test specifically of whether the finder sometimes announces "no more shares ever" and then announces a newly found share 
(The current code fails this test, ref #1191.)
 2010-09-08 21:16:54 -07:00
Zooko O'Whielacronx
9d421cb022 tests: assign the storage servers to a fixed order which triggers a bug in new downloader every time this test is run (formerly this test would detect the bug in new-downloader only sporadically) 
If you are investigating the bug in new-downloader, one way to investigate might be to change this ordering to a different fixed order (e.g. rotate by 4 instead of rotate by 5) and observe how the behavior of new-downloader differs in that case.
 2010-09-03 21:15:15 -07:00
Zooko O'Whielacronx
63fb687a44 download status: fix bug from me committing the wrong one of François's #1172 patches, fixes #1172 2010-09-02 09:15:41 -07:00
francois
485bfc0fd6 DownloadStatus: show active immutable downloads in Active Operations, Fix #1172 2010-09-02 03:17:28 -07:00
Zooko O'Whielacronx
cdc625e0f8 tests: bump up the allowed number of reads 
Kyle's OpenBSD buildslave used 41 reads when doing this test. The fact that I'm blindly bumping this number up to match the observed behavior probably means this isn't a good criterion to be testing for anyway. But perhaps someone else (Brian) could investigate why that run on Kyle's OpenBSD box took four more reads than we expected, and whether the fact that it took 41 reads to do this operation is indicative of an actual problem.
 2010-09-01 22:38:01 -07:00
Brian Warner
00e9e4e676 SegmentFetcher: use new diversity-seeking share-selection algorithm, and 
deliver all shares at once instead of feeding them out one-at-a-time.

Also fix distribution of real-number-of-segments information: now all
CommonShares (not just the ones used for the first segment) get a
correctly-sized hashtree. Previously, the late ones might not, which would
make them crash and get dropped (causing the download to fail if the initial
set were insufficient, perhaps because one of their servers went away).

Update tests, add some TODO notes, improve variable names and comments.
Improve logging: add logparents, set more appropriate levels.
 2010-08-31 18:37:02 -07:00
Brian Warner
c89a464510 Share: drop received data after each block finishes. Quick fix for the #1170 spans.py complexity bug. 2010-08-31 18:35:58 -07:00
francois
f026927f86 web: refactor rate computation, fixes #1166 2010-08-15 07:19:33 -07:00
Brian Warner
a3cf3331d2 DownloadStatus: put real numbers in progress/status rows, not placeholders. 
Improve tests.
 2010-08-09 23:06:03 -07:00
Brian Warner
dc1afc81bb web download-status: tolerate DYHBs that haven't retired yet. Fixes #1160. 
Also add a better unit test for it.
 2010-08-09 15:51:00 -07:00
Brian Warner
5267fc17fe immutable/filenode.py: put off DownloadStatus creation until first read() call 
This avoids spamming the "recent uploads and downloads" /status page from
FileNode instances that were created for a directory read but which nobody is
ever going to read from. I also cleaned up the way DownloadStatus instances
are made to only ever do it in the CiphertextFileNode, not in the
higher-level plaintext FileNode. Also fixed DownloadStatus handling of read
size, thanks to David-Sarah for the catch.
 2010-08-09 15:50:55 -07:00
Brian Warner
ed821d1504 Share: hush log entries in the main loop() after the fetch has been completed. 2010-08-09 13:43:59 -07:00
david-sarah
fd9a2205de test_runner.py: correct and simplify normalization of package directory for case-insensitive filesystems. 2010-08-08 11:50:05 -07:00
david-sarah
f88e4f0df6 test_runner.py: make test_path work for test-from-installdir. 2010-08-08 10:13:40 -07:00
david-sarah
f5bf900d64 src/allmydata/__init__.py: make the package paths more accurate when we fail to get them from setuptools. 2010-08-08 10:12:35 -07:00
david-sarah
dc53d4f5ef test_runner.py: another try at calculating the rootdir correctly for test-from-egg and test-from-prefixdir. 2010-08-08 08:43:07 -07:00
david-sarah
39fa9a81d1 test_runner.py: calculate the location of bin/tahoe correctly for test-from-prefixdir (by copying code from misc/build_helpers/run_trial.py). Also fix the false-positive check for Unicode paths in test_the_right_code, which was causing skips that should have been failures. 2010-08-07 21:28:17 -07:00
Brian Warner
4b7c94ece0 downloader.Segmentation: unregisterProducer when asked to stopProducing, this 
seems to avoid the #1155 log message which reveals the URI (and filecap).

Also add an [ERROR] marker to the flog entry, since unregisterProducer also
makes interrupted downloads appear "200 OK"; this makes it more obvious that
the download did not complete.
 2010-08-06 00:07:05 -07:00
david-sarah
a79ec1c6cd test_util.py: use SHA-256 from pycryptopp instead of MD5 from hashlib (for uses in which any hash will do), since hashlib was only added to the stdlib in Python 2.5. 2010-08-05 22:00:51 -07:00
david-sarah
600cb95fe2 test_runner.py: increase timeout to cater for Francois' ARM buildslave. 2010-08-05 21:26:01 -07:00
david-sarah
8e90255349 test_util.py: remove use of 'a if p else b' syntax that requires Python 2.5. 2010-08-05 21:16:16 -07:00
Brian Warner
a0124e95ee test_download.DownloadTest.test_simultaneous_goodguess: enable some disabled 
checks that used to fail but work now.
 2010-08-05 11:55:07 -07:00
Brian Warner
f6f9a97627 DownloadNode: fix lost-progress in fetch_failed, tolerate cancel when no segment-fetch is active. Fixes #1154. 
The lost-progress bug occurred when two simultanous read() calls fetched
different segments, and the first one failed (due to corruption, or the other
bugs in #1154): the second read() would never complete. While in this state,
cancelling the second read by having its consumer call stopProducing) would
trigger the cancel-intolerance bug. Finally, in downloader.node.Cancel,
prevent late cancels by adding an 'active' flag
 2010-08-05 11:55:07 -07:00
Brian Warner
43c5032105 util/spans.py: __nonzero__ cannot return a long either. for #1154 2010-08-05 11:55:07 -07:00
david-sarah
cd8d41584e test_storage.py: change skip note for test_large_share to say that Windows doesn't support sparse files. refs #569 2010-08-04 19:26:12 -07:00
Brian Warner
8844655705 One fix for bug #1154: webapi GETs with a 'Range' header broke new-downloader. 
The Range header causes n.read() to be called with an offset= of type 'long',
which eventually got used in a Spans/DataSpans object's __len__ method.
Apparently python doesn't permit __len__() to return longs, only ints.
Rewrote Spans/DataSpans to use s.len() instead of len(s) aka s.__len__() .
Added a test in test_download. Note that test_web didn't catch this because
it uses mock FileNodes for speed: it's probably time to rewrite that.

There is still an unresolved error-recovery problem in #1154, so I'm not
closing the ticket quite yet.
 2010-08-04 11:45:49 -07:00
Brian Warner
2bd8749849 test_download: minor cleanup 2010-08-04 10:55:55 -07:00
Brian Warner
fa34e4dd16 fetcher.py: improve comments 2010-08-04 00:28:14 -07:00
Brian Warner
2a05aa2d91 lazily create DownloadNode upon first read()/get_segment() 2010-08-04 00:28:08 -07:00