david-sarah
a817163cc6
docs: convert NEWS to NEWS.rst and change all references to it.
2011-05-16 18:02:55 -07:00
david-sarah
49fd2e6e56
scripts/common.py: don't assume that the default alias is always 'tahoe' (it is, but the API of get_alias doesn't say so). refs #1342
2011-01-30 10:49:23 -07:00
Zooko O'Whielacronx
b30a269ec2
setup: don't catch all Exception from check_requirement(), but only PackagingError and ImportError
...
I noticed this because I had accidentally inserted a bug which caused AssertionError to be raised from check_requirement().
2011-01-28 08:20:06 -07:00
Zooko O'Whielacronx
299e8ad579
replace remaining .html docs with .rst docs
...
Remove install.html (long since deprecated).
Also replace some obsolete references to install.html with references to quickstart.rst.
Fix some broken internal references within docs/historical/historical_known_issues.txt.
Thanks to Ravi Pinjala and Patrick McDonald.
refs #1227
2011-05-10 12:16:50 -07:00
david-sarah
fffb3ae75b
Fix a test failure in test_package_initialization on Python 2.4.x due to exceptions being stringified differently than in later versions of Python. refs #1389
2011-04-11 12:07:38 -07:00
david-sarah
a09d7300e0
tests: add test for including the ImportError message and traceback entry in the summary of errors from importing dependencies. refs #1389
2011-04-10 08:58:44 -07:00
david-sarah
449b7ad8c7
allmydata/__init__.py: preserve the message and last traceback entry (file, line number, function, and source line) of ImportErrors in the package versions string. fixes #1389
2011-04-10 08:57:05 -07:00
Zooko O'Whielacronx
4911510e0d
remove unused variable detected by pyflakes
2011-04-07 10:22:31 -07:00
david-sarah
f3246a9ca0
allmydata/__init__.py: Nicer reporting of unparseable version numbers in dependencies. fixes #1388
2011-04-01 13:27:50 -07:00
Brian Warner
70f9f89c66
control.py: remove all uses of s.get_serverid()
2011-02-26 19:12:03 -07:00
Brian Warner
73a54f83d1
web: remove some uses of s.get_serverid(), not all
2011-02-26 19:11:59 -07:00
Brian Warner
3834192455
immutable/downloader/fetcher.py: remove all get_serverid() calls
2011-02-26 19:11:56 -07:00
Brian Warner
a4068dd1e0
immutable/downloader/fetcher.py: fix diversity bug in server-response handling
...
When blocks terminate (either COMPLETE or CORRUPT/DEAD/BADSEGNUM), the
_shares_from_server dict was being popped incorrectly (using shnum as the
index instead of serverid). I'm still thinking through the consequences of
this bug. It was probably benign and really hard to detect. I think it would
cause us to incorrectly believe that we're pulling too many shares from a
server, and thus prefer a different server rather than asking for a second
share from the first server. The diversity code is intended to spread out the
number of shares simultaneously being requested from each server, but with
this bug, it might be spreading out the total number of shares requested at
all, not just simultaneously. (note that SegmentFetcher is scoped to a single
segment, so the effect doesn't last very long).
2011-02-26 19:11:53 -07:00
Brian Warner
9ae026d9f4
immutable/downloader/share.py: reduce get_serverid(), one left, update ext deps
...
test_download.py: create+check MyShare instances better, make sure they share
Server objects, now that finder.py cares
2011-02-26 19:11:50 -07:00
Brian Warner
baf1506b9b
immutable/downloader/finder.py: reduce use of get_serverid(), one left
2011-02-26 19:11:46 -07:00
Brian Warner
d9d55ad006
immutable/offloaded.py: reduce use of get_serverid() a bit more
2011-02-26 19:11:42 -07:00
Brian Warner
2f63d9b522
immutable/upload.py: reduce use of get_serverid()
2011-02-26 19:11:38 -07:00
Brian Warner
bdc5cfbdad
immutable/checker.py: remove some uses of s.get_serverid(), not all
2011-02-26 19:11:34 -07:00
Brian Warner
9672b34880
add remaining get_* methods to storage_client.Server, NoNetworkServer, and
...
MockIServer stubs
2011-02-26 19:11:32 -07:00
Brian Warner
36fabb4f4a
upload.py: rearrange _make_trackers a bit, no behavior changes
2011-02-26 19:11:28 -07:00
Brian Warner
ee14f2ad67
happinessutil.py: finally rename merge_peers to merge_servers
2011-02-26 19:11:24 -07:00
Brian Warner
13d284b087
test_upload.py: factor out FakeServerTracker
2011-02-26 19:11:20 -07:00
Brian Warner
9fc55bad25
test_upload.py: server-vs-tracker cleanup
2011-02-26 19:11:15 -07:00
Brian Warner
459b9d9e7c
happinessutil.py: server-vs-tracker cleanup
2011-02-26 19:11:11 -07:00
Brian Warner
43488b025c
upload.py: more tracker-vs-server cleanup
2011-02-26 19:11:07 -07:00
Brian Warner
0cf9e3b150
upload.py: fix var names to avoid confusion between 'trackers' and 'servers'
2011-02-26 19:11:03 -07:00
Brian Warner
ebfcb649f9
refactor: s/peer/server/ in immutable/upload, happinessutil.py, test_upload
...
No behavioral changes, just updating variable/method names and log messages.
The effects outside these three files should be minimal: some exception
messages changed (to say "server" instead of "peer"), and some internal class
names were changed. A few things still use "peer" to minimize external
changes, like UploadResults.timings["peer_selection"] and
happinessutil.merge_peers, which can be changed later.
2011-02-26 19:11:00 -07:00
Brian Warner
9b9ea3f739
storage_client.py: clean up test_add_server/test_add_descriptor, remove .test_servers
2011-02-26 19:10:56 -07:00
Brian Warner
cba3de04a4
test_client.py, upload.py:: remove KiB/MiB/etc constants, and other dead code
2011-02-26 19:10:51 -07:00
Zooko O'Whielacronx
578a117c5b
test: increase timeout on a network test because Francois's ARM machine hit that timeout
...
I'm skeptical that the test was proceeding correctly but ran out of time. It seems more likely that it had gotten hung. But if we raise the timeout to an even more extravagant number then we can be even more certain that the test was never going to finish.
2011-03-17 09:59:09 -07:00
Brian Warner
44466fbb1b
web/filenode.py: avoid calling req.finish() on closed HTTP connections. Closes #1366
2011-02-20 22:15:44 -08:00
david-sarah
787d12165a
Add unit tests for cross_check_pkg_resources_versus_import, and a regression test for ref #1355 . This requires a little refactoring to make it testable.
2011-02-20 17:58:17 -08:00
david-sarah
71c301ca34
allmydata/__init__.py: .name was used in place of the correct .__name__ when printing an exception. Also, robustify string formatting by using %r instead of %s in some places. fixes #1355 .
2011-02-20 18:01:25 -08:00
Brian Warner
ffd296fc5a
Refactor StorageFarmBroker handling of servers
...
Pass around IServer instance instead of (peerid, rref) tuple. Replace
"descriptor" with "server". Other replacements:
get_all_servers -> get_connected_servers/get_known_servers
get_servers_for_index -> get_servers_for_psi (now returns IServers)
This change still needs to be pushed further down: lots of code is now
getting the IServer and then distributing (peerid, rref) internally.
Instead, it ought to distribute the IServer internally and delay
extracting a serverid or rref until the last moment.
no_network.py was updated to retain parallelism.
2011-02-20 17:58:04 -08:00
david-sarah
ed67df866c
SFTP: report unknown sizes as "0" instead of "?", to satisfy some clients. fixes #1337
2011-01-27 22:22:49 -08:00
david-sarah
d318122cd2
Change tests for check_requirement to ensure that it accepts None in the comment field of a (version, location, comment) triple, and add a test that it raises ImportError when a module couldn't be imported. Also fix typo in a comment. refs #1339
2011-01-28 11:24:52 -08:00
Zooko O'Whielacronx
6f84f1b869
setup: adjust tests to use the new interface of check_requirement which has a 3-tuples instead of a 2-tuple
...
fixes #1339
2011-01-28 06:15:46 -08:00
david-sarah
c71a19b9ed
src/allmydata/__init__.py: fix #1339 , give an indication in the --version[-and-path] output of when the imported setuptools is distribute, and use a separate element in _vers_and_locs_list tuples for information other than the package name and location. This also changes slightly how the sqlite version is reported.
2011-01-27 21:41:50 -08:00
david-sarah
74b6ea36c4
CLI: improve help synopsis for 'tahoe create-key-generator'.
2011-01-22 17:50:04 -08:00
david-sarah
cb02adb110
Don't put debugging information in 'My versions' section of the Welcome page. Also remove the extra blank line between version and command output when --version[-and-path] is used. refs #1306
2011-01-21 20:02:20 -08:00
david-sarah
f84f2aa987
tests: use verlib to compare versions, and get the versions of Twisted and Nevow from get_package_versions() instead of pkg_resources. refs #1287
2011-01-21 17:42:56 -08:00
david-sarah
39eede4217
src/allmydata/web/common.py: ensure that filenames in 'humanized failures' are quoted.
2011-01-21 17:40:06 -08:00
david-sarah
0c51ce3d89
CLI: improve help synopses for some commands.
2011-01-21 17:27:00 -08:00
david-sarah
c8ccb297fa
src/allmydata/_auto_deps.py: setuptools is still an install requirement even in frozen builds, because .tac files import pkg_resources. refs #585
2011-01-21 17:24:42 -08:00
david-sarah
8587c63bdc
Make bb-freeze (and probably other static packaging tools) work. This updates various places where we assumed that the tahoe process was executed via the Python interpreter. It also allows tests to recursively invoke the same tahoe.exe, rather than bin/tahoe. refs #585
2011-01-21 00:04:29 -08:00
david-sarah
28bd80812f
Change --version and --version-and-path to not exit immediately, if a command is given.
2011-01-20 23:59:13 -08:00
david-sarah
f3b4f4181c
src/allmydata/_auto_deps.py: 'pysqlite.dbapi2' module should have been 'pysqlite2.dbapi2'. refs #1306
2011-01-20 23:32:34 -08:00
david-sarah
b6c2c9591d
src/allmydata/util/iputil.py: correct an error in the address-matching regexps introduced by the previous patch to iputil. refs #1274
2011-01-20 23:21:36 -08:00
david-sarah
b1f15a630a
Add src/allmydata/util/verlib.py, which is a copy of verlib from https://bitbucket.org/tarek/distutilsversion/src/17df9a7d96ef . It is used to normalize and compare versions of dependencies. refs #1258
2011-01-20 22:04:49 -08:00
david-sarah
29336a0916
Refactor _auto_deps.py and __init__.py, adding more robust checking of dependency versions, and not trusting pkg_resources to get the versions right. refs #1258 , #1287
2011-01-20 21:36:10 -08:00
david-sarah
d5138b3237
src/allmydata/util/iputil.py: loosen regexps and ensure that 'LANG=en_US.UTF-8' is set in the environment, to minimize problems with localized output of IP-address-finding tools. refs #1274
2011-01-20 00:48:27 -08:00
david-sarah
6dd8b6f471
Eliminate dependencies on pywin32, even via Twisted. refs #1274
2011-01-19 20:32:38 -08:00
david-sarah
fd6cdc48ae
src/allmydata/test/test_runner.py: add test_import_from_repl, which checks that we are running the right code in a bin/tahoe subprocess. refs #1258
2011-01-19 00:21:45 -08:00
david-sarah
a9fc4668c0
docs/frontends/CLI.rst, src/allmydata/test/trialtest.py: add trailing newlines. refs #1296
2011-01-19 00:19:55 -08:00
david-sarah
d21f4071c3
Eliminate direct dependencies of Tahoe-LAFS on pywin32 (rebased to trunk). refs #1274
2011-01-18 23:59:11 -08:00
david-sarah
3798d9946e
Add support to bin/tahoe for invoking a runner command prefixed with @, with the Tahoe libraries on the PYTHONPATH. This is documented in 'tahoe debug --help'.
2011-01-18 21:11:37 -08:00
david-sarah
74b1eec1d6
trivial: add comment in scripts/debug.py about trial option parsing. refs #1296
2011-01-18 22:08:08 -08:00
david-sarah
8d2983c493
Update foolscap requirement to >= 0.6.1. fixes #1329
2011-01-18 22:06:39 -08:00
david-sarah
93c1909847
bin/tahoe-script.template, src/windows/fixups.py: simplify the method of stripping initial arguments in sys.argv on Windows. This helps with bb-freeze and running tahoe via 'coverage'. Also includes some wording changes and minor refactoring of bin/tahoe-script.template. refs #585 , #1303
2011-01-18 20:53:24 -08:00
david-sarah
8f0af33ba6
src/allmydata/test/test_cli.py: add test for 'tahoe debug trial' options help. refs #1296
2011-01-18 18:42:24 -08:00
david-sarah
1819c25c88
Add src/allmydata/test/trialtest.py needed by tests for 'tahoe debug trial'. refs #1296
2011-01-18 18:02:39 -08:00
david-sarah
0d6df9c9fc
Tests for 'tahoe debug trial' (rebased and fixed to work with Twisted 10.2). refs #1296
2011-01-18 17:38:59 -08:00
david-sarah
420aadd95e
Make 'mock' a run-time rather than setup-time dependency. This is necessary in order for 'tahoe debug trial' to work. refs #1296
2011-01-18 12:51:14 -08:00
david-sarah
7a887871b0
src/allmydata/scripts/debug.py: add 'tahoe debug trial' command (rebased for trunk). refs #1296
2011-01-18 12:46:59 -08:00
david-sarah
fc98aded8a
src/allmydata/webish.py: clean-ups and correction to a comment. Also change an open and write to use fileutil.write. See ref #1286 comment 13.
2011-01-17 15:31:52 -08:00
Brian Warner
980fb0b2bb
'tahoe debug catalog-shares': sort SIs and shnums
...
Without this, SIs or shnums could be emitted in random order, depending upon
what the filesystem happens to return.
2011-01-17 01:59:32 -08:00
david-sarah
5a3acc05b7
CLI: tests for ref #1305 (v2, remove spurious extra arg to create-alias in test)
2011-01-13 20:03:27 -08:00
david-sarah
f10a2c0902
CLI: make 'tahoe create-alias' and 'tahoe add-alias' accept a trailing colon on the new alias name (v2, minor change not to rely on implicit Unicode conversion). Includes doc changes and news; tests in a separate patch. fixes #1305
2011-01-13 19:44:14 -08:00
Brian Warner
b9fb74d2a0
Improve 'tahoe ln' help text. Patch by David-Sarah. Closes #1230 .
2011-01-17 00:14:21 -08:00
Brian Warner
09a2241471
Tolerate Twisted-10.2's endpoints, patch by David-Sarah. Closes #1286 .
...
The service generated by strports.service() changed in 10.2, and the ugly
private-attribute-reading hack we used to glean a kernel-allocated port
number (e.g. when using "tcp:0", especially during unit tests) broke, causing
Tahoe to be completely unusable with Twisted-10.2 . The new ugly
private-attribute-reading hack starts by figuring out what sort of service
was generated, then reads different attributes accordingly.
This also hushes a warning when using schemeless strports strings like "0" or
"3456", by quietly prepending a "tcp:" scheme, since 10.2 complains about
those. It also adds getURL() and getPortnum() accessors to the "webish"
service, rather than having unit tests dig through _url and _portnum and such
to find out what they are.
2011-01-16 23:47:51 -08:00
david-sarah
8a3cb60bf7
Test changes to take account of ref #1311 .
2011-01-16 22:05:40 -08:00
david-sarah
d3f24736ca
create_node.py: add comments to default tahoe.cfg to clarify the meaning of each section. fixes #1311
2011-01-16 21:24:19 -08:00
david-sarah
e52d97a917
Undo the temporary hack to check the foolscap version. refs #1246
2011-01-16 21:20:42 -08:00
Brian Warner
45212d8000
Set "reserved_space=1G" in newly-created storage nodes. Closes #1208 .
2011-01-16 12:58:22 -08:00
david-sarah
727b25f622
Temporary hack to investigate whether we are getting the right version of foolscap on trunk. refs #1258
2011-01-15 20:49:59 -08:00
david-sarah
33786a6d55
Attempt to fix test-with-fake-dist build step.
2011-01-14 18:26:51 -08:00
david-sarah
e62a69e7ff
test_runner: avoid unnecessary use of non-ASCII.
2011-01-01 02:01:01 -08:00
david-sarah
2c099ffa0d
Move dependency imports from windows/depends.py (which has gone away) into src/allmydata/windows/tahoesvc.py. Also fix a pyflakes warning, and change the service display name from 'Allmydata Tahoe Node' to 'Tahoe-LAFS node'. refs #1282
2010-12-25 20:21:00 -08:00
david-sarah
3132c9b593
Remove unmaintained Windows GUI app, except for windows/tahoesvc.py which is moved to src/allmydata/windows. refs #1282
2010-12-25 20:02:37 -08:00
david-sarah
195ddbcf7c
Remove unmaintained Mac GUI app. fixes #1282
2010-12-25 18:05:08 -08:00
david-sarah
399ee5aea0
Remove src/allmydata/util/find_exe.py which is no longer used. fixes #1150
2010-12-25 18:32:06 -08:00
freestorm77
4027a136f3
status_web_pages_review.darcs.patch
...
I make some changes on status web pages
status.xhtml:
- Delete unused webform_css link
- Align tables on the left
tahoe-css:
- Do some minor changes on code synthax
- changes table.status-download-events style to look like other tables
status.py:
- Align table on the left
- Changes table header
- Add heading tags
- Modify google api graph: add image border, calculate height to feet data
signed-off-by: zooko@zooko.com
fixes #1219
2011-01-01 19:42:14 -08:00
david-sarah
5027753e23
test_storage.py: fix a pyflakes unused import warning.
2010-12-31 14:07:56 -08:00
david-sarah
0da10d4c37
test_storage.py: leave at least 512 MiB free when running test_large_share. refs #1195
2010-12-31 12:32:15 -08:00
Zooko O'Whielacronx
123a1a3f04
storage: use fileutil's version of get_disk_stats() and get_available_space(), use mockery/fakery in tests, enable large share test on platforms with sparse files and if > 4 GiB of disk space is currently available
2010-09-10 08:36:29 -08:00
Zooko O'Whielacronx
d5e71c2940
fileutil: copy in the get_disk_stats() and get_available_space() functions from storage/server.py
2010-09-10 08:35:20 -08:00
david-sarah
c82b48f3d6
Update foolscap version requirement to 0.6.0, to address http://foolscap.lothar.com/trac/ticket/167
2010-12-30 22:00:39 -08:00
david-sarah
2d5f3f694f
scripts/backupdb.py: more accurate comment about path field.
2010-12-12 09:03:20 -08:00
david-sarah
bb7c04ff9f
scripts/cli.py: fix missing 'put' in usage example for 'tahoe put'.
2010-12-12 09:02:07 -08:00
david-sarah
ee4e0d8106
scripts/cli.py: remove the disclaimer in the help for 'tahoe cp' that it does not handle non-ASCII filenames well. (At least, we intend to handle them.)
2010-11-29 16:21:45 -08:00
david-sarah
9ceb527787
Change another doc reference from .txt to .rst in a text string.
2010-11-28 09:45:07 -08:00
david-sarah
d7bdf476b9
Change a doc reference from .txt to .rst in a text string.
2010-11-28 09:34:44 -08:00
Brian Warner
69b42c6cb7
remove --multiple/-m option from all CLI commands: closes #1262
...
I personally used "tahoe start/restart -m ../MY-TESTNET/node*" all the time,
to spin up or update a local testgrid while iterating over new code. However,
with the recent switch from "subprocess.Popen(/bin/twistd)" to "import and
call twistd.run()" in scripts/startstop_node.py (yay fewer processes!),
"start -m" broke, and fixing it requires os.fork, which is unavailable on
windows (boo windows!). And I was probably the only one using -m. So in the
interests of uniformity among platforms and simpler code (yay negative code
days!), we're just removing -m from everything. I will start using a little
shell script or something to simulate the removed functionality.
This patch also cleans up CLI-function calling a bit: get the basedir from
the config dict (instead of sometimes from a separate argument), and always
return a numeric exit code.
2010-11-27 00:38:09 -08:00
Brian Warner
f36bda2780
Revert previous commit: there's an ugly corner-case on windows that fails tests.
...
Specifically, test_runner.CreateNode.test_client failed, because the
os.fork-is-present test decided that --multiple should not be allowed on
windows, even though --multiple works just fine for 'tahoe create-client'.
The only restriction on --multiple is for 'tahoe start' and 'tahoe restart'.
This needs a different approach, probably by cleaning up BasedirMixin. We
should only be withholding --multiple on windows for "start" and
"restart". (we should continue withholding --multiple on all platforms for
"run").
This reverts (git) commit f3adb037ae
:
"startstop_node.py: fix "tahoe start -m" by forking before non-final targets"
2010-11-26 16:44:11 -08:00
Brian Warner
f3adb037ae
startstop_node.py: fix "tahoe start -m" by forking before non-final targets
...
* don't advertise -m flag on tahoe start/restart/run unless os.fork is
available (i.e. windows)
* test_runner.py: add test to exercise "start/stop/restart -m"
2010-11-26 14:45:38 -08:00
david-sarah
38a0d3f5fa
tahoe_mv.py: when checking success of the DELETE operation, look at the status code from DELETE rather than from the previous PUT. fixes #1255
2010-11-09 17:09:16 -08:00
david-sarah
fdf6946669
test_cli.py: test that 'tahoe mv' reports errors from the DELETE operation. refs #1255
2010-11-11 17:46:53 -08:00
Zooko O'Whielacronx
b4c14421f7
bundled zetuptoolz: prefer locally-available distributions over remotely-downloaded distributions above all
...
This fixes #1233 . Actually the previous patches—[20101103034740-93fa1-9df33552497282eb72a84e5b434d035974bf2dbb] and [20101117080828-92b7f-dc0239f30b26e7e5d40b228114fb399c1e190ec5]—fixed it, but with them zetuptoolz would download a higher-numbered distribution from the net instead of using the locally-available (fake) pycryptopp-0.5.24, thus preventing the tests from passing. This patch changes that behavior (which is an improvement in its own right) and also fixes a bug in the tests.
2010-11-17 00:26:57 -08:00
Zooko O'Whielacronx
93dc740f01
tests: bump up the timeout on test_dirnode.Dirnode from 240s to 480s since it apparently took longer than 240s just now on François's ARM buildslave
2010-11-15 01:21:19 -08:00
david-sarah
76371ae5a9
test_cli.py: fix a stale comment that incorrectly implied that test_cli runs CLI commands in subprocesses (it actually runs them using deferToThread).
2010-11-09 18:28:19 -08:00
Zooko O'Whielacronx
59dcd5014d
setup: test whether the build system prefers binary dists which are new enough over source dists which are even newer
...
ref: #1233
2010-11-13 07:28:22 -08:00
david-sarah
cb777ad14f
tahoe_mv.py: use do_http, not urllib.openurl, in order to avoid connecting to the webapi server via a proxy. refs #1253
2010-11-09 16:53:36 -08:00
david-sarah
01a5365051
test_system.py: test for 'tahoe mv' with the http_proxy and HTTP_PROXY variables set. refs #1253
2010-11-09 19:36:49 -08:00
david-sarah
edec9c9869
test_client.py: change overzealous use of failUnlessReallyEqual on integers back to failUnlessEqual.
2010-11-09 15:08:16 -08:00
david-sarah
56f4c1218b
test_runner.py: fix test failure in test_the_right_code after applying zooko's change to test-with-fake-pkg.py
2010-11-01 20:59:05 -07:00
Zooko O'Whielacronx
a92912f513
setup: FreeStorm's WinXP-x86-py2.6 buildslave has informed us that there is yet a fourth way to spell "i386" in Python
2010-10-31 22:21:42 -07:00
david-sarah
b8905fc766
allmydata/__init__.py: move the call to require_auto_deps() to the top again, since the [4784] patch turned out not to be the cause of the failure on the 'mm netbsd5' buildslave.
2010-10-31 17:33:16 -07:00
david-sarah
63c010a9ca
allmydata/__init__.py: put the _auto_deps.require_auto_deps() call back in exactly the same place it was prior to [4784].
2010-10-31 10:46:49 -07:00
david-sarah
7e2ed2054c
allmydata/__init__.py: suppress a DeprecationWarning for the sha module on importing pycrypto.
2010-10-31 09:06:36 -07:00
david-sarah
6e76653913
allmydata/__init__.py: call require_auto_deps() after importing nevow and twisted, reverting change in [4784]. Also fix a missing 'warnings.filters.pop()'.
2010-10-31 08:38:28 -07:00
david-sarah
2ab0b1aa2c
allmydata.__init__.py: temporary hack to debug failure on midnightmagic's buildslave
2010-10-30 22:50:03 -07:00
david-sarah
6eaa7f2356
tahoe backup: perform tilde expansion in exclude-from filename (version 2). fixes #1241
2010-10-30 20:52:31 -07:00
Zooko O'Whielacronx
3b2d0d9609
setup: run require_auto_deps() before attempting to import any deps in __init__.py
...
For one thing, this makes missing-dependency failures into DistributionNotFound errors instead of ImportErrors, which might be more useful to the user. For another thing, if someone is using distributions that were installed with --multi-version, then they might be not importable until after require_auto_deps() has been run. (The docs claim that this would be the case, but we don't have an example of this happening at this time.)
2010-10-30 01:10:35 -07:00
david-sarah
1950d5a719
scripts/runner.py: fix unused import of allmydata. refs #1190
2010-10-29 17:31:49 -07:00
david-sarah
2a8f700026
scripts/runner.py: remove pkg_resources.require() calls. These are at best redundant because we have already called _auto_deps.require_auto_deps() (from allmydata.__init__) at that point, and they are causing failure of the test-from-prefixdir step on some buildslaves. refs #1190
2010-10-29 16:53:28 -07:00
david-sarah
10a5f23df9
test_runner.py: fix error in BinTahoe.test_version_no_noise introduced by last patch. refs #1235
2010-10-29 15:11:23 -07:00
david-sarah
d249589667
test_runner.py: also allow 'from pkg_resources import load_entry_point' as noise. refs #1235 .
2010-10-29 13:42:46 -07:00
david-sarah
8c95f88b4f
test_runner.py: if the only noise is 'UserWarning: Unbuilt egg for setuptools', skip instead of failing the no_noise tests. This version avoids 'any' to be compatible with Python < 2.5. refs #1235 .
2010-10-29 12:18:04 -07:00
david-sarah
4061258c85
make ResponseCache smarter to avoid memory leaks: don't record timestamps, use DataSpans to merge entries, and clear the cache when we see a new seqnum. refs #1045 , #1229
2010-10-26 21:33:02 -07:00
david-sarah
25d8103dde
windows/fixups.py: limit length of string passed in a single call to WriteConsoleW. fixes #1232 .
2010-10-26 19:16:36 -07:00
Brian Warner
c88eb097d9
scripts/tahoe_backup.py: oops, fix missing import, thanks pyflakes
...
test_cli.py: hush minor pyflakes complaint
2010-10-29 02:42:23 -07:00
Brian Warner
d61d41147c
mutable/servermap.py: update comment. Closes #1231 .
2010-10-29 02:14:24 -07:00
Francois Deppierraz
14ee763c54
tahoe_cp.py: Don't call urllib.quote with an Unicode argument, fix #1224
...
tahoe_backup.py: Fix another (potential) occurrence of calling urllib.quote()
with an Unicode parameter
2010-10-29 01:45:20 -07:00
Brian Warner
c18953c169
fix #1223 , crash+inefficiency during repair due to read overrun
...
* repairer (really the uploader) reads beyond end of input file (Uploadable)
* new-downloader does not tolerate overreads
* uploader does lots of tiny reads (inefficient)
This fixes the last two. The uploader still does a single overread at the end
of the input file, but now that's ok so we can leave it in place. The
uploader now expects the Uploadable to behave like a normal disk
file (reading beyond EOF will return less data than was asked for), and now
the new-downloadable behaves that way.
2010-10-29 01:20:36 -07:00
david-sarah
270322ad47
startstop_node.py: pyflakes import fix. refs #1190
2010-10-27 18:48:05 -07:00
Brian Warner
ac3b26ecf2
'tahoe start': use import+call rather than find+spawn
...
This removes the need to use a locally-built (dependency) bin/twistd, and
removes a big chunk of behavior differences between unix and windows. It
also happens to resolve the "client node probably started" uncertainty.
Might help with #1190 , #602 , and #71 .
2010-10-26 23:13:42 -07:00
david-sarah
59e80b2ec9
test_mutable.py: add test for ResponseCache memory leak. refs #1045 , #1129
2010-10-24 12:34:09 -07:00
david-sarah
daf2b8314a
test_encodingutil.py: test_argv_to_unicode modified the wrong encoding variable. fixes #1214
2010-10-22 20:58:10 -07:00
Zooko O'Whielacronx
0c2397523b
doc: add explanation of the motivation for the surprising and awkward API to erasure coding
2010-10-14 23:02:02 -07:00
Zooko O'Whielacronx
98ffbfb31f
setup: add --multi-version to the "setup.py develop" command-line
...
fixes #530 . I earlier tried this twice (see #530 for history) and then twice rolled it back due to some problems that arose. However, I didn't write down what the problems were in enough detail on the ticket that I can tell today whether those problems are still issues, so here goes the third attempt. (I did write down on the ticket that it would not create site.py or .pth files in the target directory with --multi-version mode, but I didn't explain why *that* was a problem.)
2010-10-05 11:23:50 -07:00
Zooko O'Whielacronx
0c6bb0178b
setup: use execfile to access _auto_deps.py in its proper location of src/allmydata/ instead of copying it into place when setup.py is executed
2010-09-05 22:57:14 -07:00
Zooko O'Whielacronx
e6a380241a
trivial: M-x whitespace-cleanup
2010-09-03 07:47:12 -07:00
Zooko O'Whielacronx
cb83f2e41c
minor: remove unused interface declaration, change allmydata.org to tahoe-lafs.org in email address, fix wording in relnotes.txt
2010-09-30 08:37:08 -07:00
Kevan Carstensen
ec4f87a98c
immutable/repairer.py: don't use the default happiness setting when repairing
2010-09-27 13:01:02 -07:00
david-sarah
6c4ba62c5d
test_download.py: simplify and possibly fix test_lost_servers; refs #1197
2010-09-11 17:32:51 -07:00
david-sarah
b23a7fcc10
test_runner.py: fix spurious failures of test_path on case-insensitive filesystems, and restore the check that the version is not 'unknown'.
2010-09-11 15:19:13 -07:00
Zooko O'Whielacronx
fd17d63c95
test: make tests stop relying on pyutil version class accepting the string 'unknown' for its version, and make them forward-compatible with the future Python Rational Version Numbering standard
2010-09-10 08:41:35 -07:00
Brian Warner
0475bd8e27
immutable download: have the finder inform its share consumer "no more shares" in a subsequent tick, thus avoiding accidentally telling it "no more shares" now and then telling it "here's another share" in a subsequent tick
...
fixes #1191
Patch by Brian. This patch description was actually written by Zooko, but I forged Brian's name on the "author" field so that he would get credit for this patch in revision control history.
2010-09-09 21:30:38 -07:00
Zooko O'Whielacronx
56a3258ff7
immutable downloader: add a test specifically of whether the finder sometimes announces "no more shares ever" and then announces a newly found share
...
(The current code fails this test, ref #1191.)
2010-09-08 21:16:54 -07:00
Zooko O'Whielacronx
9d421cb022
tests: assign the storage servers to a fixed order which triggers a bug in new downloader every time this test is run (formerly this test would detect the bug in new-downloader only sporadically)
...
If you are investigating the bug in new-downloader, one way to investigate might be to change this ordering to a different fixed order (e.g. rotate by 4 instead of rotate by 5) and observe how the behavior of new-downloader differs in that case.
2010-09-03 21:15:15 -07:00
Zooko O'Whielacronx
63fb687a44
download status: fix bug from me committing the wrong one of François's #1172 patches, fixes #1172
2010-09-02 09:15:41 -07:00
francois
485bfc0fd6
DownloadStatus: show active immutable downloads in Active Operations, Fix #1172
2010-09-02 03:17:28 -07:00
Zooko O'Whielacronx
cdc625e0f8
tests: bump up the allowed number of reads
...
Kyle's OpenBSD buildslave used 41 reads when doing this test. The fact that I'm blindly bumping this number up to match the observed behavior probably means this isn't a good criterion to be testing for anyway. But perhaps someone else (Brian) could investigate why that run on Kyle's OpenBSD box took four more reads than we expected, and whether the fact that it took 41 reads to do this operation is indicative of an actual problem.
2010-09-01 22:38:01 -07:00
Brian Warner
00e9e4e676
SegmentFetcher: use new diversity-seeking share-selection algorithm, and
...
deliver all shares at once instead of feeding them out one-at-a-time.
Also fix distribution of real-number-of-segments information: now all
CommonShares (not just the ones used for the first segment) get a
correctly-sized hashtree. Previously, the late ones might not, which would
make them crash and get dropped (causing the download to fail if the initial
set were insufficient, perhaps because one of their servers went away).
Update tests, add some TODO notes, improve variable names and comments.
Improve logging: add logparents, set more appropriate levels.
2010-08-31 18:37:02 -07:00
Brian Warner
c89a464510
Share: drop received data after each block finishes. Quick fix for the #1170 spans.py complexity bug.
2010-08-31 18:35:58 -07:00
francois
f026927f86
web: refactor rate computation, fixes #1166
2010-08-15 07:19:33 -07:00
Brian Warner
a3cf3331d2
DownloadStatus: put real numbers in progress/status rows, not placeholders.
...
Improve tests.
2010-08-09 23:06:03 -07:00
Brian Warner
dc1afc81bb
web download-status: tolerate DYHBs that haven't retired yet. Fixes #1160 .
...
Also add a better unit test for it.
2010-08-09 15:51:00 -07:00
Brian Warner
5267fc17fe
immutable/filenode.py: put off DownloadStatus creation until first read() call
...
This avoids spamming the "recent uploads and downloads" /status page from
FileNode instances that were created for a directory read but which nobody is
ever going to read from. I also cleaned up the way DownloadStatus instances
are made to only ever do it in the CiphertextFileNode, not in the
higher-level plaintext FileNode. Also fixed DownloadStatus handling of read
size, thanks to David-Sarah for the catch.
2010-08-09 15:50:55 -07:00
Brian Warner
ed821d1504
Share: hush log entries in the main loop() after the fetch has been completed.
2010-08-09 13:43:59 -07:00
david-sarah
fd9a2205de
test_runner.py: correct and simplify normalization of package directory for case-insensitive filesystems.
2010-08-08 11:50:05 -07:00
david-sarah
f88e4f0df6
test_runner.py: make test_path work for test-from-installdir.
2010-08-08 10:13:40 -07:00
david-sarah
f5bf900d64
src/allmydata/__init__.py: make the package paths more accurate when we fail to get them from setuptools.
2010-08-08 10:12:35 -07:00
david-sarah
dc53d4f5ef
test_runner.py: another try at calculating the rootdir correctly for test-from-egg and test-from-prefixdir.
2010-08-08 08:43:07 -07:00
david-sarah
39fa9a81d1
test_runner.py: calculate the location of bin/tahoe correctly for test-from-prefixdir (by copying code from misc/build_helpers/run_trial.py). Also fix the false-positive check for Unicode paths in test_the_right_code, which was causing skips that should have been failures.
2010-08-07 21:28:17 -07:00
Brian Warner
4b7c94ece0
downloader.Segmentation: unregisterProducer when asked to stopProducing, this
...
seems to avoid the #1155 log message which reveals the URI (and filecap).
Also add an [ERROR] marker to the flog entry, since unregisterProducer also
makes interrupted downloads appear "200 OK"; this makes it more obvious that
the download did not complete.
2010-08-06 00:07:05 -07:00
david-sarah
a79ec1c6cd
test_util.py: use SHA-256 from pycryptopp instead of MD5 from hashlib (for uses in which any hash will do), since hashlib was only added to the stdlib in Python 2.5.
2010-08-05 22:00:51 -07:00
david-sarah
600cb95fe2
test_runner.py: increase timeout to cater for Francois' ARM buildslave.
2010-08-05 21:26:01 -07:00
david-sarah
8e90255349
test_util.py: remove use of 'a if p else b' syntax that requires Python 2.5.
2010-08-05 21:16:16 -07:00
Brian Warner
a0124e95ee
test_download.DownloadTest.test_simultaneous_goodguess: enable some disabled
...
checks that used to fail but work now.
2010-08-05 11:55:07 -07:00
Brian Warner
f6f9a97627
DownloadNode: fix lost-progress in fetch_failed, tolerate cancel when no segment-fetch is active. Fixes #1154 .
...
The lost-progress bug occurred when two simultanous read() calls fetched
different segments, and the first one failed (due to corruption, or the other
bugs in #1154 ): the second read() would never complete. While in this state,
cancelling the second read by having its consumer call stopProducing) would
trigger the cancel-intolerance bug. Finally, in downloader.node.Cancel,
prevent late cancels by adding an 'active' flag
2010-08-05 11:55:07 -07:00
Brian Warner
43c5032105
util/spans.py: __nonzero__ cannot return a long either. for #1154
2010-08-05 11:55:07 -07:00
david-sarah
cd8d41584e
test_storage.py: change skip note for test_large_share to say that Windows doesn't support sparse files. refs #569
2010-08-04 19:26:12 -07:00
Brian Warner
8844655705
One fix for bug #1154 : webapi GETs with a 'Range' header broke new-downloader.
...
The Range header causes n.read() to be called with an offset= of type 'long',
which eventually got used in a Spans/DataSpans object's __len__ method.
Apparently python doesn't permit __len__() to return longs, only ints.
Rewrote Spans/DataSpans to use s.len() instead of len(s) aka s.__len__() .
Added a test in test_download. Note that test_web didn't catch this because
it uses mock FileNodes for speed: it's probably time to rewrite that.
There is still an unresolved error-recovery problem in #1154 , so I'm not
closing the ticket quite yet.
2010-08-04 11:45:49 -07:00
Brian Warner
2bd8749849
test_download: minor cleanup
2010-08-04 10:55:55 -07:00
Brian Warner
fa34e4dd16
fetcher.py: improve comments
2010-08-04 00:28:14 -07:00
Brian Warner
2a05aa2d91
lazily create DownloadNode upon first read()/get_segment()
2010-08-04 00:28:08 -07:00
Brian Warner
abcd6e0e96
test_hung_server: update comments, remove dead "stage_4_d" code
2010-08-04 00:28:00 -07:00
Brian Warner
919938dd95
copy the rest of David-Sarah's changes to make my tree match 1.8.0beta
2010-08-04 00:27:52 -07:00
Brian Warner
20847dd876
ShareFinder: add 10s OVERDUE timer, send new requests to replace overdue ones
...
The fixed 10-second timer will eventually be replaced with a per-server
value, calculated based on observed response times.
test_hung_server.py: enhance to exercise DYHB=OVERDUE state. Split existing
mutable+immutable tests into two pieces for clarity. Reenabled several tests.
Deleted the now-obsolete "test_failover_during_stage_4".
2010-08-04 00:27:41 -07:00
Brian Warner
63b61ce7bd
Rewrite immutable downloader ( #798 ). This patch adds and updates unit tests.
2010-08-04 00:27:10 -07:00
Brian Warner
7b7b0c9709
Rewrite immutable downloader ( #798 ). This patch includes higher-level
...
integration into the NodeMaker, and updates the web-status display to handle
the new download events.
2010-08-04 00:27:02 -07:00
Brian Warner
797828f47f
Rewrite immutable downloader ( #798 ). This patch rearranges the rest of src/allmydata/immutable/ .
2010-08-04 00:26:39 -07:00
Brian Warner
22a07e9bbe
Rewrite immutable downloader ( #798 ). This patch adds the new downloader itself.
2010-08-04 00:26:29 -07:00
Brian Warner
88d7ec2d54
util/observer.py: add EventStreamObserver
2010-08-04 00:26:12 -07:00
Brian Warner
cbcb728e7e
Add a byte-spans utility class, like perl's Set::IntSpan for .newsrc files.
...
Also a data-spans class, which records a byte (instead of a bit) for each
index.
2010-08-04 00:26:00 -07:00
david-sarah
f10f9a8646
test_storage.py: avoid spurious test failure by accepting either 'Next crawl in 59 minutes' or 'Next crawl in 60 minutes'. fixes #1140
2010-08-03 03:20:58 -07:00
david-sarah
a7c474a098
CLI: further improve consistency of basedir options and add tests. addresses #118
2010-08-03 01:54:16 -07:00
david-sarah
b730385ec8
CLI: make the synopsis for 'tahoe unlink' say unlink instead of rm.
2010-08-03 01:53:59 -07:00
david-sarah
a32f625db9
CLI: make all of the option descriptions imperative sentences.
2010-08-03 01:48:01 -07:00
david-sarah
51eeea900f
test_cli.py: make 'tahoe mkdir' tests slightly less dumb (check for 'URI:' in the output).
2010-08-03 01:47:20 -07:00
david-sarah
b9174b5fe7
test_cli.py: use u-escapes instead of UTF-8.
2010-08-03 01:35:38 -07:00
david-sarah
d81e8acf88
test_runner.py: add test_run_with_python_options, which checks that the Windows script changes haven't broken 'python <options> bin/tahoe'.
2010-08-01 23:25:58 -07:00
david-sarah
3ddb4a9bbf
test_runner.py: fix missing import of get_filesystem_encoding
2010-08-01 23:09:02 -07:00
david-sarah
3af6f19cb0
misc/build_helpers/run_trial.py: check that the root from which the module we are testing was loaded is the current directory. This version of the patch folds in later fixes to the logic for caculating the directories to compare, and improvements to error messages. addresses #1137
2010-08-01 21:55:35 -07:00
david-sarah
88b6c57a1c
Skip option arguments to the python interpreter when reconstructing Unicode argv on Windows.
2010-07-27 23:27:31 -07:00
david-sarah
1a5a33886d
windows/fixups.py: improve comments and reference some relevant Python bugs.
2010-07-27 11:19:21 -07:00
david-sarah
b5f052a0d7
windows/fixups.py: make errors reported to original_stderr have enough information to debug even if we can't see the traceback.
2010-07-26 15:19:04 -07:00
david-sarah
ec6a674a5c
windows/fixups.py: fix paste-o in name of Unicode stderr wrapper.
2010-07-26 14:47:36 -07:00
david-sarah
54bbc5325f
windows/fixups.py: Don't rely on buggy MSVCRT library for Unicode output, use the Win32 API instead. This should make it work on XP. Also, change how we handle the case where sys.stdout and sys.stderr are redirected, since the .encoding attribute isn't necessarily writeable.
2010-07-25 21:50:19 -07:00
david-sarah
1fad717aaa
test_runner.py: change to code for locating the bin/tahoe script that was missed when rebasing the patch for #1074 .
2010-07-25 11:20:08 -07:00
david-sarah
9d04b2a317
Add missing windows/fixups.py (for setting up Unicode args and output on Windows).
2010-07-25 02:28:49 -07:00
david-sarah
37b07a545f
Changes to Tahoe needed to work with new zetuptoolz (that does not use .exe wrappers on Windows), and to support Unicode arguments and stdout/stderr -- v5
2010-07-25 01:32:16 -07:00
david-sarah
54a9ba8232
scripts/common.py: fix an error introduced when rebasing to the ticket798 branch, which caused base directories to be duplicated in self.basedirs.
2010-08-01 23:49:29 -07:00
david-sarah
f952532f08
Basedir/node directory option improvements for ticket798 branch. addresses #188 , #706 , #715 , #772 , #890
2010-08-01 21:30:04 -07:00
david-sarah
16647b49f6
scripts/create_node.py: allow nickname to be Unicode. Also ensure webport is validly encoded in config file.
2010-08-01 17:02:12 -07:00
david-sarah
9aae9208e3
test_runner.py: Fix error in message arguments to 'fail' calls.
2010-08-01 18:35:26 -07:00
david-sarah
02199db615
Additional Unicode basedir changes for ticket798 branch.
2010-08-01 18:05:52 -07:00
david-sarah
d3a8ef762f
Unicode basedir changes for ticket798 branch.
2010-08-01 16:53:10 -07:00
david-sarah
013954c265
fileutil: change WindowsError to OSError in abspath_expanduser_unicode, because WindowsError might not exist.
2010-07-25 15:26:03 -07:00
david-sarah
2b33704d61
test_system: correct a failure in _test_runner caused by Unicode basedir patch on non-Unicode platforms.
2010-07-23 20:21:23 -07:00
david-sarah
f036dfaa4b
Fix test failures due to Unicode basedir patches.
2010-07-24 18:03:18 -07:00
david-sarah
28e6ad51a7
util.encodingutil: change quote_output to do less unnecessary escaping, and to use double-quotes more consistently when needed. This version avoids u-escaping for characters that are representable in the output encoding, when double quotes are used, and includes tests. fixes #1135
2010-07-23 00:53:14 -07:00
david-sarah
618db4867c
Replace uses of os.path.abspath with abspath_expanduser_unicode where necessary. This makes basedir paths consistently represented as Unicode.
2010-07-21 17:14:18 -07:00
david-sarah
11b18824c7
util.fileutil, test.test_util: add abspath_expanduser_unicode function, to work around < http://bugs.python.org/issue3426 >. util.encodingutil: add a convenience function argv_to_abspath.
2010-07-21 16:15:07 -07:00
Zooko O'Whielacronx
24c72f2942
upload: tidy up logging messages
...
reformat code to be less than 100 chars wide, refactor formatting of logging messages, add log levels to some logging messages, M-x whitespace-cleanup
2010-08-02 00:02:12 -07:00
jacob.lyles
e276050d5a
abbreviate time edge case python2.5 unit test
2010-07-29 14:06:38 -07:00
david-sarah
23c649c897
test_upload.py: rename test_problem_layout_ticket1124 to test_problem_layout_ticket_1124 -- fix .todo reference.
2010-07-29 08:29:27 -07:00
david-sarah
0ebe18cc73
test_upload.py: rename test_problem_layout_ticket1124 to test_problem_layout_ticket_1124 for consistency.
2010-07-29 07:22:50 -07:00
Zooko O'Whielacronx
28822dfb4e
tests: remove debug print
2010-08-01 23:33:39 -07:00
Zooko O'Whielacronx
69c48ebde6
immutable: tidy-up some code by using a set instead of list to hold homeless_shares
2010-08-01 23:20:04 -07:00
Zooko O'Whielacronx
fd874d6669
setup: fix a couple instances of hard-coded 'allmydata-tahoe' in the scripts, tighten the tests (as suggested by David-Sarah)
2010-08-01 09:42:07 -07:00
Zooko O'Whielacronx
44edecb36c
setup: replace hardcoded 'allmydata-tahoe' with allmydata.__appname__
2010-08-01 09:05:17 -07:00
Zooko O'Whielacronx
0f94923f22
immutable: use PrefixingLogMixin to organize logging in Tahoe2PeerSelector and add more detailed messages about peer
2010-07-19 01:20:00 -07:00
Zooko O'Whielacronx
2ebe2c2ff3
benchmarking: update bench_dirnode to be correct and use the shiniest new pyutil.benchutil features concerning what units you measure in
2010-07-18 21:49:48 -07:00
Zooko O'Whielacronx
9b4f412e49
trivial: rename and add in-line doc to clarify "used_peers" => "upload_servers"
2010-07-18 21:47:44 -07:00
jacob.lyles
d3e6e15e23
web: don't use %d formatting on a potentially large negative float -- there is a bug in Python 2.5 in that case
...
fixes #1055
2010-07-30 15:05:50 -07:00
Zooko O'Whielacronx
1c62db0db8
docs: use current cap to Zooko's wiki page in example text
...
fixes #1134
2010-07-20 18:05:43 -07:00
david-sarah
94bec75dda
__init__.py: silence DeprecationWarning about BaseException.message globally. fixes #1129
2010-07-19 18:19:39 -07:00
david-sarah
bdc6fd2f8b
test_runner: test that 'tahoe --version' outputs no noise (e.g. DeprecationWarnings).
2010-07-19 18:13:45 -07:00
Zooko O'Whielacronx
c076c0b19b
immutable: add test case of #1128 , fix test case of #1118
2010-07-19 01:16:12 -07:00
Zooko O'Whielacronx
13b5e44fbc
immutable: fix bug in which preexisting_shares and merged were shallowly referencing the same sets
...
This bug had the effect of making uploads sometimes (rarely) appear to succeed when they had actually not distributed the shares well enough to achieve the desired servers-of-happiness level.
2010-07-19 00:54:26 -07:00
david-sarah
461f946457
upload.py: fix #1118 by aborting newly-homeless buckets when reassignment runs. This makes a previously failing assert correct. This version refactors 'abort' into two methods, rather than using a default argument.
2010-07-18 21:46:55 -07:00
Zooko O'Whielacronx
1dbfcf753d
immutable: extend the tests to check that the shares that got uploaded really do make a sufficiently Happy distribution
...
This patch also renames some instances of "find_shares()" to "find_all_shares()" and other instances to "find_uri_shares()" as appropriate -- the conflation between those names confused me at first when writing these tests.
2010-07-18 21:50:47 -07:00
Zooko O'Whielacronx
2e83f243c2
immutable: test for #1118
2010-07-18 15:15:37 -07:00
Zooko O'Whielacronx
0f46766a51
immutable: test for #1124
2010-07-18 15:29:07 -07:00
Zooko O'Whielacronx
8f201c75f0
trivial: fix unused import (sorry about that, pyflakes)
2010-07-18 14:51:33 -07:00
Zooko O'Whielacronx
16bb529339
tests, NEWS, CREDITS re: #1117
...
Give Brian and Kevan promotions, move release date in NEWS to the 18th, commit Brian's test for #1117 .
fixes #1117
2010-07-18 13:32:25 -07:00
Kevan Carstensen
054c4553b5
test/test_upload.py: test to see that aborted buckets are ignored by the storage server
2010-07-15 17:10:46 -07:00
Kevan Carstensen
677f3b1f63
test/test_storage.py: test for the new remote_abort semantics.
2010-07-15 16:21:48 -07:00
Kevan Carstensen
6374f4307f
storage/immutable.py: make remote_abort btell the storage server about aborted buckets.
2010-07-15 16:21:05 -07:00
Kevan Carstensen
5becaf0309
test/test_upload.py: changes to test plumbing for #1117 tests
...
- Add a callRemoteOnly method to FakeBucketWriter.
- Change the abort method in FakeBucketWriter to not return a
RuntimeError.
2010-07-15 16:18:20 -07:00
Kevan Carstensen
6d2c216b8f
immutable/upload.py: abort buckets if peer selection fails
2010-07-15 16:17:14 -07:00
david-sarah
638a3b2af4
test_encodingutil: correct an error in the previous patch to StdlibUnicode.test_open_representable.
2010-07-18 08:14:20 -07:00
david-sarah
74c41ebb8b
test_dirnode and test_web: don't use failUnlessReallyEqual in cases where the return type from simplejson.loads can vary between unicode and str. Use to_str when comparing URIs parsed from JSON.
2010-07-18 07:29:15 -07:00
david-sarah
9a4a864573
test_encodingutil: StdlibUnicode.test_open_representable no longer uses a mock.
2010-07-18 05:54:12 -07:00
Zooko O'Whielacronx
7cc98759bd
docs: add comment clarifying #1051
2010-07-17 22:32:50 -07:00
david-sarah
d346e0853d
Add tests of caps from the future that have non-ASCII characters in them (encoded as UTF-8). The changes to test_uri.py, test_client.py, and test_dirnode.py add tests of non-ASCII future caps in addition to the current tests. The changes to test_web.py just replace the tests of all-ASCII future caps with tests of non-ASCII future caps. We also change uses of failUnlessEqual to failUnlessReallyEqual, in order to catch cases where the type of a string is not as expected.
2010-07-11 13:02:52 -07:00
Zooko O'Whielacronx
3c4fe0706f
M-x whitespace-cleanup
2010-07-17 20:27:39 -07:00
Zooko O'Whielacronx
6a7023efbe
benchmarking: update bench_dirnode.py to reflect the new directory interfaces
2010-07-17 20:17:10 -07:00
david-sarah
bdb10553eb
test_encodingutil: fix test_open_representable, which is only valid when run on a platform for which we know an unrepresentable filename.
2010-07-17 20:03:33 -07:00
francois
968edfda11
iputil.py: Add support for FreeBSD 7,8 and 9
...
committed by: zooko@zooko.com
date: 2010-07-17
and I also patched: NEWS and CREDITS
2010-07-17 19:28:32 -07:00
david-sarah
752897450d
fileutil: docstrings for non-obvious usage restrictions on methods of EncryptedTemporaryFile.
2010-07-16 22:46:47 -07:00
david-sarah
05022dca36
Move EncryptedTemporaryFile from SFTP frontend to allmydata.util.fileutil, and make the FTP frontend also use it (fixing #1083 ).
2010-07-11 14:37:21 -07:00
david-sarah
a8161c915a
Correct stringutils->encodingutil patch to be the newer version, rather than the old version that was committed in error.
2010-07-17 18:34:35 -07:00
david-sarah
0d79a4a7d1
test_cli.py: fix error that crept in when rebasing the patch for #1072 .
2010-07-17 17:01:23 -07:00
david-sarah
c59f9b162f
stringutils: add test for when sys.stdout has no encoding attribute ( fixes #1099 ).
2010-07-16 21:58:16 -07:00
david-sarah
8090785716
CLI: add 'tahoe unlink' as an alias to 'tahoe rm', for forward-compatibility.
2010-07-17 15:04:11 -07:00
Zooko O'Whielacronx
6e8477114e
minor code clean-up in dirnode.py
...
Impose micro-POLA by passing only the writekey instead of the whole node object to {{{_encrypt_rw_uri()}}}. Remove DummyImmutableFileNode in nodemaker.py, which is obviated by this. Add micro-optimization by precomputing the netstring of the empty string and branching on whether the writekey is present or not outside of {{{_encrypt_rw_uri()}}}. Add doc about writekey to docstring.
fixes #967
2010-07-13 23:02:55 -07:00
david-sarah
11077ea74d
Rename stringutils to encodingutil, and drop listdir_unicode and open_unicode (since the Python stdlib functions work fine with Unicode paths). Also move some utility functions to fileutil.
2010-07-11 17:30:15 -07:00
david-sarah
fa0fd66e17
Allow URIs passed in the initial JSON for t=mkdir-with-children, t=mkdir-immutable to be Unicode. Also pass the name of each child into nodemaker.create_from_cap for error reporting.
2010-07-11 12:55:25 -07:00
david-sarah
e2ef2c2691
cli.py: make command descriptions consistently end with a full stop.
2010-07-13 18:45:38 -07:00
david-sarah
15ddab08ed
SFTP: address some of the comments in zooko's review ( #1106 ).
2010-07-11 19:55:37 -07:00
david-sarah
3b1b0147a8
test_storage.py: potential fix for failures when logging is enabled.
1970-07-12 21:05:46 -07:00
terrellrussell
d0381e679e
upcase_since_on_welcome
2010-07-08 12:39:03 -07:00
freestorm77
35ec8f6ac2
server_version_on_welcome_page.dpatch.txt
...
- The storage server version is 0 for all storage nodes in the Welcome Page
2010-06-05 12:17:21 -07:00
freestorm77
496b91717a
directory_html_top_banner.dpatch
...
The div tag with the link "Return to Welcome page" on the directory.xhtml page is not correct
2010-06-22 13:53:01 -07:00
freestorm77
eb63ba9a26
tahoe_css_toolbar.dpatch
...
CSS modification to be correctly diplayed with Internet Explorer 8
The links on the top of page directory.xhtml are not diplayed in the same line as display with Firefox.
2010-06-22 14:00:46 -07:00
freestorm77
97579af73c
runnin_test_tahoe_css.dpatch
...
Runnin test for changes in tahoe.css file
2010-06-22 14:47:14 -07:00
freestorm77
0304918a77
runnin_test_directory_xhtml.dpatch
...
Runnin test for diretory.xhtml top banner
2010-06-22 13:14:03 -07:00
david-sarah
b8df25a8d1
stringutils.py: tolerate sys.stdout having no 'encoding' attribute.
2010-06-25 21:08:17 -07:00
david-sarah
88169328be
SFTP: don't call .stopProducing on the producer registered with OverwriteableFileConsumer (which breaks with warner's new downloader).
2010-06-28 16:19:26 -07:00
Zooko O'Whielacronx
ab80719198
trivial: tiny update to in-line comment
...
(I'm actually committing this patch as a test of the new eager-annotation-computation of trac-darcs.)
2010-06-13 21:57:15 -07:00
david-sarah
6b2f99fa9a
test_backupdb.py: skip test_unicode if we can't represent the test filenames.
2010-06-18 19:26:20 -07:00
david-sarah
a2caf6f4b2
test_web.py: correct a test that was missed in the change to not write ctime/mtime.
2010-06-18 19:17:18 -07:00
david-sarah
4712875193
dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.
2010-06-18 16:01:19 -07:00
david-sarah
72e395d878
test_storage.py: print more information on test failures.
2010-06-16 20:46:23 -07:00
david-sarah
390fc78a9a
test_cli.py: fix test failure in CLI.test_listdir_unicode_good due to filenames returned from listdir_unicode no longer being normalized.
2010-06-17 21:51:10 -07:00
david-sarah
178401eb4e
tahoe backup: unicode tests.
2010-06-17 20:52:11 -07:00
david-sarah
1a0674bf37
CLI: allow Unicode patterns in exclude option to 'tahoe backup'.
2010-06-16 20:39:01 -07:00
david-sarah
a9fe3792de
dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.
2010-06-17 17:02:49 -07:00
david-sarah
718870a796
dirnode.py: comments about normalization changes.
2010-06-16 21:14:11 -07:00
david-sarah
7e7644589a
stringutils.py: remove unused import.
2010-06-16 20:44:40 -07:00
david-sarah
5ada31034b
test_stringutils.py: take account of the output of listdir_unicode no longer being normalized. Also use Unicode escapes, not UTF-8.
2010-06-16 20:44:09 -07:00
david-sarah
025aede9e4
test_dirnode.py: partial tests for normalization changes.
2010-06-16 20:40:25 -07:00
david-sarah
c1412c8a81
SFTP: get 'ctime' attribute from 'tahoe:linkmotime'.
2010-06-16 20:37:44 -07:00
david-sarah
9f5488b2d1
stringutils.py: don't NFC-normalize the output of listdir_unicode.
2010-06-16 18:55:37 -07:00
david-sarah
e2c7ad1d88
stringutils.py: Add encoding argument to quote_output. Also work around a bug in locale.getpreferredencoding on older Pythons.
2010-06-15 21:20:12 -07:00
david-sarah
c8d99b77a3
Provisional patch to NFC-normalize filenames going in and out of Tahoe directories.
2010-06-15 20:14:50 -07:00
david-sarah
408b12a7e6
test_system.py: investigate failure in allmydata.test.test_system.SystemTest.test_upload_and_download_random_key due to bytes_sent not being an int
2010-06-15 17:16:48 -07:00
david-sarah
a3acb2b773
SFTP: remove a dubious use of 'pragma: no cover'.
2010-06-13 09:43:56 -07:00
david-sarah
11e88c340a
SFTP: test that renaming onto a just-opened file fails.
2010-06-11 20:37:09 -07:00
david-sarah
b67f8b66c8
SFTP: further small improvements to test coverage. Also ensure that after a test failure, later tests don't fail spuriously due to the checks for heisenfile leaks.
2010-06-11 20:07:37 -07:00
david-sarah
b4e074d250
SFTP: further improve test coverage (paths containing '.', bad data for posix-rename extension, and error in test of openShell).
2010-06-11 14:31:42 -07:00
david-sarah
52f87904ed
SFTP: improve test coverage for no-write on mutable files, and check for heisenfile table leaks in all relevant tests. Delete test_memory_leak since it is now redundant.
2010-06-11 13:57:52 -07:00
david-sarah
546c3d2ed4
SFTP: add test for extension of file opened with FXF_APPEND.
2010-06-10 11:26:47 -07:00
Zooko O'Whielacronx
f19c240ca8
tests: bump up the timeout on this iputil test from 2s to 4s
2010-06-09 07:30:17 -07:00
david-sarah
3c44389440
SFTP: fix most significant memory leak described in #1045 (due to a file being added to all_heisenfiles under more than one direntry when renamed).
2010-06-09 01:00:03 -07:00
david-sarah
5786874d38
test_stringutils.py: Fix test failure on CentOS builder, possibly Python 2.4.3-related.
2010-06-08 23:50:56 -07:00
david-sarah
ffd1c5aca5
Fix for Unicode-related test failures on Zooko's OS X 10.6 machine.
2010-06-08 22:54:48 -07:00
david-sarah
80a89d3931
stringutils.py, sftpd.py: Portability fixes for Python <= 2.5.
2010-06-08 18:33:02 -07:00
david-sarah
32d9deace3
Back out Windows-specific Unicode argument support for v1.7.
2010-06-08 17:08:03 -07:00
david-sarah
4b5927fdb8
SFTP: suppress NoSuchChildError if heisenfile attributes have been updated in setAttrs, in the case where the parent is available.
2010-06-07 23:37:53 -07:00
david-sarah
195b4afa8f
SFTP: ignore permissions when opening a file (needed for sshfs interoperability).
2010-06-07 22:57:00 -07:00
david-sarah
a67e745b26
test_web.py: fix pyflakes warnings introduced by byterange patch.
2010-06-07 21:20:12 -07:00
Jeremy Fitzhardinge
63b28d707b
Improve HTTP/1.1 byterange handling
...
Fix parsing of a Range: header to support:
- multiple ranges (parsed, but not returned)
- suffix byte ranges ("-2139")
- correct handling of incorrectly formatted range headers
(correct behaviour is to ignore the header and return the full
file)
- return appropriate error for ranges outside the file
Multiple ranges are parsed, but only the first range is returned.
Returning multiple ranges requires using the multipart/byterange
content type.
2010-03-09 20:59:13 -07:00
Zooko O'Whielacronx
d13784b649
tests: bump up the timeout on these tests; MM's buildslave is sometimes extremely slow on tests, but it will complete them if given enough time. MM is working on making that buildslave more predictable in how long it takes to run tests.
2010-06-07 20:37:54 -07:00
david-sarah
3c883e6e44
test_cli.py: remove invalid 'test_listdir_unicode_bad' test.
2010-06-07 11:37:30 -07:00
david-sarah
71f8757bd1
check_memory.py: adapt to servers-of-happiness changes.
2010-06-07 18:35:28 -07:00
david-sarah
7092de1b6f
Remove the 'tahoe debug consolidate' subcommand.
2010-06-07 11:37:57 -07:00
david-sarah
731e3d68df
common_http.py, tahoe_cp.py: Fix an error in calling the superclass constructor in HTTPError and MissingSourceError (introduced by the Unicode fixes).
2010-06-07 10:47:14 -07:00
Zooko O'Whielacronx
faade8b20a
tests: drastically increase timeout of this very time-consuming test in honor of François's ARM box
2010-06-07 04:59:29 -07:00
david-sarah
8b014372b1
test_stringutils.py: trivial error in exception message for skipped test.
2010-06-06 23:14:55 -07:00
david-sarah
529c9f673a
More Unicode test fixes.
2010-06-06 22:33:58 -07:00
david-sarah
65b6f4e3ce
Unicode fixes for platforms with non-native-Unicode filesystems.
2010-06-06 21:32:38 -07:00
david-sarah
80252debcd
Unicode fixes.
2010-06-06 18:02:15 -07:00
Zooko O'Whielacronx
5bcca5151e
unicode tests: fix missing import
2010-06-04 07:26:30 -07:00
Zooko O'Whielacronx
db8a6f3aa6
unicode: make test_cli test a non-ascii argument, and make the fallback term encoding be locale.getpreferredencoding()
2010-06-04 07:12:51 -07:00
Zooko O'Whielacronx
442008a690
unicode: always decode json manifest as utf-8 then encode for stdout
...
pyflakes pointed out that the exception handler fallback called an un-imported function, showing that the fallback wasn't being exercised.
I'm not 100% sure that this patch is right and would appreciate François or someone reviewing it.
2010-06-04 01:48:40 -07:00
Zooko O'Whielacronx
952f3bf3ef
fix flakes
2010-06-04 00:58:45 -07:00
Zooko O'Whielacronx
08058c1073
fix syntax of assertion handling that isn't portable to older versions of Python
2010-06-04 00:58:05 -07:00
Francois Deppierraz
0eb4d83937
test_stringutils.py: Skip test test_listdir_unicode_good if filesystem supports only ASCII filenames
2010-05-21 09:08:39 -07:00
Francois Deppierraz
01da7f06e6
test_stringutils.py: Skip test_listdir_unicode on mocked platform which cannot store non-ASCII filenames
2010-05-21 09:05:59 -07:00
Francois Deppierraz
58e9df9f5a
test_stringutils.py: Add a test class for OpenBSD 4.1 with LANG=C
2010-05-21 07:00:53 -07:00
Francois Deppierraz
1aeff62161
test_stringutils.py: Mock the open() call in test_open_unicode
...
This test ensure that open(a_unicode_string) is used on Unicode platforms
(Windows or MacOS X) and that open(a_correctly_encoded_bytestring) on other
platforms such as Unix.
2010-05-21 06:58:17 -07:00
Francois Deppierraz
eecaaca6b7
test_stringutils.py: Fix a trivial Python 2.4 syntax incompatibility
2010-05-21 02:33:45 -07:00
Francois Deppierraz
6c70fb7b3c
test_cli.py: Fix tests when sys.stdout.encoding=None and refactor this code into functions
2010-05-20 01:44:47 -07:00
Francois Deppierraz
496802420e
Fix handling of correctly encoded unicode filenames ( #534 )
...
Tahoe CLI commands working on local files, for instance 'tahoe cp' or 'tahoe
backup', have been improved to correctly handle filenames containing non-ASCII
characters.
In the case where Tahoe encounters a filename which cannot be decoded using the
system encoding, an error will be returned and the operation will fail. Under
Linux, this typically happens when the filesystem contains filenames encoded
with another encoding, for instance latin1, than the system locale, for
instance UTF-8. In such case, you'll need to fix your system with tools such
as 'convmv' before using Tahoe CLI.
All CLI commands have been improved to support non-ASCII parameters such as
filenames and aliases on all supported Operating Systems except Windows as of
now.
2010-05-19 17:43:56 -07:00
Francois Deppierraz
d0ed14e1bb
stringutils.py: Unicode helper functions + associated tests
...
This file contains a bunch of helper functions which converts
unicode string from and to argv, filenames and stdout.
2010-05-19 17:41:05 -07:00
david-sarah
a2cff856ff
Resolve merge conflict for sftpd.py
2010-06-03 11:25:37 -07:00
david-sarah
2dcb62de90
SFTP: possible fix for metadata times being shown as the epoch.
2010-06-02 16:45:14 -07:00
david-sarah
c9984c40e9
SFTP: further improvements to test coverage.
2010-06-02 16:44:22 -07:00
david-sarah
2a791b0d05
SFTP: improve test coverage. Also make creating a directory fail when permissions are read-only (rather than ignoring the permissions).
2010-06-01 21:19:34 -07:00
david-sarah
29a06457d2
dirnode.py: fix a bug in the no-write change for Adder, and improve test coverage. Add a 'metadata' argument to create_subdirectory, with documentation. Also update some comments in test_dirnode.py made stale by the ctime/mtime change.
2010-06-01 20:26:41 -07:00
david-sarah
db394671e9
SFTP: fix a bug that caused the temporary files underlying EncryptedTemporaryFiles not to be closed.
2010-05-31 22:53:10 -07:00
david-sarah
de95140b7b
SFTP: changes for #1063 ('no-write' field) including comment:1 (clearing owner write permission diminishes to a read cap). Includes documentation changes, but not tests for the new behaviour.
2010-05-31 22:11:39 -07:00
david-sarah
9faa02deb5
SFTP: the same bug as in _sync_heisenfiles also occurred in two other places.
2010-05-29 23:01:27 -07:00
david-sarah
cccdd32806
SFTP: another try at fixing the _sync_heisenfiles bug.
2010-05-29 22:52:54 -07:00
david-sarah
51b0d8bde3
SFTP: fix silly bug in _sync_heisenfiles ('f is not ignore' vs 'not (f is ignore)').
2010-05-29 22:38:07 -07:00
david-sarah
b9d3cf12b8
SFTP: log when a sync completes.
2010-05-29 22:18:40 -07:00
david-sarah
c4a8af4d27
SFTP: fix bug in previous logging patch.
2010-05-29 22:00:00 -07:00
david-sarah
3395a7e154
SFTP: more logging to track down OpenOffice hang.
2010-05-29 21:08:09 -07:00
david-sarah
4be24a89df
SFTP: avoid blocking close on a heisenfile that has been abandoned or never changed. Also, improve the logging to help track down a case where OpenOffice hangs on opening a file with FXF_READ|FXF_WRITE.
2010-05-29 19:55:44 -07:00
david-sarah
8ed28a95e7
Move suppression of DeprecationWarning about BaseException.message from sftpd.py to main __init__.py. Also, remove the global suppression of the 'integer argument expected, got float' warning, which turned out to be a bug.
2010-05-28 22:05:37 -07:00
david-sarah
e867985539
SFTP: cater to clients that assume a file is created as soon as they have made an open request; also, fix some race conditions associated with closing a file at about the same time as renaming or removing it.
2010-05-28 21:52:53 -07:00
david-sarah
0b888f8201
SFTP: 'sync' any open files at a direntry before opening any new file at that direntry. This works around the sshfs misbehaviour of returning success to clients immediately on close.
2010-05-25 16:02:57 -07:00
david-sarah
a143b1297b
SFTP: handle removing a file while it is open. Also some simplifications of the logout handling.
2010-05-25 11:42:10 -07:00
david-sarah
e81ce2bbd2
SFTP: a posix-rename response should actually return an FXP_STATUS reply, not an FXP_EXTENDED_REPLY as Twisted Conch assumes. Work around this by raising an SFTPError with code FX_OK.
2010-05-24 20:33:23 -07:00
david-sarah
8f86490d48
SFTP: fix problem with posix-rename code returning a Deferred for the renamed filenode, not for the result of the request (an empty string).
2010-05-24 19:02:09 -07:00
david-sarah
210578da63
SFTP: fix time handling to make sure floats are not passed into twisted.conch, and to print times in the future less ambiguously in directory listings.
2010-05-24 16:04:12 -07:00
david-sarah
549dccd608
SFTP: name of the POSIX rename extension should be 'posix-rename@openssh.com', not 'extposix-rename@openssh.com'.
2010-05-23 19:11:56 -07:00
david-sarah
6e114e44b5
SFTP: avoid race condition where .write could be called on an OverwriteableFileConsumer after it had been closed.
2010-05-23 16:38:30 -07:00
david-sarah
c6b414c815
SFTP: log tracebacks for RAISEd exceptions.
2010-05-23 15:15:35 -07:00
david-sarah
725daee788
SFTP: more logging to investigate behaviour of getAttrs(path).
2010-05-23 13:42:36 -07:00
david-sarah
75357fd2d5
SFTP: fix pyflakes warnings; drop 'noisy' versions of eventually_callback and eventually_errback; robustify conversion of exception messages to UTF-8.
2010-05-23 07:09:05 -07:00
david-sarah
38964fb35a
SFTP: fixes and test cases for renaming of open files.
2010-05-22 20:25:49 -07:00
david-sarah
42ecd77f4e
SFTP: Increase test_sftp timeout to cater for francois' ARM buildslave.
2010-05-22 12:16:39 -07:00
david-sarah
5974773969
SFTP: Fix error in support for getAttrs on an open file, to index open files by directory entry rather than path. Extend that support to renaming open files. Also, implement the extposix-rename@openssh.org extension, and some other minor refactoring.
2010-05-21 20:58:36 -07:00
david-sarah
5545a678bf
SFTP tests: fix test_openDirectory_and_attrs that was failing in timezones west of UTC.
2010-05-20 11:10:27 -07:00
david-sarah
ce3872d10e
SFTP: allow getAttrs to succeed on a file that has been opened for creation but not yet uploaded or linked (part of #1050 ).
2010-05-19 20:56:13 -07:00