ExternalVendorCode/tahoe-lafs
1
0
Fork 0
mirror of https://github.com/tahoe-lafs/tahoe-lafs.git synced 2024-12-27 16:28:53 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity
5,168 Commits 185 Branches 90 Tags 89 MiB
1576c35d38
Commit Graph

2940 Commits

Author SHA1 Message Date
david-sarah
420aadd95e Make 'mock' a run-time rather than setup-time dependency. This is necessary in order for 'tahoe debug trial' to work. refs #1296 2011-01-18 12:51:14 -08:00
david-sarah
7a887871b0 src/allmydata/scripts/debug.py: add 'tahoe debug trial' command (rebased for trunk). refs #1296 2011-01-18 12:46:59 -08:00
david-sarah
fc98aded8a src/allmydata/webish.py: clean-ups and correction to a comment. Also change an open and write to use fileutil.write. See ref #1286 comment 13. 2011-01-17 15:31:52 -08:00
Brian Warner
980fb0b2bb 'tahoe debug catalog-shares': sort SIs and shnums 
Without this, SIs or shnums could be emitted in random order, depending upon
what the filesystem happens to return.
 2011-01-17 01:59:32 -08:00
david-sarah
5a3acc05b7 CLI: tests for ref #1305 (v2, remove spurious extra arg to create-alias in test) 2011-01-13 20:03:27 -08:00
david-sarah
f10a2c0902 CLI: make 'tahoe create-alias' and 'tahoe add-alias' accept a trailing colon on the new alias name (v2, minor change not to rely on implicit Unicode conversion). Includes doc changes and news; tests in a separate patch. fixes #1305 2011-01-13 19:44:14 -08:00
Brian Warner
b9fb74d2a0 Improve 'tahoe ln' help text. Patch by David-Sarah. Closes #1230. 2011-01-17 00:14:21 -08:00
Brian Warner
09a2241471 Tolerate Twisted-10.2's endpoints, patch by David-Sarah. Closes #1286. 
The service generated by strports.service() changed in 10.2, and the ugly
private-attribute-reading hack we used to glean a kernel-allocated port
number (e.g. when using "tcp:0", especially during unit tests) broke, causing
Tahoe to be completely unusable with Twisted-10.2 . The new ugly
private-attribute-reading hack starts by figuring out what sort of service
was generated, then reads different attributes accordingly.

This also hushes a warning when using schemeless strports strings like "0" or
"3456", by quietly prepending a "tcp:" scheme, since 10.2 complains about
those. It also adds getURL() and getPortnum() accessors to the "webish"
service, rather than having unit tests dig through _url and _portnum and such
to find out what they are.
 2011-01-16 23:47:51 -08:00
david-sarah
8a3cb60bf7 Test changes to take account of ref #1311. 2011-01-16 22:05:40 -08:00
david-sarah
d3f24736ca create_node.py: add comments to default tahoe.cfg to clarify the meaning of each section. fixes #1311 2011-01-16 21:24:19 -08:00
david-sarah
e52d97a917 Undo the temporary hack to check the foolscap version. refs #1246 2011-01-16 21:20:42 -08:00
Brian Warner
45212d8000 Set "reserved_space=1G" in newly-created storage nodes. Closes #1208. 2011-01-16 12:58:22 -08:00
david-sarah
727b25f622 Temporary hack to investigate whether we are getting the right version of foolscap on trunk. refs #1258 2011-01-15 20:49:59 -08:00
david-sarah
e62a69e7ff test_runner: avoid unnecessary use of non-ASCII. 2011-01-01 02:01:01 -08:00
david-sarah
2c099ffa0d Move dependency imports from windows/depends.py (which has gone away) into src/allmydata/windows/tahoesvc.py. Also fix a pyflakes warning, and change the service display name from 'Allmydata Tahoe Node' to 'Tahoe-LAFS node'. refs #1282 2010-12-25 20:21:00 -08:00
david-sarah
3132c9b593 Remove unmaintained Windows GUI app, except for windows/tahoesvc.py which is moved to src/allmydata/windows. refs #1282 2010-12-25 20:02:37 -08:00
david-sarah
195ddbcf7c Remove unmaintained Mac GUI app. fixes #1282 2010-12-25 18:05:08 -08:00
david-sarah
399ee5aea0 Remove src/allmydata/util/find_exe.py which is no longer used. fixes #1150 2010-12-25 18:32:06 -08:00
freestorm77
4027a136f3 status_web_pages_review.darcs.patch 
I make some changes on status web pages

status.xhtml:
- Delete unused webform_css link
- Align tables on the left

tahoe-css:
- Do some minor changes on code synthax
- changes table.status-download-events style to look like other tables

status.py:
- Align table on the left
- Changes table header
- Add heading tags
- Modify google api graph: add image border, calculate height to feet data

signed-off-by: zooko@zooko.com
fixes #1219
 2011-01-01 19:42:14 -08:00
david-sarah
5027753e23 test_storage.py: fix a pyflakes unused import warning. 2010-12-31 14:07:56 -08:00
david-sarah
0da10d4c37 test_storage.py: leave at least 512 MiB free when running test_large_share. refs #1195 2010-12-31 12:32:15 -08:00
Zooko O'Whielacronx
123a1a3f04 storage: use fileutil's version of get_disk_stats() and get_available_space(), use mockery/fakery in tests, enable large share test on platforms with sparse files and if > 4 GiB of disk space is currently available 2010-09-10 08:36:29 -08:00
Zooko O'Whielacronx
d5e71c2940 fileutil: copy in the get_disk_stats() and get_available_space() functions from storage/server.py 2010-09-10 08:35:20 -08:00
david-sarah
c82b48f3d6 Update foolscap version requirement to 0.6.0, to address http://foolscap.lothar.com/trac/ticket/167 2010-12-30 22:00:39 -08:00
david-sarah
2d5f3f694f scripts/backupdb.py: more accurate comment about path field. 2010-12-12 09:03:20 -08:00
david-sarah
bb7c04ff9f scripts/cli.py: fix missing 'put' in usage example for 'tahoe put'. 2010-12-12 09:02:07 -08:00
david-sarah
ee4e0d8106 scripts/cli.py: remove the disclaimer in the help for 'tahoe cp' that it does not handle non-ASCII filenames well. (At least, we intend to handle them.) 2010-11-29 16:21:45 -08:00
david-sarah
9ceb527787 Change another doc reference from .txt to .rst in a text string. 2010-11-28 09:45:07 -08:00
david-sarah
d7bdf476b9 Change a doc reference from .txt to .rst in a text string. 2010-11-28 09:34:44 -08:00
Brian Warner
69b42c6cb7 remove --multiple/-m option from all CLI commands: closes #1262 
I personally used "tahoe start/restart -m ../MY-TESTNET/node*" all the time,
to spin up or update a local testgrid while iterating over new code. However,
with the recent switch from "subprocess.Popen(/bin/twistd)" to "import and
call twistd.run()" in scripts/startstop_node.py (yay fewer processes!),
"start -m" broke, and fixing it requires os.fork, which is unavailable on
windows (boo windows!). And I was probably the only one using -m. So in the
interests of uniformity among platforms and simpler code (yay negative code
days!), we're just removing -m from everything. I will start using a little
shell script or something to simulate the removed functionality.

This patch also cleans up CLI-function calling a bit: get the basedir from
the config dict (instead of sometimes from a separate argument), and always
return a numeric exit code.
 2010-11-27 00:38:09 -08:00
Brian Warner
f36bda2780 Revert previous commit: there's an ugly corner-case on windows that fails tests. 
Specifically, test_runner.CreateNode.test_client failed, because the
os.fork-is-present test decided that --multiple should not be allowed on
windows, even though --multiple works just fine for 'tahoe create-client'.
The only restriction on --multiple is for 'tahoe start' and 'tahoe restart'.

This needs a different approach, probably by cleaning up BasedirMixin. We
should only be withholding --multiple on windows for "start" and
"restart". (we should continue withholding --multiple on all platforms for
"run").

This reverts (git) commit f3adb037ae:
 "startstop_node.py: fix "tahoe start -m" by forking before non-final targets"
 2010-11-26 16:44:11 -08:00
Brian Warner
f3adb037ae startstop_node.py: fix "tahoe start -m" by forking before non-final targets 
* don't advertise -m flag on tahoe start/restart/run unless os.fork is
  available (i.e. windows)
* test_runner.py: add test to exercise "start/stop/restart -m"
 2010-11-26 14:45:38 -08:00
david-sarah
38a0d3f5fa tahoe_mv.py: when checking success of the DELETE operation, look at the status code from DELETE rather than from the previous PUT. fixes #1255 2010-11-09 17:09:16 -08:00
david-sarah
fdf6946669 test_cli.py: test that 'tahoe mv' reports errors from the DELETE operation. refs #1255 2010-11-11 17:46:53 -08:00
Zooko O'Whielacronx
93dc740f01 tests: bump up the timeout on test_dirnode.Dirnode from 240s to 480s since it apparently took longer than 240s just now on François's ARM buildslave 2010-11-15 01:21:19 -08:00
david-sarah
76371ae5a9 test_cli.py: fix a stale comment that incorrectly implied that test_cli runs CLI commands in subprocesses (it actually runs them using deferToThread). 2010-11-09 18:28:19 -08:00
david-sarah
cb777ad14f tahoe_mv.py: use do_http, not urllib.openurl, in order to avoid connecting to the webapi server via a proxy. refs #1253 2010-11-09 16:53:36 -08:00
david-sarah
01a5365051 test_system.py: test for 'tahoe mv' with the http_proxy and HTTP_PROXY variables set. refs #1253 2010-11-09 19:36:49 -08:00
david-sarah
edec9c9869 test_client.py: change overzealous use of failUnlessReallyEqual on integers back to failUnlessEqual. 2010-11-09 15:08:16 -08:00
david-sarah
56f4c1218b test_runner.py: fix test failure in test_the_right_code after applying zooko's change to test-with-fake-pkg.py 2010-11-01 20:59:05 -07:00
Zooko O'Whielacronx
a92912f513 setup: FreeStorm's WinXP-x86-py2.6 buildslave has informed us that there is yet a fourth way to spell "i386" in Python 2010-10-31 22:21:42 -07:00
david-sarah
b8905fc766 allmydata/__init__.py: move the call to require_auto_deps() to the top again, since the [4784] patch turned out not to be the cause of the failure on the 'mm netbsd5' buildslave. 2010-10-31 17:33:16 -07:00
david-sarah
63c010a9ca allmydata/__init__.py: put the _auto_deps.require_auto_deps() call back in exactly the same place it was prior to [4784]. 2010-10-31 10:46:49 -07:00
david-sarah
7e2ed2054c allmydata/__init__.py: suppress a DeprecationWarning for the sha module on importing pycrypto. 2010-10-31 09:06:36 -07:00
david-sarah
6e76653913 allmydata/__init__.py: call require_auto_deps() after importing nevow and twisted, reverting change in [4784]. Also fix a missing 'warnings.filters.pop()'. 2010-10-31 08:38:28 -07:00
david-sarah
2ab0b1aa2c allmydata.__init__.py: temporary hack to debug failure on midnightmagic's buildslave 2010-10-30 22:50:03 -07:00
david-sarah
6eaa7f2356 tahoe backup: perform tilde expansion in exclude-from filename (version 2). fixes #1241 2010-10-30 20:52:31 -07:00
Zooko O'Whielacronx
3b2d0d9609 setup: run require_auto_deps() before attempting to import any deps in __init__.py 
For one thing, this makes missing-dependency failures into DistributionNotFound errors instead of ImportErrors, which might be more useful to the user. For another thing, if someone is using distributions that were installed with --multi-version, then they might be not importable until after require_auto_deps() has been run. (The docs claim that this would be the case, but we don't have an example of this happening at this time.)
 2010-10-30 01:10:35 -07:00
david-sarah
1950d5a719 scripts/runner.py: fix unused import of allmydata. refs #1190 2010-10-29 17:31:49 -07:00
david-sarah
2a8f700026 scripts/runner.py: remove pkg_resources.require() calls. These are at best redundant because we have already called _auto_deps.require_auto_deps() (from allmydata.__init__) at that point, and they are causing failure of the test-from-prefixdir step on some buildslaves. refs #1190 2010-10-29 16:53:28 -07:00
david-sarah
10a5f23df9 test_runner.py: fix error in BinTahoe.test_version_no_noise introduced by last patch. refs #1235 2010-10-29 15:11:23 -07:00
david-sarah
d249589667 test_runner.py: also allow 'from pkg_resources import load_entry_point' as noise. refs #1235. 2010-10-29 13:42:46 -07:00
david-sarah
8c95f88b4f test_runner.py: if the only noise is 'UserWarning: Unbuilt egg for setuptools', skip instead of failing the no_noise tests. This version avoids 'any' to be compatible with Python < 2.5. refs #1235. 2010-10-29 12:18:04 -07:00
david-sarah
4061258c85 make ResponseCache smarter to avoid memory leaks: don't record timestamps, use DataSpans to merge entries, and clear the cache when we see a new seqnum. refs #1045, #1229 2010-10-26 21:33:02 -07:00
david-sarah
25d8103dde windows/fixups.py: limit length of string passed in a single call to WriteConsoleW. fixes #1232. 2010-10-26 19:16:36 -07:00
Brian Warner
c88eb097d9 scripts/tahoe_backup.py: oops, fix missing import, thanks pyflakes 
test_cli.py: hush minor pyflakes complaint
 2010-10-29 02:42:23 -07:00
Brian Warner
d61d41147c mutable/servermap.py: update comment. Closes #1231. 2010-10-29 02:14:24 -07:00
Francois Deppierraz
14ee763c54 tahoe_cp.py: Don't call urllib.quote with an Unicode argument, fix #1224 
tahoe_backup.py: Fix another (potential) occurrence of calling urllib.quote()
with an Unicode parameter
 2010-10-29 01:45:20 -07:00
Brian Warner
c18953c169 fix #1223, crash+inefficiency during repair due to read overrun 
* repairer (really the uploader) reads beyond end of input file (Uploadable)
* new-downloader does not tolerate overreads
* uploader does lots of tiny reads (inefficient)

This fixes the last two. The uploader still does a single overread at the end
of the input file, but now that's ok so we can leave it in place. The
uploader now expects the Uploadable to behave like a normal disk
file (reading beyond EOF will return less data than was asked for), and now
the new-downloadable behaves that way.
 2010-10-29 01:20:36 -07:00
david-sarah
270322ad47 startstop_node.py: pyflakes import fix. refs #1190 2010-10-27 18:48:05 -07:00
Brian Warner
ac3b26ecf2 'tahoe start': use import+call rather than find+spawn 
This removes the need to use a locally-built (dependency) bin/twistd, and
removes a big chunk of behavior differences between unix and windows. It
also happens to resolve the "client node probably started" uncertainty.
Might help with #1190, #602, and #71.
 2010-10-26 23:13:42 -07:00
david-sarah
59e80b2ec9 test_mutable.py: add test for ResponseCache memory leak. refs #1045, #1129 2010-10-24 12:34:09 -07:00
david-sarah
daf2b8314a test_encodingutil.py: test_argv_to_unicode modified the wrong encoding variable. fixes #1214 2010-10-22 20:58:10 -07:00
Zooko O'Whielacronx
0c2397523b doc: add explanation of the motivation for the surprising and awkward API to erasure coding 2010-10-14 23:02:02 -07:00
Zooko O'Whielacronx
98ffbfb31f setup: add --multi-version to the "setup.py develop" command-line 
fixes #530. I earlier tried this twice (see #530 for history) and then twice rolled it back due to some problems that arose. However, I didn't write down what the problems were in enough detail on the ticket that I can tell today whether those problems are still issues, so here goes the third attempt. (I did write down on the ticket that it would not create site.py or .pth files in the target directory with --multi-version mode, but I didn't explain why *that* was a problem.)
 2010-10-05 11:23:50 -07:00
Zooko O'Whielacronx
0c6bb0178b setup: use execfile to access _auto_deps.py in its proper location of src/allmydata/ instead of copying it into place when setup.py is executed 2010-09-05 22:57:14 -07:00
Zooko O'Whielacronx
e6a380241a trivial: M-x whitespace-cleanup 2010-09-03 07:47:12 -07:00
Zooko O'Whielacronx
cb83f2e41c minor: remove unused interface declaration, change allmydata.org to tahoe-lafs.org in email address, fix wording in relnotes.txt 2010-09-30 08:37:08 -07:00
Kevan Carstensen
ec4f87a98c immutable/repairer.py: don't use the default happiness setting when repairing 2010-09-27 13:01:02 -07:00
david-sarah
6c4ba62c5d test_download.py: simplify and possibly fix test_lost_servers; refs #1197 2010-09-11 17:32:51 -07:00
david-sarah
b23a7fcc10 test_runner.py: fix spurious failures of test_path on case-insensitive filesystems, and restore the check that the version is not 'unknown'. 2010-09-11 15:19:13 -07:00
Zooko O'Whielacronx
fd17d63c95 test: make tests stop relying on pyutil version class accepting the string 'unknown' for its version, and make them forward-compatible with the future Python Rational Version Numbering standard 2010-09-10 08:41:35 -07:00
Brian Warner
0475bd8e27 immutable download: have the finder inform its share consumer "no more shares" in a subsequent tick, thus avoiding accidentally telling it "no more shares" now and then telling it "here's another share" in a subsequent tick 
fixes #1191
Patch by Brian. This patch description was actually written by Zooko, but I forged Brian's name on the "author" field so that he would get credit for this patch in revision control history.
 2010-09-09 21:30:38 -07:00
Zooko O'Whielacronx
56a3258ff7 immutable downloader: add a test specifically of whether the finder sometimes announces "no more shares ever" and then announces a newly found share 
(The current code fails this test, ref #1191.)
 2010-09-08 21:16:54 -07:00
Zooko O'Whielacronx
9d421cb022 tests: assign the storage servers to a fixed order which triggers a bug in new downloader every time this test is run (formerly this test would detect the bug in new-downloader only sporadically) 
If you are investigating the bug in new-downloader, one way to investigate might be to change this ordering to a different fixed order (e.g. rotate by 4 instead of rotate by 5) and observe how the behavior of new-downloader differs in that case.
 2010-09-03 21:15:15 -07:00
Zooko O'Whielacronx
63fb687a44 download status: fix bug from me committing the wrong one of François's #1172 patches, fixes #1172 2010-09-02 09:15:41 -07:00
francois
485bfc0fd6 DownloadStatus: show active immutable downloads in Active Operations, Fix #1172 2010-09-02 03:17:28 -07:00
Zooko O'Whielacronx
cdc625e0f8 tests: bump up the allowed number of reads 
Kyle's OpenBSD buildslave used 41 reads when doing this test. The fact that I'm blindly bumping this number up to match the observed behavior probably means this isn't a good criterion to be testing for anyway. But perhaps someone else (Brian) could investigate why that run on Kyle's OpenBSD box took four more reads than we expected, and whether the fact that it took 41 reads to do this operation is indicative of an actual problem.
 2010-09-01 22:38:01 -07:00
Brian Warner
00e9e4e676 SegmentFetcher: use new diversity-seeking share-selection algorithm, and 
deliver all shares at once instead of feeding them out one-at-a-time.

Also fix distribution of real-number-of-segments information: now all
CommonShares (not just the ones used for the first segment) get a
correctly-sized hashtree. Previously, the late ones might not, which would
make them crash and get dropped (causing the download to fail if the initial
set were insufficient, perhaps because one of their servers went away).

Update tests, add some TODO notes, improve variable names and comments.
Improve logging: add logparents, set more appropriate levels.
 2010-08-31 18:37:02 -07:00
Brian Warner
c89a464510 Share: drop received data after each block finishes. Quick fix for the #1170 spans.py complexity bug. 2010-08-31 18:35:58 -07:00
francois
f026927f86 web: refactor rate computation, fixes #1166 2010-08-15 07:19:33 -07:00
Brian Warner
a3cf3331d2 DownloadStatus: put real numbers in progress/status rows, not placeholders. 
Improve tests.
 2010-08-09 23:06:03 -07:00
Brian Warner
dc1afc81bb web download-status: tolerate DYHBs that haven't retired yet. Fixes #1160. 
Also add a better unit test for it.
 2010-08-09 15:51:00 -07:00
Brian Warner
5267fc17fe immutable/filenode.py: put off DownloadStatus creation until first read() call 
This avoids spamming the "recent uploads and downloads" /status page from
FileNode instances that were created for a directory read but which nobody is
ever going to read from. I also cleaned up the way DownloadStatus instances
are made to only ever do it in the CiphertextFileNode, not in the
higher-level plaintext FileNode. Also fixed DownloadStatus handling of read
size, thanks to David-Sarah for the catch.
 2010-08-09 15:50:55 -07:00
Brian Warner
ed821d1504 Share: hush log entries in the main loop() after the fetch has been completed. 2010-08-09 13:43:59 -07:00
david-sarah
fd9a2205de test_runner.py: correct and simplify normalization of package directory for case-insensitive filesystems. 2010-08-08 11:50:05 -07:00
david-sarah
f88e4f0df6 test_runner.py: make test_path work for test-from-installdir. 2010-08-08 10:13:40 -07:00
david-sarah
f5bf900d64 src/allmydata/__init__.py: make the package paths more accurate when we fail to get them from setuptools. 2010-08-08 10:12:35 -07:00
david-sarah
dc53d4f5ef test_runner.py: another try at calculating the rootdir correctly for test-from-egg and test-from-prefixdir. 2010-08-08 08:43:07 -07:00
david-sarah
39fa9a81d1 test_runner.py: calculate the location of bin/tahoe correctly for test-from-prefixdir (by copying code from misc/build_helpers/run_trial.py). Also fix the false-positive check for Unicode paths in test_the_right_code, which was causing skips that should have been failures. 2010-08-07 21:28:17 -07:00
Brian Warner
4b7c94ece0 downloader.Segmentation: unregisterProducer when asked to stopProducing, this 
seems to avoid the #1155 log message which reveals the URI (and filecap).

Also add an [ERROR] marker to the flog entry, since unregisterProducer also
makes interrupted downloads appear "200 OK"; this makes it more obvious that
the download did not complete.
 2010-08-06 00:07:05 -07:00
david-sarah
a79ec1c6cd test_util.py: use SHA-256 from pycryptopp instead of MD5 from hashlib (for uses in which any hash will do), since hashlib was only added to the stdlib in Python 2.5. 2010-08-05 22:00:51 -07:00
david-sarah
600cb95fe2 test_runner.py: increase timeout to cater for Francois' ARM buildslave. 2010-08-05 21:26:01 -07:00
david-sarah
8e90255349 test_util.py: remove use of 'a if p else b' syntax that requires Python 2.5. 2010-08-05 21:16:16 -07:00
Brian Warner
a0124e95ee test_download.DownloadTest.test_simultaneous_goodguess: enable some disabled 
checks that used to fail but work now.
 2010-08-05 11:55:07 -07:00
Brian Warner
f6f9a97627 DownloadNode: fix lost-progress in fetch_failed, tolerate cancel when no segment-fetch is active. Fixes #1154. 
The lost-progress bug occurred when two simultanous read() calls fetched
different segments, and the first one failed (due to corruption, or the other
bugs in #1154): the second read() would never complete. While in this state,
cancelling the second read by having its consumer call stopProducing) would
trigger the cancel-intolerance bug. Finally, in downloader.node.Cancel,
prevent late cancels by adding an 'active' flag
 2010-08-05 11:55:07 -07:00
Brian Warner
43c5032105 util/spans.py: __nonzero__ cannot return a long either. for #1154 2010-08-05 11:55:07 -07:00
david-sarah
cd8d41584e test_storage.py: change skip note for test_large_share to say that Windows doesn't support sparse files. refs #569 2010-08-04 19:26:12 -07:00
Brian Warner
8844655705 One fix for bug #1154: webapi GETs with a 'Range' header broke new-downloader. 
The Range header causes n.read() to be called with an offset= of type 'long',
which eventually got used in a Spans/DataSpans object's __len__ method.
Apparently python doesn't permit __len__() to return longs, only ints.
Rewrote Spans/DataSpans to use s.len() instead of len(s) aka s.__len__() .
Added a test in test_download. Note that test_web didn't catch this because
it uses mock FileNodes for speed: it's probably time to rewrite that.

There is still an unresolved error-recovery problem in #1154, so I'm not
closing the ticket quite yet.
 2010-08-04 11:45:49 -07:00
Brian Warner
2bd8749849 test_download: minor cleanup 2010-08-04 10:55:55 -07:00
Brian Warner
fa34e4dd16 fetcher.py: improve comments 2010-08-04 00:28:14 -07:00
Brian Warner
2a05aa2d91 lazily create DownloadNode upon first read()/get_segment() 2010-08-04 00:28:08 -07:00
Brian Warner
abcd6e0e96 test_hung_server: update comments, remove dead "stage_4_d" code 2010-08-04 00:28:00 -07:00
Brian Warner
919938dd95 copy the rest of David-Sarah's changes to make my tree match 1.8.0beta 2010-08-04 00:27:52 -07:00
Brian Warner
20847dd876 ShareFinder: add 10s OVERDUE timer, send new requests to replace overdue ones 
The fixed 10-second timer will eventually be replaced with a per-server
value, calculated based on observed response times.

test_hung_server.py: enhance to exercise DYHB=OVERDUE state. Split existing
mutable+immutable tests into two pieces for clarity. Reenabled several tests.
Deleted the now-obsolete "test_failover_during_stage_4".
 2010-08-04 00:27:41 -07:00
Brian Warner
63b61ce7bd Rewrite immutable downloader (#798). This patch adds and updates unit tests. 2010-08-04 00:27:10 -07:00
Brian Warner
7b7b0c9709 Rewrite immutable downloader (#798). This patch includes higher-level 
integration into the NodeMaker, and updates the web-status display to handle
the new download events.
 2010-08-04 00:27:02 -07:00
Brian Warner
797828f47f Rewrite immutable downloader (#798). This patch rearranges the rest of src/allmydata/immutable/ . 2010-08-04 00:26:39 -07:00
Brian Warner
22a07e9bbe Rewrite immutable downloader (#798). This patch adds the new downloader itself. 2010-08-04 00:26:29 -07:00
Brian Warner
88d7ec2d54 util/observer.py: add EventStreamObserver 2010-08-04 00:26:12 -07:00
Brian Warner
cbcb728e7e Add a byte-spans utility class, like perl's Set::IntSpan for .newsrc files. 
Also a data-spans class, which records a byte (instead of a bit) for each
index.
 2010-08-04 00:26:00 -07:00
david-sarah
f10f9a8646 test_storage.py: avoid spurious test failure by accepting either 'Next crawl in 59 minutes' or 'Next crawl in 60 minutes'. fixes #1140 2010-08-03 03:20:58 -07:00
david-sarah
a7c474a098 CLI: further improve consistency of basedir options and add tests. addresses #118 2010-08-03 01:54:16 -07:00
david-sarah
b730385ec8 CLI: make the synopsis for 'tahoe unlink' say unlink instead of rm. 2010-08-03 01:53:59 -07:00
david-sarah
a32f625db9 CLI: make all of the option descriptions imperative sentences. 2010-08-03 01:48:01 -07:00
david-sarah
51eeea900f test_cli.py: make 'tahoe mkdir' tests slightly less dumb (check for 'URI:' in the output). 2010-08-03 01:47:20 -07:00
david-sarah
b9174b5fe7 test_cli.py: use u-escapes instead of UTF-8. 2010-08-03 01:35:38 -07:00
david-sarah
d81e8acf88 test_runner.py: add test_run_with_python_options, which checks that the Windows script changes haven't broken 'python <options> bin/tahoe'. 2010-08-01 23:25:58 -07:00
david-sarah
3ddb4a9bbf test_runner.py: fix missing import of get_filesystem_encoding 2010-08-01 23:09:02 -07:00
david-sarah
3af6f19cb0 misc/build_helpers/run_trial.py: check that the root from which the module we are testing was loaded is the current directory. This version of the patch folds in later fixes to the logic for caculating the directories to compare, and improvements to error messages. addresses #1137 2010-08-01 21:55:35 -07:00
david-sarah
88b6c57a1c Skip option arguments to the python interpreter when reconstructing Unicode argv on Windows. 2010-07-27 23:27:31 -07:00
david-sarah
1a5a33886d windows/fixups.py: improve comments and reference some relevant Python bugs. 2010-07-27 11:19:21 -07:00
david-sarah
b5f052a0d7 windows/fixups.py: make errors reported to original_stderr have enough information to debug even if we can't see the traceback. 2010-07-26 15:19:04 -07:00
david-sarah
ec6a674a5c windows/fixups.py: fix paste-o in name of Unicode stderr wrapper. 2010-07-26 14:47:36 -07:00
david-sarah
54bbc5325f windows/fixups.py: Don't rely on buggy MSVCRT library for Unicode output, use the Win32 API instead. This should make it work on XP. Also, change how we handle the case where sys.stdout and sys.stderr are redirected, since the .encoding attribute isn't necessarily writeable. 2010-07-25 21:50:19 -07:00
david-sarah
1fad717aaa test_runner.py: change to code for locating the bin/tahoe script that was missed when rebasing the patch for #1074. 2010-07-25 11:20:08 -07:00
david-sarah
9d04b2a317 Add missing windows/fixups.py (for setting up Unicode args and output on Windows). 2010-07-25 02:28:49 -07:00
david-sarah
37b07a545f Changes to Tahoe needed to work with new zetuptoolz (that does not use .exe wrappers on Windows), and to support Unicode arguments and stdout/stderr -- v5 2010-07-25 01:32:16 -07:00
david-sarah
54a9ba8232 scripts/common.py: fix an error introduced when rebasing to the ticket798 branch, which caused base directories to be duplicated in self.basedirs. 2010-08-01 23:49:29 -07:00
david-sarah
f952532f08 Basedir/node directory option improvements for ticket798 branch. addresses #188, #706, #715, #772, #890 2010-08-01 21:30:04 -07:00
david-sarah
16647b49f6 scripts/create_node.py: allow nickname to be Unicode. Also ensure webport is validly encoded in config file. 2010-08-01 17:02:12 -07:00
david-sarah
9aae9208e3 test_runner.py: Fix error in message arguments to 'fail' calls. 2010-08-01 18:35:26 -07:00
david-sarah
02199db615 Additional Unicode basedir changes for ticket798 branch. 2010-08-01 18:05:52 -07:00
david-sarah
d3a8ef762f Unicode basedir changes for ticket798 branch. 2010-08-01 16:53:10 -07:00
david-sarah
013954c265 fileutil: change WindowsError to OSError in abspath_expanduser_unicode, because WindowsError might not exist. 2010-07-25 15:26:03 -07:00
david-sarah
2b33704d61 test_system: correct a failure in _test_runner caused by Unicode basedir patch on non-Unicode platforms. 2010-07-23 20:21:23 -07:00
david-sarah
f036dfaa4b Fix test failures due to Unicode basedir patches. 2010-07-24 18:03:18 -07:00
david-sarah
28e6ad51a7 util.encodingutil: change quote_output to do less unnecessary escaping, and to use double-quotes more consistently when needed. This version avoids u-escaping for characters that are representable in the output encoding, when double quotes are used, and includes tests. fixes #1135 2010-07-23 00:53:14 -07:00
david-sarah
618db4867c Replace uses of os.path.abspath with abspath_expanduser_unicode where necessary. This makes basedir paths consistently represented as Unicode. 2010-07-21 17:14:18 -07:00
david-sarah
11b18824c7 util.fileutil, test.test_util: add abspath_expanduser_unicode function, to work around <http://bugs.python.org/issue3426>. util.encodingutil: add a convenience function argv_to_abspath. 2010-07-21 16:15:07 -07:00
Zooko O'Whielacronx
24c72f2942 upload: tidy up logging messages 
reformat code to be less than 100 chars wide, refactor formatting of logging messages, add log levels to some logging messages, M-x whitespace-cleanup
 2010-08-02 00:02:12 -07:00
jacob.lyles
e276050d5a abbreviate time edge case python2.5 unit test 2010-07-29 14:06:38 -07:00
david-sarah
23c649c897 test_upload.py: rename test_problem_layout_ticket1124 to test_problem_layout_ticket_1124 -- fix .todo reference. 2010-07-29 08:29:27 -07:00
david-sarah
0ebe18cc73 test_upload.py: rename test_problem_layout_ticket1124 to test_problem_layout_ticket_1124 for consistency. 2010-07-29 07:22:50 -07:00
Zooko O'Whielacronx
28822dfb4e tests: remove debug print 2010-08-01 23:33:39 -07:00
Zooko O'Whielacronx
69c48ebde6 immutable: tidy-up some code by using a set instead of list to hold homeless_shares 2010-08-01 23:20:04 -07:00
Zooko O'Whielacronx
fd874d6669 setup: fix a couple instances of hard-coded 'allmydata-tahoe' in the scripts, tighten the tests (as suggested by David-Sarah) 2010-08-01 09:42:07 -07:00
Zooko O'Whielacronx
44edecb36c setup: replace hardcoded 'allmydata-tahoe' with allmydata.__appname__ 2010-08-01 09:05:17 -07:00
Zooko O'Whielacronx
0f94923f22 immutable: use PrefixingLogMixin to organize logging in Tahoe2PeerSelector and add more detailed messages about peer 2010-07-19 01:20:00 -07:00
Zooko O'Whielacronx
2ebe2c2ff3 benchmarking: update bench_dirnode to be correct and use the shiniest new pyutil.benchutil features concerning what units you measure in 2010-07-18 21:49:48 -07:00
Zooko O'Whielacronx
9b4f412e49 trivial: rename and add in-line doc to clarify "used_peers" => "upload_servers" 2010-07-18 21:47:44 -07:00
jacob.lyles
d3e6e15e23 web: don't use %d formatting on a potentially large negative float -- there is a bug in Python 2.5 in that case 
fixes #1055
 2010-07-30 15:05:50 -07:00
Zooko O'Whielacronx
1c62db0db8 docs: use current cap to Zooko's wiki page in example text 
fixes #1134
 2010-07-20 18:05:43 -07:00
david-sarah
94bec75dda __init__.py: silence DeprecationWarning about BaseException.message globally. fixes #1129 2010-07-19 18:19:39 -07:00
david-sarah
bdc6fd2f8b test_runner: test that 'tahoe --version' outputs no noise (e.g. DeprecationWarnings). 2010-07-19 18:13:45 -07:00
Zooko O'Whielacronx
c076c0b19b immutable: add test case of #1128, fix test case of #1118 2010-07-19 01:16:12 -07:00
Zooko O'Whielacronx
13b5e44fbc immutable: fix bug in which preexisting_shares and merged were shallowly referencing the same sets 
This bug had the effect of making uploads sometimes (rarely) appear to succeed when they had actually not distributed the shares well enough to achieve the desired servers-of-happiness level.
 2010-07-19 00:54:26 -07:00
david-sarah
461f946457 upload.py: fix #1118 by aborting newly-homeless buckets when reassignment runs. This makes a previously failing assert correct. This version refactors 'abort' into two methods, rather than using a default argument. 2010-07-18 21:46:55 -07:00
Zooko O'Whielacronx
1dbfcf753d immutable: extend the tests to check that the shares that got uploaded really do make a sufficiently Happy distribution 
This patch also renames some instances of "find_shares()" to "find_all_shares()" and other instances to "find_uri_shares()" as appropriate -- the conflation between those names confused me at first when writing these tests.
 2010-07-18 21:50:47 -07:00
Zooko O'Whielacronx
2e83f243c2 immutable: test for #1118 2010-07-18 15:15:37 -07:00
Zooko O'Whielacronx
0f46766a51 immutable: test for #1124 2010-07-18 15:29:07 -07:00
Zooko O'Whielacronx
8f201c75f0 trivial: fix unused import (sorry about that, pyflakes) 2010-07-18 14:51:33 -07:00
Zooko O'Whielacronx
16bb529339 tests, NEWS, CREDITS re: #1117 
Give Brian and Kevan promotions, move release date in NEWS to the 18th, commit Brian's test for #1117.
fixes #1117
 2010-07-18 13:32:25 -07:00
Kevan Carstensen
054c4553b5 test/test_upload.py: test to see that aborted buckets are ignored by the storage server 2010-07-15 17:10:46 -07:00
Kevan Carstensen
677f3b1f63 test/test_storage.py: test for the new remote_abort semantics. 2010-07-15 16:21:48 -07:00
Kevan Carstensen
6374f4307f storage/immutable.py: make remote_abort btell the storage server about aborted buckets. 2010-07-15 16:21:05 -07:00
Kevan Carstensen
5becaf0309 test/test_upload.py: changes to test plumbing for #1117 tests 
- Add a callRemoteOnly method to FakeBucketWriter.
    - Change the abort method in FakeBucketWriter to not return a
      RuntimeError.
 2010-07-15 16:18:20 -07:00
Kevan Carstensen
6d2c216b8f immutable/upload.py: abort buckets if peer selection fails 2010-07-15 16:17:14 -07:00
david-sarah
638a3b2af4 test_encodingutil: correct an error in the previous patch to StdlibUnicode.test_open_representable. 2010-07-18 08:14:20 -07:00
david-sarah
74c41ebb8b test_dirnode and test_web: don't use failUnlessReallyEqual in cases where the return type from simplejson.loads can vary between unicode and str. Use to_str when comparing URIs parsed from JSON. 2010-07-18 07:29:15 -07:00
david-sarah
9a4a864573 test_encodingutil: StdlibUnicode.test_open_representable no longer uses a mock. 2010-07-18 05:54:12 -07:00
Zooko O'Whielacronx
7cc98759bd docs: add comment clarifying #1051 2010-07-17 22:32:50 -07:00
david-sarah
d346e0853d Add tests of caps from the future that have non-ASCII characters in them (encoded as UTF-8). The changes to test_uri.py, test_client.py, and test_dirnode.py add tests of non-ASCII future caps in addition to the current tests. The changes to test_web.py just replace the tests of all-ASCII future caps with tests of non-ASCII future caps. We also change uses of failUnlessEqual to failUnlessReallyEqual, in order to catch cases where the type of a string is not as expected. 2010-07-11 13:02:52 -07:00
Zooko O'Whielacronx
3c4fe0706f M-x whitespace-cleanup 2010-07-17 20:27:39 -07:00
Zooko O'Whielacronx
6a7023efbe benchmarking: update bench_dirnode.py to reflect the new directory interfaces 2010-07-17 20:17:10 -07:00
david-sarah
bdb10553eb test_encodingutil: fix test_open_representable, which is only valid when run on a platform for which we know an unrepresentable filename. 2010-07-17 20:03:33 -07:00
francois
968edfda11 iputil.py: Add support for FreeBSD 7,8 and 9 
committed by: zooko@zooko.com
date: 2010-07-17
and I also patched: NEWS and CREDITS
 2010-07-17 19:28:32 -07:00
david-sarah
752897450d fileutil: docstrings for non-obvious usage restrictions on methods of EncryptedTemporaryFile. 2010-07-16 22:46:47 -07:00
david-sarah
05022dca36 Move EncryptedTemporaryFile from SFTP frontend to allmydata.util.fileutil, and make the FTP frontend also use it (fixing #1083). 2010-07-11 14:37:21 -07:00
david-sarah
a8161c915a Correct stringutils->encodingutil patch to be the newer version, rather than the old version that was committed in error. 2010-07-17 18:34:35 -07:00
david-sarah
0d79a4a7d1 test_cli.py: fix error that crept in when rebasing the patch for #1072. 2010-07-17 17:01:23 -07:00
david-sarah
c59f9b162f stringutils: add test for when sys.stdout has no encoding attribute (fixes #1099). 2010-07-16 21:58:16 -07:00
david-sarah
8090785716 CLI: add 'tahoe unlink' as an alias to 'tahoe rm', for forward-compatibility. 2010-07-17 15:04:11 -07:00
Zooko O'Whielacronx
6e8477114e minor code clean-up in dirnode.py 
Impose micro-POLA by passing only the writekey instead of the whole node object to {{{_encrypt_rw_uri()}}}. Remove DummyImmutableFileNode in nodemaker.py, which is obviated by this. Add micro-optimization by precomputing the netstring of the empty string and branching on whether the writekey is present or not outside of {{{_encrypt_rw_uri()}}}. Add doc about writekey to docstring.
fixes #967
 2010-07-13 23:02:55 -07:00
david-sarah
11077ea74d Rename stringutils to encodingutil, and drop listdir_unicode and open_unicode (since the Python stdlib functions work fine with Unicode paths). Also move some utility functions to fileutil. 2010-07-11 17:30:15 -07:00
david-sarah
fa0fd66e17 Allow URIs passed in the initial JSON for t=mkdir-with-children, t=mkdir-immutable to be Unicode. Also pass the name of each child into nodemaker.create_from_cap for error reporting. 2010-07-11 12:55:25 -07:00
david-sarah
e2ef2c2691 cli.py: make command descriptions consistently end with a full stop. 2010-07-13 18:45:38 -07:00
david-sarah
15ddab08ed SFTP: address some of the comments in zooko's review (#1106). 2010-07-11 19:55:37 -07:00
david-sarah
3b1b0147a8 test_storage.py: potential fix for failures when logging is enabled. 1970-07-12 21:05:46 -07:00
terrellrussell
d0381e679e upcase_since_on_welcome 2010-07-08 12:39:03 -07:00
freestorm77
35ec8f6ac2 server_version_on_welcome_page.dpatch.txt 
- The storage server version is 0 for all storage nodes in the Welcome Page
 2010-06-05 12:17:21 -07:00
freestorm77
496b91717a directory_html_top_banner.dpatch 
The div tag with the link "Return to Welcome page" on the directory.xhtml page is not correct
 2010-06-22 13:53:01 -07:00
freestorm77
eb63ba9a26 tahoe_css_toolbar.dpatch 
CSS modification to be correctly diplayed with Internet Explorer 8

The links on the top of page directory.xhtml are not diplayed in the same line as display with Firefox.
 2010-06-22 14:00:46 -07:00
freestorm77
97579af73c runnin_test_tahoe_css.dpatch 
Runnin test for changes in tahoe.css file
 2010-06-22 14:47:14 -07:00
freestorm77
0304918a77 runnin_test_directory_xhtml.dpatch 
Runnin test for diretory.xhtml top banner
 2010-06-22 13:14:03 -07:00
david-sarah
b8df25a8d1 stringutils.py: tolerate sys.stdout having no 'encoding' attribute. 2010-06-25 21:08:17 -07:00
david-sarah
88169328be SFTP: don't call .stopProducing on the producer registered with OverwriteableFileConsumer (which breaks with warner's new downloader). 2010-06-28 16:19:26 -07:00
Zooko O'Whielacronx
ab80719198 trivial: tiny update to in-line comment 
(I'm actually committing this patch as a test of the new eager-annotation-computation of trac-darcs.)
 2010-06-13 21:57:15 -07:00
david-sarah
6b2f99fa9a test_backupdb.py: skip test_unicode if we can't represent the test filenames. 2010-06-18 19:26:20 -07:00
david-sarah
a2caf6f4b2 test_web.py: correct a test that was missed in the change to not write ctime/mtime. 2010-06-18 19:17:18 -07:00