3349 Commits

Author	SHA1	Message	Date
david-sarah	b8905fc766	allmydata/__init__.py: move the call to require_auto_deps() to the top again, since the [4784] patch turned out not to be the cause of the failure on the 'mm netbsd5' buildslave.	2010-10-31 17:33:16 -07:00
david-sarah	63c010a9ca	allmydata/__init__.py: put the _auto_deps.require_auto_deps() call back in exactly the same place it was prior to [4784].	2010-10-31 10:46:49 -07:00
david-sarah	7e2ed2054c	allmydata/__init__.py: suppress a DeprecationWarning for the sha module on importing pycrypto.	2010-10-31 09:06:36 -07:00
david-sarah	6e76653913	allmydata/__init__.py: call require_auto_deps() after importing nevow and twisted, reverting change in [4784]. Also fix a missing 'warnings.filters.pop()'.	2010-10-31 08:38:28 -07:00
david-sarah	2ab0b1aa2c	allmydata.__init__.py: temporary hack to debug failure on midnightmagic's buildslave	2010-10-30 22:50:03 -07:00
david-sarah	6eaa7f2356	tahoe backup: perform tilde expansion in exclude-from filename (version 2). fixes #1241	2010-10-30 20:52:31 -07:00
Zooko O'Whielacronx	3b2d0d9609	setup: run require_auto_deps() before attempting to import any deps in __init__.py ... For one thing, this makes missing-dependency failures into DistributionNotFound errors instead of ImportErrors, which might be more useful to the user. For another thing, if someone is using distributions that were installed with --multi-version, then they might be not importable until after require_auto_deps() has been run. (The docs claim that this would be the case, but we don't have an example of this happening at this time.)	2010-10-30 01:10:35 -07:00
david-sarah	1950d5a719	scripts/runner.py: fix unused import of allmydata. refs #1190	2010-10-29 17:31:49 -07:00
david-sarah	2a8f700026	scripts/runner.py: remove pkg_resources.require() calls. These are at best redundant because we have already called _auto_deps.require_auto_deps() (from allmydata.__init__) at that point, and they are causing failure of the test-from-prefixdir step on some buildslaves. refs #1190	2010-10-29 16:53:28 -07:00
david-sarah	10a5f23df9	test_runner.py: fix error in BinTahoe.test_version_no_noise introduced by last patch. refs #1235	2010-10-29 15:11:23 -07:00
david-sarah	d249589667	test_runner.py: also allow 'from pkg_resources import load_entry_point' as noise. refs #1235.	2010-10-29 13:42:46 -07:00
david-sarah	8c95f88b4f	test_runner.py: if the only noise is 'UserWarning: Unbuilt egg for setuptools', skip instead of failing the no_noise tests. This version avoids 'any' to be compatible with Python < 2.5. refs #1235.	2010-10-29 12:18:04 -07:00
david-sarah	4061258c85	make ResponseCache smarter to avoid memory leaks: don't record timestamps, use DataSpans to merge entries, and clear the cache when we see a new seqnum. refs #1045, #1229	2010-10-26 21:33:02 -07:00
david-sarah	25d8103dde	windows/fixups.py: limit length of string passed in a single call to WriteConsoleW. fixes #1232.	2010-10-26 19:16:36 -07:00
Brian Warner	c88eb097d9	scripts/tahoe_backup.py: oops, fix missing import, thanks pyflakes ... test_cli.py: hush minor pyflakes complaint	2010-10-29 02:42:23 -07:00
Brian Warner	d61d41147c	mutable/servermap.py: update comment. Closes #1231.	2010-10-29 02:14:24 -07:00
Francois Deppierraz	14ee763c54	tahoe_cp.py: Don't call urllib.quote with an Unicode argument, fix #1224 ... tahoe_backup.py: Fix another (potential) occurrence of calling urllib.quote() with an Unicode parameter	2010-10-29 01:45:20 -07:00
Brian Warner	c18953c169	fix #1223, crash+inefficiency during repair due to read overrun ... * repairer (really the uploader) reads beyond end of input file (Uploadable) * new-downloader does not tolerate overreads * uploader does lots of tiny reads (inefficient) This fixes the last two. The uploader still does a single overread at the end of the input file, but now that's ok so we can leave it in place. The uploader now expects the Uploadable to behave like a normal disk file (reading beyond EOF will return less data than was asked for), and now the new-downloadable behaves that way.	2010-10-29 01:20:36 -07:00
david-sarah	270322ad47	startstop_node.py: pyflakes import fix. refs #1190	2010-10-27 18:48:05 -07:00
Brian Warner	ac3b26ecf2	'tahoe start': use import+call rather than find+spawn ... This removes the need to use a locally-built (dependency) bin/twistd, and removes a big chunk of behavior differences between unix and windows. It also happens to resolve the "client node probably started" uncertainty. Might help with #1190, #602, and #71.	2010-10-26 23:13:42 -07:00
david-sarah	59e80b2ec9	test_mutable.py: add test for ResponseCache memory leak. refs #1045, #1129	2010-10-24 12:34:09 -07:00
david-sarah	daf2b8314a	test_encodingutil.py: test_argv_to_unicode modified the wrong encoding variable. fixes #1214	2010-10-22 20:58:10 -07:00
Zooko O'Whielacronx	0c2397523b	doc: add explanation of the motivation for the surprising and awkward API to erasure coding	2010-10-14 23:02:02 -07:00
Zooko O'Whielacronx	98ffbfb31f	setup: add --multi-version to the "setup.py develop" command-line ... fixes #530. I earlier tried this twice (see #530 for history) and then twice rolled it back due to some problems that arose. However, I didn't write down what the problems were in enough detail on the ticket that I can tell today whether those problems are still issues, so here goes the third attempt. (I did write down on the ticket that it would not create site.py or .pth files in the target directory with --multi-version mode, but I didn't explain why *that* was a problem.)	2010-10-05 11:23:50 -07:00
Zooko O'Whielacronx	0c6bb0178b	setup: use execfile to access _auto_deps.py in its proper location of src/allmydata/ instead of copying it into place when setup.py is executed	2010-09-05 22:57:14 -07:00
Zooko O'Whielacronx	e6a380241a	trivial: M-x whitespace-cleanup	2010-09-03 07:47:12 -07:00
Zooko O'Whielacronx	cb83f2e41c	minor: remove unused interface declaration, change allmydata.org to tahoe-lafs.org in email address, fix wording in relnotes.txt	2010-09-30 08:37:08 -07:00
Kevan Carstensen	ec4f87a98c	immutable/repairer.py: don't use the default happiness setting when repairing	2010-09-27 13:01:02 -07:00
david-sarah	6c4ba62c5d	test_download.py: simplify and possibly fix test_lost_servers; refs #1197	2010-09-11 17:32:51 -07:00
david-sarah	b23a7fcc10	test_runner.py: fix spurious failures of test_path on case-insensitive filesystems, and restore the check that the version is not 'unknown'.	2010-09-11 15:19:13 -07:00
Zooko O'Whielacronx	fd17d63c95	test: make tests stop relying on pyutil version class accepting the string 'unknown' for its version, and make them forward-compatible with the future Python Rational Version Numbering standard	2010-09-10 08:41:35 -07:00
Brian Warner	0475bd8e27	immutable download: have the finder inform its share consumer "no more shares" in a subsequent tick, thus avoiding accidentally telling it "no more shares" now and then telling it "here's another share" in a subsequent tick ... fixes #1191 Patch by Brian. This patch description was actually written by Zooko, but I forged Brian's name on the "author" field so that he would get credit for this patch in revision control history.	2010-09-09 21:30:38 -07:00
Zooko O'Whielacronx	56a3258ff7	immutable downloader: add a test specifically of whether the finder sometimes announces "no more shares ever" and then announces a newly found share ... (The current code fails this test, ref #1191.)	2010-09-08 21:16:54 -07:00
Zooko O'Whielacronx	9d421cb022	tests: assign the storage servers to a fixed order which triggers a bug in new downloader every time this test is run (formerly this test would detect the bug in new-downloader only sporadically) ... If you are investigating the bug in new-downloader, one way to investigate might be to change this ordering to a different fixed order (e.g. rotate by 4 instead of rotate by 5) and observe how the behavior of new-downloader differs in that case.	2010-09-03 21:15:15 -07:00
Zooko O'Whielacronx	63fb687a44	download status: fix bug from me committing the wrong one of François's #1172 patches, fixes #1172	2010-09-02 09:15:41 -07:00
francois	485bfc0fd6	DownloadStatus: show active immutable downloads in Active Operations, Fix #1172	2010-09-02 03:17:28 -07:00
Zooko O'Whielacronx	cdc625e0f8	tests: bump up the allowed number of reads ... Kyle's OpenBSD buildslave used 41 reads when doing this test. The fact that I'm blindly bumping this number up to match the observed behavior probably means this isn't a good criterion to be testing for anyway. But perhaps someone else (Brian) could investigate why that run on Kyle's OpenBSD box took four more reads than we expected, and whether the fact that it took 41 reads to do this operation is indicative of an actual problem.	2010-09-01 22:38:01 -07:00
Brian Warner	00e9e4e676	SegmentFetcher: use new diversity-seeking share-selection algorithm, and ... deliver all shares at once instead of feeding them out one-at-a-time. Also fix distribution of real-number-of-segments information: now all CommonShares (not just the ones used for the first segment) get a correctly-sized hashtree. Previously, the late ones might not, which would make them crash and get dropped (causing the download to fail if the initial set were insufficient, perhaps because one of their servers went away). Update tests, add some TODO notes, improve variable names and comments. Improve logging: add logparents, set more appropriate levels.	2010-08-31 18:37:02 -07:00
Brian Warner	c89a464510	Share: drop received data after each block finishes. Quick fix for the #1170 spans.py complexity bug.	2010-08-31 18:35:58 -07:00
francois	f026927f86	web: refactor rate computation, fixes #1166	2010-08-15 07:19:33 -07:00
Brian Warner	a3cf3331d2	DownloadStatus: put real numbers in progress/status rows, not placeholders. ... Improve tests.	2010-08-09 23:06:03 -07:00
Brian Warner	dc1afc81bb	web download-status: tolerate DYHBs that haven't retired yet. Fixes #1160. ... Also add a better unit test for it.	2010-08-09 15:51:00 -07:00
Brian Warner	5267fc17fe	immutable/filenode.py: put off DownloadStatus creation until first read() call ... This avoids spamming the "recent uploads and downloads" /status page from FileNode instances that were created for a directory read but which nobody is ever going to read from. I also cleaned up the way DownloadStatus instances are made to only ever do it in the CiphertextFileNode, not in the higher-level plaintext FileNode. Also fixed DownloadStatus handling of read size, thanks to David-Sarah for the catch.	2010-08-09 15:50:55 -07:00
Brian Warner	ed821d1504	Share: hush log entries in the main loop() after the fetch has been completed.	2010-08-09 13:43:59 -07:00
david-sarah	fd9a2205de	test_runner.py: correct and simplify normalization of package directory for case-insensitive filesystems.	2010-08-08 11:50:05 -07:00
david-sarah	f88e4f0df6	test_runner.py: make test_path work for test-from-installdir.	2010-08-08 10:13:40 -07:00
david-sarah	f5bf900d64	src/allmydata/__init__.py: make the package paths more accurate when we fail to get them from setuptools.	2010-08-08 10:12:35 -07:00
david-sarah	dc53d4f5ef	test_runner.py: another try at calculating the rootdir correctly for test-from-egg and test-from-prefixdir.	2010-08-08 08:43:07 -07:00
david-sarah	39fa9a81d1	test_runner.py: calculate the location of bin/tahoe correctly for test-from-prefixdir (by copying code from misc/build_helpers/run_trial.py). Also fix the false-positive check for Unicode paths in test_the_right_code, which was causing skips that should have been failures.	2010-08-07 21:28:17 -07:00
Brian Warner	4b7c94ece0	downloader.Segmentation: unregisterProducer when asked to stopProducing, this ... seems to avoid the #1155 log message which reveals the URI (and filecap). Also add an [ERROR] marker to the flog entry, since unregisterProducer also makes interrupted downloads appear "200 OK"; this makes it more obvious that the download did not complete.	2010-08-06 00:07:05 -07:00
david-sarah	a79ec1c6cd	test_util.py: use SHA-256 from pycryptopp instead of MD5 from hashlib (for uses in which any hash will do), since hashlib was only added to the stdlib in Python 2.5.	2010-08-05 22:00:51 -07:00
david-sarah	600cb95fe2	test_runner.py: increase timeout to cater for Francois' ARM buildslave.	2010-08-05 21:26:01 -07:00
david-sarah	8e90255349	test_util.py: remove use of 'a if p else b' syntax that requires Python 2.5.	2010-08-05 21:16:16 -07:00
Brian Warner	a0124e95ee	test_download.DownloadTest.test_simultaneous_goodguess: enable some disabled ... checks that used to fail but work now.	2010-08-05 11:55:07 -07:00
Brian Warner	f6f9a97627	DownloadNode: fix lost-progress in fetch_failed, tolerate cancel when no segment-fetch is active. Fixes #1154. ... The lost-progress bug occurred when two simultanous read() calls fetched different segments, and the first one failed (due to corruption, or the other bugs in #1154): the second read() would never complete. While in this state, cancelling the second read by having its consumer call stopProducing) would trigger the cancel-intolerance bug. Finally, in downloader.node.Cancel, prevent late cancels by adding an 'active' flag	2010-08-05 11:55:07 -07:00
Brian Warner	43c5032105	util/spans.py: __nonzero__ cannot return a long either. for #1154	2010-08-05 11:55:07 -07:00
david-sarah	cd8d41584e	test_storage.py: change skip note for test_large_share to say that Windows doesn't support sparse files. refs #569	2010-08-04 19:26:12 -07:00
Brian Warner	8844655705	One fix for bug #1154: webapi GETs with a 'Range' header broke new-downloader. ... The Range header causes n.read() to be called with an offset= of type 'long', which eventually got used in a Spans/DataSpans object's __len__ method. Apparently python doesn't permit __len__() to return longs, only ints. Rewrote Spans/DataSpans to use s.len() instead of len(s) aka s.__len__() . Added a test in test_download. Note that test_web didn't catch this because it uses mock FileNodes for speed: it's probably time to rewrite that. There is still an unresolved error-recovery problem in #1154, so I'm not closing the ticket quite yet.	2010-08-04 11:45:49 -07:00
Brian Warner	2bd8749849	test_download: minor cleanup	2010-08-04 10:55:55 -07:00
Brian Warner	fa34e4dd16	fetcher.py: improve comments	2010-08-04 00:28:14 -07:00
Brian Warner	2a05aa2d91	lazily create DownloadNode upon first read()/get_segment()	2010-08-04 00:28:08 -07:00
Brian Warner	abcd6e0e96	test_hung_server: update comments, remove dead "stage_4_d" code	2010-08-04 00:28:00 -07:00
Brian Warner	919938dd95	copy the rest of David-Sarah's changes to make my tree match 1.8.0beta	2010-08-04 00:27:52 -07:00
Brian Warner	20847dd876	ShareFinder: add 10s OVERDUE timer, send new requests to replace overdue ones ... The fixed 10-second timer will eventually be replaced with a per-server value, calculated based on observed response times. test_hung_server.py: enhance to exercise DYHB=OVERDUE state. Split existing mutable+immutable tests into two pieces for clarity. Reenabled several tests. Deleted the now-obsolete "test_failover_during_stage_4".	2010-08-04 00:27:41 -07:00
Brian Warner	63b61ce7bd	Rewrite immutable downloader (#798). This patch adds and updates unit tests.	2010-08-04 00:27:10 -07:00
Brian Warner	7b7b0c9709	Rewrite immutable downloader (#798). This patch includes higher-level ... integration into the NodeMaker, and updates the web-status display to handle the new download events.	2010-08-04 00:27:02 -07:00
Brian Warner	797828f47f	Rewrite immutable downloader (#798). This patch rearranges the rest of src/allmydata/immutable/ .	2010-08-04 00:26:39 -07:00
Brian Warner	22a07e9bbe	Rewrite immutable downloader (#798). This patch adds the new downloader itself.	2010-08-04 00:26:29 -07:00
Brian Warner	88d7ec2d54	util/observer.py: add EventStreamObserver	2010-08-04 00:26:12 -07:00
Brian Warner	cbcb728e7e	Add a byte-spans utility class, like perl's Set::IntSpan for .newsrc files. ... Also a data-spans class, which records a byte (instead of a bit) for each index.	2010-08-04 00:26:00 -07:00
david-sarah	f10f9a8646	test_storage.py: avoid spurious test failure by accepting either 'Next crawl in 59 minutes' or 'Next crawl in 60 minutes'. fixes #1140	2010-08-03 03:20:58 -07:00
david-sarah	a7c474a098	CLI: further improve consistency of basedir options and add tests. addresses #118	2010-08-03 01:54:16 -07:00
david-sarah	b730385ec8	CLI: make the synopsis for 'tahoe unlink' say unlink instead of rm.	2010-08-03 01:53:59 -07:00
david-sarah	a32f625db9	CLI: make all of the option descriptions imperative sentences.	2010-08-03 01:48:01 -07:00
david-sarah	51eeea900f	test_cli.py: make 'tahoe mkdir' tests slightly less dumb (check for 'URI:' in the output).	2010-08-03 01:47:20 -07:00
david-sarah	b9174b5fe7	test_cli.py: use u-escapes instead of UTF-8.	2010-08-03 01:35:38 -07:00
david-sarah	d81e8acf88	test_runner.py: add test_run_with_python_options, which checks that the Windows script changes haven't broken 'python <options> bin/tahoe'.	2010-08-01 23:25:58 -07:00
david-sarah	3ddb4a9bbf	test_runner.py: fix missing import of get_filesystem_encoding	2010-08-01 23:09:02 -07:00
david-sarah	3af6f19cb0	misc/build_helpers/run_trial.py: check that the root from which the module we are testing was loaded is the current directory. This version of the patch folds in later fixes to the logic for caculating the directories to compare, and improvements to error messages. addresses #1137	2010-08-01 21:55:35 -07:00
david-sarah	88b6c57a1c	Skip option arguments to the python interpreter when reconstructing Unicode argv on Windows.	2010-07-27 23:27:31 -07:00
david-sarah	1a5a33886d	windows/fixups.py: improve comments and reference some relevant Python bugs.	2010-07-27 11:19:21 -07:00
david-sarah	b5f052a0d7	windows/fixups.py: make errors reported to original_stderr have enough information to debug even if we can't see the traceback.	2010-07-26 15:19:04 -07:00
david-sarah	ec6a674a5c	windows/fixups.py: fix paste-o in name of Unicode stderr wrapper.	2010-07-26 14:47:36 -07:00
david-sarah	54bbc5325f	windows/fixups.py: Don't rely on buggy MSVCRT library for Unicode output, use the Win32 API instead. This should make it work on XP. Also, change how we handle the case where sys.stdout and sys.stderr are redirected, since the .encoding attribute isn't necessarily writeable.	2010-07-25 21:50:19 -07:00
david-sarah	1fad717aaa	test_runner.py: change to code for locating the bin/tahoe script that was missed when rebasing the patch for #1074.	2010-07-25 11:20:08 -07:00
david-sarah	9d04b2a317	Add missing windows/fixups.py (for setting up Unicode args and output on Windows).	2010-07-25 02:28:49 -07:00
david-sarah	37b07a545f	Changes to Tahoe needed to work with new zetuptoolz (that does not use .exe wrappers on Windows), and to support Unicode arguments and stdout/stderr -- v5	2010-07-25 01:32:16 -07:00
david-sarah	54a9ba8232	scripts/common.py: fix an error introduced when rebasing to the ticket798 branch, which caused base directories to be duplicated in self.basedirs.	2010-08-01 23:49:29 -07:00
david-sarah	f952532f08	Basedir/node directory option improvements for ticket798 branch. addresses #188, #706, #715, #772, #890	2010-08-01 21:30:04 -07:00
david-sarah	16647b49f6	scripts/create_node.py: allow nickname to be Unicode. Also ensure webport is validly encoded in config file.	2010-08-01 17:02:12 -07:00
david-sarah	9aae9208e3	test_runner.py: Fix error in message arguments to 'fail' calls.	2010-08-01 18:35:26 -07:00
david-sarah	02199db615	Additional Unicode basedir changes for ticket798 branch.	2010-08-01 18:05:52 -07:00
david-sarah	d3a8ef762f	Unicode basedir changes for ticket798 branch.	2010-08-01 16:53:10 -07:00
david-sarah	013954c265	fileutil: change WindowsError to OSError in abspath_expanduser_unicode, because WindowsError might not exist.	2010-07-25 15:26:03 -07:00
david-sarah	2b33704d61	test_system: correct a failure in _test_runner caused by Unicode basedir patch on non-Unicode platforms.	2010-07-23 20:21:23 -07:00
david-sarah	f036dfaa4b	Fix test failures due to Unicode basedir patches.	2010-07-24 18:03:18 -07:00
david-sarah	28e6ad51a7	util.encodingutil: change quote_output to do less unnecessary escaping, and to use double-quotes more consistently when needed. This version avoids u-escaping for characters that are representable in the output encoding, when double quotes are used, and includes tests. fixes #1135	2010-07-23 00:53:14 -07:00
david-sarah	618db4867c	Replace uses of os.path.abspath with abspath_expanduser_unicode where necessary. This makes basedir paths consistently represented as Unicode.	2010-07-21 17:14:18 -07:00
david-sarah	11b18824c7	util.fileutil, test.test_util: add abspath_expanduser_unicode function, to work around <http://bugs.python.org/issue3426>. util.encodingutil: add a convenience function argv_to_abspath.	2010-07-21 16:15:07 -07:00
Zooko O'Whielacronx	24c72f2942	upload: tidy up logging messages ... reformat code to be less than 100 chars wide, refactor formatting of logging messages, add log levels to some logging messages, M-x whitespace-cleanup	2010-08-02 00:02:12 -07:00
jacob.lyles	e276050d5a	abbreviate time edge case python2.5 unit test	2010-07-29 14:06:38 -07:00
david-sarah	23c649c897	test_upload.py: rename test_problem_layout_ticket1124 to test_problem_layout_ticket_1124 -- fix .todo reference.	2010-07-29 08:29:27 -07:00
david-sarah	0ebe18cc73	test_upload.py: rename test_problem_layout_ticket1124 to test_problem_layout_ticket_1124 for consistency.	2010-07-29 07:22:50 -07:00
Zooko O'Whielacronx	28822dfb4e	tests: remove debug print	2010-08-01 23:33:39 -07:00
Zooko O'Whielacronx	69c48ebde6	immutable: tidy-up some code by using a set instead of list to hold homeless_shares	2010-08-01 23:20:04 -07:00
Zooko O'Whielacronx	fd874d6669	setup: fix a couple instances of hard-coded 'allmydata-tahoe' in the scripts, tighten the tests (as suggested by David-Sarah)	2010-08-01 09:42:07 -07:00
Zooko O'Whielacronx	44edecb36c	setup: replace hardcoded 'allmydata-tahoe' with allmydata.__appname__	2010-08-01 09:05:17 -07:00
Zooko O'Whielacronx	0f94923f22	immutable: use PrefixingLogMixin to organize logging in Tahoe2PeerSelector and add more detailed messages about peer	2010-07-19 01:20:00 -07:00
Zooko O'Whielacronx	2ebe2c2ff3	benchmarking: update bench_dirnode to be correct and use the shiniest new pyutil.benchutil features concerning what units you measure in	2010-07-18 21:49:48 -07:00
Zooko O'Whielacronx	9b4f412e49	trivial: rename and add in-line doc to clarify "used_peers" => "upload_servers"	2010-07-18 21:47:44 -07:00
jacob.lyles	d3e6e15e23	web: don't use %d formatting on a potentially large negative float -- there is a bug in Python 2.5 in that case ... fixes #1055	2010-07-30 15:05:50 -07:00
Zooko O'Whielacronx	1c62db0db8	docs: use current cap to Zooko's wiki page in example text ... fixes #1134	2010-07-20 18:05:43 -07:00
david-sarah	94bec75dda	__init__.py: silence DeprecationWarning about BaseException.message globally. fixes #1129	2010-07-19 18:19:39 -07:00
david-sarah	bdc6fd2f8b	test_runner: test that 'tahoe --version' outputs no noise (e.g. DeprecationWarnings).	2010-07-19 18:13:45 -07:00
Zooko O'Whielacronx	c076c0b19b	immutable: add test case of #1128, fix test case of #1118	2010-07-19 01:16:12 -07:00
Zooko O'Whielacronx	13b5e44fbc	immutable: fix bug in which preexisting_shares and merged were shallowly referencing the same sets ... This bug had the effect of making uploads sometimes (rarely) appear to succeed when they had actually not distributed the shares well enough to achieve the desired servers-of-happiness level.	2010-07-19 00:54:26 -07:00
david-sarah	461f946457	upload.py: fix #1118 by aborting newly-homeless buckets when reassignment runs. This makes a previously failing assert correct. This version refactors 'abort' into two methods, rather than using a default argument.	2010-07-18 21:46:55 -07:00
Zooko O'Whielacronx	1dbfcf753d	immutable: extend the tests to check that the shares that got uploaded really do make a sufficiently Happy distribution ... This patch also renames some instances of "find_shares()" to "find_all_shares()" and other instances to "find_uri_shares()" as appropriate -- the conflation between those names confused me at first when writing these tests.	2010-07-18 21:50:47 -07:00
Zooko O'Whielacronx	2e83f243c2	immutable: test for #1118	2010-07-18 15:15:37 -07:00
Zooko O'Whielacronx	0f46766a51	immutable: test for #1124	2010-07-18 15:29:07 -07:00
Zooko O'Whielacronx	8f201c75f0	trivial: fix unused import (sorry about that, pyflakes)	2010-07-18 14:51:33 -07:00
Zooko O'Whielacronx	16bb529339	tests, NEWS, CREDITS re: #1117 ... Give Brian and Kevan promotions, move release date in NEWS to the 18th, commit Brian's test for #1117. fixes #1117	2010-07-18 13:32:25 -07:00
Kevan Carstensen	054c4553b5	test/test_upload.py: test to see that aborted buckets are ignored by the storage server	2010-07-15 17:10:46 -07:00
Kevan Carstensen	677f3b1f63	test/test_storage.py: test for the new remote_abort semantics.	2010-07-15 16:21:48 -07:00
Kevan Carstensen	6374f4307f	storage/immutable.py: make remote_abort btell the storage server about aborted buckets.	2010-07-15 16:21:05 -07:00
Kevan Carstensen	5becaf0309	test/test_upload.py: changes to test plumbing for #1117 tests ... - Add a callRemoteOnly method to FakeBucketWriter. - Change the abort method in FakeBucketWriter to not return a RuntimeError.	2010-07-15 16:18:20 -07:00
Kevan Carstensen	6d2c216b8f	immutable/upload.py: abort buckets if peer selection fails	2010-07-15 16:17:14 -07:00
david-sarah	638a3b2af4	test_encodingutil: correct an error in the previous patch to StdlibUnicode.test_open_representable.	2010-07-18 08:14:20 -07:00
david-sarah	74c41ebb8b	test_dirnode and test_web: don't use failUnlessReallyEqual in cases where the return type from simplejson.loads can vary between unicode and str. Use to_str when comparing URIs parsed from JSON.	2010-07-18 07:29:15 -07:00
david-sarah	9a4a864573	test_encodingutil: StdlibUnicode.test_open_representable no longer uses a mock.	2010-07-18 05:54:12 -07:00
Zooko O'Whielacronx	7cc98759bd	docs: add comment clarifying #1051	2010-07-17 22:32:50 -07:00
david-sarah	d346e0853d	Add tests of caps from the future that have non-ASCII characters in them (encoded as UTF-8). The changes to test_uri.py, test_client.py, and test_dirnode.py add tests of non-ASCII future caps in addition to the current tests. The changes to test_web.py just replace the tests of all-ASCII future caps with tests of non-ASCII future caps. We also change uses of failUnlessEqual to failUnlessReallyEqual, in order to catch cases where the type of a string is not as expected.	2010-07-11 13:02:52 -07:00
Zooko O'Whielacronx	3c4fe0706f	M-x whitespace-cleanup	2010-07-17 20:27:39 -07:00
Zooko O'Whielacronx	6a7023efbe	benchmarking: update bench_dirnode.py to reflect the new directory interfaces	2010-07-17 20:17:10 -07:00
david-sarah	bdb10553eb	test_encodingutil: fix test_open_representable, which is only valid when run on a platform for which we know an unrepresentable filename.	2010-07-17 20:03:33 -07:00
francois	968edfda11	iputil.py: Add support for FreeBSD 7,8 and 9 ... committed by: zooko@zooko.com date: 2010-07-17 and I also patched: NEWS and CREDITS	2010-07-17 19:28:32 -07:00
david-sarah	752897450d	fileutil: docstrings for non-obvious usage restrictions on methods of EncryptedTemporaryFile.	2010-07-16 22:46:47 -07:00
david-sarah	05022dca36	Move EncryptedTemporaryFile from SFTP frontend to allmydata.util.fileutil, and make the FTP frontend also use it (fixing #1083).	2010-07-11 14:37:21 -07:00
david-sarah	a8161c915a	Correct stringutils->encodingutil patch to be the newer version, rather than the old version that was committed in error.	2010-07-17 18:34:35 -07:00
david-sarah	0d79a4a7d1	test_cli.py: fix error that crept in when rebasing the patch for #1072.	2010-07-17 17:01:23 -07:00
david-sarah	c59f9b162f	stringutils: add test for when sys.stdout has no encoding attribute (fixes #1099).	2010-07-16 21:58:16 -07:00
david-sarah	8090785716	CLI: add 'tahoe unlink' as an alias to 'tahoe rm', for forward-compatibility.	2010-07-17 15:04:11 -07:00
Zooko O'Whielacronx	6e8477114e	minor code clean-up in dirnode.py ... Impose micro-POLA by passing only the writekey instead of the whole node object to {{{_encrypt_rw_uri()}}}. Remove DummyImmutableFileNode in nodemaker.py, which is obviated by this. Add micro-optimization by precomputing the netstring of the empty string and branching on whether the writekey is present or not outside of {{{_encrypt_rw_uri()}}}. Add doc about writekey to docstring. fixes #967	2010-07-13 23:02:55 -07:00
david-sarah	11077ea74d	Rename stringutils to encodingutil, and drop listdir_unicode and open_unicode (since the Python stdlib functions work fine with Unicode paths). Also move some utility functions to fileutil.	2010-07-11 17:30:15 -07:00
david-sarah	fa0fd66e17	Allow URIs passed in the initial JSON for t=mkdir-with-children, t=mkdir-immutable to be Unicode. Also pass the name of each child into nodemaker.create_from_cap for error reporting.	2010-07-11 12:55:25 -07:00
david-sarah	e2ef2c2691	cli.py: make command descriptions consistently end with a full stop.	2010-07-13 18:45:38 -07:00
david-sarah	15ddab08ed	SFTP: address some of the comments in zooko's review (#1106).	2010-07-11 19:55:37 -07:00
david-sarah	3b1b0147a8	test_storage.py: potential fix for failures when logging is enabled.	1970-07-12 21:05:46 -07:00
terrellrussell	d0381e679e	upcase_since_on_welcome	2010-07-08 12:39:03 -07:00
freestorm77	35ec8f6ac2	server_version_on_welcome_page.dpatch.txt ... - The storage server version is 0 for all storage nodes in the Welcome Page	2010-06-05 12:17:21 -07:00
freestorm77	496b91717a	directory_html_top_banner.dpatch ... The div tag with the link "Return to Welcome page" on the directory.xhtml page is not correct	2010-06-22 13:53:01 -07:00
freestorm77	eb63ba9a26	tahoe_css_toolbar.dpatch ... CSS modification to be correctly diplayed with Internet Explorer 8 The links on the top of page directory.xhtml are not diplayed in the same line as display with Firefox.	2010-06-22 14:00:46 -07:00
freestorm77	97579af73c	runnin_test_tahoe_css.dpatch ... Runnin test for changes in tahoe.css file	2010-06-22 14:47:14 -07:00
freestorm77	0304918a77	runnin_test_directory_xhtml.dpatch ... Runnin test for diretory.xhtml top banner	2010-06-22 13:14:03 -07:00
david-sarah	b8df25a8d1	stringutils.py: tolerate sys.stdout having no 'encoding' attribute.	2010-06-25 21:08:17 -07:00
david-sarah	88169328be	SFTP: don't call .stopProducing on the producer registered with OverwriteableFileConsumer (which breaks with warner's new downloader).	2010-06-28 16:19:26 -07:00
Zooko O'Whielacronx	ab80719198	trivial: tiny update to in-line comment ... (I'm actually committing this patch as a test of the new eager-annotation-computation of trac-darcs.)	2010-06-13 21:57:15 -07:00
david-sarah	6b2f99fa9a	test_backupdb.py: skip test_unicode if we can't represent the test filenames.	2010-06-18 19:26:20 -07:00
david-sarah	a2caf6f4b2	test_web.py: correct a test that was missed in the change to not write ctime/mtime.	2010-06-18 19:17:18 -07:00
david-sarah	4712875193	dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.	2010-06-18 16:01:19 -07:00
david-sarah	72e395d878	test_storage.py: print more information on test failures.	2010-06-16 20:46:23 -07:00
david-sarah	390fc78a9a	test_cli.py: fix test failure in CLI.test_listdir_unicode_good due to filenames returned from listdir_unicode no longer being normalized.	2010-06-17 21:51:10 -07:00
david-sarah	178401eb4e	tahoe backup: unicode tests.	2010-06-17 20:52:11 -07:00
david-sarah	1a0674bf37	CLI: allow Unicode patterns in exclude option to 'tahoe backup'.	2010-06-16 20:39:01 -07:00
david-sarah	a9fe3792de	dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.	2010-06-17 17:02:49 -07:00
david-sarah	718870a796	dirnode.py: comments about normalization changes.	2010-06-16 21:14:11 -07:00
david-sarah	7e7644589a	stringutils.py: remove unused import.	2010-06-16 20:44:40 -07:00
david-sarah	5ada31034b	test_stringutils.py: take account of the output of listdir_unicode no longer being normalized. Also use Unicode escapes, not UTF-8.	2010-06-16 20:44:09 -07:00
david-sarah	025aede9e4	test_dirnode.py: partial tests for normalization changes.	2010-06-16 20:40:25 -07:00
david-sarah	c1412c8a81	SFTP: get 'ctime' attribute from 'tahoe:linkmotime'.	2010-06-16 20:37:44 -07:00
david-sarah	9f5488b2d1	stringutils.py: don't NFC-normalize the output of listdir_unicode.	2010-06-16 18:55:37 -07:00
david-sarah	e2c7ad1d88	stringutils.py: Add encoding argument to quote_output. Also work around a bug in locale.getpreferredencoding on older Pythons.	2010-06-15 21:20:12 -07:00
david-sarah	c8d99b77a3	Provisional patch to NFC-normalize filenames going in and out of Tahoe directories.	2010-06-15 20:14:50 -07:00
david-sarah	408b12a7e6	test_system.py: investigate failure in allmydata.test.test_system.SystemTest.test_upload_and_download_random_key due to bytes_sent not being an int	2010-06-15 17:16:48 -07:00
david-sarah	a3acb2b773	SFTP: remove a dubious use of 'pragma: no cover'.	2010-06-13 09:43:56 -07:00
david-sarah	11e88c340a	SFTP: test that renaming onto a just-opened file fails.	2010-06-11 20:37:09 -07:00
david-sarah	b67f8b66c8	SFTP: further small improvements to test coverage. Also ensure that after a test failure, later tests don't fail spuriously due to the checks for heisenfile leaks.	2010-06-11 20:07:37 -07:00
david-sarah	b4e074d250	SFTP: further improve test coverage (paths containing '.', bad data for posix-rename extension, and error in test of openShell).	2010-06-11 14:31:42 -07:00
david-sarah	52f87904ed	SFTP: improve test coverage for no-write on mutable files, and check for heisenfile table leaks in all relevant tests. Delete test_memory_leak since it is now redundant.	2010-06-11 13:57:52 -07:00
david-sarah	546c3d2ed4	SFTP: add test for extension of file opened with FXF_APPEND.	2010-06-10 11:26:47 -07:00
Zooko O'Whielacronx	f19c240ca8	tests: bump up the timeout on this iputil test from 2s to 4s	2010-06-09 07:30:17 -07:00
david-sarah	3c44389440	SFTP: fix most significant memory leak described in #1045 (due to a file being added to all_heisenfiles under more than one direntry when renamed).	2010-06-09 01:00:03 -07:00
david-sarah	5786874d38	test_stringutils.py: Fix test failure on CentOS builder, possibly Python 2.4.3-related.	2010-06-08 23:50:56 -07:00
david-sarah	ffd1c5aca5	Fix for Unicode-related test failures on Zooko's OS X 10.6 machine.	2010-06-08 22:54:48 -07:00
david-sarah	80a89d3931	stringutils.py, sftpd.py: Portability fixes for Python <= 2.5.	2010-06-08 18:33:02 -07:00
david-sarah	32d9deace3	Back out Windows-specific Unicode argument support for v1.7.	2010-06-08 17:08:03 -07:00
david-sarah	4b5927fdb8	SFTP: suppress NoSuchChildError if heisenfile attributes have been updated in setAttrs, in the case where the parent is available.	2010-06-07 23:37:53 -07:00
david-sarah	195b4afa8f	SFTP: ignore permissions when opening a file (needed for sshfs interoperability).	2010-06-07 22:57:00 -07:00
david-sarah	a67e745b26	test_web.py: fix pyflakes warnings introduced by byterange patch.	2010-06-07 21:20:12 -07:00
Jeremy Fitzhardinge	63b28d707b	Improve HTTP/1.1 byterange handling ... Fix parsing of a Range: header to support: - multiple ranges (parsed, but not returned) - suffix byte ranges ("-2139") - correct handling of incorrectly formatted range headers (correct behaviour is to ignore the header and return the full file) - return appropriate error for ranges outside the file Multiple ranges are parsed, but only the first range is returned. Returning multiple ranges requires using the multipart/byterange content type.	2010-03-09 20:59:13 -07:00
Zooko O'Whielacronx	d13784b649	tests: bump up the timeout on these tests; MM's buildslave is sometimes extremely slow on tests, but it will complete them if given enough time. MM is working on making that buildslave more predictable in how long it takes to run tests.	2010-06-07 20:37:54 -07:00
david-sarah	3c883e6e44	test_cli.py: remove invalid 'test_listdir_unicode_bad' test.	2010-06-07 11:37:30 -07:00
david-sarah	71f8757bd1	check_memory.py: adapt to servers-of-happiness changes.	2010-06-07 18:35:28 -07:00
david-sarah	7092de1b6f	Remove the 'tahoe debug consolidate' subcommand.	2010-06-07 11:37:57 -07:00
david-sarah	731e3d68df	common_http.py, tahoe_cp.py: Fix an error in calling the superclass constructor in HTTPError and MissingSourceError (introduced by the Unicode fixes).	2010-06-07 10:47:14 -07:00
Zooko O'Whielacronx	faade8b20a	tests: drastically increase timeout of this very time-consuming test in honor of François's ARM box	2010-06-07 04:59:29 -07:00
david-sarah	8b014372b1	test_stringutils.py: trivial error in exception message for skipped test.	2010-06-06 23:14:55 -07:00
david-sarah	529c9f673a	More Unicode test fixes.	2010-06-06 22:33:58 -07:00
david-sarah	65b6f4e3ce	Unicode fixes for platforms with non-native-Unicode filesystems.	2010-06-06 21:32:38 -07:00
david-sarah	80252debcd	Unicode fixes.	2010-06-06 18:02:15 -07:00
Zooko O'Whielacronx	5bcca5151e	unicode tests: fix missing import	2010-06-04 07:26:30 -07:00
Zooko O'Whielacronx	db8a6f3aa6	unicode: make test_cli test a non-ascii argument, and make the fallback term encoding be locale.getpreferredencoding()	2010-06-04 07:12:51 -07:00
Zooko O'Whielacronx	442008a690	unicode: always decode json manifest as utf-8 then encode for stdout ... pyflakes pointed out that the exception handler fallback called an un-imported function, showing that the fallback wasn't being exercised. I'm not 100% sure that this patch is right and would appreciate François or someone reviewing it.	2010-06-04 01:48:40 -07:00
Zooko O'Whielacronx	952f3bf3ef	fix flakes	2010-06-04 00:58:45 -07:00
Zooko O'Whielacronx	08058c1073	fix syntax of assertion handling that isn't portable to older versions of Python	2010-06-04 00:58:05 -07:00
Francois Deppierraz	0eb4d83937	test_stringutils.py: Skip test test_listdir_unicode_good if filesystem supports only ASCII filenames	2010-05-21 09:08:39 -07:00
Francois Deppierraz	01da7f06e6	test_stringutils.py: Skip test_listdir_unicode on mocked platform which cannot store non-ASCII filenames	2010-05-21 09:05:59 -07:00
Francois Deppierraz	58e9df9f5a	test_stringutils.py: Add a test class for OpenBSD 4.1 with LANG=C	2010-05-21 07:00:53 -07:00
Francois Deppierraz	1aeff62161	test_stringutils.py: Mock the open() call in test_open_unicode ... This test ensure that open(a_unicode_string) is used on Unicode platforms (Windows or MacOS X) and that open(a_correctly_encoded_bytestring) on other platforms such as Unix.	2010-05-21 06:58:17 -07:00
Francois Deppierraz	eecaaca6b7	test_stringutils.py: Fix a trivial Python 2.4 syntax incompatibility	2010-05-21 02:33:45 -07:00
Francois Deppierraz	6c70fb7b3c	test_cli.py: Fix tests when sys.stdout.encoding=None and refactor this code into functions	2010-05-20 01:44:47 -07:00
Francois Deppierraz	496802420e	Fix handling of correctly encoded unicode filenames (#534) ... Tahoe CLI commands working on local files, for instance 'tahoe cp' or 'tahoe backup', have been improved to correctly handle filenames containing non-ASCII characters. In the case where Tahoe encounters a filename which cannot be decoded using the system encoding, an error will be returned and the operation will fail. Under Linux, this typically happens when the filesystem contains filenames encoded with another encoding, for instance latin1, than the system locale, for instance UTF-8. In such case, you'll need to fix your system with tools such as 'convmv' before using Tahoe CLI. All CLI commands have been improved to support non-ASCII parameters such as filenames and aliases on all supported Operating Systems except Windows as of now.	2010-05-19 17:43:56 -07:00
Francois Deppierraz	d0ed14e1bb	stringutils.py: Unicode helper functions + associated tests ... This file contains a bunch of helper functions which converts unicode string from and to argv, filenames and stdout.	2010-05-19 17:41:05 -07:00
david-sarah	a2cff856ff	Resolve merge conflict for sftpd.py	2010-06-03 11:25:37 -07:00
david-sarah	2dcb62de90	SFTP: possible fix for metadata times being shown as the epoch.	2010-06-02 16:45:14 -07:00
david-sarah	c9984c40e9	SFTP: further improvements to test coverage.	2010-06-02 16:44:22 -07:00
david-sarah	2a791b0d05	SFTP: improve test coverage. Also make creating a directory fail when permissions are read-only (rather than ignoring the permissions).	2010-06-01 21:19:34 -07:00
david-sarah	29a06457d2	dirnode.py: fix a bug in the no-write change for Adder, and improve test coverage. Add a 'metadata' argument to create_subdirectory, with documentation. Also update some comments in test_dirnode.py made stale by the ctime/mtime change.	2010-06-01 20:26:41 -07:00
david-sarah	db394671e9	SFTP: fix a bug that caused the temporary files underlying EncryptedTemporaryFiles not to be closed.	2010-05-31 22:53:10 -07:00
david-sarah	de95140b7b	SFTP: changes for #1063 ('no-write' field) including comment:1 (clearing owner write permission diminishes to a read cap). Includes documentation changes, but not tests for the new behaviour.	2010-05-31 22:11:39 -07:00
david-sarah	9faa02deb5	SFTP: the same bug as in _sync_heisenfiles also occurred in two other places.	2010-05-29 23:01:27 -07:00
david-sarah	cccdd32806	SFTP: another try at fixing the _sync_heisenfiles bug.	2010-05-29 22:52:54 -07:00
david-sarah	51b0d8bde3	SFTP: fix silly bug in _sync_heisenfiles ('f is not ignore' vs 'not (f is ignore)').	2010-05-29 22:38:07 -07:00
david-sarah	b9d3cf12b8	SFTP: log when a sync completes.	2010-05-29 22:18:40 -07:00
david-sarah	c4a8af4d27	SFTP: fix bug in previous logging patch.	2010-05-29 22:00:00 -07:00
david-sarah	3395a7e154	SFTP: more logging to track down OpenOffice hang.	2010-05-29 21:08:09 -07:00
david-sarah	4be24a89df	SFTP: avoid blocking close on a heisenfile that has been abandoned or never changed. Also, improve the logging to help track down a case where OpenOffice hangs on opening a file with FXF_READ|FXF_WRITE.	2010-05-29 19:55:44 -07:00
david-sarah	8ed28a95e7	Move suppression of DeprecationWarning about BaseException.message from sftpd.py to main __init__.py. Also, remove the global suppression of the 'integer argument expected, got float' warning, which turned out to be a bug.	2010-05-28 22:05:37 -07:00
david-sarah	e867985539	SFTP: cater to clients that assume a file is created as soon as they have made an open request; also, fix some race conditions associated with closing a file at about the same time as renaming or removing it.	2010-05-28 21:52:53 -07:00
david-sarah	0b888f8201	SFTP: 'sync' any open files at a direntry before opening any new file at that direntry. This works around the sshfs misbehaviour of returning success to clients immediately on close.	2010-05-25 16:02:57 -07:00
david-sarah	a143b1297b	SFTP: handle removing a file while it is open. Also some simplifications of the logout handling.	2010-05-25 11:42:10 -07:00
david-sarah	e81ce2bbd2	SFTP: a posix-rename response should actually return an FXP_STATUS reply, not an FXP_EXTENDED_REPLY as Twisted Conch assumes. Work around this by raising an SFTPError with code FX_OK.	2010-05-24 20:33:23 -07:00
david-sarah	8f86490d48	SFTP: fix problem with posix-rename code returning a Deferred for the renamed filenode, not for the result of the request (an empty string).	2010-05-24 19:02:09 -07:00
david-sarah	210578da63	SFTP: fix time handling to make sure floats are not passed into twisted.conch, and to print times in the future less ambiguously in directory listings.	2010-05-24 16:04:12 -07:00
david-sarah	549dccd608	SFTP: name of the POSIX rename extension should be 'posix-rename@openssh.com', not 'extposix-rename@openssh.com'.	2010-05-23 19:11:56 -07:00
david-sarah	6e114e44b5	SFTP: avoid race condition where .write could be called on an OverwriteableFileConsumer after it had been closed.	2010-05-23 16:38:30 -07:00
david-sarah	c6b414c815	SFTP: log tracebacks for RAISEd exceptions.	2010-05-23 15:15:35 -07:00
david-sarah	725daee788	SFTP: more logging to investigate behaviour of getAttrs(path).	2010-05-23 13:42:36 -07:00
david-sarah	75357fd2d5	SFTP: fix pyflakes warnings; drop 'noisy' versions of eventually_callback and eventually_errback; robustify conversion of exception messages to UTF-8.	2010-05-23 07:09:05 -07:00
david-sarah	38964fb35a	SFTP: fixes and test cases for renaming of open files.	2010-05-22 20:25:49 -07:00
david-sarah	42ecd77f4e	SFTP: Increase test_sftp timeout to cater for francois' ARM buildslave.	2010-05-22 12:16:39 -07:00
david-sarah	5974773969	SFTP: Fix error in support for getAttrs on an open file, to index open files by directory entry rather than path. Extend that support to renaming open files. Also, implement the extposix-rename@openssh.org extension, and some other minor refactoring.	2010-05-21 20:58:36 -07:00
david-sarah	5545a678bf	SFTP tests: fix test_openDirectory_and_attrs that was failing in timezones west of UTC.	2010-05-20 11:10:27 -07:00
david-sarah	ce3872d10e	SFTP: allow getAttrs to succeed on a file that has been opened for creation but not yet uploaded or linked (part of #1050).	2010-05-19 20:56:13 -07:00
david-sarah	7f9e351bce	SFTP: improve logging so that results of requests are (usually) logged.	2010-05-19 17:36:52 -07:00
david-sarah	a62733ed08	SFTP: add tests for more combinations of open flags.	2010-05-18 22:39:33 -07:00
david-sarah	f0945526ce	SFTP: allow FXF_WRITE | FXF_TRUNC (#1050).	2010-05-18 21:32:40 -07:00
david-sarah	bde27cab57	SFTP: remove another case where we were logging data.	2010-05-18 18:27:13 -07:00
david-sarah	d8b096a054	SFTP: avoid logging all data passed to callbacks.	2010-05-18 17:06:51 -07:00
david-sarah	819eaa74c1	SFTP: fixes related to reporting of permissions (needed for sshfs).	2010-05-17 22:45:21 -07:00
david-sarah	8d118e28f1	SFTP: change error code returned for ExistingChildError to FX_FAILURE (fixes gvfs with some picky programs such as gedit).	2010-05-17 17:42:05 -07:00
david-sarah	57699fd1eb	SFTP: fixed bugs that caused hangs during write (#1037).	2010-05-16 21:42:28 -07:00
david-sarah	5f9c10901b	SFTP: work around a probable bug in twisted.conch.ssh.session:loseConnection(). Also some minor error handling cleanups.	2010-05-16 18:26:06 -07:00
david-sarah	6a2b0229f0	SFTP: Support statvfs extensions, avoid logging actual data, and decline shell sessions politely.	2010-05-16 08:43:47 -07:00
david-sarah	31994aaf74	SFTP: fix error in SFTPUserHandler arguments introduced by execCommand patch.	2010-05-15 18:40:45 -07:00
david-sarah	8028bf01bc	SFTP: implement execCommand to interoperate with clients that issue a 'df -P -k /' command. Also eliminate use of Zope adaptation.	2010-05-15 18:27:54 -07:00
david-sarah	6ba6c97a7a	sftpd.py: 'log.OPERATIONAL' should be just 'OPERATIONAL'.	2010-05-15 08:55:33 -07:00
david-sarah	5ab4a376b7	Attempt to fix #1040 by making SFTPUser implement ISession.	2010-05-14 17:57:19 -07:00
david-sarah	7409d1e019	Eliminate Windows newlines from sftpd.py.	2010-05-14 17:56:56 -07:00
david-sarah	14d0721246	Update SFTP implementation and tests: fix #1038 and switch to foolscap logging; also some code reorganization.	2010-05-13 21:31:13 -07:00
david-sarah	5fdd63d0ad	Tests for new SFTP implementation	2010-05-11 23:05:52 -07:00
david-sarah	5248c808ac	New SFTP implementation: mutable files, read/write support, streaming download, Unicode filenames, and more	2010-05-11 22:54:07 -07:00
david-sarah	9214dbda50	Add must_exist, must_be_directory, and must_be_file arguments to DirectoryNode.delete. This will be used to fixes a minor condition in the SFTP frontend.	2010-05-27 12:45:29 -07:00
david-sarah	4556702044	Fix test failures in test_web caused by changes to web page titles in #1062. Also, change a 'target' field to '_blank' instead of 'blank' in welcome.xhtml.	2010-06-03 16:21:05 -07:00
david-sarah	4f58355a0e	dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.	2010-06-01 18:46:44 -07:00
david-sarah	53f7d2c7fe	dirnode.py: Fix #1034 (MetadataSetter does not enforce restriction on setting 'tahoe' subkeys), and expose the metadata updater for use by SFTP. Also, support diminishing a child cap to read-only if 'no-write' is set in the metadata.	2010-05-31 21:54:28 -07:00
david-sarah	027e7701bd	Change doc comments in interfaces.py to take into account unknown nodes.	2010-05-28 10:19:22 -07:00
david-sarah	6d91b1e00e	Trivial whitespace changes.	2010-05-27 12:41:14 -07:00
david-sarah	aac94a8d89	Suppress 'integer argument expected, got float' DeprecationWarning everywhere	2010-05-23 15:11:57 -07:00
david-sarah	e706f043f0	Change shouldFail to avoid Unicode errors when converting Failure to str	2010-05-11 23:07:54 -07:00
david-sarah	b8641f8f9b	allmydata.org -> tahoe-lafs.org in __init__.py	2010-06-02 23:35:30 -07:00
david-sarah	90e9fe28af	Resolve conflict in patch to change imports to absolute.	2010-06-02 22:46:08 -07:00
freestorm77	59a77f82d7	title_rename_xhtml.dpatch.txt ... - Renamed xhtml Title from "Allmydata - Tahoe" to "Tahoe-LAFS" - Renamed Tahoe to Tahoe-LAFS in page content - Changed Tahoe-LAFS home page link to http://tahoe-lafs.org (added target="blank") - Deleted commented css script in info.xhtml	2010-05-29 10:25:42 -07:00
Zooko O'Whielacronx	9e2da05837	tests: refactor test_web.py to have less duplication of literal caps-from-the-future ... This is a prelude to a patch which will add tests of caps from the future which have non-ascii chars in them.	2010-05-18 22:51:46 -07:00
david-sarah	a80f19a084	Correct harmless indentation errors found by pylint	2010-02-25 23:21:51 -07:00
david-sarah	e76092e16c	Change relative imports to absolute	2010-02-26 01:14:33 -07:00
david-sarah	e880fc2ce4	Suppress deprecation warning for twisted.web.error.NoResource when using Twisted >= 9.0.0.	2010-05-16 13:56:25 -07:00
Zooko O'Whielacronx	d4f6d4092c	test: show stdout and stderr in case of non-zero exit code from "tahoe" command	2010-05-24 00:33:48 -07:00
Zooko O'Whielacronx	294c0a6279	tests: fix pyflakes warnings in bench_dirnode.py	2010-05-21 13:25:11 -07:00
Zooko O'Whielacronx	35efdf04ca	tests: don't let bench_dirnode.py do stuff and have side-effects at import time (unless __name__ == '__main__')	2010-05-21 05:20:52 -07:00
Zooko O'Whielacronx	a48979516e	tests: increase timeout to give François's ARM buildslave a chance to complete the tests	2010-05-20 06:45:26 -07:00
Kevan Carstensen	d0706d27ea	Hush pyflakes warnings	2010-05-15 11:43:44 -07:00
Kevan Carstensen	9179dd099a	Improve code coverage of the Tahoe2PeerSelector tests.	2010-05-14 20:29:13 -07:00
Kevan Carstensen	55a2849f38	Remove a comment that no longer makes sense.	2010-05-14 13:35:16 -07:00
Kevan Carstensen	e225f573b9	Fix up the behavior of #778, per reviewers' comments ... - Make some important utility functions clearer and more thoroughly documented. - Assert in upload.servers_of_happiness that the buckets attributes of PeerTrackers passed to it are mutually disjoint. - Get rid of some silly non-Pythonisms that I didn't see when I first wrote these patches. - Make sure that should_add_server returns true when queried about a shnum that it doesn't know about yet. - Change Tahoe2PeerSelector.preexisting_shares to map a shareid to a set of peerids, alter dependencies to deal with that. - Remove upload.should_add_servers, because it is no longer necessary - Move upload.shares_of_happiness and upload.shares_by_server to a utility file. - Change some points in Tahoe2PeerSelector. - Compute servers_of_happiness using a bipartite matching algorithm that we know is optimal instead of an ad-hoc greedy algorithm that isn't. - Change servers_of_happiness to just take a sharemap as an argument, change its callers to merge existing_shares and used_peers before calling it. - Change an error message in the encoder to be more appropriate for servers of happiness. - Clarify the wording of an error message in immutable/upload.py - Refactor a happiness failure message to happinessutil.py, and make immutable/upload.py and immutable/encode.py use it. - Move the word "only" as far to the right as possible in failure messages. - Use a better definition of progress during peer selection. - Do read-only peer share detection queries in parallel, not sequentially. - Clean up logging semantics; print the query statistics whenever an upload is unsuccessful, not just in one case.	2010-05-13 17:49:17 -07:00
Kevan Carstensen	9bc71d3da0	Alter the error message when an upload fails, per some comments in #778. ... When I first implemented #778, I just altered the error messages to refer to servers where they referred to shares. The resulting error messages weren't very good. These are a bit better.	2009-12-30 15:03:44 -07:00
Kevan Carstensen	8bcc771e26	Change "UploadHappinessError" to "UploadUnhappinessError"	2009-12-04 22:30:37 -07:00
Kevan Carstensen	68fb556e93	Alter the error message returned when peer selection fails ... The Tahoe2PeerSelector returned either NoSharesError or NotEnoughSharesError for a variety of error conditions that weren't informatively described by them. This patch creates a new error, UploadHappinessError, replaces uses of NoSharesError and NotEnoughSharesError with it, and alters the error message raised with the errors to be more in line with the new servers_of_happiness behavior. See ticket #834 for more information.	2009-11-22 18:24:05 -07:00
Kevan Carstensen	320582be5a	Eliminate overcounting iof servers_of_happiness in Tahoe2PeerSelector; also reorganize some things.	2009-11-17 19:45:42 -07:00
Kevan Carstensen	4e29060847	Change stray "shares_of_happiness" to "servers_of_happiness"	2009-11-16 15:24:59 -07:00
Kevan Carstensen	a816de3f23	Alter Tahoe2PeerSelector to make sure that it recognizes existing shares on readonly servers, fixing an issue in #778	2009-11-16 13:28:05 -07:00
Kevan Carstensen	8c71df53f9	Alter 'immutable/encode.py' and 'immutable/upload.py' to use servers_of_happiness instead of shares_of_happiness.	2009-11-04 05:12:22 -07:00
Kevan Carstensen	b2d8a7cec2	Alter the signature of set_shareholders in IEncoder to add a 'servermap' parameter, which gives IEncoders enough information to perform a sane check for servers_of_happiness.	2009-11-03 21:32:41 -07:00
Kevan Carstensen	89a8da9dd2	Alter CiphertextDownloader to work with servers_of_happiness	2009-09-23 21:19:32 -07:00
Kevan Carstensen	5683112a02	Revisions of the #778 tests, per reviewers' comments ... - Fix comments and confusing naming. - Add tests for the new error messages suggested by David-Sarah and Zooko. - Alter existing tests for new error messages. - Make sure that the tests continue to work with the trunk. - Add a test for a mutual disjointedness assertion that I added to upload.servers_of_happiness. - Fix the comments to correctly reflect read-onlyness - Add a test for an edge case in should_add_server - Add an assertion to make sure that share redistribution works as it should - Alter tests to work with revised servers_of_happiness semantics - Remove tests for should_add_server, since that function no longer exists. - Alter tests to know about merge_peers, and to use it before calling servers_of_happiness. - Add tests for merge_peers. - Add Zooko's puzzles to the tests. - Edit encoding tests to expect the new kind of failure message. - Edit tests to expect error messages with the word "only" moved as far to the right as possible. - Extended and cleaned up some helper functions. - Changed some tests to call more appropriate helper functions. - Added a test for the failing redistribution algorithm - Added a test for the progress message - Added a test for the upper bound on readonly peer share discovery.	2010-05-13 18:25:42 -07:00
Kevan Carstensen	d360ee0d2e	Alter various unit tests to work with the new happy behavior	2010-01-07 12:13:25 -07:00
Kevan Carstensen	953c9555ca	Replace "UploadHappinessError" with "UploadUnhappinessError" in tests.	2009-12-04 22:34:53 -07:00
Kevan Carstensen	c0f3dbb919	Add tests for the behavior described in #834.	2009-11-22 19:20:08 -07:00
Kevan Carstensen	c3b11dedea	Re-work 'test_upload.py' to be more readable; add more tests for #778	2009-11-16 13:23:34 -07:00
Kevan Carstensen	9590690450	Test Tahoe2PeerSelector to make sure that it recognizeses existing shares on readonly servers	2009-11-08 18:37:35 -07:00
Kevan Carstensen	324a72e4a3	Add more tests for comment:53 in ticket #778	2009-11-04 05:28:49 -07:00
Kevan Carstensen	697f1cfae8	Add a test for upload.shares_by_server	2009-11-04 05:13:24 -07:00
Kevan Carstensen	0d8e70bfe5	Minor tweak to an existing test -- make the first server read-write, instead of read-only	2009-11-03 21:42:32 -07:00
Kevan Carstensen	5ad29d4e8b	Alter tests to use the new form of set_shareholders	2009-11-03 21:36:02 -07:00
"Kevan Carstensen"	ee9690b357	Refactor some behavior into a mixin, and add tests for the behavior described in #778	2009-10-30 02:19:08 -07:00
Kevan Carstensen	362f204075	Alter NoNetworkGrid to allow the creation of readonly servers for testing purposes.	2009-10-17 18:30:13 -07:00
"Kevan Carstensen"	5fe125ed74	Alter wording in 'interfaces.py' to be correct wrt #778	2009-12-04 21:40:05 -07:00
Zooko O'Whielacronx	6037df55b3	web: add a simple test that you can load directory.xhtml	2010-05-09 23:37:29 -07:00
Francois Deppierraz	dd95e8e71b	CLI: Support for https url in option --node-url ... This patch modifies the regular expression used for verifying of '--node-url' parameter. Support for accessing a Tahoe gateway over HTTPS was already present, thanks to Python's urllib.	2010-04-30 11:56:09 -07:00
Brian Warner	e2031aff0f	backupdb.did_create_directory: use REPLACE INTO, not INSERT INTO + ignore error ... This handles the case where we upload a new tahoe directory for a previously-processed local directory, possibly creating a new dircap (if the metadata had changed). Now we replace the old dirhash->dircap record. The previous behavior left the old record in place (with the old dircap and timestamps), so we'd never stop creating new directories and never converge on a null backup.	2010-04-27 22:08:03 -07:00
Brian Warner	f6f8256b35	"tahoe webopen": add --info flag, to get ?t=info ... Also fix some trailing whitespace.	2010-04-24 16:30:03 -07:00
Zooko O'Whielacronx	80ae6f2706	setup: fix error in licensing declaration from Shawn Willden, as noted by the FSF compliance division	2010-03-09 08:37:36 -08:00
Zooko O'Whielacronx	34a50b857a	docs: a few small edits to try to guide newcomers through the docs ... These edits were suggested by my watching over Jake Appelbaum's shoulder as he completely ignored/skipped/missed install.html and also as he decided that debian.txt wouldn't help him with basic installation. Then I threw in a few docs edits that have been sitting around in my sandbox asking to be committed for months.	2010-03-03 15:19:02 -08:00
david-sarah	40edf8f419	Change code that gives a base32 SI or an empty string to be more straightforward. (#948)	2010-02-26 22:55:51 -08:00
david-sarah	a2ed17f2a0	Additional test for DIR2-LIT directories in test_web.py, fixed version (#948)	2010-02-24 20:18:24 -08:00
david-sarah	d7b50a3b86	Additional fixes for DIR2-LIT More Info page and deep-check/manifest operations (#948)	2010-02-24 00:02:20 -08:00
david-sarah	e6aee33bb7	directories: add DIR2-LIT directories to test_deepcheck.py (#948)	2010-02-23 23:54:33 -08:00
david-sarah	187d837c1d	dirnode: add tests of literal dirnodes (current and fix for #948)	2010-02-23 20:33:45 -08:00
david-sarah	fec9185f2a	Additional fix for abbrev_si, with test	2010-02-21 19:36:52 -08:00
david-sarah	d29ec184a6	Additions to test_web.py for #948	2010-02-21 18:53:52 -08:00
david-sarah	973f0afdd3	Change direct accesses to an_uri.storage_index to calls to .get_storage_index() (fixes #948)	2010-02-21 18:45:04 -08:00
david-sarah	6a7feea455	Tweak to 'tahoe ls --help' output (#837)	2010-02-23 19:02:31 -08:00
david-sarah	9741b9655f	Test behaviour of 'tahoe ls' for unknown objects (#837)	2010-02-23 18:59:13 -08:00
david-sarah	03134eedb5	Improve behaviour of 'tahoe ls' for unknown objects, addressing kevan's comments	2010-02-19 22:13:13 -08:00
Brian Warner	1de4d2c594	test_util.py: improve coverage of util.time_format	2010-02-23 23:46:37 -05:00
Kevan Carstensen	81ad52d6eb	Change OphandleTable to use a deterministic clock, so we can test it ... To test the changes for #577, we need a deterministic way to simulate the passage of long periods of time. twisted.internet.task.Clock seems, from my Googling, to be the way to go for this functionality. I changed a few things so that OphandleTable would use twisted.internet.task.Clock when testing: * WebishServer.__init___ now takes an optional 'clock' parameter, * which it passes to the root.Root instance it creates. * root.Root.__init__ now takes an optional 'clock' parameter, which it passes to the OphandleTable.__init__ method. * OphandleTable.__init__ now takes an optional 'clock' parameter. If it is provided, and it isn't None, its callLater method will be used to schedule ophandle expirations (as opposed to using reactor.callLater, which is what OphandleTable does normally). * The WebMixin object in test_web.py now sets a self.clock parameter, which is a twisted.internet.task.Clock that it feeds to the WebishServer it creates. Tests using the WebMixin can control the passage of time in OphandleTable by accessing self.clock.	2010-02-20 13:07:13 -08:00
Kevan Carstensen	bcdc78bd44	Add tests for the ophandle expiration behavior in #577	2010-02-20 17:04:55 -08:00
Kevan Carstensen	e9b59a4949	Increase ophandle expiration times, per #577	2010-02-20 17:05:12 -08:00
david-sarah	85a50feeaa	More cleanups to test_cli using new utilities for reading and writing files.	2010-02-05 17:38:55 -08:00
david-sarah	c984a09fe7	Fix race conditions and missing callback in allmydata.test.test_cli.Cp.test_copy_using_filecap, add utilities for one-liner reading and writing of files, and fix cases in test_cli where files were not being closed after writing.	2010-02-05 17:37:27 -08:00
Kevan Carstensen	c00a62a2e7	Add tests for #939	2010-02-11 22:21:37 -08:00
Kevan Carstensen	63c8c65747	Alter CLI utilities to handle nonexistent aliases better	2010-02-10 18:43:18 -08:00
Brian Warner	4040b1000b	web/storage.py: display total-seen on the last-complete-cycle line. For #940.	2010-02-07 16:20:10 -08:00
Brian Warner	880f824103	code coverage: replace figleaf with coverage.py, should work on py2.6 now. ... It still lacks the right HTML report (the builtin report is very pretty, but lacks the "lines uncovered" numbers that I want), and the half-finished delta-from-last-run measurements.	2010-02-03 08:54:21 -08:00
Zooko O'Whielacronx	3e4342ecb3	immutable: downloader accepts notifications of buckets even if those notifications arrive after he has begun downloading shares. ... This can be useful if one of the ones that he has already begun downloading fails. See #287 for discussion. This fixes part of #287 which part was a regression caused by #928, namely this fixes fail-over in case a share is corrupted (or the server returns an error or disconnects). This does not fix the related issue mentioned in #287 if a server hangs and doesn't reply to requests for blocks.	2010-01-31 22:16:10 -08:00
Zooko O'Whielacronx	e4e2599017	tests: don't require tahoe to run with no noise if we are using an old twisted that emits DeprecationWarnings	2010-01-31 21:23:23 -08:00
david-sarah	100548257a	Use if instead of assert to check for twisted ftp patch	2010-01-26 17:55:29 -08:00
Zooko O'Whielacronx	7d32fafc44	tests: stop being surprised that Nevow no longer prints out warnings when it tries to find its static files ... Unless we are using a sufficiently new version of Nevow, in which case if it prints out warnings then this is a hard test failure. :-)	2010-01-31 20:11:44 -08:00
david-sarah	6215ebd95e	cli: suppress DeprecationWarnings emitted from importing nevow and twisted. Fixes #859	2010-01-31 16:44:29 -08:00
david-sarah	ea3954372a	Improvements to test_unknownnode to cover invalid cap URIs with known prefixes	2010-01-29 22:39:08 -08:00
david-sarah	37a242e01a	Improvements to test_hung_server, and fix for status updates in download.py	2010-01-29 22:43:03 -08:00
Zooko O'Whielacronx	d62428c1e6	immutable: fix bug in tests, change line-endings to unix style, add comment	2010-01-29 10:42:37 -08:00
david-sarah	baa11a0ad4	New tests for #928	2010-01-29 04:38:45 -08:00
Zooko O'Whielacronx	2bd9dfa5bd	immutable: download from the first servers which provide at least K buckets instead of waiting for all servers to reply ... This should put an end to the phenomenon I've been seeing that a single hung server can cause all downloads on a grid to hang. Also it should speed up all downloads by (a) not-waiting for responses to queries that it doesn't need, and (b) downloading shares from the servers which answered the initial query the fastest. Also, do not count how many buckets you've gotten when deciding whether the download has enough shares or not -- instead count how many buckets to *unique* shares that you've gotten. This appears to improve a slightly weird behavior in the current download code in which receiving >= K different buckets all to the same sharenumber would make it think it had enough to download the file when in fact it hadn't. This patch needs tests before it is actually ready for trunk.	2010-01-27 15:34:17 -08:00
david-sarah	14280b009c	Eliminate 'foo if test else bar' syntax that isn't supported by Python 2.4	2010-01-28 19:52:10 -08:00
david-sarah	3e35959e9b	Add mutable field to t=json output for unknown nodes, when mutability is known	2010-01-28 19:14:24 -08:00
david-sarah	4560e021a9	Show -IMM and -RO suffixes for types of immutable and read-only unknown nodes in directory listings	2010-01-28 14:08:00 -08:00
david-sarah	1bb627405f	Fix inaccurate comment in test_mutant_dirnodes_are_omitted	2010-01-28 12:24:56 -08:00
david-sarah	b9eda4de6a	Address comments by Kevan on 833 and add test for stripping spaces	2010-01-27 15:06:42 -08:00
david-sarah	56c00cb381	Miscellaneous documentation, test, and code formatting tweaks.	2010-01-26 23:03:09 -08:00
david-sarah	6057bc02cc	Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.	2010-01-26 22:44:30 -08:00
Brian Warner	3880486f91	test_runner: cleanup, refactor common code into a non-executable method ... Having both test_node() and test_client() (one of which calls the other) felt confusing to me, so I changed it to have test_node(), test_client(), and a common do_create() helper method.	2010-01-27 14:40:40 -08:00
Brian Warner	5045a8721a	scripts/runner.py: simplify David-Sarah's clever grouped-commands usage trick	2010-01-27 14:37:58 -08:00
Brian Warner	e769bbb6dd	tahoe backup: skip all symlinks, with warning. Fixes #850, addresses #641.	2010-01-27 14:35:17 -08:00
Brian Warner	52185053c3	"tahoe backup": fix --exclude-vcs docs to include Git	2010-01-27 12:10:44 -08:00
david-sarah	5c5a6fe610	Patch to accept t=set-children as well as t=set_children	2010-01-23 19:00:20 -08:00
Zooko O'Whielacronx	4a4305e7e6	ftpd: clearer error message if Twisted needs a patch (by Nils Durner)	2010-01-26 06:34:11 -08:00
francois	b03406af9d	tahoe_backup.py: display warnings on errors instead of stopping the whole backup. Fix #729. ... This patch displays a warning to the user in two cases: 1. When special files like symlinks, fifos, devices, etc. are found in the local source. 2. If files or directories are not readables by the user running the 'tahoe backup' command. In verbose mode, the number of skipped files and directories is printed at the end of the backup. Exit status returned by 'tahoe backup': - 0 everything went fine - 1 the backup failed - 2 files were skipped during the backup	2010-01-20 01:42:49 -08:00
david-sarah	174d356692	Message saying that we couldn't find bin/tahoe should say where we looked	2010-01-16 12:45:56 -08:00
Zooko O'Whielacronx	a1444d9367	cli: merge the better version of David-Sarah's split-usage-and-help patch with the earlier version that I mistakenly committed	2010-01-25 20:45:59 -08:00
david-sarah	b079f32da2	Split tahoe --help options into groups.	2010-01-11 20:39:35 -08:00
Zooko O'Whielacronx	b94b9af189	cli: split usage strings into groups (patch by David-Sarah Hopwood)	2010-01-25 20:39:21 -08:00
david-sarah	87f1bae7fe	Add create-node CLI command, and make create-client equivalent to create-node --no-storage (fixes #760)	2010-01-15 21:20:55 -08:00
david-sarah	26ab58e006	Remove replace= parameter to mkdir-immutable and mkdir-with-children	2010-01-24 14:43:25 -08:00
Brian Warner	de14791caf	Fix webapi t=mkdir with multpart/form-data, as on the Welcome page. Closes #919.	2010-01-20 22:50:52 -08:00
Brian Warner	bb4883bbd4	tahoe_add_alias.py: minor refactoring	2010-01-14 22:42:20 -08:00
Brian Warner	9ab7524f0d	test_dirnode.py: reduce scope of a Client instance, suggested by Kevan.	2010-01-14 22:27:13 -08:00
Brian Warner	01a7630334	test_provisioning: STAN is not always a list. Fix by David-Sarah Hopwood.	2010-01-14 17:46:32 -08:00
Brian Warner	2e098e2a4d	web/directory.py mkdir-immutable: hush pyflakes, add TODO for #903 behavior	2010-01-14 14:28:04 -08:00
Brian Warner	e9223fcd0e	hush pyflakes-0.4.0 warnings: slightly less-trivial fixes. Closes #900. ... This includes one fix (in test_web) which was testing the wrong thing.	2010-01-14 14:17:19 -08:00
Brian Warner	731d15e56f	hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900.	2010-01-14 14:15:29 -08:00
Brian Warner	874a979a8e	tahoe add-alias/create-alias: don't corrupt non-newline-terminated alias ... file. Closes #741.	2010-01-14 13:02:46 -08:00
Brian Warner	d3d1293d2f	change docs and --help to use "grid" instead of "virtual drive": closes #892. ... Thanks to David-Sarah Hopwood for the patch.	2010-01-14 12:11:19 -08:00
Brian Warner	fb879ddea4	client.py: fix/update comments on KeyGenerator	2010-01-12 16:42:26 -08:00
Brian Warner	d888bf3377	Clean up log.err calls, for one of the issues in #889. ... allmydata.util.log.err() either takes a Failure as the first positional argument, or takes no positional arguments and must be invoked in an exception handler. Fixed its signature to match both foolscap.logging.log.err and twisted.python.log.err . Included a brief unit test.	2010-01-11 17:33:43 -08:00
Brian Warner	bacb6fe5aa	tidy up DeadReferenceError handling, ignore them in add_lease calls ... Stop checking separately for ConnectionDone/ConnectionLost, since those have been folded into DeadReferenceError since foolscap-0.3.1 . Write rrefutil.trap_deadref() in terms of rrefutil.trap_and_discard() to improve code coverage.	2010-01-11 16:07:23 -08:00
Brian Warner	1ed5bbdcb6	test_repairer: add (commented-out) test_each_byte, to see exactly what the ... Verifier misses The results (described in #819) match our expectations: it misses corruption in unused share fields and in most container fields (which are only visible to the storage server, not the client). 1265 bytes of a 2753 byte share (hosting a 56-byte file with an artifically small segment size) are unused, mostly in the unused tail of the overallocated UEB space (765 bytes), and the allocated-but-unwritten plaintext_hash_tree (480 bytes).	2010-01-10 12:35:52 -08:00
Zooko O'Whielacronx	e7a9c000bf	repairer: fix some wrong offsets in the randomized verifier tests, debugged by Brian ... fixes #819	2010-01-10 12:37:21 -08:00
Brian Warner	1ff49a15bb	test_repairer: fix colliding basedir names, which caused test inconsistencies	2010-01-10 00:46:19 -08:00
Zooko O'Whielacronx	876c4a153b	repairer: add deterministic test for #819, mark as TODO	2010-01-09 17:36:19 -08:00
Brian Warner	419a076f59	mutable/publish: don't loop() right away upon DeadReferenceError. Closes #877 ... The bug was that a disconnected server could cause us to re-enter the initial loop() call, sending multiple queries to a single server, provoking an incorrect UCWE. To fix it, stall the loop() with an eventual.fireEventually()	2010-01-02 14:08:41 -08:00
Brian Warner	db19b62702	immutable/checker.py: oops, forgot some imports. Also hush pyflakes.	2009-12-29 15:39:09 -08:00
Brian Warner	ba0690c9d7	mutable repair: return successful=False when numshares<k (thus repair fails), ... instead of weird errors. Closes #874 and #786. Previously, if the file had 0 shares, this would raise TypeError as it tried to call download_version(None). If the file had some shares but fewer than 'k', it would incorrectly raise MustForceRepairError. Added get_successful() to the IRepairResults API, to give repair() a place to report non-code-bug problems like this.	2009-12-29 15:37:46 -08:00
Brian Warner	acd211765c	node.py/interfaces.py: minor docs fixes	2009-12-29 15:04:09 -08:00
Brian Warner	794e32738f	checker: don't let failures in add-lease affect checker results. Closes #875. ... Mutable servermap updates and the immutable checker, when run with add_lease=True, send both the do-you-have-block and add-lease commands in parallel, to avoid an extra round trip time. Many older servers have problems with add-lease and raise various exceptions, which don't generally matter. The client-side code was catching+ignoring some of them, but unrecognized exceptions were passed through to the DYHB code, concealing the DYHB results from the checker, making it think the server had no shares. The fix is to separate the code paths. Both commands are sent at the same time, but the errback path from add-lease is handled separately. Known exceptions are ignored, the others (both unknown-remote and all-local) are logged (log.WEIRD, which will trigger an Incident), but neither will affect the DYHB results. The add-lease message is sent first, and we know that the server handles them synchronously. So when the checker is done, we can be sure that all the add-lease messages have been retired. This makes life easier for unit tests.	2009-12-29 15:01:08 -08:00
Brian Warner	a50fdca6ab	test_cli: verify fix for "tahoe get" not creating empty file on error (#121)	2009-12-27 18:54:44 -05:00
Brian Warner	00d0ca3902	addendum to "Fix 'tahoe ls' on files (#771)" ... tahoe_ls.py: tolerate missing metadata web/filenode.py: minor cleanups test_cli.py: test 'tahoe ls FILECAP'	2009-12-27 18:21:49 -05:00
Brian Warner	a8a768ef9d	Fix 'tahoe ls' on files (#771). Patch adapted from Kevan Carstensen. ... web/filenode.py: also serve edge metadata when using t=json on a DIRCAP/childname object. tahoe_ls.py: list file objects as if we were listing one-entry directories. Show edge metadata if we have it, which will be true when doing 'tahoe ls DIRCAP/filename' and false when doing 'tahoe ls FILECAP'	2009-12-27 17:54:43 -05:00
Brian Warner	931ab76588	tahoe_get: don't create the output file on error. Closes #121.	2009-12-27 17:04:04 -05:00
Brian Warner	499add09e6	webapi: don't accept zero-length childnames during traversal. Closes #358, #676. ... This forbids operations that would implicitly create a directory with a zero-length (empty string) name, like what you'd get if you did "tahoe put local /oops/blah" (#358) or "POST /uri/CAP//?t=mkdir" (#676). The error message is fairly friendly too. Also added code to "tahoe put" to catch this error beforehand and suggest the correct syntax (i.e. without the leading slash).	2009-12-27 15:10:43 -05:00
Brian Warner	2e0a61a953	CLI: send 'Accept:' header to ask for text/plain tracebacks. Closes #646. ... The webapi has been looking for an Accept header since 1.4.0, but it treats a missing header as equal to */* (to honor RFC2616). This change finally modifies our CLI tools to ask for "text/plain, application/octet-stream", which seems roughly correct (we either want a plain-text traceback or error message, or an uninterpreted chunk of binary data to save to disk). Some day we'll figure out how JSON fits into this scheme.	2009-12-27 14:58:28 -05:00
Brian Warner	a4a6c02ef8	more #859: avoid deprecation warning for unit tests too, hush pyflakes ... * factor maybe-import-sha logic into util.hashutil	2009-12-14 16:01:47 -08:00
Zooko O'Whielacronx	b69e1c600d	use hashlib module if available, thus avoiding a DeprecationWarning for importing the old sha module; fixes #859	2009-12-14 13:27:03 -08:00
Brian Warner	47cba4b51a	mutable/retrieve.py: stop reaching into private MutableFileNode attributes	2009-12-08 09:29:21 -08:00
Brian Warner	aa3caf812d	mutable/servermap.py: stop reaching into private MutableFileNode attributes	2009-12-08 09:26:08 -08:00
Brian Warner	eb18686284	mutable/servermap.py: oops, query N+e servers in MODE_WRITE, not k+e ... under normal conditions, this wouldn't cause any problems, but if the shares are really sparse (perhaps because new servers were added), then file-modifies might stop looking too early and leave old shares in place	2009-12-08 09:11:56 -08:00
Brian Warner	696cc000d0	control.py: fix speedtest: use download_best_version (not read) on mutable nodes	2009-12-06 22:05:12 -08:00
Zooko O'Whielacronx	4706859967	setup: ignore _darcs in the "test-clean" test and make the "clean" step remove all .egg's in the root dir	2009-12-06 10:48:35 -08:00
Brian Warner	a980ea337b	remove MutableFileNode.download(), prefer download_best_version() instead	2009-12-01 17:54:38 -05:00
Brian Warner	96834da0a2	Simplify immutable download API: use just filenode.read(consumer, offset, size) ... * remove Downloader.download_to_data/download_to_filename/download_to_filehandle * remove download.Data/FileName/FileHandle targets * remove filenode.download/download_to_data/download_to_filename methods * leave Downloader.download (the whole Downloader will go away eventually) * add util.consumer.MemoryConsumer/download_to_data, for convenience (this is mostly used by unit tests, but it gets used by enough non-test code to warrant putting it in allmydata.util) * update tests * removes about 180 lines of code. Yay negative code days! Overall plan is to rewrite immutable/download.py and leave filenode.read() as the sole read-side API.	2009-12-01 17:53:30 -05:00
Brian Warner	1059db51f2	server.py: undo my bogus 'correction' of David-Sarah's comment fix ... and move it to a better line	2009-11-30 21:46:07 -05:00
Brian Warner	74974b27fe	Implement more coherent behavior when copying with dircaps/filecaps (closes #761). Patch by Kevan Carstensen.	2009-11-30 13:10:09 -08:00
Brian Warner	91e7cb484b	storage.py: update comment	2009-11-30 11:59:13 -08:00
david-sarah	ef002c935a	storage server: detect disk space usage on Windows too (fixes #637)	2009-11-20 21:56:44 -08:00
david-sarah	c4d38ad4c5	make status of finished operations consistently "Finished"	2009-11-20 22:15:43 -08:00
Brian Warner	9901a08fd3	Add dirnodes to backupdb and "tahoe backup", closes #606. ... * backups now share dirnodes with any previous backup, in any location, so renames and moves are handled very efficiently * "tahoe backup" no longer bothers reading the previous snapshot * if you switch grids, you should delete ~/.tahoe/private/backupdb.sqlite, to force new uploads of all files and directories	2009-11-26 15:42:57 -08:00
Brian Warner	366a309795	webapi: fix t=check for DIR2-LIT (i.e. empty immutable directories)	2009-11-26 15:27:31 -08:00
Brian Warner	41bcc9f39e	PipelineError: fix str() on python2.4 . Closes #842.	2009-11-24 13:25:12 -08:00
Brian Warner	d918ab407e	test_uri.py: s/NewDirnode/Dirnode/ , now that they aren't "new" anymore	2009-11-19 23:55:53 -08:00
Brian Warner	0cf320c2ab	interface name cleanups: IFileNode, IImmutableFileNode, IMutableFileNode ... The proper hierarchy is: IFilesystemNode +IFileNode ++IMutableFileNode ++IImmutableFileNode +IDirectoryNode Also expand test_client.py (NodeMaker) to hit all IFilesystemNode types.	2009-11-19 23:52:55 -08:00
Brian Warner	d2badbea78	class name cleanups: s/FileNode/ImmutableFileNode/ ... also fix test/bench_dirnode.py for recent dirnode changes	2009-11-19 23:22:39 -08:00
Brian Warner	6e7fb1006d	Use DIR-IMM and t=mkdir-immutable for "tahoe backup", for #828	2009-11-18 11:28:13 -08:00
Brian Warner	834b20210a	web/directory.py: use "DIR-IMM" to describe immutable directories, not DIR-RO	2009-11-18 11:18:32 -08:00
Brian Warner	e5452290f7	web/info.py: hush pyflakes	2009-11-18 11:17:36 -08:00
Brian Warner	e046744f40	make get_size/get_current_size consistent for all IFilesystemNode classes ... * stop caching most_recent_size in dirnode, rely upon backing filenode for it * start caching most_recent_size in MutableFileNode * return None when you don't know, not "?" * only render None as "?" in the web "more info" page * add get_size/get_current_size to UnknownNode	2009-11-18 11:16:24 -08:00
Brian Warner	512fe3ad62	ImmutableDirectoryURIVerifier: fix verifycap handling	2009-11-18 08:42:38 -08:00
Brian Warner	f85690697a	Add t=mkdir-immutable to the webapi. Closes #607. ... * change t=mkdir-with-children to not use multipart/form encoding. Instead, the request body is all JSON. t=mkdir-immutable uses this format too. * make nodemaker.create_immutable_dirnode() get convergence from SecretHolder, but let callers override it * raise NotDeepImmutableError instead of using assert() * add mutable= argument to DirectoryNode.create_subdirectory(), default True	2009-11-17 23:09:00 -08:00
Brian Warner	d2426ea3bd	move convergence secret into SecretHolder, next to lease secret	2009-11-17 17:54:44 -08:00
Brian Warner	5fe713fc52	nodemaker: implement immutable directories (internal interface), for #607 ... * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet	2009-11-11 16:22:33 -08:00
Brian Warner	cc422f8dc0	stop using IURI()/etc as an adapter	2009-11-11 14:45:42 -08:00
Brian Warner	131e05b155	clean up uri-vs-cap terminology, emphasize cap instances instead of URI strings ... * "cap" means a python instance which encapsulates a filecap/dircap (uri.py) * "uri" means a string with a "URI:" prefix * FileNode instances are created with (and retain) a cap instance, and generate uri strings on demand * .get_cap/get_readcap/get_verifycap/get_repaircap return cap instances * .get_uri/get_readonly_uri return uri strings * add filenode.download_to_filename() for control.py, should find a better way * use MutableFileNode.init_from_cap, not .init_from_uri * directory URI instances: use get_filenode_cap, not get_filenode_uri * update/cleanup bench_dirnode.py to match, add Makefile target to run it	2009-11-11 14:26:19 -08:00
Brian Warner	f47672d12a	add parser for immutable directory caps: DIR2-CHK, DIR2-LIT, DIR2-CHK-Verifier	2009-11-04 10:13:51 -08:00
Zooko O'Whielacronx	9733201c0b	wui: s/TahoeLAFS/Tahoe-LAFS/	2009-10-28 18:50:50 -08:00
Zooko O'Whielacronx	1be4d45779	tests: bump up the timeout on test_repairer to see if 120 seconds was too short for François's ARM box to do the test even when it was doing it right.	2009-10-27 13:48:00 -08:00
Brian Warner	2695af91a7	dirnode.pack_children(): add deep_immutable= argument ... This will be used by DIR2:CHK to enforce the deep-immutability requirement.	2009-10-26 09:28:09 -07:00
Brian Warner	768c76aa5f	webapi: use t=mkdir-with-children instead of a children= arg to t=mkdir . ... This is safer: in the earlier API, an old webapi server would silently ignore the initial children, and clients trying to set them would have to fetch the newly-created directory to discover the incompatibility. In the new API, clients using t=mkdir-with-children against an old webapi server will get a clear error.	2009-10-25 18:13:21 -07:00
Brian Warner	1273b5c233	nodemaker.create_new_mutable_directory: pack_children() in initial_contents= ... instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts	2009-10-19 17:51:18 -07:00
Brian Warner	59ee02e82f	test_dirnode.py: add tests of initial_children= args to client.create_dirnode ... and nodemaker.create_new_mutable_directory	2009-10-17 12:41:59 -07:00
Brian Warner	b4ec86c95a	update many dirnode interfaces to accept dict-of-nodes instead of dict-of-caps ... interfaces.py: define INodeMaker, document argument values, change create_new_mutable_directory() to take dict-of-nodes. Change dirnode.set_nodes() and dirnode.create_subdirectory() too. nodemaker.py: use INodeMaker, update create_new_mutable_directory() client.py: have create_dirnode() delegate initial_children= to nodemaker dirnode.py (Adder): take dict-of-nodes instead of list-of-nodes, which updates set_nodes() and create_subdirectory() web/common.py (convert_initial_children_json): create dict-of-nodes web/directory.py: same web/unlinked.py: same test_dirnode.py: update tests to match	2009-10-17 12:28:29 -07:00
Brian Warner	8a7c980e37	dirnode.py: move pack_children() out to a function, for eventual use by others	2009-10-17 11:07:07 -07:00
Brian Warner	ea373de042	move dirnode.CachingDict to dictutil.AuxValueDict, generalize method names, ... improve tests. Let dirnode _pack_children accept either dict or AuxValueDict.	2009-10-17 11:00:05 -07:00
Brian Warner	e63f59f50c	test/common.py: update FakeMutableFileNode to new contents= callable scheme	2009-10-12 22:21:54 -07:00
Brian Warner	480e1d318d	The initial_children= argument to nodemaker.create_new_mutable_directory is ... now enabled.	2009-10-12 20:19:22 -07:00
Brian Warner	c2520e4ec7	client.create_mutable_file(contents=) now accepts a callable, which is ... invoked with the new MutableFileNode and is supposed to return the initial contents. This can be used by e.g. a new dirnode which needs the filenode's writekey to encrypt its initial children. create_mutable_file() still accepts a bytestring too, or None for an empty file.	2009-10-12 20:12:32 -07:00
Brian Warner	b30041c5ec	webapi: t=mkdir now accepts initial children, using the same JSON that t=json ... emits. client.create_dirnode(initial_children=) now works.	2009-10-12 19:34:44 -07:00
Brian Warner	cf65cc2ae3	replace dirnode.create_empty_directory() with create_subdirectory(), which ... takes an initial_children= argument	2009-10-12 19:15:20 -07:00
Brian Warner	d079eb45f6	dirnode.set_children: change return value: fire with self instead of None	2009-10-12 18:50:26 -07:00
Brian Warner	f871c3bb3d	dirnode.set_nodes: change return value: fire with self instead of None	2009-10-12 18:45:46 -07:00
Brian Warner	304aadd4f7	dirnode.set_children: take a dict, not a list	2009-10-12 17:24:40 -07:00
Brian Warner	e2ffc3dc03	dirnode.set_uri/set_children: change signature to take writecap+readcap ... instead of a single cap. The webapi t=set_children call benefits too.	2009-10-12 16:51:26 -07:00
Brian Warner	3ee740628a	replace Client.create_empty_dirnode() with create_dirnode(), in anticipation ... of adding initial_children= argument. Includes stubbed-out initial_children= support.	2009-10-12 15:45:06 -07:00
Brian Warner	576c47d61e	test_web.py: use a less-fake client, making test harness smaller	2009-10-12 15:28:08 -07:00
Brian Warner	f4aa418086	Verifier: check the full cryptext-hash tree on each share. Removed .todos ... from the last few test_repairer tests that were waiting on this.	2009-10-05 15:18:49 -07:00
Brian Warner	504c767d03	Verifier: check the full block-hash-tree on each share ... Removed the .todo from two test_repairer tests that check this. The only remaining .todos are on the three crypttext-hash-tree tests.	2009-10-05 14:48:44 -07:00
Brian Warner	e8f56af5a7	Verifier: check the full share-hash chain on each share ... Removed the .todo from two test_repairer tests that check this.	2009-10-05 14:34:43 -07:00
Brian Warner	bbd00de495	test_repairer: rename Verifier test cases to be more precise and less verbose	2009-10-05 13:11:15 -07:00
Brian Warner	be95129833	immutable/checker.py: rearrange code a little bit, make it easier to follow	2009-10-05 13:02:52 -07:00
Brian Warner	9dba4ec641	test/common.py: wrap docstrings to 80cols so I can read them more easily	2009-10-05 13:01:43 -07:00
Brian Warner	19d336513c	immutable/download.py: wrap to 80cols, no functional changes	2009-10-05 12:25:42 -07:00
Zooko O'Whielacronx	00a44c057a	docs: a few licensing clarifications requested by Ubuntu	2009-09-26 20:32:26 -07:00
Zooko O'Whielacronx	aaaa633f18	setup: remove a convenience copy of figleaf, to ease inclusion into Ubuntu Karmic Koala ... We need to carefully document the licence of figleaf in order to get Tahoe-LAFS into Ubuntu Karmic Koala. However, figleaf isn't really a part of Tahoe-LAFS per se -- this is just a "convenience copy" of a development tool. The quickest way to make Tahoe-LAFS acceptable for Karmic then, is to remove figleaf from the Tahoe-LAFS tarball itself. People who want to run figleaf on Tahoe-LAFS (as everyone should want) can install figleaf themselves. I haven't tested this -- there may be incompatibilities between upstream figleaf and the copy that we had here...	2009-09-23 22:32:15 -07:00
Zooko O'Whielacronx	9f0da4012e	docs: Shawn Willden grants permission for his contributions under GPL2+|TGPPL1+	2009-09-21 09:46:51 -07:00
Zooko O'Whielacronx	603e08c4a9	doc: licensing cleanups ... Use nice utf-8 © instead of "(c)". Remove licensing statements on utility modules that have been assigned to allmydata.com by their original authors. (Nattraverso was not assigned to allmydata.com -- it was LGPL'ed -- but I checked and src/allmydata/util/iputil.py was completely rewritten and doesn't contain any line of code from nattraverso.) Add notes to misc/debian/copyright about licensing on files that aren't just allmydata.com-licensed.	2009-09-20 10:16:31 -07:00
Brian Warner	14cb5ca436	test/check_speed.py: fix shbang line	2009-08-17 17:59:48 -07:00
Brian Warner	5283d4c19e	de-Service-ify Helper, pass in storage_broker and secret_holder directly. ... This makes it more obvious that the Helper currently generates leases with the Helper's own secrets, rather than getting values from the client, which is arguably a bug that will likely be resolved with the Accounting project.	2009-08-15 13:17:37 -07:00
Brian Warner	4a4a4f9520	immutable.Downloader: pass StorageBroker to constructor, stop being a Service ... child of the client, access with client.downloader instead of client.getServiceNamed("downloader"). The single "Downloader" instance is scheduled for demolition anyways, to be replaced by individual filenode.download calls.	2009-08-15 12:25:43 -07:00
Zooko O'Whielacronx	22c962bbc2	tests: double the timeout on test_runner.RunNode.test_introducer since feisty hit a timeout ... I'm not sure if this is an actual timing issue (feisty is running on an overloaded VM if I recall correctly), or it there is a deeper bug.	2009-08-15 09:05:12 -07:00
Brian Warner	d214f7eebe	stop making History be a Service, it wasn't necessary	2009-08-15 04:44:15 -07:00
Brian Warner	0d5dc51617	Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. ... * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo	2009-08-15 04:28:46 -07:00
Zooko O'Whielacronx	2a63fc9159	docs: update relnotes.txt, relnotes-short.txt, and others documentation bits for v1.5.0 release!	2009-08-01 19:57:10 -07:00
Brian Warner	ad45511156	ftpd: update docs, point to Twisted ticket for the proposed patch	2009-07-31 11:32:26 -07:00
Brian Warner	0d0e0ba2fb	test_runner.py: tolerate symlinks in the source tree directory's ancestor path	2009-07-30 10:17:24 -07:00
Brian Warner	bf1e61c8f3	Touch up #705 changes: ... webapi.txt: clarify replace=only-files argument, mention replace= on POST t=uri test_cli.py: insert whitespace between logical operations web.common.parse_replace_arg: make it case-insensitive, to match the docs	2009-07-20 11:38:03 -04:00
Zooko O'Whielacronx	b76ca4549a	iputil: added "openbsd3" => "bsd" and "openbsd4" => "bsd" per tdelaet's report; should fix ticket #768	2009-07-20 06:13:14 -07:00
kevan	0d8b1e29fa	Add tests for tahoe mv behavior	2009-07-19 20:46:09 -07:00
kevan	4331326b21	Add tests for tahoe mv help text	2009-07-19 20:45:53 -07:00
kevan	52aceb1a8e	Implement more clearly defined moving semantics in tahoe_mv.py	2009-07-19 20:45:23 -07:00
kevan	36f2e01275	Update tahoe mv help text.	2009-07-19 20:45:03 -07:00
kevan	8eb7ddab6b	Add tests for parse_replace_arg()	2009-07-19 20:47:05 -07:00
kevan	40360a7a9c	Add tests for new PUT behavior	2009-07-19 20:46:32 -07:00
kevan	18a80d99b1	Alter Adder + Adder tests to look for 'only-files' instead of 'only_files'	2009-07-19 20:43:18 -07:00
kevan	74207d8334	Alter filenode.py to use parse_replace_arg	2009-07-19 20:48:31 -07:00
kevan	90677745b3	Alter directory.py to use parse_replace_arg()	2009-07-19 20:47:46 -07:00
kevan	7ab92c7511	Add a function to parse arguments for the replace parameter	2009-07-19 20:47:23 -07:00
Zooko O'Whielacronx	7607268a58	trivial: remove obsolete comment text	2009-07-20 06:08:43 -07:00
Zooko O'Whielacronx	c6fce3a1c6	directories: minor change: fix warning message from the benchmarking script	2009-07-19 17:21:14 -07:00
kevan	ca4de9ee97	Add unit tests for the Adder in dirnode.py	2009-07-18 12:50:49 -07:00
kevan	c476c66b0e	Add 'only_files' option to the overwrite field in Adder	2009-07-17 20:00:10 -07:00
Brian Warner	531cc7899f	rename NewDirectoryNode to DirectoryNode, NewDirectoryURI to DirectoryURI	2009-07-17 17:15:49 -05:00
Brian Warner	8536db4e64	interfaces: remove dead code, FileNode_ and EncryptedThing constraints	2009-07-17 17:11:39 -05:00
Zooko O'Whielacronx	d2d3bb2f80	introduction/storage-broker: add assertions of serverids being the right type and of internal consistency between the number of storage servers known and the number of storage servers	2009-07-17 06:52:21 -07:00
Brian Warner	1192b61dfe	upload: fix #758 recursion-loop in peer-selection when servers report errors. ... The bug was in the code that handles a third-or-later pass, and was previously untested.	2009-07-17 00:07:09 -05:00
Brian Warner	efa608a135	'tahoe stats': tolerate empty directories. Closes #693.	2009-07-15 00:51:09 -07:00
Brian Warner	7950fc0f2e	node.py: record pid in twisted log at startup. Closes #476.	2009-07-15 00:29:29 -07:00
Brian Warner	d8ba8c2eb5	Allow tests to pass with -OO by turning some AssertionErrors (the ones that ... we actually exercise during tests) into more specific exceptions, so they don't get optimized away. The best rule to follow is probably this: if an exception is worth testing, then it's part of the API, and AssertionError should never be part of the API. Closes #749.	2009-07-14 23:45:10 -07:00
Zooko O'Whielacronx	10ad3ad7d6	wui: fix test for welcome page	2009-07-14 20:34:29 -07:00
Zooko O'Whielacronx	eb5ecc931d	wui: adjust headers/titles and "Attach something to this directory" text in accordance with #691 ... Also name it as "TahoeLAFS" in this text.	2009-07-14 19:58:14 -07:00
Zooko O'Whielacronx	2f704ed001	dirnode: finish renaming "iv" to "salt" in the code and the hash tag	2009-07-12 17:13:20 -07:00
Brian Warner	7f1d8b7c46	dirnode.py/_encrypt_rwcap: rename IV to "salt", which is more accurate	2009-07-13 00:50:25 +01:00
Brian Warner	c1d5717cf0	dirnode.py: security bug: also use child writecap to derive child enc key, ... not just the dirnode writecap. The previous code (which only hashed the dirnode writecap) would use the same key for all children, which is very bad. This is the correct implementation of #750.	2009-07-13 00:47:50 +01:00
kevan	d71adaf1ca	Use CachingDict instead of dict in dirnode.py	2009-07-03 20:43:01 -07:00
Zooko O'Whielacronx	34213cd2c7	directories: fix semantic conflict between my "keep track of position" optimization patch and Kevan's "cache serialized entries" optimization patch	2009-07-09 20:20:28 -07:00
kevan	44c652fbbe	Modify bench_dirnode.py to use CachingDict.	2009-07-05 15:31:42 -07:00
kevan	325793457d	Add tests for CachingDict, _pack_contents, _unpack_contents	2009-07-03 20:43:28 -07:00
Zooko O'Whielacronx	0e2d005146	trivial: removed unused import noticed by pyflakes	2009-07-09 06:05:13 -07:00
Zooko O'Whielacronx	786ed012b3	directories: make the IV for the writecaps in directory entries be computed from the secure hash of the writecap itself ... This makes encoding of directory entries deterministic, and it is also a tad faster on Macbook Pro than getting a random IV with os.urandom(16).	2009-07-04 19:48:15 -07:00
Zooko O'Whielacronx	fcedca966e	tests: raise the timeout on pollmixin from 100s to 1000s -- it looks like it may have triggered too eagerly on Zandr's 266 MHz armel	2009-07-08 21:47:29 -07:00
Zooko O'Whielacronx	22d390acbb	immutable: base32-encode the keys to generate cache filenames that will work on all platforms	2009-07-08 08:26:33 -07:00
Zooko O'Whielacronx	c0d1e7deae	directories: make initialization of the download cache lazy ... If you open up a directory containing thousands of files, it currently computes the cache filename and checks for the cache file on disk immediately for each immutble file in that directory. With this patch, it delays those steps until you try to do something with an immutable file that could use the cache.	2009-07-07 17:40:40 -07:00
kevan	903005a528	Add CachingDict dict subclass to dirnode.py	2009-07-05 14:23:45 -07:00
Zooko O'Whielacronx	c678e8c77a	directories: rename internal data member download_cache to download_cache_dirman (benchmarks set this internal member and use the new name, so changing this makes the bench_dirnode.py work again)	2009-07-07 07:34:04 -07:00
Zooko O'Whielacronx	efafcfb91a	directories: keep track of your position as you decode netstring after netstring from an input buffer instead of copying the trailing part ... This makes decoding linear in the number of netstrings instead of O(N^2).	2009-07-04 19:51:09 -07:00
Zooko O'Whielacronx	4206a2c1c7	directories: in bench_dirnode.py, use a real CacheDirectoryManager instead of a fake one (because CacheDirectoryManager is a significant user of CPU and/or time)	2009-07-06 20:41:19 -07:00
Zooko O'Whielacronx	0df5b446eb	directories: make the profiling behavior of bench_dirnode.py accessible by adding '--profile' to the cmdline	2009-07-06 20:30:35 -07:00
Zooko O'Whielacronx	e414c73877	directories: update the directory benchmarks to exercise the unpack-and-repack functionality, and add optional profiling	2009-07-05 09:29:53 -07:00
Zooko O'Whielacronx	859d7ba8b0	test/benchmark: benchmark the time to pack and unpack dirnodes ... See also the prof_benchmarks() function in this file which will run the benchmarks under profiling.	2009-07-04 15:43:00 -07:00
Brian Warner	ef1b6ae8e3	Tolerate unknown URI types in directory structures. Part of #683. ... The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).	2009-07-02 18:07:49 -07:00
Brian Warner	4a46e91192	test_client.py: minor refactoring of BASECONFIG usage	2009-07-02 17:54:50 -07:00
Brian Warner	6237aeabd7	create_node_from_uri: take both writecap+readcap, move logic out of dirnode.py	2009-07-02 15:25:37 -07:00
Brian Warner	656277ac98	dirnode.py: prepare to preserve both rwcap+rocap when copying ... This will make it easier to tolerate unknown nodes safely.	2009-07-02 14:12:54 -07:00
Brian Warner	3dedfed9de	interfaces.py: wrap some lines to 80cols	2009-07-01 18:57:28 -07:00
Brian Warner	b640baf500	cli: webopen: when called with no arguments, open the Welcome page	2009-07-01 13:05:48 -07:00
Brian Warner	e330abc3c3	mutable repairer: skip repair of readcaps instead of throwing an exception. ... This should improve the behavior of #625 a bit: at least all the files will get repaired.	2009-06-30 18:13:43 -07:00
Brian Warner	4194565b3d	interfaces.py: improve ICheckAndRepairResults docs a bit	2009-06-30 17:19:25 -07:00
Brian Warner	8fca155a66	repairer.py: wrap to 80cols. No code changes.	2009-06-30 17:00:47 -07:00
Brian Warner	52fa421430	use 522-bit RSA keys in all unit tests (except one) ... This reduces the total test time on my laptop from 400s to 283s. * src/allmydata/test/test_system.py (SystemTest.test_mutable._test_debug): Remove assertion about container_size/data_size, this changes with keysize and was too variable anyways. * src/allmydata/mutable/filenode.py (MutableFileNode.create): add keysize= * src/allmydata/dirnode.py (NewDirectoryNode.create): same * src/allmydata/client.py (Client.DEFAULT_MUTABLE_KEYSIZE): add default, this overrides the one in MutableFileNode	2009-06-29 15:31:24 -07:00
Brian Warner	c6ae255847	remove trailing whitespace	2009-06-29 13:03:58 -07:00
Brian Warner	dddc2d0378	remove trailing whitespace from some util classes	2009-06-29 13:03:41 -07:00
Brian Warner	d0f80579a5	repairer: raise a better exception when faced with a readonly filenode. Still ... produces an error, though.	2009-06-25 23:32:30 -07:00
Brian Warner	97ffc8a418	mutable/filenode.py: set _writekey to None, rather than leaving it missing ... This will at least turn the really really weird error when a repair of a readonly mutable file is attempted into a merely really weird assertion that mentions "repair currently requires a writecap".	2009-06-25 23:20:22 -07:00
Brian Warner	b43672c9f5	one last tweak	2009-06-25 22:26:27 -07:00
Brian Warner	08effc6499	more 'tahoe cp' docs tweaks	2009-06-25 22:16:52 -07:00
Brian Warner	b30db031ab	cli.py: minor tweaks to test posthooks	2009-06-25 19:45:57 -07:00
Brian Warner	df1ddf83f5	cli.py: update comments on 'tahoe cp' --help a bit	2009-06-25 19:38:37 -07:00
Brian Warner	8d5cc756f0	cli: add some --help text to 'tahoe cp'	2009-06-25 16:57:51 -07:00
Brian Warner	2adc184668	make it possible to add/renew-lease from the WUI ... add add/renew-lease checkbox on the "more info" page check/deep-check forms	2009-06-25 16:18:24 -07:00
Brian Warner	2947da457e	hush pyflakes	2009-06-24 19:18:09 -07:00
Brian Warner	bd6ecc9f44	Split out NoSharesError, stop adding attributes to NotEnoughSharesError, change humanize_failure to include the original exception string, update tests, behave better if humanize_failure fails.	2009-06-24 19:17:07 -07:00
Brian Warner	837733641f	remove introducer/old.py, will create something similar when the RIIntroducer changes	2009-06-24 12:40:38 -07:00
midnightmagic	96cda877ff	Basically just a trivial platform detection patch for NetBSD.	2009-06-17 16:36:47 -07:00
Brian Warner	f573bb2929	check_grid.py: print stderr when a subcommand fails	2009-06-22 19:28:33 -07:00
Brian Warner	8df15e9f30	big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging	2009-06-22 19:10:47 -07:00
Brian Warner	546266c806	web/welcome.xhtml: remove trailing whitespace	2009-06-22 19:09:09 -07:00
Brian Warner	aa23ff9180	rrefutil: add trap_deafref utility, to make the callRemote-plus-ignore-DeadReferenceError-plug-log-other-errors pattern easier	2009-06-22 19:08:26 -07:00
Brian Warner	699510c8f1	PollMixin: snoop trial's error observer to halt the test early if an error is seen. This turns a lot of timeouts into fast failures.	2009-06-22 19:07:31 -07:00
Brian Warner	711c09bc5d	clean up storage_broker interface: should fix #732	2009-06-21 16:51:19 -07:00
Brian Warner	f14004eeb3	hush pyflakes with recent FileTooLarge removal	2009-06-21 16:17:57 -07:00
kevan	efcc45951d	change max filesize limit tests ... Instead of testing to see that the previous SDMF filesize limit was being obeyed, we now test to make sure that we can insert files larger than that limit.	2009-06-20 14:28:22 -07:00
kevan	db939750a8	remove upper limit on SDMF filesize	2009-06-20 14:31:30 -07:00
Brian Warner	a6caae9b5d	immutable/download: instrument do-you-have-block responses to investigate #732	2009-06-20 21:12:09 -07:00
Brian Warner	2c5aa12151	test_util: add known-answer tests for hashutil tags	2009-06-17 21:57:09 -07:00
Zooko O'Whielacronx	6a93caf7a3	tests: bump up a timeout that expired on Zandr's box	2009-06-13 12:57:03 -07:00
Zooko O'Whielacronx	cc2953e663	util: hooray! A clean implementation of this simple utility! Black Dew pointed out that the inverse of time.gmtime() is hidden in the "calendar" module.	2009-06-13 09:01:12 -07:00
Zooko O'Whielacronx	45928315f6	util: Brian's horrible hack to figure out how much localtime and utctime differ. Now we'll see if it works on Windows.	2009-06-12 13:45:56 -07:00
Zooko O'Whielacronx	4a30c5899c	util: oops, time.tzset() doesn't work on Windows -- hopefully the new "London" unit test passes on Windows when we skip tzset() on platforms that don't have it	2009-06-11 17:09:20 -07:00
Zooko O'Whielacronx	8978cb0738	util: fix time_format.iso_utc_time_to_seconds() so that it works even in London	2009-06-11 15:11:29 -07:00
Zooko O'Whielacronx	0ed65fba34	tests: significantly increase timeouts that triggered on Zandr's ARM box	2009-06-10 09:10:43 -07:00
Zooko O'Whielacronx	69a3b23b34	test: multiple by 10 or so all timeouts that Zandr's ARM box just overran	2009-06-10 05:56:39 -07:00
Zooko O'Whielacronx	bdfb80834f	tests: bump up timeout on a test that timed out on draco	2009-06-09 21:46:28 -07:00
Zooko O'Whielacronx	974295f963	tests: raise the timeout for test_cli since Zandr's ARM machine totally burst through the old one	2009-06-09 14:05:09 -07:00
Brian Warner	2e35648901	test_cli.Backup: increase timeout massively, it takes 1200s on zandr's ARM linkstation	2009-06-08 22:28:01 -07:00
Zooko O'Whielacronx	3cfc00f5cf	tests: double the timeouts on some tests which time-out on Francois's box	2009-06-08 19:17:53 -07:00
Zooko O'Whielacronx	0bc116b3a2	tests: bump up timeouts so that the tests can finish before timeout on Francois's little arm box	2009-06-08 15:55:57 -07:00
Zooko O'Whielacronx	c3e9d36d97	tests: increase timeouts on some other tests that timed-out on Francois's arm box	2009-06-05 07:34:37 -07:00
Zooko O'Whielacronx	63c8f0c424	tests: bump up the timeout on a bunch of tests that took longer than the default timeout (120s) on François Lenny-armv5tel	2009-06-04 20:14:44 -07:00
Zooko O'Whielacronx	8c18ac38a9	backup: remove the --no-backupdb command, the handling of "can't import sqlite", and the related tests, and change an error message to more correctly indicate failure to load the database from disk rather than failure to import sqlite module ... Fixes #728.	2009-06-04 10:31:31 -07:00
Zooko O'Whielacronx	2841d6a266	setup: add pysqlite and sqlite to get_package_versions()	2009-06-04 08:37:28 -07:00
Brian Warner	35b3f7f426	more refactoring: move get_all_serverids() and get_nickname_for_serverid() from Client to storage_broker	2009-06-01 20:07:50 -07:00
Brian Warner	b1290633b8	more storage_broker refactoring: downloader gets a broker instead of a client, ... use Client.get_storage_broker() accessor instead of direct attribute access.	2009-06-01 19:25:11 -07:00
Brian Warner	3a1196701f	test_runner.py: remove test_client_no_noise: the issue in question is ... ticketed in http://divmod.org/trac/ticket/2830 and doesn't need a Tahoe-side change, plus this test fails on win32 for unrelated reasons (and test_client is the place to think about the win32 issue).	2009-06-01 15:50:07 -07:00
Brian Warner	4177a3616b	remove plaintext-hashing code from the helper interface, to close #722 ... and deny the Helper the ability to mount a partial-information-guessing attack. This will probably break compatibility between new clients and very old (pre-1.0) helpers.	2009-06-01 15:49:16 -07:00
Brian Warner	c516361fd2	start to factor server-connection-management into a distinct 'StorageServerFarmBroker' object, separate from the client and the introducer. This is the starting point for #467: static server selection	2009-06-01 14:06:04 -07:00
Brian Warner	d29281c9c5	mutable: catch and display first error, so code bugs which break all servers get displayed better	2009-06-01 14:04:07 -07:00
Kevin Reid	23441389a5	Modify markup of Tahoe web pages to be more amenable to styling; some minor changes of wording.	2009-05-26 16:25:45 -07:00
Kevin Reid	a71c914e50	Tweak wording in directory page: not-read-only is "modifiable", mention creating a directory _in this directory_.	2009-05-26 16:24:14 -07:00
Kevin Reid	3e80676a29	Comment on duplication of code/markup found during styling project.	2009-05-03 13:34:42 -07:00
Kevin Reid	d0b4fd4689	Add CSS styles to spiff up the Tahoe WUI's appearance, particularly the welcome page and directories.	2009-05-03 13:31:42 -07:00
Kevin Reid	35d3671f4e	Link all Tahoe web pages to the /tahoe_css stylesheet which already exists.	2009-05-03 13:25:33 -07:00
Kevin Reid	2f3842954b	Fix broken link from Provisioning to Reliability page.	2009-05-01 12:10:50 -07:00
Brian Warner	de83526acd	immutable/encode.py: tolerate immediate _remove_shareholder by copying the ... landlord list before iterating over it. This can probably only happen in unit tests, but cleaning it up makes certain test failures easier to analyze.	2009-05-22 11:44:24 -07:00
Brian Warner	1863aee0aa	switch to using RemoteException instead of 'wrapped' RemoteReferences. Should fix #653, the rref-EQ problem	2009-05-21 17:46:32 -07:00
Brian Warner	c9803d5217	switch all foolscap imports to use foolscap.api or foolscap.logging	2009-05-21 17:38:23 -07:00
Brian Warner	b542d5aca6	test_runner.py: fix minor typo	2009-05-19 20:36:20 -07:00
Brian Warner	9f0bc04d32	test_introducer.py: add a test for the python2.4.0/2.4.1 bug in base64.b32decode	2009-05-18 20:41:01 -07:00
Brian Warner	79437baade	immutable WriteBucketProxy: use pipeline to speed up uploads by overlapping roundtrips, for #392	2009-05-18 16:44:22 -07:00
Brian Warner	e76c6b606f	util/pipeline.py: new utility class to manage size-limited work pipelines, for #392	2009-05-18 16:43:26 -07:00
Zooko O'Whielacronx	718537ab8c	tests: mark test_runner as coded in utf-8 instead of ascii	2009-05-07 15:31:51 -07:00
Zooko O'Whielacronx	04ff9a3f97	tests: raise timeout on test_runner.RunNode.test_introducer from 120s to 240s, since it hit the 120s time-out on François Lenny-armv5tel	2009-05-07 14:50:12 -07:00
Zooko O'Whielacronx	77b578928e	trivial: fix comment	2009-04-13 10:41:38 -07:00
Zooko O'Whielacronx	6350634ee3	tests: remove obsolete test (it tests functionality that is long gone) which occasionally ERRORs now that we have more metadata (since [20090411225205-92b7f-7adfb89cb4db4ac7d28427934dea3d2c108f6476])	2009-04-12 19:36:21 -07:00
Zooko O'Whielacronx	6b9899ade1	trivial: remove trailing whitespace and unused import	2009-04-11 19:17:42 -07:00
Zooko O'Whielacronx	9729753692	dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API ... Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.	2009-04-11 15:52:05 -07:00
Zooko O'Whielacronx	5b02896307	rollback [20090226150237-b2345-1e916a746a7f4627b050f02f0e442fae5caf69d4] for 1.4.0 release; #645	2009-04-11 11:19:06 -07:00
Brian Warner	54952e9bea	#622: disable 'Repair' button on check-results page until we make it work correctly	2009-04-09 16:59:59 -07:00
Brian Warner	922b3034be	web: make sure that PUT /uri?mutable=false really means immutable, fixes #675	2009-04-07 19:13:40 -07:00
Zooko O'Whielacronx	37c6e77764	wui: edit some of the human-readable parts of the wui such as button labels ... (The word "parent" suggests that you can go up a directory hierarchy -- perhaps that word is vestigial.)	2009-04-07 11:54:59 -07:00
Zooko O'Whielacronx	8b1cd154da	wui: fix bug in which empty directory is marked as "unreadable", add test, remove exclamation point	2009-04-07 11:28:34 -07:00
Brian Warner	40c78297e8	hashtree.py: another micro-optimization, expand on a comment	2009-04-03 21:35:27 -07:00
Zooko O'Whielacronx	de8475389b	hashtree: fix tests of depth_of	2009-04-03 16:56:14 -07:00
Zooko O'Whielacronx	3ac5f427ba	trivial: source code metadata	2009-04-03 16:33:15 -07:00
Zooko O'Whielacronx	a07e746d9c	trivial: whitespace	2009-04-03 16:33:02 -07:00
Zooko O'Whielacronx	d381bdc905	hashtree: a couple of tiny improvements ... * calculate depth-first with math instead of traversing the actual tree * don't mark a node with a red dot if you instead compare it with an extant hash value (tiny optimization) * edit a comment about checking the root node	2009-04-03 16:30:37 -07:00
Zooko O'Whielacronx	b12a7f9ee8	leases, time_format: modify time stamping in lease description ... * emit lease expiry date in ISO-8601'ish format as well as Brian's format * rename iso_utc_time_to_localseconds() to iso_utc_time_to_seconds() * add iso_utc_date() * simplify the body of iso_utc_time_to_seconds()	2009-04-03 15:59:04 -07:00
Brian Warner	466014f66f	hashtree: fix O(N**2) behavior, to improve fatal alacrity problems in a 10GB file (#670). Also improve docstring.	2009-03-31 13:21:27 -07:00
Brian Warner	bd93430c53	expirer: include crawler progress in the JSON status output	2009-03-24 13:51:37 -07:00
Brian Warner	b5d4972a7a	move GC docs out of proposed/, since it's all implemented now. Add reference to configuration.txt . Add expire.*= suggestions to tahoe.cfg .	2009-03-23 16:08:20 -07:00
Brian Warner	5e8c31c3b6	storage: use constant-time comparison for write-enablers and lease-secrets	2009-03-22 20:21:28 -07:00
Brian Warner	01e2032669	hashutil: add constant-time comparison function, to avoid timing attacks when python's short-circuiting data-dependent == operator is used to, say, check a write-enabler	2009-03-22 20:20:55 -07:00
Brian Warner	6599eae6f9	WUI: fix display of empty directories, it threw an exception before	2009-03-20 16:58:09 -07:00
Brian Warner	4f4d748fd9	storage webstatus: insert spaces when we're configured to expire multiple sharetypes	2009-03-20 15:44:50 -07:00
Brian Warner	3b65607926	expirer: tolerate empty buckets, refactor bucketsize-counting code a bit, don't increment -mutable/-immutable counters unless we actually know the sharetype	2009-03-20 12:18:16 -07:00
Brian Warner	8645738c77	storage: improve wording of status message	2009-03-19 11:48:37 -07:00
Brian Warner	66fe8ca24c	parse_date: insist that it returns an int	2009-03-19 11:44:43 -07:00
Brian Warner	186b6a8c01	storage status: report expiration-cutoff-date like 19-Mar-2009 (as opposed to the tahoe.cfg input format of 2009-03-19), for redundancy: someone who gets the month and day switched will have a better chance to spot the problem in the storage-status output if it's in a different format	2009-03-19 11:07:56 -07:00
Brian Warner	f0071c2571	expirer: clean up constructor args, add tahoe.cfg controls, use cutoff_date instead of date_cutoff	2009-03-18 18:00:09 -07:00
Brian Warner	babcf632da	util/time_format: new routine to parse dates like 2009-03-18, switch expirer to use it. I'd prefer to use 18-Mar-2009, but it is surprisingly non-trivial to build a parser that will take UTC dates instead of local dates	2009-03-18 17:58:14 -07:00
Brian Warner	8eaee28550	expirer: change setup, config options, in preparation for adding tahoe.cfg controls	2009-03-18 17:21:38 -07:00
Brian Warner	fffab0d724	expirer: track mutable-vs-immutable sharecounts and sizes, report them on the web status page for comparison	2009-03-18 13:25:04 -07:00
Brian Warner	406fdba61f	add utility function to parse durations, for lease-expiration config	2009-03-17 00:01:17 -07:00
Brian Warner	24ab5ec26f	expirer: add mode to expire only-mutable or only-immutable shares	2009-03-16 23:51:18 -07:00
Brian Warner	475e7b8804	tahoe_ls.py: add comment about error cases to improve	2009-03-16 22:12:06 -07:00
Brian Warner	c7254c5f1d	GC: add date-cutoff -based expiration, add proposed docs	2009-03-16 22:10:41 -07:00
Brian Warner	2b525a42d3	test_deepcheck: remove the 10s timeout: our dapper buildslave requires 30s, and the reduced timeout was only there because this tests fails by timeout rather than explicitly	2009-03-15 17:51:34 -07:00
Brian Warner	f0b0ad1c8f	tahoe cp -r: add --caps-only flag, to write filecaps into local files instead of actual file contents. Used only for debugging and as a quick tree-comparison tool.	2009-03-15 16:19:58 -07:00
Brian Warner	6e57576f2e	dirnode deep_traverse: insert a turn break (fireEventually) at least once every 100 files, otherwise a CHK followed by more than 158 LITs can overflow the stack, sort of like #237.	2009-03-13 16:31:35 -07:00
Brian Warner	e59164e0bb	consolidate: remove pointless 'else' after for loop	2009-03-13 01:27:51 -07:00
Brian Warner	ae64ddf8e6	consolidate: add eta, flush stdout	2009-03-13 01:24:51 -07:00
Brian Warner	06bf443e91	consolidate: tolerate unicode dirnames	2009-03-12 23:54:02 -07:00
Brian Warner	1a741fdb03	dirnode.py: when doing deep-traverse, walk each directory in alphabetical order, to make things like 'manifest' more predictable	2009-03-12 23:50:46 -07:00
Brian Warner	a18f8d4cc7	consolidator: add progress to scan-old-directory passes	2009-03-12 22:47:28 -07:00
Brian Warner	ddb2f1a6e3	consolidator: fix cycle detection to not trigger on merely shared directories, add snapshot counter to progress	2009-03-12 21:22:29 -07:00
Brian Warner	ddc9a7ae74	consolidator: re-use more directories, add total directories seen-vs-used counts	2009-03-12 20:48:01 -07:00
Brian Warner	809ec25ffa	tahoe_backup.py: tolerate more time formats	2009-03-12 18:16:00 -07:00
Brian Warner	1c24707f19	consolidator: add more verbose traversal of directories	2009-03-12 16:29:00 -07:00
Brian Warner	760688a224	consolidate: create multiple numbered backups of the original Archives directory, not just the first time	2009-03-12 16:04:27 -07:00
Brian Warner	c4c6a62954	add 'tahoe debug consolidate' command, to merge directories created by repeated 'tahoe cp -r' or the allmydata win32 backup tool, into the form that would have been created by 'tahoe backup'.	2009-03-12 13:56:06 -07:00
Brian Warner	969b758bf5	tahoe_cp.py: return 0 for success, instead of None	2009-03-12 13:53:45 -07:00
Brian Warner	1ccd426a34	expirer: fix prediction math, thanks to Zandr for the catch	2009-03-09 13:42:17 -07:00
Brian Warner	8708045a98	storage.expirer: oops, fix upgrade-handler code	2009-03-08 20:55:16 -07:00
Brian Warner	df3f7f93e0	storage.expirer: handle upgrades better	2009-03-08 20:42:20 -07:00
Brian Warner	a68ad06254	storage.expirer: exercise the last missing line of webstatus code	2009-03-08 20:38:28 -07:00
Brian Warner	6d6049430c	expirer: tolerate corrupt shares, add them to the state and history for future examination	2009-03-08 20:08:40 -07:00
Brian Warner	1a98521c3d	storage/immutable: raise a specific error upon seeing a bad version number, instead of using assert. Also wrap to 80cols.	2009-03-08 20:07:32 -07:00
Brian Warner	02b40ec499	storage: add test coverage for BucketReader.__repr__	2009-03-08 20:05:27 -07:00
Brian Warner	6d7319c588	storage/mutable: raise a specific error upon seeing bad magic, instead of using assert	2009-03-08 19:02:01 -07:00
Zooko O'Whielacronx	7e7167f2e5	nodeadmin: node stops itself if a hotline file hasn't been touched in 120 seconds now, instead of in 60 seconds ... A test failed on draco (MacPPC) because it took 67.1 seconds to get around to running the test, and the node had already stopped itself when the hotline file was 60 seconds old.	2009-03-07 20:50:39 -07:00
Brian Warner	5675b4e7e0	expirer: make web display a bit more consistent	2009-03-07 16:14:42 -07:00
Brian Warner	df045650e0	web/storage.py: tolerate unknown-future displays, I'm not sure why LeaseCrawler.test_unpredictable_future didn't catch this	2009-03-07 16:02:43 -07:00
Brian Warner	192c161be7	tahoe_cp.py: improve error reporting slightly: don't json-interpret HTTP errors, pass through tahoe webapi error messages	2009-03-07 05:40:51 -07:00
Brian Warner	2c2c3c2295	test_cli.py: assert that 'ls' on an unrecoverable file now gives a better error message	2009-03-07 05:08:15 -07:00
Brian Warner	67d7600781	test_web: test behavior of broken-dirnode GET, both html and json	2009-03-07 04:57:07 -07:00
Brian Warner	950200fece	web: when a dirnode can't be read, emit a regular HTML page but with the child-table and upload-forms replaced with an apologetic message. Make sure to include the 'get info' links so the user can do a filecheck	2009-03-07 04:56:01 -07:00
Brian Warner	badd79671c	web/common: split out exception-to-explanation+code mapping to a separate humanize_failure() function, so it can be used by other code. Add explanation for mutable UnrecoverableFileError.	2009-03-07 04:54:08 -07:00
Brian Warner	9c7fafd5af	test_cli: use explicit (and stable) testdirs, instead of using self.mktemp	2009-03-07 03:04:28 -07:00
Brian Warner	091cfd04ca	test_storage: solaris doesn't appear to give the same block count as other platforms, so don't assert as much about 'diskbytes' recovered	2009-03-07 02:45:18 -07:00
Brian Warner	24ebdb3954	tahoe_cp.py: don't be verbose unless --verbose is passed: default is one line of success	2009-03-07 02:44:21 -07:00
Brian Warner	0dee2a6036	storage: add a lease-checker-and-expirer crawler, plus web status page. ... This walks slowly through all shares, examining their leases, deciding which are still valid and which have expired. Once enabled, it will then remove the expired leases, and delete shares which no longer have any valid leases. Note that there is not yet a tahoe.cfg option to enable lease-deletion: the current code is read-only. A subsequent patch will add a tahoe.cfg knob to control this, as well as docs. Some other minor items included in this patch: tahoe debug dump-share has a new --leases-only flag storage sharefile/leaseinfo code is cleaned up storage web status page (/storage) has more info, more tests coverage space-left measurement on OS-X should be more accurate (it was off by 2048x) (use stat .f_frsize instead of f_bsize)	2009-03-06 22:45:17 -07:00
Zooko O'Whielacronx	e9199a89ab	trivial: remove unused import detected by pyflakes, and remove trailing whitespace	2009-03-05 16:32:04 -07:00
Zooko O'Whielacronx	5e90d82a02	trivial: use more specific function for ascii-encoding storage index	2009-02-22 11:57:51 -07:00
Brian Warner	c5ecb7b3ba	test_cli: validate non-HTML error response of 'tahoe get' on an unrecoverable file	2009-03-03 22:11:46 -07:00