3349 Commits

Author	SHA1	Message	Date
jacob.lyles	e276050d5a	abbreviate time edge case python2.5 unit test	2010-07-29 14:06:38 -07:00
david-sarah	23c649c897	test_upload.py: rename test_problem_layout_ticket1124 to test_problem_layout_ticket_1124 -- fix .todo reference.	2010-07-29 08:29:27 -07:00
david-sarah	0ebe18cc73	test_upload.py: rename test_problem_layout_ticket1124 to test_problem_layout_ticket_1124 for consistency.	2010-07-29 07:22:50 -07:00
Zooko O'Whielacronx	28822dfb4e	tests: remove debug print	2010-08-01 23:33:39 -07:00
Zooko O'Whielacronx	69c48ebde6	immutable: tidy-up some code by using a set instead of list to hold homeless_shares	2010-08-01 23:20:04 -07:00
Zooko O'Whielacronx	fd874d6669	setup: fix a couple instances of hard-coded 'allmydata-tahoe' in the scripts, tighten the tests (as suggested by David-Sarah)	2010-08-01 09:42:07 -07:00
Zooko O'Whielacronx	44edecb36c	setup: replace hardcoded 'allmydata-tahoe' with allmydata.__appname__	2010-08-01 09:05:17 -07:00
Zooko O'Whielacronx	0f94923f22	immutable: use PrefixingLogMixin to organize logging in Tahoe2PeerSelector and add more detailed messages about peer	2010-07-19 01:20:00 -07:00
Zooko O'Whielacronx	2ebe2c2ff3	benchmarking: update bench_dirnode to be correct and use the shiniest new pyutil.benchutil features concerning what units you measure in	2010-07-18 21:49:48 -07:00
Zooko O'Whielacronx	9b4f412e49	trivial: rename and add in-line doc to clarify "used_peers" => "upload_servers"	2010-07-18 21:47:44 -07:00
jacob.lyles	d3e6e15e23	web: don't use %d formatting on a potentially large negative float -- there is a bug in Python 2.5 in that case ... fixes #1055	2010-07-30 15:05:50 -07:00
Zooko O'Whielacronx	1c62db0db8	docs: use current cap to Zooko's wiki page in example text ... fixes #1134	2010-07-20 18:05:43 -07:00
david-sarah	94bec75dda	__init__.py: silence DeprecationWarning about BaseException.message globally. fixes #1129	2010-07-19 18:19:39 -07:00
david-sarah	bdc6fd2f8b	test_runner: test that 'tahoe --version' outputs no noise (e.g. DeprecationWarnings).	2010-07-19 18:13:45 -07:00
Zooko O'Whielacronx	c076c0b19b	immutable: add test case of #1128, fix test case of #1118	2010-07-19 01:16:12 -07:00
Zooko O'Whielacronx	13b5e44fbc	immutable: fix bug in which preexisting_shares and merged were shallowly referencing the same sets ... This bug had the effect of making uploads sometimes (rarely) appear to succeed when they had actually not distributed the shares well enough to achieve the desired servers-of-happiness level.	2010-07-19 00:54:26 -07:00
david-sarah	461f946457	upload.py: fix #1118 by aborting newly-homeless buckets when reassignment runs. This makes a previously failing assert correct. This version refactors 'abort' into two methods, rather than using a default argument.	2010-07-18 21:46:55 -07:00
Zooko O'Whielacronx	1dbfcf753d	immutable: extend the tests to check that the shares that got uploaded really do make a sufficiently Happy distribution ... This patch also renames some instances of "find_shares()" to "find_all_shares()" and other instances to "find_uri_shares()" as appropriate -- the conflation between those names confused me at first when writing these tests.	2010-07-18 21:50:47 -07:00
Zooko O'Whielacronx	2e83f243c2	immutable: test for #1118	2010-07-18 15:15:37 -07:00
Zooko O'Whielacronx	0f46766a51	immutable: test for #1124	2010-07-18 15:29:07 -07:00
Zooko O'Whielacronx	8f201c75f0	trivial: fix unused import (sorry about that, pyflakes)	2010-07-18 14:51:33 -07:00
Zooko O'Whielacronx	16bb529339	tests, NEWS, CREDITS re: #1117 ... Give Brian and Kevan promotions, move release date in NEWS to the 18th, commit Brian's test for #1117. fixes #1117	2010-07-18 13:32:25 -07:00
Kevan Carstensen	054c4553b5	test/test_upload.py: test to see that aborted buckets are ignored by the storage server	2010-07-15 17:10:46 -07:00
Kevan Carstensen	677f3b1f63	test/test_storage.py: test for the new remote_abort semantics.	2010-07-15 16:21:48 -07:00
Kevan Carstensen	6374f4307f	storage/immutable.py: make remote_abort btell the storage server about aborted buckets.	2010-07-15 16:21:05 -07:00
Kevan Carstensen	5becaf0309	test/test_upload.py: changes to test plumbing for #1117 tests ... - Add a callRemoteOnly method to FakeBucketWriter. - Change the abort method in FakeBucketWriter to not return a RuntimeError.	2010-07-15 16:18:20 -07:00
Kevan Carstensen	6d2c216b8f	immutable/upload.py: abort buckets if peer selection fails	2010-07-15 16:17:14 -07:00
david-sarah	638a3b2af4	test_encodingutil: correct an error in the previous patch to StdlibUnicode.test_open_representable.	2010-07-18 08:14:20 -07:00
david-sarah	74c41ebb8b	test_dirnode and test_web: don't use failUnlessReallyEqual in cases where the return type from simplejson.loads can vary between unicode and str. Use to_str when comparing URIs parsed from JSON.	2010-07-18 07:29:15 -07:00
david-sarah	9a4a864573	test_encodingutil: StdlibUnicode.test_open_representable no longer uses a mock.	2010-07-18 05:54:12 -07:00
Zooko O'Whielacronx	7cc98759bd	docs: add comment clarifying #1051	2010-07-17 22:32:50 -07:00
david-sarah	d346e0853d	Add tests of caps from the future that have non-ASCII characters in them (encoded as UTF-8). The changes to test_uri.py, test_client.py, and test_dirnode.py add tests of non-ASCII future caps in addition to the current tests. The changes to test_web.py just replace the tests of all-ASCII future caps with tests of non-ASCII future caps. We also change uses of failUnlessEqual to failUnlessReallyEqual, in order to catch cases where the type of a string is not as expected.	2010-07-11 13:02:52 -07:00
Zooko O'Whielacronx	3c4fe0706f	M-x whitespace-cleanup	2010-07-17 20:27:39 -07:00
Zooko O'Whielacronx	6a7023efbe	benchmarking: update bench_dirnode.py to reflect the new directory interfaces	2010-07-17 20:17:10 -07:00
david-sarah	bdb10553eb	test_encodingutil: fix test_open_representable, which is only valid when run on a platform for which we know an unrepresentable filename.	2010-07-17 20:03:33 -07:00
francois	968edfda11	iputil.py: Add support for FreeBSD 7,8 and 9 ... committed by: zooko@zooko.com date: 2010-07-17 and I also patched: NEWS and CREDITS	2010-07-17 19:28:32 -07:00
david-sarah	752897450d	fileutil: docstrings for non-obvious usage restrictions on methods of EncryptedTemporaryFile.	2010-07-16 22:46:47 -07:00
david-sarah	05022dca36	Move EncryptedTemporaryFile from SFTP frontend to allmydata.util.fileutil, and make the FTP frontend also use it (fixing #1083).	2010-07-11 14:37:21 -07:00
david-sarah	a8161c915a	Correct stringutils->encodingutil patch to be the newer version, rather than the old version that was committed in error.	2010-07-17 18:34:35 -07:00
david-sarah	0d79a4a7d1	test_cli.py: fix error that crept in when rebasing the patch for #1072.	2010-07-17 17:01:23 -07:00
david-sarah	c59f9b162f	stringutils: add test for when sys.stdout has no encoding attribute (fixes #1099).	2010-07-16 21:58:16 -07:00
david-sarah	8090785716	CLI: add 'tahoe unlink' as an alias to 'tahoe rm', for forward-compatibility.	2010-07-17 15:04:11 -07:00
Zooko O'Whielacronx	6e8477114e	minor code clean-up in dirnode.py ... Impose micro-POLA by passing only the writekey instead of the whole node object to {{{_encrypt_rw_uri()}}}. Remove DummyImmutableFileNode in nodemaker.py, which is obviated by this. Add micro-optimization by precomputing the netstring of the empty string and branching on whether the writekey is present or not outside of {{{_encrypt_rw_uri()}}}. Add doc about writekey to docstring. fixes #967	2010-07-13 23:02:55 -07:00
david-sarah	11077ea74d	Rename stringutils to encodingutil, and drop listdir_unicode and open_unicode (since the Python stdlib functions work fine with Unicode paths). Also move some utility functions to fileutil.	2010-07-11 17:30:15 -07:00
david-sarah	fa0fd66e17	Allow URIs passed in the initial JSON for t=mkdir-with-children, t=mkdir-immutable to be Unicode. Also pass the name of each child into nodemaker.create_from_cap for error reporting.	2010-07-11 12:55:25 -07:00
david-sarah	e2ef2c2691	cli.py: make command descriptions consistently end with a full stop.	2010-07-13 18:45:38 -07:00
david-sarah	15ddab08ed	SFTP: address some of the comments in zooko's review (#1106).	2010-07-11 19:55:37 -07:00
david-sarah	3b1b0147a8	test_storage.py: potential fix for failures when logging is enabled.	1970-07-12 21:05:46 -07:00
terrellrussell	d0381e679e	upcase_since_on_welcome	2010-07-08 12:39:03 -07:00
freestorm77	35ec8f6ac2	server_version_on_welcome_page.dpatch.txt ... - The storage server version is 0 for all storage nodes in the Welcome Page	2010-06-05 12:17:21 -07:00
freestorm77	496b91717a	directory_html_top_banner.dpatch ... The div tag with the link "Return to Welcome page" on the directory.xhtml page is not correct	2010-06-22 13:53:01 -07:00
freestorm77	eb63ba9a26	tahoe_css_toolbar.dpatch ... CSS modification to be correctly diplayed with Internet Explorer 8 The links on the top of page directory.xhtml are not diplayed in the same line as display with Firefox.	2010-06-22 14:00:46 -07:00
freestorm77	97579af73c	runnin_test_tahoe_css.dpatch ... Runnin test for changes in tahoe.css file	2010-06-22 14:47:14 -07:00
freestorm77	0304918a77	runnin_test_directory_xhtml.dpatch ... Runnin test for diretory.xhtml top banner	2010-06-22 13:14:03 -07:00
david-sarah	b8df25a8d1	stringutils.py: tolerate sys.stdout having no 'encoding' attribute.	2010-06-25 21:08:17 -07:00
david-sarah	88169328be	SFTP: don't call .stopProducing on the producer registered with OverwriteableFileConsumer (which breaks with warner's new downloader).	2010-06-28 16:19:26 -07:00
Zooko O'Whielacronx	ab80719198	trivial: tiny update to in-line comment ... (I'm actually committing this patch as a test of the new eager-annotation-computation of trac-darcs.)	2010-06-13 21:57:15 -07:00
david-sarah	6b2f99fa9a	test_backupdb.py: skip test_unicode if we can't represent the test filenames.	2010-06-18 19:26:20 -07:00
david-sarah	a2caf6f4b2	test_web.py: correct a test that was missed in the change to not write ctime/mtime.	2010-06-18 19:17:18 -07:00
david-sarah	4712875193	dirnode.py: stop writing 'ctime' and 'mtime' fields. Includes documentation and test changes.	2010-06-18 16:01:19 -07:00
david-sarah	72e395d878	test_storage.py: print more information on test failures.	2010-06-16 20:46:23 -07:00
david-sarah	390fc78a9a	test_cli.py: fix test failure in CLI.test_listdir_unicode_good due to filenames returned from listdir_unicode no longer being normalized.	2010-06-17 21:51:10 -07:00
david-sarah	178401eb4e	tahoe backup: unicode tests.	2010-06-17 20:52:11 -07:00
david-sarah	1a0674bf37	CLI: allow Unicode patterns in exclude option to 'tahoe backup'.	2010-06-16 20:39:01 -07:00
david-sarah	a9fe3792de	dirnodes: fix normalization hole where childnames in directories created by nodemaker.create_mutable/immutable_directory would not be normalized. Add a test that we normalize names coming out of a directory.	2010-06-17 17:02:49 -07:00
david-sarah	718870a796	dirnode.py: comments about normalization changes.	2010-06-16 21:14:11 -07:00
david-sarah	7e7644589a	stringutils.py: remove unused import.	2010-06-16 20:44:40 -07:00
david-sarah	5ada31034b	test_stringutils.py: take account of the output of listdir_unicode no longer being normalized. Also use Unicode escapes, not UTF-8.	2010-06-16 20:44:09 -07:00
david-sarah	025aede9e4	test_dirnode.py: partial tests for normalization changes.	2010-06-16 20:40:25 -07:00
david-sarah	c1412c8a81	SFTP: get 'ctime' attribute from 'tahoe:linkmotime'.	2010-06-16 20:37:44 -07:00
david-sarah	9f5488b2d1	stringutils.py: don't NFC-normalize the output of listdir_unicode.	2010-06-16 18:55:37 -07:00
david-sarah	e2c7ad1d88	stringutils.py: Add encoding argument to quote_output. Also work around a bug in locale.getpreferredencoding on older Pythons.	2010-06-15 21:20:12 -07:00
david-sarah	c8d99b77a3	Provisional patch to NFC-normalize filenames going in and out of Tahoe directories.	2010-06-15 20:14:50 -07:00
david-sarah	408b12a7e6	test_system.py: investigate failure in allmydata.test.test_system.SystemTest.test_upload_and_download_random_key due to bytes_sent not being an int	2010-06-15 17:16:48 -07:00
david-sarah	a3acb2b773	SFTP: remove a dubious use of 'pragma: no cover'.	2010-06-13 09:43:56 -07:00
david-sarah	11e88c340a	SFTP: test that renaming onto a just-opened file fails.	2010-06-11 20:37:09 -07:00
david-sarah	b67f8b66c8	SFTP: further small improvements to test coverage. Also ensure that after a test failure, later tests don't fail spuriously due to the checks for heisenfile leaks.	2010-06-11 20:07:37 -07:00
david-sarah	b4e074d250	SFTP: further improve test coverage (paths containing '.', bad data for posix-rename extension, and error in test of openShell).	2010-06-11 14:31:42 -07:00
david-sarah	52f87904ed	SFTP: improve test coverage for no-write on mutable files, and check for heisenfile table leaks in all relevant tests. Delete test_memory_leak since it is now redundant.	2010-06-11 13:57:52 -07:00
david-sarah	546c3d2ed4	SFTP: add test for extension of file opened with FXF_APPEND.	2010-06-10 11:26:47 -07:00
Zooko O'Whielacronx	f19c240ca8	tests: bump up the timeout on this iputil test from 2s to 4s	2010-06-09 07:30:17 -07:00
david-sarah	3c44389440	SFTP: fix most significant memory leak described in #1045 (due to a file being added to all_heisenfiles under more than one direntry when renamed).	2010-06-09 01:00:03 -07:00
david-sarah	5786874d38	test_stringutils.py: Fix test failure on CentOS builder, possibly Python 2.4.3-related.	2010-06-08 23:50:56 -07:00
david-sarah	ffd1c5aca5	Fix for Unicode-related test failures on Zooko's OS X 10.6 machine.	2010-06-08 22:54:48 -07:00
david-sarah	80a89d3931	stringutils.py, sftpd.py: Portability fixes for Python <= 2.5.	2010-06-08 18:33:02 -07:00
david-sarah	32d9deace3	Back out Windows-specific Unicode argument support for v1.7.	2010-06-08 17:08:03 -07:00
david-sarah	4b5927fdb8	SFTP: suppress NoSuchChildError if heisenfile attributes have been updated in setAttrs, in the case where the parent is available.	2010-06-07 23:37:53 -07:00
david-sarah	195b4afa8f	SFTP: ignore permissions when opening a file (needed for sshfs interoperability).	2010-06-07 22:57:00 -07:00
david-sarah	a67e745b26	test_web.py: fix pyflakes warnings introduced by byterange patch.	2010-06-07 21:20:12 -07:00
Jeremy Fitzhardinge	63b28d707b	Improve HTTP/1.1 byterange handling ... Fix parsing of a Range: header to support: - multiple ranges (parsed, but not returned) - suffix byte ranges ("-2139") - correct handling of incorrectly formatted range headers (correct behaviour is to ignore the header and return the full file) - return appropriate error for ranges outside the file Multiple ranges are parsed, but only the first range is returned. Returning multiple ranges requires using the multipart/byterange content type.	2010-03-09 20:59:13 -07:00
Zooko O'Whielacronx	d13784b649	tests: bump up the timeout on these tests; MM's buildslave is sometimes extremely slow on tests, but it will complete them if given enough time. MM is working on making that buildslave more predictable in how long it takes to run tests.	2010-06-07 20:37:54 -07:00
david-sarah	3c883e6e44	test_cli.py: remove invalid 'test_listdir_unicode_bad' test.	2010-06-07 11:37:30 -07:00
david-sarah	71f8757bd1	check_memory.py: adapt to servers-of-happiness changes.	2010-06-07 18:35:28 -07:00
david-sarah	7092de1b6f	Remove the 'tahoe debug consolidate' subcommand.	2010-06-07 11:37:57 -07:00
david-sarah	731e3d68df	common_http.py, tahoe_cp.py: Fix an error in calling the superclass constructor in HTTPError and MissingSourceError (introduced by the Unicode fixes).	2010-06-07 10:47:14 -07:00
Zooko O'Whielacronx	faade8b20a	tests: drastically increase timeout of this very time-consuming test in honor of François's ARM box	2010-06-07 04:59:29 -07:00
david-sarah	8b014372b1	test_stringutils.py: trivial error in exception message for skipped test.	2010-06-06 23:14:55 -07:00
david-sarah	529c9f673a	More Unicode test fixes.	2010-06-06 22:33:58 -07:00
david-sarah	65b6f4e3ce	Unicode fixes for platforms with non-native-Unicode filesystems.	2010-06-06 21:32:38 -07:00
david-sarah	80252debcd	Unicode fixes.	2010-06-06 18:02:15 -07:00
Zooko O'Whielacronx	5bcca5151e	unicode tests: fix missing import	2010-06-04 07:26:30 -07:00
Zooko O'Whielacronx	db8a6f3aa6	unicode: make test_cli test a non-ascii argument, and make the fallback term encoding be locale.getpreferredencoding()	2010-06-04 07:12:51 -07:00
Zooko O'Whielacronx	442008a690	unicode: always decode json manifest as utf-8 then encode for stdout ... pyflakes pointed out that the exception handler fallback called an un-imported function, showing that the fallback wasn't being exercised. I'm not 100% sure that this patch is right and would appreciate François or someone reviewing it.	2010-06-04 01:48:40 -07:00
Zooko O'Whielacronx	952f3bf3ef	fix flakes	2010-06-04 00:58:45 -07:00
Zooko O'Whielacronx	08058c1073	fix syntax of assertion handling that isn't portable to older versions of Python	2010-06-04 00:58:05 -07:00
Francois Deppierraz	0eb4d83937	test_stringutils.py: Skip test test_listdir_unicode_good if filesystem supports only ASCII filenames	2010-05-21 09:08:39 -07:00
Francois Deppierraz	01da7f06e6	test_stringutils.py: Skip test_listdir_unicode on mocked platform which cannot store non-ASCII filenames	2010-05-21 09:05:59 -07:00
Francois Deppierraz	58e9df9f5a	test_stringutils.py: Add a test class for OpenBSD 4.1 with LANG=C	2010-05-21 07:00:53 -07:00
Francois Deppierraz	1aeff62161	test_stringutils.py: Mock the open() call in test_open_unicode ... This test ensure that open(a_unicode_string) is used on Unicode platforms (Windows or MacOS X) and that open(a_correctly_encoded_bytestring) on other platforms such as Unix.	2010-05-21 06:58:17 -07:00
Francois Deppierraz	eecaaca6b7	test_stringutils.py: Fix a trivial Python 2.4 syntax incompatibility	2010-05-21 02:33:45 -07:00
Francois Deppierraz	6c70fb7b3c	test_cli.py: Fix tests when sys.stdout.encoding=None and refactor this code into functions	2010-05-20 01:44:47 -07:00
Francois Deppierraz	496802420e	Fix handling of correctly encoded unicode filenames (#534) ... Tahoe CLI commands working on local files, for instance 'tahoe cp' or 'tahoe backup', have been improved to correctly handle filenames containing non-ASCII characters. In the case where Tahoe encounters a filename which cannot be decoded using the system encoding, an error will be returned and the operation will fail. Under Linux, this typically happens when the filesystem contains filenames encoded with another encoding, for instance latin1, than the system locale, for instance UTF-8. In such case, you'll need to fix your system with tools such as 'convmv' before using Tahoe CLI. All CLI commands have been improved to support non-ASCII parameters such as filenames and aliases on all supported Operating Systems except Windows as of now.	2010-05-19 17:43:56 -07:00
Francois Deppierraz	d0ed14e1bb	stringutils.py: Unicode helper functions + associated tests ... This file contains a bunch of helper functions which converts unicode string from and to argv, filenames and stdout.	2010-05-19 17:41:05 -07:00
david-sarah	a2cff856ff	Resolve merge conflict for sftpd.py	2010-06-03 11:25:37 -07:00
david-sarah	2dcb62de90	SFTP: possible fix for metadata times being shown as the epoch.	2010-06-02 16:45:14 -07:00
david-sarah	c9984c40e9	SFTP: further improvements to test coverage.	2010-06-02 16:44:22 -07:00
david-sarah	2a791b0d05	SFTP: improve test coverage. Also make creating a directory fail when permissions are read-only (rather than ignoring the permissions).	2010-06-01 21:19:34 -07:00
david-sarah	29a06457d2	dirnode.py: fix a bug in the no-write change for Adder, and improve test coverage. Add a 'metadata' argument to create_subdirectory, with documentation. Also update some comments in test_dirnode.py made stale by the ctime/mtime change.	2010-06-01 20:26:41 -07:00
david-sarah	db394671e9	SFTP: fix a bug that caused the temporary files underlying EncryptedTemporaryFiles not to be closed.	2010-05-31 22:53:10 -07:00
david-sarah	de95140b7b	SFTP: changes for #1063 ('no-write' field) including comment:1 (clearing owner write permission diminishes to a read cap). Includes documentation changes, but not tests for the new behaviour.	2010-05-31 22:11:39 -07:00
david-sarah	9faa02deb5	SFTP: the same bug as in _sync_heisenfiles also occurred in two other places.	2010-05-29 23:01:27 -07:00
david-sarah	cccdd32806	SFTP: another try at fixing the _sync_heisenfiles bug.	2010-05-29 22:52:54 -07:00
david-sarah	51b0d8bde3	SFTP: fix silly bug in _sync_heisenfiles ('f is not ignore' vs 'not (f is ignore)').	2010-05-29 22:38:07 -07:00
david-sarah	b9d3cf12b8	SFTP: log when a sync completes.	2010-05-29 22:18:40 -07:00
david-sarah	c4a8af4d27	SFTP: fix bug in previous logging patch.	2010-05-29 22:00:00 -07:00
david-sarah	3395a7e154	SFTP: more logging to track down OpenOffice hang.	2010-05-29 21:08:09 -07:00
david-sarah	4be24a89df	SFTP: avoid blocking close on a heisenfile that has been abandoned or never changed. Also, improve the logging to help track down a case where OpenOffice hangs on opening a file with FXF_READ|FXF_WRITE.	2010-05-29 19:55:44 -07:00
david-sarah	8ed28a95e7	Move suppression of DeprecationWarning about BaseException.message from sftpd.py to main __init__.py. Also, remove the global suppression of the 'integer argument expected, got float' warning, which turned out to be a bug.	2010-05-28 22:05:37 -07:00
david-sarah	e867985539	SFTP: cater to clients that assume a file is created as soon as they have made an open request; also, fix some race conditions associated with closing a file at about the same time as renaming or removing it.	2010-05-28 21:52:53 -07:00
david-sarah	0b888f8201	SFTP: 'sync' any open files at a direntry before opening any new file at that direntry. This works around the sshfs misbehaviour of returning success to clients immediately on close.	2010-05-25 16:02:57 -07:00
david-sarah	a143b1297b	SFTP: handle removing a file while it is open. Also some simplifications of the logout handling.	2010-05-25 11:42:10 -07:00
david-sarah	e81ce2bbd2	SFTP: a posix-rename response should actually return an FXP_STATUS reply, not an FXP_EXTENDED_REPLY as Twisted Conch assumes. Work around this by raising an SFTPError with code FX_OK.	2010-05-24 20:33:23 -07:00
david-sarah	8f86490d48	SFTP: fix problem with posix-rename code returning a Deferred for the renamed filenode, not for the result of the request (an empty string).	2010-05-24 19:02:09 -07:00
david-sarah	210578da63	SFTP: fix time handling to make sure floats are not passed into twisted.conch, and to print times in the future less ambiguously in directory listings.	2010-05-24 16:04:12 -07:00
david-sarah	549dccd608	SFTP: name of the POSIX rename extension should be 'posix-rename@openssh.com', not 'extposix-rename@openssh.com'.	2010-05-23 19:11:56 -07:00
david-sarah	6e114e44b5	SFTP: avoid race condition where .write could be called on an OverwriteableFileConsumer after it had been closed.	2010-05-23 16:38:30 -07:00
david-sarah	c6b414c815	SFTP: log tracebacks for RAISEd exceptions.	2010-05-23 15:15:35 -07:00
david-sarah	725daee788	SFTP: more logging to investigate behaviour of getAttrs(path).	2010-05-23 13:42:36 -07:00
david-sarah	75357fd2d5	SFTP: fix pyflakes warnings; drop 'noisy' versions of eventually_callback and eventually_errback; robustify conversion of exception messages to UTF-8.	2010-05-23 07:09:05 -07:00
david-sarah	38964fb35a	SFTP: fixes and test cases for renaming of open files.	2010-05-22 20:25:49 -07:00
david-sarah	42ecd77f4e	SFTP: Increase test_sftp timeout to cater for francois' ARM buildslave.	2010-05-22 12:16:39 -07:00
david-sarah	5974773969	SFTP: Fix error in support for getAttrs on an open file, to index open files by directory entry rather than path. Extend that support to renaming open files. Also, implement the extposix-rename@openssh.org extension, and some other minor refactoring.	2010-05-21 20:58:36 -07:00
david-sarah	5545a678bf	SFTP tests: fix test_openDirectory_and_attrs that was failing in timezones west of UTC.	2010-05-20 11:10:27 -07:00
david-sarah	ce3872d10e	SFTP: allow getAttrs to succeed on a file that has been opened for creation but not yet uploaded or linked (part of #1050).	2010-05-19 20:56:13 -07:00
david-sarah	7f9e351bce	SFTP: improve logging so that results of requests are (usually) logged.	2010-05-19 17:36:52 -07:00
david-sarah	a62733ed08	SFTP: add tests for more combinations of open flags.	2010-05-18 22:39:33 -07:00
david-sarah	f0945526ce	SFTP: allow FXF_WRITE | FXF_TRUNC (#1050).	2010-05-18 21:32:40 -07:00
david-sarah	bde27cab57	SFTP: remove another case where we were logging data.	2010-05-18 18:27:13 -07:00
david-sarah	d8b096a054	SFTP: avoid logging all data passed to callbacks.	2010-05-18 17:06:51 -07:00
david-sarah	819eaa74c1	SFTP: fixes related to reporting of permissions (needed for sshfs).	2010-05-17 22:45:21 -07:00
david-sarah	8d118e28f1	SFTP: change error code returned for ExistingChildError to FX_FAILURE (fixes gvfs with some picky programs such as gedit).	2010-05-17 17:42:05 -07:00
david-sarah	57699fd1eb	SFTP: fixed bugs that caused hangs during write (#1037).	2010-05-16 21:42:28 -07:00
david-sarah	5f9c10901b	SFTP: work around a probable bug in twisted.conch.ssh.session:loseConnection(). Also some minor error handling cleanups.	2010-05-16 18:26:06 -07:00
david-sarah	6a2b0229f0	SFTP: Support statvfs extensions, avoid logging actual data, and decline shell sessions politely.	2010-05-16 08:43:47 -07:00
david-sarah	31994aaf74	SFTP: fix error in SFTPUserHandler arguments introduced by execCommand patch.	2010-05-15 18:40:45 -07:00
david-sarah	8028bf01bc	SFTP: implement execCommand to interoperate with clients that issue a 'df -P -k /' command. Also eliminate use of Zope adaptation.	2010-05-15 18:27:54 -07:00
david-sarah	6ba6c97a7a	sftpd.py: 'log.OPERATIONAL' should be just 'OPERATIONAL'.	2010-05-15 08:55:33 -07:00
david-sarah	5ab4a376b7	Attempt to fix #1040 by making SFTPUser implement ISession.	2010-05-14 17:57:19 -07:00
david-sarah	7409d1e019	Eliminate Windows newlines from sftpd.py.	2010-05-14 17:56:56 -07:00
david-sarah	14d0721246	Update SFTP implementation and tests: fix #1038 and switch to foolscap logging; also some code reorganization.	2010-05-13 21:31:13 -07:00
david-sarah	5fdd63d0ad	Tests for new SFTP implementation	2010-05-11 23:05:52 -07:00
david-sarah	5248c808ac	New SFTP implementation: mutable files, read/write support, streaming download, Unicode filenames, and more	2010-05-11 22:54:07 -07:00
david-sarah	9214dbda50	Add must_exist, must_be_directory, and must_be_file arguments to DirectoryNode.delete. This will be used to fixes a minor condition in the SFTP frontend.	2010-05-27 12:45:29 -07:00
david-sarah	4556702044	Fix test failures in test_web caused by changes to web page titles in #1062. Also, change a 'target' field to '_blank' instead of 'blank' in welcome.xhtml.	2010-06-03 16:21:05 -07:00
david-sarah	4f58355a0e	dirnode.py: Fix bug that caused 'tahoe' fields, 'ctime' and 'mtime' not to be updated when new metadata is present.	2010-06-01 18:46:44 -07:00
david-sarah	53f7d2c7fe	dirnode.py: Fix #1034 (MetadataSetter does not enforce restriction on setting 'tahoe' subkeys), and expose the metadata updater for use by SFTP. Also, support diminishing a child cap to read-only if 'no-write' is set in the metadata.	2010-05-31 21:54:28 -07:00
david-sarah	027e7701bd	Change doc comments in interfaces.py to take into account unknown nodes.	2010-05-28 10:19:22 -07:00
david-sarah	6d91b1e00e	Trivial whitespace changes.	2010-05-27 12:41:14 -07:00
david-sarah	aac94a8d89	Suppress 'integer argument expected, got float' DeprecationWarning everywhere	2010-05-23 15:11:57 -07:00
david-sarah	e706f043f0	Change shouldFail to avoid Unicode errors when converting Failure to str	2010-05-11 23:07:54 -07:00
david-sarah	b8641f8f9b	allmydata.org -> tahoe-lafs.org in __init__.py	2010-06-02 23:35:30 -07:00
david-sarah	90e9fe28af	Resolve conflict in patch to change imports to absolute.	2010-06-02 22:46:08 -07:00
freestorm77	59a77f82d7	title_rename_xhtml.dpatch.txt ... - Renamed xhtml Title from "Allmydata - Tahoe" to "Tahoe-LAFS" - Renamed Tahoe to Tahoe-LAFS in page content - Changed Tahoe-LAFS home page link to http://tahoe-lafs.org (added target="blank") - Deleted commented css script in info.xhtml	2010-05-29 10:25:42 -07:00
Zooko O'Whielacronx	9e2da05837	tests: refactor test_web.py to have less duplication of literal caps-from-the-future ... This is a prelude to a patch which will add tests of caps from the future which have non-ascii chars in them.	2010-05-18 22:51:46 -07:00
david-sarah	a80f19a084	Correct harmless indentation errors found by pylint	2010-02-25 23:21:51 -07:00
david-sarah	e76092e16c	Change relative imports to absolute	2010-02-26 01:14:33 -07:00
david-sarah	e880fc2ce4	Suppress deprecation warning for twisted.web.error.NoResource when using Twisted >= 9.0.0.	2010-05-16 13:56:25 -07:00
Zooko O'Whielacronx	d4f6d4092c	test: show stdout and stderr in case of non-zero exit code from "tahoe" command	2010-05-24 00:33:48 -07:00
Zooko O'Whielacronx	294c0a6279	tests: fix pyflakes warnings in bench_dirnode.py	2010-05-21 13:25:11 -07:00
Zooko O'Whielacronx	35efdf04ca	tests: don't let bench_dirnode.py do stuff and have side-effects at import time (unless __name__ == '__main__')	2010-05-21 05:20:52 -07:00
Zooko O'Whielacronx	a48979516e	tests: increase timeout to give François's ARM buildslave a chance to complete the tests	2010-05-20 06:45:26 -07:00
Kevan Carstensen	d0706d27ea	Hush pyflakes warnings	2010-05-15 11:43:44 -07:00
Kevan Carstensen	9179dd099a	Improve code coverage of the Tahoe2PeerSelector tests.	2010-05-14 20:29:13 -07:00
Kevan Carstensen	55a2849f38	Remove a comment that no longer makes sense.	2010-05-14 13:35:16 -07:00
Kevan Carstensen	e225f573b9	Fix up the behavior of #778, per reviewers' comments ... - Make some important utility functions clearer and more thoroughly documented. - Assert in upload.servers_of_happiness that the buckets attributes of PeerTrackers passed to it are mutually disjoint. - Get rid of some silly non-Pythonisms that I didn't see when I first wrote these patches. - Make sure that should_add_server returns true when queried about a shnum that it doesn't know about yet. - Change Tahoe2PeerSelector.preexisting_shares to map a shareid to a set of peerids, alter dependencies to deal with that. - Remove upload.should_add_servers, because it is no longer necessary - Move upload.shares_of_happiness and upload.shares_by_server to a utility file. - Change some points in Tahoe2PeerSelector. - Compute servers_of_happiness using a bipartite matching algorithm that we know is optimal instead of an ad-hoc greedy algorithm that isn't. - Change servers_of_happiness to just take a sharemap as an argument, change its callers to merge existing_shares and used_peers before calling it. - Change an error message in the encoder to be more appropriate for servers of happiness. - Clarify the wording of an error message in immutable/upload.py - Refactor a happiness failure message to happinessutil.py, and make immutable/upload.py and immutable/encode.py use it. - Move the word "only" as far to the right as possible in failure messages. - Use a better definition of progress during peer selection. - Do read-only peer share detection queries in parallel, not sequentially. - Clean up logging semantics; print the query statistics whenever an upload is unsuccessful, not just in one case.	2010-05-13 17:49:17 -07:00
Kevan Carstensen	9bc71d3da0	Alter the error message when an upload fails, per some comments in #778. ... When I first implemented #778, I just altered the error messages to refer to servers where they referred to shares. The resulting error messages weren't very good. These are a bit better.	2009-12-30 15:03:44 -07:00
Kevan Carstensen	8bcc771e26	Change "UploadHappinessError" to "UploadUnhappinessError"	2009-12-04 22:30:37 -07:00
Kevan Carstensen	68fb556e93	Alter the error message returned when peer selection fails ... The Tahoe2PeerSelector returned either NoSharesError or NotEnoughSharesError for a variety of error conditions that weren't informatively described by them. This patch creates a new error, UploadHappinessError, replaces uses of NoSharesError and NotEnoughSharesError with it, and alters the error message raised with the errors to be more in line with the new servers_of_happiness behavior. See ticket #834 for more information.	2009-11-22 18:24:05 -07:00
Kevan Carstensen	320582be5a	Eliminate overcounting iof servers_of_happiness in Tahoe2PeerSelector; also reorganize some things.	2009-11-17 19:45:42 -07:00
Kevan Carstensen	4e29060847	Change stray "shares_of_happiness" to "servers_of_happiness"	2009-11-16 15:24:59 -07:00
Kevan Carstensen	a816de3f23	Alter Tahoe2PeerSelector to make sure that it recognizes existing shares on readonly servers, fixing an issue in #778	2009-11-16 13:28:05 -07:00
Kevan Carstensen	8c71df53f9	Alter 'immutable/encode.py' and 'immutable/upload.py' to use servers_of_happiness instead of shares_of_happiness.	2009-11-04 05:12:22 -07:00
Kevan Carstensen	b2d8a7cec2	Alter the signature of set_shareholders in IEncoder to add a 'servermap' parameter, which gives IEncoders enough information to perform a sane check for servers_of_happiness.	2009-11-03 21:32:41 -07:00
Kevan Carstensen	89a8da9dd2	Alter CiphertextDownloader to work with servers_of_happiness	2009-09-23 21:19:32 -07:00
Kevan Carstensen	5683112a02	Revisions of the #778 tests, per reviewers' comments ... - Fix comments and confusing naming. - Add tests for the new error messages suggested by David-Sarah and Zooko. - Alter existing tests for new error messages. - Make sure that the tests continue to work with the trunk. - Add a test for a mutual disjointedness assertion that I added to upload.servers_of_happiness. - Fix the comments to correctly reflect read-onlyness - Add a test for an edge case in should_add_server - Add an assertion to make sure that share redistribution works as it should - Alter tests to work with revised servers_of_happiness semantics - Remove tests for should_add_server, since that function no longer exists. - Alter tests to know about merge_peers, and to use it before calling servers_of_happiness. - Add tests for merge_peers. - Add Zooko's puzzles to the tests. - Edit encoding tests to expect the new kind of failure message. - Edit tests to expect error messages with the word "only" moved as far to the right as possible. - Extended and cleaned up some helper functions. - Changed some tests to call more appropriate helper functions. - Added a test for the failing redistribution algorithm - Added a test for the progress message - Added a test for the upper bound on readonly peer share discovery.	2010-05-13 18:25:42 -07:00
Kevan Carstensen	d360ee0d2e	Alter various unit tests to work with the new happy behavior	2010-01-07 12:13:25 -07:00
Kevan Carstensen	953c9555ca	Replace "UploadHappinessError" with "UploadUnhappinessError" in tests.	2009-12-04 22:34:53 -07:00
Kevan Carstensen	c0f3dbb919	Add tests for the behavior described in #834.	2009-11-22 19:20:08 -07:00
Kevan Carstensen	c3b11dedea	Re-work 'test_upload.py' to be more readable; add more tests for #778	2009-11-16 13:23:34 -07:00
Kevan Carstensen	9590690450	Test Tahoe2PeerSelector to make sure that it recognizeses existing shares on readonly servers	2009-11-08 18:37:35 -07:00
Kevan Carstensen	324a72e4a3	Add more tests for comment:53 in ticket #778	2009-11-04 05:28:49 -07:00
Kevan Carstensen	697f1cfae8	Add a test for upload.shares_by_server	2009-11-04 05:13:24 -07:00
Kevan Carstensen	0d8e70bfe5	Minor tweak to an existing test -- make the first server read-write, instead of read-only	2009-11-03 21:42:32 -07:00
Kevan Carstensen	5ad29d4e8b	Alter tests to use the new form of set_shareholders	2009-11-03 21:36:02 -07:00
"Kevan Carstensen"	ee9690b357	Refactor some behavior into a mixin, and add tests for the behavior described in #778	2009-10-30 02:19:08 -07:00
Kevan Carstensen	362f204075	Alter NoNetworkGrid to allow the creation of readonly servers for testing purposes.	2009-10-17 18:30:13 -07:00
"Kevan Carstensen"	5fe125ed74	Alter wording in 'interfaces.py' to be correct wrt #778	2009-12-04 21:40:05 -07:00
Zooko O'Whielacronx	6037df55b3	web: add a simple test that you can load directory.xhtml	2010-05-09 23:37:29 -07:00
Francois Deppierraz	dd95e8e71b	CLI: Support for https url in option --node-url ... This patch modifies the regular expression used for verifying of '--node-url' parameter. Support for accessing a Tahoe gateway over HTTPS was already present, thanks to Python's urllib.	2010-04-30 11:56:09 -07:00
Brian Warner	e2031aff0f	backupdb.did_create_directory: use REPLACE INTO, not INSERT INTO + ignore error ... This handles the case where we upload a new tahoe directory for a previously-processed local directory, possibly creating a new dircap (if the metadata had changed). Now we replace the old dirhash->dircap record. The previous behavior left the old record in place (with the old dircap and timestamps), so we'd never stop creating new directories and never converge on a null backup.	2010-04-27 22:08:03 -07:00
Brian Warner	f6f8256b35	"tahoe webopen": add --info flag, to get ?t=info ... Also fix some trailing whitespace.	2010-04-24 16:30:03 -07:00
Zooko O'Whielacronx	80ae6f2706	setup: fix error in licensing declaration from Shawn Willden, as noted by the FSF compliance division	2010-03-09 08:37:36 -08:00
Zooko O'Whielacronx	34a50b857a	docs: a few small edits to try to guide newcomers through the docs ... These edits were suggested by my watching over Jake Appelbaum's shoulder as he completely ignored/skipped/missed install.html and also as he decided that debian.txt wouldn't help him with basic installation. Then I threw in a few docs edits that have been sitting around in my sandbox asking to be committed for months.	2010-03-03 15:19:02 -08:00
david-sarah	40edf8f419	Change code that gives a base32 SI or an empty string to be more straightforward. (#948)	2010-02-26 22:55:51 -08:00
david-sarah	a2ed17f2a0	Additional test for DIR2-LIT directories in test_web.py, fixed version (#948)	2010-02-24 20:18:24 -08:00
david-sarah	d7b50a3b86	Additional fixes for DIR2-LIT More Info page and deep-check/manifest operations (#948)	2010-02-24 00:02:20 -08:00
david-sarah	e6aee33bb7	directories: add DIR2-LIT directories to test_deepcheck.py (#948)	2010-02-23 23:54:33 -08:00
david-sarah	187d837c1d	dirnode: add tests of literal dirnodes (current and fix for #948)	2010-02-23 20:33:45 -08:00
david-sarah	fec9185f2a	Additional fix for abbrev_si, with test	2010-02-21 19:36:52 -08:00
david-sarah	d29ec184a6	Additions to test_web.py for #948	2010-02-21 18:53:52 -08:00
david-sarah	973f0afdd3	Change direct accesses to an_uri.storage_index to calls to .get_storage_index() (fixes #948)	2010-02-21 18:45:04 -08:00
david-sarah	6a7feea455	Tweak to 'tahoe ls --help' output (#837)	2010-02-23 19:02:31 -08:00
david-sarah	9741b9655f	Test behaviour of 'tahoe ls' for unknown objects (#837)	2010-02-23 18:59:13 -08:00
david-sarah	03134eedb5	Improve behaviour of 'tahoe ls' for unknown objects, addressing kevan's comments	2010-02-19 22:13:13 -08:00
Brian Warner	1de4d2c594	test_util.py: improve coverage of util.time_format	2010-02-23 23:46:37 -05:00
Kevan Carstensen	81ad52d6eb	Change OphandleTable to use a deterministic clock, so we can test it ... To test the changes for #577, we need a deterministic way to simulate the passage of long periods of time. twisted.internet.task.Clock seems, from my Googling, to be the way to go for this functionality. I changed a few things so that OphandleTable would use twisted.internet.task.Clock when testing: * WebishServer.__init___ now takes an optional 'clock' parameter, * which it passes to the root.Root instance it creates. * root.Root.__init__ now takes an optional 'clock' parameter, which it passes to the OphandleTable.__init__ method. * OphandleTable.__init__ now takes an optional 'clock' parameter. If it is provided, and it isn't None, its callLater method will be used to schedule ophandle expirations (as opposed to using reactor.callLater, which is what OphandleTable does normally). * The WebMixin object in test_web.py now sets a self.clock parameter, which is a twisted.internet.task.Clock that it feeds to the WebishServer it creates. Tests using the WebMixin can control the passage of time in OphandleTable by accessing self.clock.	2010-02-20 13:07:13 -08:00
Kevan Carstensen	bcdc78bd44	Add tests for the ophandle expiration behavior in #577	2010-02-20 17:04:55 -08:00
Kevan Carstensen	e9b59a4949	Increase ophandle expiration times, per #577	2010-02-20 17:05:12 -08:00
david-sarah	85a50feeaa	More cleanups to test_cli using new utilities for reading and writing files.	2010-02-05 17:38:55 -08:00
david-sarah	c984a09fe7	Fix race conditions and missing callback in allmydata.test.test_cli.Cp.test_copy_using_filecap, add utilities for one-liner reading and writing of files, and fix cases in test_cli where files were not being closed after writing.	2010-02-05 17:37:27 -08:00
Kevan Carstensen	c00a62a2e7	Add tests for #939	2010-02-11 22:21:37 -08:00
Kevan Carstensen	63c8c65747	Alter CLI utilities to handle nonexistent aliases better	2010-02-10 18:43:18 -08:00
Brian Warner	4040b1000b	web/storage.py: display total-seen on the last-complete-cycle line. For #940.	2010-02-07 16:20:10 -08:00
Brian Warner	880f824103	code coverage: replace figleaf with coverage.py, should work on py2.6 now. ... It still lacks the right HTML report (the builtin report is very pretty, but lacks the "lines uncovered" numbers that I want), and the half-finished delta-from-last-run measurements.	2010-02-03 08:54:21 -08:00
Zooko O'Whielacronx	3e4342ecb3	immutable: downloader accepts notifications of buckets even if those notifications arrive after he has begun downloading shares. ... This can be useful if one of the ones that he has already begun downloading fails. See #287 for discussion. This fixes part of #287 which part was a regression caused by #928, namely this fixes fail-over in case a share is corrupted (or the server returns an error or disconnects). This does not fix the related issue mentioned in #287 if a server hangs and doesn't reply to requests for blocks.	2010-01-31 22:16:10 -08:00
Zooko O'Whielacronx	e4e2599017	tests: don't require tahoe to run with no noise if we are using an old twisted that emits DeprecationWarnings	2010-01-31 21:23:23 -08:00
david-sarah	100548257a	Use if instead of assert to check for twisted ftp patch	2010-01-26 17:55:29 -08:00
Zooko O'Whielacronx	7d32fafc44	tests: stop being surprised that Nevow no longer prints out warnings when it tries to find its static files ... Unless we are using a sufficiently new version of Nevow, in which case if it prints out warnings then this is a hard test failure. :-)	2010-01-31 20:11:44 -08:00
david-sarah	6215ebd95e	cli: suppress DeprecationWarnings emitted from importing nevow and twisted. Fixes #859	2010-01-31 16:44:29 -08:00
david-sarah	ea3954372a	Improvements to test_unknownnode to cover invalid cap URIs with known prefixes	2010-01-29 22:39:08 -08:00
david-sarah	37a242e01a	Improvements to test_hung_server, and fix for status updates in download.py	2010-01-29 22:43:03 -08:00
Zooko O'Whielacronx	d62428c1e6	immutable: fix bug in tests, change line-endings to unix style, add comment	2010-01-29 10:42:37 -08:00
david-sarah	baa11a0ad4	New tests for #928	2010-01-29 04:38:45 -08:00
Zooko O'Whielacronx	2bd9dfa5bd	immutable: download from the first servers which provide at least K buckets instead of waiting for all servers to reply ... This should put an end to the phenomenon I've been seeing that a single hung server can cause all downloads on a grid to hang. Also it should speed up all downloads by (a) not-waiting for responses to queries that it doesn't need, and (b) downloading shares from the servers which answered the initial query the fastest. Also, do not count how many buckets you've gotten when deciding whether the download has enough shares or not -- instead count how many buckets to *unique* shares that you've gotten. This appears to improve a slightly weird behavior in the current download code in which receiving >= K different buckets all to the same sharenumber would make it think it had enough to download the file when in fact it hadn't. This patch needs tests before it is actually ready for trunk.	2010-01-27 15:34:17 -08:00
david-sarah	14280b009c	Eliminate 'foo if test else bar' syntax that isn't supported by Python 2.4	2010-01-28 19:52:10 -08:00
david-sarah	3e35959e9b	Add mutable field to t=json output for unknown nodes, when mutability is known	2010-01-28 19:14:24 -08:00
david-sarah	4560e021a9	Show -IMM and -RO suffixes for types of immutable and read-only unknown nodes in directory listings	2010-01-28 14:08:00 -08:00
david-sarah	1bb627405f	Fix inaccurate comment in test_mutant_dirnodes_are_omitted	2010-01-28 12:24:56 -08:00
david-sarah	b9eda4de6a	Address comments by Kevan on 833 and add test for stripping spaces	2010-01-27 15:06:42 -08:00
david-sarah	56c00cb381	Miscellaneous documentation, test, and code formatting tweaks.	2010-01-26 23:03:09 -08:00
david-sarah	6057bc02cc	Prevent mutable objects from being retrieved from an immutable directory, and associated forward-compatibility improvements.	2010-01-26 22:44:30 -08:00
Brian Warner	3880486f91	test_runner: cleanup, refactor common code into a non-executable method ... Having both test_node() and test_client() (one of which calls the other) felt confusing to me, so I changed it to have test_node(), test_client(), and a common do_create() helper method.	2010-01-27 14:40:40 -08:00
Brian Warner	5045a8721a	scripts/runner.py: simplify David-Sarah's clever grouped-commands usage trick	2010-01-27 14:37:58 -08:00
Brian Warner	e769bbb6dd	tahoe backup: skip all symlinks, with warning. Fixes #850, addresses #641.	2010-01-27 14:35:17 -08:00
Brian Warner	52185053c3	"tahoe backup": fix --exclude-vcs docs to include Git	2010-01-27 12:10:44 -08:00
david-sarah	5c5a6fe610	Patch to accept t=set-children as well as t=set_children	2010-01-23 19:00:20 -08:00
Zooko O'Whielacronx	4a4305e7e6	ftpd: clearer error message if Twisted needs a patch (by Nils Durner)	2010-01-26 06:34:11 -08:00
francois	b03406af9d	tahoe_backup.py: display warnings on errors instead of stopping the whole backup. Fix #729. ... This patch displays a warning to the user in two cases: 1. When special files like symlinks, fifos, devices, etc. are found in the local source. 2. If files or directories are not readables by the user running the 'tahoe backup' command. In verbose mode, the number of skipped files and directories is printed at the end of the backup. Exit status returned by 'tahoe backup': - 0 everything went fine - 1 the backup failed - 2 files were skipped during the backup	2010-01-20 01:42:49 -08:00
david-sarah	174d356692	Message saying that we couldn't find bin/tahoe should say where we looked	2010-01-16 12:45:56 -08:00
Zooko O'Whielacronx	a1444d9367	cli: merge the better version of David-Sarah's split-usage-and-help patch with the earlier version that I mistakenly committed	2010-01-25 20:45:59 -08:00
david-sarah	b079f32da2	Split tahoe --help options into groups.	2010-01-11 20:39:35 -08:00
Zooko O'Whielacronx	b94b9af189	cli: split usage strings into groups (patch by David-Sarah Hopwood)	2010-01-25 20:39:21 -08:00
david-sarah	87f1bae7fe	Add create-node CLI command, and make create-client equivalent to create-node --no-storage (fixes #760)	2010-01-15 21:20:55 -08:00
david-sarah	26ab58e006	Remove replace= parameter to mkdir-immutable and mkdir-with-children	2010-01-24 14:43:25 -08:00
Brian Warner	de14791caf	Fix webapi t=mkdir with multpart/form-data, as on the Welcome page. Closes #919.	2010-01-20 22:50:52 -08:00
Brian Warner	bb4883bbd4	tahoe_add_alias.py: minor refactoring	2010-01-14 22:42:20 -08:00
Brian Warner	9ab7524f0d	test_dirnode.py: reduce scope of a Client instance, suggested by Kevan.	2010-01-14 22:27:13 -08:00
Brian Warner	01a7630334	test_provisioning: STAN is not always a list. Fix by David-Sarah Hopwood.	2010-01-14 17:46:32 -08:00
Brian Warner	2e098e2a4d	web/directory.py mkdir-immutable: hush pyflakes, add TODO for #903 behavior	2010-01-14 14:28:04 -08:00
Brian Warner	e9223fcd0e	hush pyflakes-0.4.0 warnings: slightly less-trivial fixes. Closes #900. ... This includes one fix (in test_web) which was testing the wrong thing.	2010-01-14 14:17:19 -08:00
Brian Warner	731d15e56f	hush pyflakes-0.4.0 warnings: remove trivial unused variables. For #900.	2010-01-14 14:15:29 -08:00
Brian Warner	874a979a8e	tahoe add-alias/create-alias: don't corrupt non-newline-terminated alias ... file. Closes #741.	2010-01-14 13:02:46 -08:00
Brian Warner	d3d1293d2f	change docs and --help to use "grid" instead of "virtual drive": closes #892. ... Thanks to David-Sarah Hopwood for the patch.	2010-01-14 12:11:19 -08:00
Brian Warner	fb879ddea4	client.py: fix/update comments on KeyGenerator	2010-01-12 16:42:26 -08:00
Brian Warner	d888bf3377	Clean up log.err calls, for one of the issues in #889. ... allmydata.util.log.err() either takes a Failure as the first positional argument, or takes no positional arguments and must be invoked in an exception handler. Fixed its signature to match both foolscap.logging.log.err and twisted.python.log.err . Included a brief unit test.	2010-01-11 17:33:43 -08:00
Brian Warner	bacb6fe5aa	tidy up DeadReferenceError handling, ignore them in add_lease calls ... Stop checking separately for ConnectionDone/ConnectionLost, since those have been folded into DeadReferenceError since foolscap-0.3.1 . Write rrefutil.trap_deadref() in terms of rrefutil.trap_and_discard() to improve code coverage.	2010-01-11 16:07:23 -08:00
Brian Warner	1ed5bbdcb6	test_repairer: add (commented-out) test_each_byte, to see exactly what the ... Verifier misses The results (described in #819) match our expectations: it misses corruption in unused share fields and in most container fields (which are only visible to the storage server, not the client). 1265 bytes of a 2753 byte share (hosting a 56-byte file with an artifically small segment size) are unused, mostly in the unused tail of the overallocated UEB space (765 bytes), and the allocated-but-unwritten plaintext_hash_tree (480 bytes).	2010-01-10 12:35:52 -08:00
Zooko O'Whielacronx	e7a9c000bf	repairer: fix some wrong offsets in the randomized verifier tests, debugged by Brian ... fixes #819	2010-01-10 12:37:21 -08:00
Brian Warner	1ff49a15bb	test_repairer: fix colliding basedir names, which caused test inconsistencies	2010-01-10 00:46:19 -08:00
Zooko O'Whielacronx	876c4a153b	repairer: add deterministic test for #819, mark as TODO	2010-01-09 17:36:19 -08:00
Brian Warner	419a076f59	mutable/publish: don't loop() right away upon DeadReferenceError. Closes #877 ... The bug was that a disconnected server could cause us to re-enter the initial loop() call, sending multiple queries to a single server, provoking an incorrect UCWE. To fix it, stall the loop() with an eventual.fireEventually()	2010-01-02 14:08:41 -08:00
Brian Warner	db19b62702	immutable/checker.py: oops, forgot some imports. Also hush pyflakes.	2009-12-29 15:39:09 -08:00
Brian Warner	ba0690c9d7	mutable repair: return successful=False when numshares<k (thus repair fails), ... instead of weird errors. Closes #874 and #786. Previously, if the file had 0 shares, this would raise TypeError as it tried to call download_version(None). If the file had some shares but fewer than 'k', it would incorrectly raise MustForceRepairError. Added get_successful() to the IRepairResults API, to give repair() a place to report non-code-bug problems like this.	2009-12-29 15:37:46 -08:00
Brian Warner	acd211765c	node.py/interfaces.py: minor docs fixes	2009-12-29 15:04:09 -08:00
Brian Warner	794e32738f	checker: don't let failures in add-lease affect checker results. Closes #875. ... Mutable servermap updates and the immutable checker, when run with add_lease=True, send both the do-you-have-block and add-lease commands in parallel, to avoid an extra round trip time. Many older servers have problems with add-lease and raise various exceptions, which don't generally matter. The client-side code was catching+ignoring some of them, but unrecognized exceptions were passed through to the DYHB code, concealing the DYHB results from the checker, making it think the server had no shares. The fix is to separate the code paths. Both commands are sent at the same time, but the errback path from add-lease is handled separately. Known exceptions are ignored, the others (both unknown-remote and all-local) are logged (log.WEIRD, which will trigger an Incident), but neither will affect the DYHB results. The add-lease message is sent first, and we know that the server handles them synchronously. So when the checker is done, we can be sure that all the add-lease messages have been retired. This makes life easier for unit tests.	2009-12-29 15:01:08 -08:00
Brian Warner	a50fdca6ab	test_cli: verify fix for "tahoe get" not creating empty file on error (#121)	2009-12-27 18:54:44 -05:00
Brian Warner	00d0ca3902	addendum to "Fix 'tahoe ls' on files (#771)" ... tahoe_ls.py: tolerate missing metadata web/filenode.py: minor cleanups test_cli.py: test 'tahoe ls FILECAP'	2009-12-27 18:21:49 -05:00
Brian Warner	a8a768ef9d	Fix 'tahoe ls' on files (#771). Patch adapted from Kevan Carstensen. ... web/filenode.py: also serve edge metadata when using t=json on a DIRCAP/childname object. tahoe_ls.py: list file objects as if we were listing one-entry directories. Show edge metadata if we have it, which will be true when doing 'tahoe ls DIRCAP/filename' and false when doing 'tahoe ls FILECAP'	2009-12-27 17:54:43 -05:00
Brian Warner	931ab76588	tahoe_get: don't create the output file on error. Closes #121.	2009-12-27 17:04:04 -05:00
Brian Warner	499add09e6	webapi: don't accept zero-length childnames during traversal. Closes #358, #676. ... This forbids operations that would implicitly create a directory with a zero-length (empty string) name, like what you'd get if you did "tahoe put local /oops/blah" (#358) or "POST /uri/CAP//?t=mkdir" (#676). The error message is fairly friendly too. Also added code to "tahoe put" to catch this error beforehand and suggest the correct syntax (i.e. without the leading slash).	2009-12-27 15:10:43 -05:00
Brian Warner	2e0a61a953	CLI: send 'Accept:' header to ask for text/plain tracebacks. Closes #646. ... The webapi has been looking for an Accept header since 1.4.0, but it treats a missing header as equal to */* (to honor RFC2616). This change finally modifies our CLI tools to ask for "text/plain, application/octet-stream", which seems roughly correct (we either want a plain-text traceback or error message, or an uninterpreted chunk of binary data to save to disk). Some day we'll figure out how JSON fits into this scheme.	2009-12-27 14:58:28 -05:00
Brian Warner	a4a6c02ef8	more #859: avoid deprecation warning for unit tests too, hush pyflakes ... * factor maybe-import-sha logic into util.hashutil	2009-12-14 16:01:47 -08:00
Zooko O'Whielacronx	b69e1c600d	use hashlib module if available, thus avoiding a DeprecationWarning for importing the old sha module; fixes #859	2009-12-14 13:27:03 -08:00
Brian Warner	47cba4b51a	mutable/retrieve.py: stop reaching into private MutableFileNode attributes	2009-12-08 09:29:21 -08:00
Brian Warner	aa3caf812d	mutable/servermap.py: stop reaching into private MutableFileNode attributes	2009-12-08 09:26:08 -08:00
Brian Warner	eb18686284	mutable/servermap.py: oops, query N+e servers in MODE_WRITE, not k+e ... under normal conditions, this wouldn't cause any problems, but if the shares are really sparse (perhaps because new servers were added), then file-modifies might stop looking too early and leave old shares in place	2009-12-08 09:11:56 -08:00
Brian Warner	696cc000d0	control.py: fix speedtest: use download_best_version (not read) on mutable nodes	2009-12-06 22:05:12 -08:00
Zooko O'Whielacronx	4706859967	setup: ignore _darcs in the "test-clean" test and make the "clean" step remove all .egg's in the root dir	2009-12-06 10:48:35 -08:00
Brian Warner	a980ea337b	remove MutableFileNode.download(), prefer download_best_version() instead	2009-12-01 17:54:38 -05:00
Brian Warner	96834da0a2	Simplify immutable download API: use just filenode.read(consumer, offset, size) ... * remove Downloader.download_to_data/download_to_filename/download_to_filehandle * remove download.Data/FileName/FileHandle targets * remove filenode.download/download_to_data/download_to_filename methods * leave Downloader.download (the whole Downloader will go away eventually) * add util.consumer.MemoryConsumer/download_to_data, for convenience (this is mostly used by unit tests, but it gets used by enough non-test code to warrant putting it in allmydata.util) * update tests * removes about 180 lines of code. Yay negative code days! Overall plan is to rewrite immutable/download.py and leave filenode.read() as the sole read-side API.	2009-12-01 17:53:30 -05:00
Brian Warner	1059db51f2	server.py: undo my bogus 'correction' of David-Sarah's comment fix ... and move it to a better line	2009-11-30 21:46:07 -05:00
Brian Warner	74974b27fe	Implement more coherent behavior when copying with dircaps/filecaps (closes #761). Patch by Kevan Carstensen.	2009-11-30 13:10:09 -08:00
Brian Warner	91e7cb484b	storage.py: update comment	2009-11-30 11:59:13 -08:00
david-sarah	ef002c935a	storage server: detect disk space usage on Windows too (fixes #637)	2009-11-20 21:56:44 -08:00
david-sarah	c4d38ad4c5	make status of finished operations consistently "Finished"	2009-11-20 22:15:43 -08:00
Brian Warner	9901a08fd3	Add dirnodes to backupdb and "tahoe backup", closes #606. ... * backups now share dirnodes with any previous backup, in any location, so renames and moves are handled very efficiently * "tahoe backup" no longer bothers reading the previous snapshot * if you switch grids, you should delete ~/.tahoe/private/backupdb.sqlite, to force new uploads of all files and directories	2009-11-26 15:42:57 -08:00
Brian Warner	366a309795	webapi: fix t=check for DIR2-LIT (i.e. empty immutable directories)	2009-11-26 15:27:31 -08:00
Brian Warner	41bcc9f39e	PipelineError: fix str() on python2.4 . Closes #842.	2009-11-24 13:25:12 -08:00
Brian Warner	d918ab407e	test_uri.py: s/NewDirnode/Dirnode/ , now that they aren't "new" anymore	2009-11-19 23:55:53 -08:00
Brian Warner	0cf320c2ab	interface name cleanups: IFileNode, IImmutableFileNode, IMutableFileNode ... The proper hierarchy is: IFilesystemNode +IFileNode ++IMutableFileNode ++IImmutableFileNode +IDirectoryNode Also expand test_client.py (NodeMaker) to hit all IFilesystemNode types.	2009-11-19 23:52:55 -08:00
Brian Warner	d2badbea78	class name cleanups: s/FileNode/ImmutableFileNode/ ... also fix test/bench_dirnode.py for recent dirnode changes	2009-11-19 23:22:39 -08:00
Brian Warner	6e7fb1006d	Use DIR-IMM and t=mkdir-immutable for "tahoe backup", for #828	2009-11-18 11:28:13 -08:00
Brian Warner	834b20210a	web/directory.py: use "DIR-IMM" to describe immutable directories, not DIR-RO	2009-11-18 11:18:32 -08:00
Brian Warner	e5452290f7	web/info.py: hush pyflakes	2009-11-18 11:17:36 -08:00
Brian Warner	e046744f40	make get_size/get_current_size consistent for all IFilesystemNode classes ... * stop caching most_recent_size in dirnode, rely upon backing filenode for it * start caching most_recent_size in MutableFileNode * return None when you don't know, not "?" * only render None as "?" in the web "more info" page * add get_size/get_current_size to UnknownNode	2009-11-18 11:16:24 -08:00
Brian Warner	512fe3ad62	ImmutableDirectoryURIVerifier: fix verifycap handling	2009-11-18 08:42:38 -08:00
Brian Warner	f85690697a	Add t=mkdir-immutable to the webapi. Closes #607. ... * change t=mkdir-with-children to not use multipart/form encoding. Instead, the request body is all JSON. t=mkdir-immutable uses this format too. * make nodemaker.create_immutable_dirnode() get convergence from SecretHolder, but let callers override it * raise NotDeepImmutableError instead of using assert() * add mutable= argument to DirectoryNode.create_subdirectory(), default True	2009-11-17 23:09:00 -08:00
Brian Warner	d2426ea3bd	move convergence secret into SecretHolder, next to lease secret	2009-11-17 17:54:44 -08:00
Brian Warner	5fe713fc52	nodemaker: implement immutable directories (internal interface), for #607 ... * nodemaker.create_from_cap() now handles DIR2-CHK and DIR2-LIT * client.create_immutable_dirnode() is used to create them * no webapi yet	2009-11-11 16:22:33 -08:00
Brian Warner	cc422f8dc0	stop using IURI()/etc as an adapter	2009-11-11 14:45:42 -08:00
Brian Warner	131e05b155	clean up uri-vs-cap terminology, emphasize cap instances instead of URI strings ... * "cap" means a python instance which encapsulates a filecap/dircap (uri.py) * "uri" means a string with a "URI:" prefix * FileNode instances are created with (and retain) a cap instance, and generate uri strings on demand * .get_cap/get_readcap/get_verifycap/get_repaircap return cap instances * .get_uri/get_readonly_uri return uri strings * add filenode.download_to_filename() for control.py, should find a better way * use MutableFileNode.init_from_cap, not .init_from_uri * directory URI instances: use get_filenode_cap, not get_filenode_uri * update/cleanup bench_dirnode.py to match, add Makefile target to run it	2009-11-11 14:26:19 -08:00
Brian Warner	f47672d12a	add parser for immutable directory caps: DIR2-CHK, DIR2-LIT, DIR2-CHK-Verifier	2009-11-04 10:13:51 -08:00
Zooko O'Whielacronx	9733201c0b	wui: s/TahoeLAFS/Tahoe-LAFS/	2009-10-28 18:50:50 -08:00
Zooko O'Whielacronx	1be4d45779	tests: bump up the timeout on test_repairer to see if 120 seconds was too short for François's ARM box to do the test even when it was doing it right.	2009-10-27 13:48:00 -08:00
Brian Warner	2695af91a7	dirnode.pack_children(): add deep_immutable= argument ... This will be used by DIR2:CHK to enforce the deep-immutability requirement.	2009-10-26 09:28:09 -07:00
Brian Warner	768c76aa5f	webapi: use t=mkdir-with-children instead of a children= arg to t=mkdir . ... This is safer: in the earlier API, an old webapi server would silently ignore the initial children, and clients trying to set them would have to fetch the newly-created directory to discover the incompatibility. In the new API, clients using t=mkdir-with-children against an old webapi server will get a clear error.	2009-10-25 18:13:21 -07:00
Brian Warner	1273b5c233	nodemaker.create_new_mutable_directory: pack_children() in initial_contents= ... instead of creating an empty file and then adding the children later. This should speed up mkdir(initial_children) considerably, removing two roundtrips and an entire read-modify-write cycle, probably bringing it down to a single roundtrip. A quick test (against the volunteergrid) suggests a 30% speedup. test_dirnode: add new tests to enforce the restrictions that interfaces.py claims for create_new_mutable_directory(): no UnknownNodes, metadata dicts	2009-10-19 17:51:18 -07:00
Brian Warner	59ee02e82f	test_dirnode.py: add tests of initial_children= args to client.create_dirnode ... and nodemaker.create_new_mutable_directory	2009-10-17 12:41:59 -07:00
Brian Warner	b4ec86c95a	update many dirnode interfaces to accept dict-of-nodes instead of dict-of-caps ... interfaces.py: define INodeMaker, document argument values, change create_new_mutable_directory() to take dict-of-nodes. Change dirnode.set_nodes() and dirnode.create_subdirectory() too. nodemaker.py: use INodeMaker, update create_new_mutable_directory() client.py: have create_dirnode() delegate initial_children= to nodemaker dirnode.py (Adder): take dict-of-nodes instead of list-of-nodes, which updates set_nodes() and create_subdirectory() web/common.py (convert_initial_children_json): create dict-of-nodes web/directory.py: same web/unlinked.py: same test_dirnode.py: update tests to match	2009-10-17 12:28:29 -07:00
Brian Warner	8a7c980e37	dirnode.py: move pack_children() out to a function, for eventual use by others	2009-10-17 11:07:07 -07:00
Brian Warner	ea373de042	move dirnode.CachingDict to dictutil.AuxValueDict, generalize method names, ... improve tests. Let dirnode _pack_children accept either dict or AuxValueDict.	2009-10-17 11:00:05 -07:00
Brian Warner	e63f59f50c	test/common.py: update FakeMutableFileNode to new contents= callable scheme	2009-10-12 22:21:54 -07:00
Brian Warner	480e1d318d	The initial_children= argument to nodemaker.create_new_mutable_directory is ... now enabled.	2009-10-12 20:19:22 -07:00
Brian Warner	c2520e4ec7	client.create_mutable_file(contents=) now accepts a callable, which is ... invoked with the new MutableFileNode and is supposed to return the initial contents. This can be used by e.g. a new dirnode which needs the filenode's writekey to encrypt its initial children. create_mutable_file() still accepts a bytestring too, or None for an empty file.	2009-10-12 20:12:32 -07:00
Brian Warner	b30041c5ec	webapi: t=mkdir now accepts initial children, using the same JSON that t=json ... emits. client.create_dirnode(initial_children=) now works.	2009-10-12 19:34:44 -07:00
Brian Warner	cf65cc2ae3	replace dirnode.create_empty_directory() with create_subdirectory(), which ... takes an initial_children= argument	2009-10-12 19:15:20 -07:00
Brian Warner	d079eb45f6	dirnode.set_children: change return value: fire with self instead of None	2009-10-12 18:50:26 -07:00
Brian Warner	f871c3bb3d	dirnode.set_nodes: change return value: fire with self instead of None	2009-10-12 18:45:46 -07:00
Brian Warner	304aadd4f7	dirnode.set_children: take a dict, not a list	2009-10-12 17:24:40 -07:00
Brian Warner	e2ffc3dc03	dirnode.set_uri/set_children: change signature to take writecap+readcap ... instead of a single cap. The webapi t=set_children call benefits too.	2009-10-12 16:51:26 -07:00
Brian Warner	3ee740628a	replace Client.create_empty_dirnode() with create_dirnode(), in anticipation ... of adding initial_children= argument. Includes stubbed-out initial_children= support.	2009-10-12 15:45:06 -07:00
Brian Warner	576c47d61e	test_web.py: use a less-fake client, making test harness smaller	2009-10-12 15:28:08 -07:00
Brian Warner	f4aa418086	Verifier: check the full cryptext-hash tree on each share. Removed .todos ... from the last few test_repairer tests that were waiting on this.	2009-10-05 15:18:49 -07:00
Brian Warner	504c767d03	Verifier: check the full block-hash-tree on each share ... Removed the .todo from two test_repairer tests that check this. The only remaining .todos are on the three crypttext-hash-tree tests.	2009-10-05 14:48:44 -07:00
Brian Warner	e8f56af5a7	Verifier: check the full share-hash chain on each share ... Removed the .todo from two test_repairer tests that check this.	2009-10-05 14:34:43 -07:00
Brian Warner	bbd00de495	test_repairer: rename Verifier test cases to be more precise and less verbose	2009-10-05 13:11:15 -07:00
Brian Warner	be95129833	immutable/checker.py: rearrange code a little bit, make it easier to follow	2009-10-05 13:02:52 -07:00
Brian Warner	9dba4ec641	test/common.py: wrap docstrings to 80cols so I can read them more easily	2009-10-05 13:01:43 -07:00
Brian Warner	19d336513c	immutable/download.py: wrap to 80cols, no functional changes	2009-10-05 12:25:42 -07:00
Zooko O'Whielacronx	00a44c057a	docs: a few licensing clarifications requested by Ubuntu	2009-09-26 20:32:26 -07:00
Zooko O'Whielacronx	aaaa633f18	setup: remove a convenience copy of figleaf, to ease inclusion into Ubuntu Karmic Koala ... We need to carefully document the licence of figleaf in order to get Tahoe-LAFS into Ubuntu Karmic Koala. However, figleaf isn't really a part of Tahoe-LAFS per se -- this is just a "convenience copy" of a development tool. The quickest way to make Tahoe-LAFS acceptable for Karmic then, is to remove figleaf from the Tahoe-LAFS tarball itself. People who want to run figleaf on Tahoe-LAFS (as everyone should want) can install figleaf themselves. I haven't tested this -- there may be incompatibilities between upstream figleaf and the copy that we had here...	2009-09-23 22:32:15 -07:00
Zooko O'Whielacronx	9f0da4012e	docs: Shawn Willden grants permission for his contributions under GPL2+|TGPPL1+	2009-09-21 09:46:51 -07:00
Zooko O'Whielacronx	603e08c4a9	doc: licensing cleanups ... Use nice utf-8 © instead of "(c)". Remove licensing statements on utility modules that have been assigned to allmydata.com by their original authors. (Nattraverso was not assigned to allmydata.com -- it was LGPL'ed -- but I checked and src/allmydata/util/iputil.py was completely rewritten and doesn't contain any line of code from nattraverso.) Add notes to misc/debian/copyright about licensing on files that aren't just allmydata.com-licensed.	2009-09-20 10:16:31 -07:00
Brian Warner	14cb5ca436	test/check_speed.py: fix shbang line	2009-08-17 17:59:48 -07:00
Brian Warner	5283d4c19e	de-Service-ify Helper, pass in storage_broker and secret_holder directly. ... This makes it more obvious that the Helper currently generates leases with the Helper's own secrets, rather than getting values from the client, which is arguably a bug that will likely be resolved with the Accounting project.	2009-08-15 13:17:37 -07:00
Brian Warner	4a4a4f9520	immutable.Downloader: pass StorageBroker to constructor, stop being a Service ... child of the client, access with client.downloader instead of client.getServiceNamed("downloader"). The single "Downloader" instance is scheduled for demolition anyways, to be replaced by individual filenode.download calls.	2009-08-15 12:25:43 -07:00
Zooko O'Whielacronx	22c962bbc2	tests: double the timeout on test_runner.RunNode.test_introducer since feisty hit a timeout ... I'm not sure if this is an actual timing issue (feisty is running on an overloaded VM if I recall correctly), or it there is a deeper bug.	2009-08-15 09:05:12 -07:00
Brian Warner	d214f7eebe	stop making History be a Service, it wasn't necessary	2009-08-15 04:44:15 -07:00
Brian Warner	0d5dc51617	Overhaul IFilesystemNode handling, to simplify tests and use POLA internally. ... * stop using IURI as an adapter * pass cap strings around instead of URI instances * move filenode/dirnode creation duties from Client to new NodeMaker class * move other Client duties to KeyGenerator, SecretHolder, History classes * stop passing Client reference to dirnode/filenode constructors - pass less-powerful references instead, like StorageBroker or Uploader * always create DirectoryNodes by wrapping a filenode (mutable for now) * remove some specialized mock classes from unit tests Detailed list of changes (done one at a time, then merged together) always pass a string to create_node_from_uri(), not an IURI instance always pass a string to IFilesystemNode constructors, not an IURI instance stop using IURI() as an adapter, switch on cap prefix in create_node_from_uri() client.py: move SecretHolder code out to a separate class test_web.py: hush pyflakes client.py: move NodeMaker functionality out into a separate object LiteralFileNode: stop storing a Client reference immutable Checker: remove Client reference, it only needs a SecretHolder immutable Upload: remove Client reference, leave SecretHolder and StorageBroker immutable Repairer: replace Client reference with StorageBroker and SecretHolder immutable FileNode: remove Client reference mutable.Publish: stop passing Client mutable.ServermapUpdater: get StorageBroker in constructor, not by peeking into Client reference MutableChecker: reference StorageBroker and History directly, not through Client mutable.FileNode: removed unused indirection to checker classes mutable.FileNode: remove Client reference client.py: move RSA key generation into a separate class, so it can be passed to the nodemaker move create_mutable_file() into NodeMaker test_dirnode.py: stop using FakeClient mockups, use NoNetworkGrid instead. This simplifies the code, but takes longer to run (17s instead of 6s). This should come down later when other cleanups make it possible to use simpler (non-RSA) fake mutable files for dirnode tests. test_mutable.py: clean up basedir names client.py: move create_empty_dirnode() into NodeMaker dirnode.py: get rid of DirectoryNode.create remove DirectoryNode.init_from_uri, refactor NodeMaker for customization, simplify test_web's mock Client to match stop passing Client to DirectoryNode, make DirectoryNode.create_with_mutablefile the normal DirectoryNode constructor, start removing client from NodeMaker remove Client from NodeMaker move helper status into History, pass History to web.Status instead of Client test_mutable.py: fix minor typo	2009-08-15 04:28:46 -07:00
Zooko O'Whielacronx	2a63fc9159	docs: update relnotes.txt, relnotes-short.txt, and others documentation bits for v1.5.0 release!	2009-08-01 19:57:10 -07:00
Brian Warner	ad45511156	ftpd: update docs, point to Twisted ticket for the proposed patch	2009-07-31 11:32:26 -07:00
Brian Warner	0d0e0ba2fb	test_runner.py: tolerate symlinks in the source tree directory's ancestor path	2009-07-30 10:17:24 -07:00
Brian Warner	bf1e61c8f3	Touch up #705 changes: ... webapi.txt: clarify replace=only-files argument, mention replace= on POST t=uri test_cli.py: insert whitespace between logical operations web.common.parse_replace_arg: make it case-insensitive, to match the docs	2009-07-20 11:38:03 -04:00
Zooko O'Whielacronx	b76ca4549a	iputil: added "openbsd3" => "bsd" and "openbsd4" => "bsd" per tdelaet's report; should fix ticket #768	2009-07-20 06:13:14 -07:00
kevan	0d8b1e29fa	Add tests for tahoe mv behavior	2009-07-19 20:46:09 -07:00
kevan	4331326b21	Add tests for tahoe mv help text	2009-07-19 20:45:53 -07:00
kevan	52aceb1a8e	Implement more clearly defined moving semantics in tahoe_mv.py	2009-07-19 20:45:23 -07:00
kevan	36f2e01275	Update tahoe mv help text.	2009-07-19 20:45:03 -07:00
kevan	8eb7ddab6b	Add tests for parse_replace_arg()	2009-07-19 20:47:05 -07:00
kevan	40360a7a9c	Add tests for new PUT behavior	2009-07-19 20:46:32 -07:00
kevan	18a80d99b1	Alter Adder + Adder tests to look for 'only-files' instead of 'only_files'	2009-07-19 20:43:18 -07:00
kevan	74207d8334	Alter filenode.py to use parse_replace_arg	2009-07-19 20:48:31 -07:00
kevan	90677745b3	Alter directory.py to use parse_replace_arg()	2009-07-19 20:47:46 -07:00
kevan	7ab92c7511	Add a function to parse arguments for the replace parameter	2009-07-19 20:47:23 -07:00
Zooko O'Whielacronx	7607268a58	trivial: remove obsolete comment text	2009-07-20 06:08:43 -07:00
Zooko O'Whielacronx	c6fce3a1c6	directories: minor change: fix warning message from the benchmarking script	2009-07-19 17:21:14 -07:00
kevan	ca4de9ee97	Add unit tests for the Adder in dirnode.py	2009-07-18 12:50:49 -07:00
kevan	c476c66b0e	Add 'only_files' option to the overwrite field in Adder	2009-07-17 20:00:10 -07:00
Brian Warner	531cc7899f	rename NewDirectoryNode to DirectoryNode, NewDirectoryURI to DirectoryURI	2009-07-17 17:15:49 -05:00
Brian Warner	8536db4e64	interfaces: remove dead code, FileNode_ and EncryptedThing constraints	2009-07-17 17:11:39 -05:00
Zooko O'Whielacronx	d2d3bb2f80	introduction/storage-broker: add assertions of serverids being the right type and of internal consistency between the number of storage servers known and the number of storage servers	2009-07-17 06:52:21 -07:00
Brian Warner	1192b61dfe	upload: fix #758 recursion-loop in peer-selection when servers report errors. ... The bug was in the code that handles a third-or-later pass, and was previously untested.	2009-07-17 00:07:09 -05:00
Brian Warner	efa608a135	'tahoe stats': tolerate empty directories. Closes #693.	2009-07-15 00:51:09 -07:00
Brian Warner	7950fc0f2e	node.py: record pid in twisted log at startup. Closes #476.	2009-07-15 00:29:29 -07:00
Brian Warner	d8ba8c2eb5	Allow tests to pass with -OO by turning some AssertionErrors (the ones that ... we actually exercise during tests) into more specific exceptions, so they don't get optimized away. The best rule to follow is probably this: if an exception is worth testing, then it's part of the API, and AssertionError should never be part of the API. Closes #749.	2009-07-14 23:45:10 -07:00
Zooko O'Whielacronx	10ad3ad7d6	wui: fix test for welcome page	2009-07-14 20:34:29 -07:00
Zooko O'Whielacronx	eb5ecc931d	wui: adjust headers/titles and "Attach something to this directory" text in accordance with #691 ... Also name it as "TahoeLAFS" in this text.	2009-07-14 19:58:14 -07:00
Zooko O'Whielacronx	2f704ed001	dirnode: finish renaming "iv" to "salt" in the code and the hash tag	2009-07-12 17:13:20 -07:00
Brian Warner	7f1d8b7c46	dirnode.py/_encrypt_rwcap: rename IV to "salt", which is more accurate	2009-07-13 00:50:25 +01:00
Brian Warner	c1d5717cf0	dirnode.py: security bug: also use child writecap to derive child enc key, ... not just the dirnode writecap. The previous code (which only hashed the dirnode writecap) would use the same key for all children, which is very bad. This is the correct implementation of #750.	2009-07-13 00:47:50 +01:00
kevan	d71adaf1ca	Use CachingDict instead of dict in dirnode.py	2009-07-03 20:43:01 -07:00
Zooko O'Whielacronx	34213cd2c7	directories: fix semantic conflict between my "keep track of position" optimization patch and Kevan's "cache serialized entries" optimization patch	2009-07-09 20:20:28 -07:00
kevan	44c652fbbe	Modify bench_dirnode.py to use CachingDict.	2009-07-05 15:31:42 -07:00
kevan	325793457d	Add tests for CachingDict, _pack_contents, _unpack_contents	2009-07-03 20:43:28 -07:00
Zooko O'Whielacronx	0e2d005146	trivial: removed unused import noticed by pyflakes	2009-07-09 06:05:13 -07:00
Zooko O'Whielacronx	786ed012b3	directories: make the IV for the writecaps in directory entries be computed from the secure hash of the writecap itself ... This makes encoding of directory entries deterministic, and it is also a tad faster on Macbook Pro than getting a random IV with os.urandom(16).	2009-07-04 19:48:15 -07:00
Zooko O'Whielacronx	fcedca966e	tests: raise the timeout on pollmixin from 100s to 1000s -- it looks like it may have triggered too eagerly on Zandr's 266 MHz armel	2009-07-08 21:47:29 -07:00
Zooko O'Whielacronx	22d390acbb	immutable: base32-encode the keys to generate cache filenames that will work on all platforms	2009-07-08 08:26:33 -07:00
Zooko O'Whielacronx	c0d1e7deae	directories: make initialization of the download cache lazy ... If you open up a directory containing thousands of files, it currently computes the cache filename and checks for the cache file on disk immediately for each immutble file in that directory. With this patch, it delays those steps until you try to do something with an immutable file that could use the cache.	2009-07-07 17:40:40 -07:00
kevan	903005a528	Add CachingDict dict subclass to dirnode.py	2009-07-05 14:23:45 -07:00
Zooko O'Whielacronx	c678e8c77a	directories: rename internal data member download_cache to download_cache_dirman (benchmarks set this internal member and use the new name, so changing this makes the bench_dirnode.py work again)	2009-07-07 07:34:04 -07:00
Zooko O'Whielacronx	efafcfb91a	directories: keep track of your position as you decode netstring after netstring from an input buffer instead of copying the trailing part ... This makes decoding linear in the number of netstrings instead of O(N^2).	2009-07-04 19:51:09 -07:00
Zooko O'Whielacronx	4206a2c1c7	directories: in bench_dirnode.py, use a real CacheDirectoryManager instead of a fake one (because CacheDirectoryManager is a significant user of CPU and/or time)	2009-07-06 20:41:19 -07:00
Zooko O'Whielacronx	0df5b446eb	directories: make the profiling behavior of bench_dirnode.py accessible by adding '--profile' to the cmdline	2009-07-06 20:30:35 -07:00
Zooko O'Whielacronx	e414c73877	directories: update the directory benchmarks to exercise the unpack-and-repack functionality, and add optional profiling	2009-07-05 09:29:53 -07:00
Zooko O'Whielacronx	859d7ba8b0	test/benchmark: benchmark the time to pack and unpack dirnodes ... See also the prof_benchmarks() function in this file which will run the benchmarks under profiling.	2009-07-04 15:43:00 -07:00
Brian Warner	ef1b6ae8e3	Tolerate unknown URI types in directory structures. Part of #683. ... The idea is that future versions of Tahoe will add new URI types that this version won't recognize, but might store them in directories that we *can* read. We should handle these "objects from the future" as best we can. Previous releases of Tahoe would just explode. With this change, we'll continue to be able to work with everything else in the directory. The code change is to wrap anything we don't recognize as an UnknownNode instance (as opposed to a FileNode or DirectoryNode). Then webapi knows how to render these (mostly by leaving fields blank), deep-check knows to skip over them, deep-stats counts them in "count-unknown". You can rename and delete these things, but you can't add new ones (because we wouldn't know how to generate a readcap to put into the dirnode's rocap slot, and because this lets us catch typos better).	2009-07-02 18:07:49 -07:00
Brian Warner	4a46e91192	test_client.py: minor refactoring of BASECONFIG usage	2009-07-02 17:54:50 -07:00
Brian Warner	6237aeabd7	create_node_from_uri: take both writecap+readcap, move logic out of dirnode.py	2009-07-02 15:25:37 -07:00
Brian Warner	656277ac98	dirnode.py: prepare to preserve both rwcap+rocap when copying ... This will make it easier to tolerate unknown nodes safely.	2009-07-02 14:12:54 -07:00
Brian Warner	3dedfed9de	interfaces.py: wrap some lines to 80cols	2009-07-01 18:57:28 -07:00
Brian Warner	b640baf500	cli: webopen: when called with no arguments, open the Welcome page	2009-07-01 13:05:48 -07:00
Brian Warner	e330abc3c3	mutable repairer: skip repair of readcaps instead of throwing an exception. ... This should improve the behavior of #625 a bit: at least all the files will get repaired.	2009-06-30 18:13:43 -07:00
Brian Warner	4194565b3d	interfaces.py: improve ICheckAndRepairResults docs a bit	2009-06-30 17:19:25 -07:00
Brian Warner	8fca155a66	repairer.py: wrap to 80cols. No code changes.	2009-06-30 17:00:47 -07:00
Brian Warner	52fa421430	use 522-bit RSA keys in all unit tests (except one) ... This reduces the total test time on my laptop from 400s to 283s. * src/allmydata/test/test_system.py (SystemTest.test_mutable._test_debug): Remove assertion about container_size/data_size, this changes with keysize and was too variable anyways. * src/allmydata/mutable/filenode.py (MutableFileNode.create): add keysize= * src/allmydata/dirnode.py (NewDirectoryNode.create): same * src/allmydata/client.py (Client.DEFAULT_MUTABLE_KEYSIZE): add default, this overrides the one in MutableFileNode	2009-06-29 15:31:24 -07:00
Brian Warner	c6ae255847	remove trailing whitespace	2009-06-29 13:03:58 -07:00
Brian Warner	dddc2d0378	remove trailing whitespace from some util classes	2009-06-29 13:03:41 -07:00
Brian Warner	d0f80579a5	repairer: raise a better exception when faced with a readonly filenode. Still ... produces an error, though.	2009-06-25 23:32:30 -07:00
Brian Warner	97ffc8a418	mutable/filenode.py: set _writekey to None, rather than leaving it missing ... This will at least turn the really really weird error when a repair of a readonly mutable file is attempted into a merely really weird assertion that mentions "repair currently requires a writecap".	2009-06-25 23:20:22 -07:00
Brian Warner	b43672c9f5	one last tweak	2009-06-25 22:26:27 -07:00
Brian Warner	08effc6499	more 'tahoe cp' docs tweaks	2009-06-25 22:16:52 -07:00
Brian Warner	b30db031ab	cli.py: minor tweaks to test posthooks	2009-06-25 19:45:57 -07:00
Brian Warner	df1ddf83f5	cli.py: update comments on 'tahoe cp' --help a bit	2009-06-25 19:38:37 -07:00
Brian Warner	8d5cc756f0	cli: add some --help text to 'tahoe cp'	2009-06-25 16:57:51 -07:00
Brian Warner	2adc184668	make it possible to add/renew-lease from the WUI ... add add/renew-lease checkbox on the "more info" page check/deep-check forms	2009-06-25 16:18:24 -07:00
Brian Warner	2947da457e	hush pyflakes	2009-06-24 19:18:09 -07:00
Brian Warner	bd6ecc9f44	Split out NoSharesError, stop adding attributes to NotEnoughSharesError, change humanize_failure to include the original exception string, update tests, behave better if humanize_failure fails.	2009-06-24 19:17:07 -07:00
Brian Warner	837733641f	remove introducer/old.py, will create something similar when the RIIntroducer changes	2009-06-24 12:40:38 -07:00
midnightmagic	96cda877ff	Basically just a trivial platform detection patch for NetBSD.	2009-06-17 16:36:47 -07:00
Brian Warner	f573bb2929	check_grid.py: print stderr when a subcommand fails	2009-06-22 19:28:33 -07:00
Brian Warner	8df15e9f30	big rework of introducer client: change local API, split division of responsibilites better, remove old-code testing, improve error logging	2009-06-22 19:10:47 -07:00
Brian Warner	546266c806	web/welcome.xhtml: remove trailing whitespace	2009-06-22 19:09:09 -07:00
Brian Warner	aa23ff9180	rrefutil: add trap_deafref utility, to make the callRemote-plus-ignore-DeadReferenceError-plug-log-other-errors pattern easier	2009-06-22 19:08:26 -07:00
Brian Warner	699510c8f1	PollMixin: snoop trial's error observer to halt the test early if an error is seen. This turns a lot of timeouts into fast failures.	2009-06-22 19:07:31 -07:00
Brian Warner	711c09bc5d	clean up storage_broker interface: should fix #732	2009-06-21 16:51:19 -07:00
Brian Warner	f14004eeb3	hush pyflakes with recent FileTooLarge removal	2009-06-21 16:17:57 -07:00
kevan	efcc45951d	change max filesize limit tests ... Instead of testing to see that the previous SDMF filesize limit was being obeyed, we now test to make sure that we can insert files larger than that limit.	2009-06-20 14:28:22 -07:00
kevan	db939750a8	remove upper limit on SDMF filesize	2009-06-20 14:31:30 -07:00
Brian Warner	a6caae9b5d	immutable/download: instrument do-you-have-block responses to investigate #732	2009-06-20 21:12:09 -07:00
Brian Warner	2c5aa12151	test_util: add known-answer tests for hashutil tags	2009-06-17 21:57:09 -07:00
Zooko O'Whielacronx	6a93caf7a3	tests: bump up a timeout that expired on Zandr's box	2009-06-13 12:57:03 -07:00
Zooko O'Whielacronx	cc2953e663	util: hooray! A clean implementation of this simple utility! Black Dew pointed out that the inverse of time.gmtime() is hidden in the "calendar" module.	2009-06-13 09:01:12 -07:00
Zooko O'Whielacronx	45928315f6	util: Brian's horrible hack to figure out how much localtime and utctime differ. Now we'll see if it works on Windows.	2009-06-12 13:45:56 -07:00
Zooko O'Whielacronx	4a30c5899c	util: oops, time.tzset() doesn't work on Windows -- hopefully the new "London" unit test passes on Windows when we skip tzset() on platforms that don't have it	2009-06-11 17:09:20 -07:00
Zooko O'Whielacronx	8978cb0738	util: fix time_format.iso_utc_time_to_seconds() so that it works even in London	2009-06-11 15:11:29 -07:00
Zooko O'Whielacronx	0ed65fba34	tests: significantly increase timeouts that triggered on Zandr's ARM box	2009-06-10 09:10:43 -07:00
Zooko O'Whielacronx	69a3b23b34	test: multiple by 10 or so all timeouts that Zandr's ARM box just overran	2009-06-10 05:56:39 -07:00
Zooko O'Whielacronx	bdfb80834f	tests: bump up timeout on a test that timed out on draco	2009-06-09 21:46:28 -07:00
Zooko O'Whielacronx	974295f963	tests: raise the timeout for test_cli since Zandr's ARM machine totally burst through the old one	2009-06-09 14:05:09 -07:00
Brian Warner	2e35648901	test_cli.Backup: increase timeout massively, it takes 1200s on zandr's ARM linkstation	2009-06-08 22:28:01 -07:00
Zooko O'Whielacronx	3cfc00f5cf	tests: double the timeouts on some tests which time-out on Francois's box	2009-06-08 19:17:53 -07:00
Zooko O'Whielacronx	0bc116b3a2	tests: bump up timeouts so that the tests can finish before timeout on Francois's little arm box	2009-06-08 15:55:57 -07:00
Zooko O'Whielacronx	c3e9d36d97	tests: increase timeouts on some other tests that timed-out on Francois's arm box	2009-06-05 07:34:37 -07:00
Zooko O'Whielacronx	63c8f0c424	tests: bump up the timeout on a bunch of tests that took longer than the default timeout (120s) on François Lenny-armv5tel	2009-06-04 20:14:44 -07:00
Zooko O'Whielacronx	8c18ac38a9	backup: remove the --no-backupdb command, the handling of "can't import sqlite", and the related tests, and change an error message to more correctly indicate failure to load the database from disk rather than failure to import sqlite module ... Fixes #728.	2009-06-04 10:31:31 -07:00
Zooko O'Whielacronx	2841d6a266	setup: add pysqlite and sqlite to get_package_versions()	2009-06-04 08:37:28 -07:00
Brian Warner	35b3f7f426	more refactoring: move get_all_serverids() and get_nickname_for_serverid() from Client to storage_broker	2009-06-01 20:07:50 -07:00
Brian Warner	b1290633b8	more storage_broker refactoring: downloader gets a broker instead of a client, ... use Client.get_storage_broker() accessor instead of direct attribute access.	2009-06-01 19:25:11 -07:00
Brian Warner	3a1196701f	test_runner.py: remove test_client_no_noise: the issue in question is ... ticketed in http://divmod.org/trac/ticket/2830 and doesn't need a Tahoe-side change, plus this test fails on win32 for unrelated reasons (and test_client is the place to think about the win32 issue).	2009-06-01 15:50:07 -07:00
Brian Warner	4177a3616b	remove plaintext-hashing code from the helper interface, to close #722 ... and deny the Helper the ability to mount a partial-information-guessing attack. This will probably break compatibility between new clients and very old (pre-1.0) helpers.	2009-06-01 15:49:16 -07:00
Brian Warner	c516361fd2	start to factor server-connection-management into a distinct 'StorageServerFarmBroker' object, separate from the client and the introducer. This is the starting point for #467: static server selection	2009-06-01 14:06:04 -07:00
Brian Warner	d29281c9c5	mutable: catch and display first error, so code bugs which break all servers get displayed better	2009-06-01 14:04:07 -07:00
Kevin Reid	23441389a5	Modify markup of Tahoe web pages to be more amenable to styling; some minor changes of wording.	2009-05-26 16:25:45 -07:00
Kevin Reid	a71c914e50	Tweak wording in directory page: not-read-only is "modifiable", mention creating a directory _in this directory_.	2009-05-26 16:24:14 -07:00
Kevin Reid	3e80676a29	Comment on duplication of code/markup found during styling project.	2009-05-03 13:34:42 -07:00
Kevin Reid	d0b4fd4689	Add CSS styles to spiff up the Tahoe WUI's appearance, particularly the welcome page and directories.	2009-05-03 13:31:42 -07:00
Kevin Reid	35d3671f4e	Link all Tahoe web pages to the /tahoe_css stylesheet which already exists.	2009-05-03 13:25:33 -07:00
Kevin Reid	2f3842954b	Fix broken link from Provisioning to Reliability page.	2009-05-01 12:10:50 -07:00
Brian Warner	de83526acd	immutable/encode.py: tolerate immediate _remove_shareholder by copying the ... landlord list before iterating over it. This can probably only happen in unit tests, but cleaning it up makes certain test failures easier to analyze.	2009-05-22 11:44:24 -07:00
Brian Warner	1863aee0aa	switch to using RemoteException instead of 'wrapped' RemoteReferences. Should fix #653, the rref-EQ problem	2009-05-21 17:46:32 -07:00
Brian Warner	c9803d5217	switch all foolscap imports to use foolscap.api or foolscap.logging	2009-05-21 17:38:23 -07:00
Brian Warner	b542d5aca6	test_runner.py: fix minor typo	2009-05-19 20:36:20 -07:00
Brian Warner	9f0bc04d32	test_introducer.py: add a test for the python2.4.0/2.4.1 bug in base64.b32decode	2009-05-18 20:41:01 -07:00
Brian Warner	79437baade	immutable WriteBucketProxy: use pipeline to speed up uploads by overlapping roundtrips, for #392	2009-05-18 16:44:22 -07:00
Brian Warner	e76c6b606f	util/pipeline.py: new utility class to manage size-limited work pipelines, for #392	2009-05-18 16:43:26 -07:00
Zooko O'Whielacronx	718537ab8c	tests: mark test_runner as coded in utf-8 instead of ascii	2009-05-07 15:31:51 -07:00
Zooko O'Whielacronx	04ff9a3f97	tests: raise timeout on test_runner.RunNode.test_introducer from 120s to 240s, since it hit the 120s time-out on François Lenny-armv5tel	2009-05-07 14:50:12 -07:00
Zooko O'Whielacronx	77b578928e	trivial: fix comment	2009-04-13 10:41:38 -07:00
Zooko O'Whielacronx	6350634ee3	tests: remove obsolete test (it tests functionality that is long gone) which occasionally ERRORs now that we have more metadata (since [20090411225205-92b7f-7adfb89cb4db4ac7d28427934dea3d2c108f6476])	2009-04-12 19:36:21 -07:00
Zooko O'Whielacronx	6b9899ade1	trivial: remove trailing whitespace and unused import	2009-04-11 19:17:42 -07:00
Zooko O'Whielacronx	9729753692	dirnode: add 'tahoe'/'linkcrtime' and 'tahoe'/'linkmotime' to take the place of what 'mtime'/'ctime' originally did, and make the 'tahoe' subdict be unwritable through the set_children API ... Also add extensive documentation in docs/frontends/webapi.txt about the behaviors of these values. See ticket #628.	2009-04-11 15:52:05 -07:00
Zooko O'Whielacronx	5b02896307	rollback [20090226150237-b2345-1e916a746a7f4627b050f02f0e442fae5caf69d4] for 1.4.0 release; #645	2009-04-11 11:19:06 -07:00
Brian Warner	54952e9bea	#622: disable 'Repair' button on check-results page until we make it work correctly	2009-04-09 16:59:59 -07:00
Brian Warner	922b3034be	web: make sure that PUT /uri?mutable=false really means immutable, fixes #675	2009-04-07 19:13:40 -07:00
Zooko O'Whielacronx	37c6e77764	wui: edit some of the human-readable parts of the wui such as button labels ... (The word "parent" suggests that you can go up a directory hierarchy -- perhaps that word is vestigial.)	2009-04-07 11:54:59 -07:00
Zooko O'Whielacronx	8b1cd154da	wui: fix bug in which empty directory is marked as "unreadable", add test, remove exclamation point	2009-04-07 11:28:34 -07:00
Brian Warner	40c78297e8	hashtree.py: another micro-optimization, expand on a comment	2009-04-03 21:35:27 -07:00
Zooko O'Whielacronx	de8475389b	hashtree: fix tests of depth_of	2009-04-03 16:56:14 -07:00
Zooko O'Whielacronx	3ac5f427ba	trivial: source code metadata	2009-04-03 16:33:15 -07:00
Zooko O'Whielacronx	a07e746d9c	trivial: whitespace	2009-04-03 16:33:02 -07:00
Zooko O'Whielacronx	d381bdc905	hashtree: a couple of tiny improvements ... * calculate depth-first with math instead of traversing the actual tree * don't mark a node with a red dot if you instead compare it with an extant hash value (tiny optimization) * edit a comment about checking the root node	2009-04-03 16:30:37 -07:00
Zooko O'Whielacronx	b12a7f9ee8	leases, time_format: modify time stamping in lease description ... * emit lease expiry date in ISO-8601'ish format as well as Brian's format * rename iso_utc_time_to_localseconds() to iso_utc_time_to_seconds() * add iso_utc_date() * simplify the body of iso_utc_time_to_seconds()	2009-04-03 15:59:04 -07:00
Brian Warner	466014f66f	hashtree: fix O(N**2) behavior, to improve fatal alacrity problems in a 10GB file (#670). Also improve docstring.	2009-03-31 13:21:27 -07:00
Brian Warner	bd93430c53	expirer: include crawler progress in the JSON status output	2009-03-24 13:51:37 -07:00
Brian Warner	b5d4972a7a	move GC docs out of proposed/, since it's all implemented now. Add reference to configuration.txt . Add expire.*= suggestions to tahoe.cfg .	2009-03-23 16:08:20 -07:00
Brian Warner	5e8c31c3b6	storage: use constant-time comparison for write-enablers and lease-secrets	2009-03-22 20:21:28 -07:00
Brian Warner	01e2032669	hashutil: add constant-time comparison function, to avoid timing attacks when python's short-circuiting data-dependent == operator is used to, say, check a write-enabler	2009-03-22 20:20:55 -07:00
Brian Warner	6599eae6f9	WUI: fix display of empty directories, it threw an exception before	2009-03-20 16:58:09 -07:00
Brian Warner	4f4d748fd9	storage webstatus: insert spaces when we're configured to expire multiple sharetypes	2009-03-20 15:44:50 -07:00
Brian Warner	3b65607926	expirer: tolerate empty buckets, refactor bucketsize-counting code a bit, don't increment -mutable/-immutable counters unless we actually know the sharetype	2009-03-20 12:18:16 -07:00
Brian Warner	8645738c77	storage: improve wording of status message	2009-03-19 11:48:37 -07:00
Brian Warner	66fe8ca24c	parse_date: insist that it returns an int	2009-03-19 11:44:43 -07:00
Brian Warner	186b6a8c01	storage status: report expiration-cutoff-date like 19-Mar-2009 (as opposed to the tahoe.cfg input format of 2009-03-19), for redundancy: someone who gets the month and day switched will have a better chance to spot the problem in the storage-status output if it's in a different format	2009-03-19 11:07:56 -07:00
Brian Warner	f0071c2571	expirer: clean up constructor args, add tahoe.cfg controls, use cutoff_date instead of date_cutoff	2009-03-18 18:00:09 -07:00
Brian Warner	babcf632da	util/time_format: new routine to parse dates like 2009-03-18, switch expirer to use it. I'd prefer to use 18-Mar-2009, but it is surprisingly non-trivial to build a parser that will take UTC dates instead of local dates	2009-03-18 17:58:14 -07:00
Brian Warner	8eaee28550	expirer: change setup, config options, in preparation for adding tahoe.cfg controls	2009-03-18 17:21:38 -07:00
Brian Warner	fffab0d724	expirer: track mutable-vs-immutable sharecounts and sizes, report them on the web status page for comparison	2009-03-18 13:25:04 -07:00
Brian Warner	406fdba61f	add utility function to parse durations, for lease-expiration config	2009-03-17 00:01:17 -07:00
Brian Warner	24ab5ec26f	expirer: add mode to expire only-mutable or only-immutable shares	2009-03-16 23:51:18 -07:00
Brian Warner	475e7b8804	tahoe_ls.py: add comment about error cases to improve	2009-03-16 22:12:06 -07:00
Brian Warner	c7254c5f1d	GC: add date-cutoff -based expiration, add proposed docs	2009-03-16 22:10:41 -07:00
Brian Warner	2b525a42d3	test_deepcheck: remove the 10s timeout: our dapper buildslave requires 30s, and the reduced timeout was only there because this tests fails by timeout rather than explicitly	2009-03-15 17:51:34 -07:00
Brian Warner	f0b0ad1c8f	tahoe cp -r: add --caps-only flag, to write filecaps into local files instead of actual file contents. Used only for debugging and as a quick tree-comparison tool.	2009-03-15 16:19:58 -07:00
Brian Warner	6e57576f2e	dirnode deep_traverse: insert a turn break (fireEventually) at least once every 100 files, otherwise a CHK followed by more than 158 LITs can overflow the stack, sort of like #237.	2009-03-13 16:31:35 -07:00
Brian Warner	e59164e0bb	consolidate: remove pointless 'else' after for loop	2009-03-13 01:27:51 -07:00
Brian Warner	ae64ddf8e6	consolidate: add eta, flush stdout	2009-03-13 01:24:51 -07:00
Brian Warner	06bf443e91	consolidate: tolerate unicode dirnames	2009-03-12 23:54:02 -07:00
Brian Warner	1a741fdb03	dirnode.py: when doing deep-traverse, walk each directory in alphabetical order, to make things like 'manifest' more predictable	2009-03-12 23:50:46 -07:00
Brian Warner	a18f8d4cc7	consolidator: add progress to scan-old-directory passes	2009-03-12 22:47:28 -07:00
Brian Warner	ddb2f1a6e3	consolidator: fix cycle detection to not trigger on merely shared directories, add snapshot counter to progress	2009-03-12 21:22:29 -07:00
Brian Warner	ddc9a7ae74	consolidator: re-use more directories, add total directories seen-vs-used counts	2009-03-12 20:48:01 -07:00
Brian Warner	809ec25ffa	tahoe_backup.py: tolerate more time formats	2009-03-12 18:16:00 -07:00
Brian Warner	1c24707f19	consolidator: add more verbose traversal of directories	2009-03-12 16:29:00 -07:00
Brian Warner	760688a224	consolidate: create multiple numbered backups of the original Archives directory, not just the first time	2009-03-12 16:04:27 -07:00
Brian Warner	c4c6a62954	add 'tahoe debug consolidate' command, to merge directories created by repeated 'tahoe cp -r' or the allmydata win32 backup tool, into the form that would have been created by 'tahoe backup'.	2009-03-12 13:56:06 -07:00
Brian Warner	969b758bf5	tahoe_cp.py: return 0 for success, instead of None	2009-03-12 13:53:45 -07:00
Brian Warner	1ccd426a34	expirer: fix prediction math, thanks to Zandr for the catch	2009-03-09 13:42:17 -07:00
Brian Warner	8708045a98	storage.expirer: oops, fix upgrade-handler code	2009-03-08 20:55:16 -07:00
Brian Warner	df3f7f93e0	storage.expirer: handle upgrades better	2009-03-08 20:42:20 -07:00
Brian Warner	a68ad06254	storage.expirer: exercise the last missing line of webstatus code	2009-03-08 20:38:28 -07:00
Brian Warner	6d6049430c	expirer: tolerate corrupt shares, add them to the state and history for future examination	2009-03-08 20:08:40 -07:00
Brian Warner	1a98521c3d	storage/immutable: raise a specific error upon seeing a bad version number, instead of using assert. Also wrap to 80cols.	2009-03-08 20:07:32 -07:00
Brian Warner	02b40ec499	storage: add test coverage for BucketReader.__repr__	2009-03-08 20:05:27 -07:00
Brian Warner	6d7319c588	storage/mutable: raise a specific error upon seeing bad magic, instead of using assert	2009-03-08 19:02:01 -07:00
Zooko O'Whielacronx	7e7167f2e5	nodeadmin: node stops itself if a hotline file hasn't been touched in 120 seconds now, instead of in 60 seconds ... A test failed on draco (MacPPC) because it took 67.1 seconds to get around to running the test, and the node had already stopped itself when the hotline file was 60 seconds old.	2009-03-07 20:50:39 -07:00
Brian Warner	5675b4e7e0	expirer: make web display a bit more consistent	2009-03-07 16:14:42 -07:00
Brian Warner	df045650e0	web/storage.py: tolerate unknown-future displays, I'm not sure why LeaseCrawler.test_unpredictable_future didn't catch this	2009-03-07 16:02:43 -07:00
Brian Warner	192c161be7	tahoe_cp.py: improve error reporting slightly: don't json-interpret HTTP errors, pass through tahoe webapi error messages	2009-03-07 05:40:51 -07:00
Brian Warner	2c2c3c2295	test_cli.py: assert that 'ls' on an unrecoverable file now gives a better error message	2009-03-07 05:08:15 -07:00
Brian Warner	67d7600781	test_web: test behavior of broken-dirnode GET, both html and json	2009-03-07 04:57:07 -07:00
Brian Warner	950200fece	web: when a dirnode can't be read, emit a regular HTML page but with the child-table and upload-forms replaced with an apologetic message. Make sure to include the 'get info' links so the user can do a filecheck	2009-03-07 04:56:01 -07:00
Brian Warner	badd79671c	web/common: split out exception-to-explanation+code mapping to a separate humanize_failure() function, so it can be used by other code. Add explanation for mutable UnrecoverableFileError.	2009-03-07 04:54:08 -07:00
Brian Warner	9c7fafd5af	test_cli: use explicit (and stable) testdirs, instead of using self.mktemp	2009-03-07 03:04:28 -07:00
Brian Warner	091cfd04ca	test_storage: solaris doesn't appear to give the same block count as other platforms, so don't assert as much about 'diskbytes' recovered	2009-03-07 02:45:18 -07:00
Brian Warner	24ebdb3954	tahoe_cp.py: don't be verbose unless --verbose is passed: default is one line of success	2009-03-07 02:44:21 -07:00
Brian Warner	0dee2a6036	storage: add a lease-checker-and-expirer crawler, plus web status page. ... This walks slowly through all shares, examining their leases, deciding which are still valid and which have expired. Once enabled, it will then remove the expired leases, and delete shares which no longer have any valid leases. Note that there is not yet a tahoe.cfg option to enable lease-deletion: the current code is read-only. A subsequent patch will add a tahoe.cfg knob to control this, as well as docs. Some other minor items included in this patch: tahoe debug dump-share has a new --leases-only flag storage sharefile/leaseinfo code is cleaned up storage web status page (/storage) has more info, more tests coverage space-left measurement on OS-X should be more accurate (it was off by 2048x) (use stat .f_frsize instead of f_bsize)	2009-03-06 22:45:17 -07:00
Zooko O'Whielacronx	e9199a89ab	trivial: remove unused import detected by pyflakes, and remove trailing whitespace	2009-03-05 16:32:04 -07:00
Zooko O'Whielacronx	5e90d82a02	trivial: use more specific function for ascii-encoding storage index	2009-02-22 11:57:51 -07:00
Brian Warner	c5ecb7b3ba	test_cli: validate non-HTML error response of 'tahoe get' on an unrecoverable file	2009-03-03 22:11:46 -07:00
Brian Warner	f42e3bb107	web: full patch for HTML-vs-plaintext traceback renderings, improve test coverage of exception rendering	2009-03-03 21:56:30 -07:00
Brian Warner	90226f335f	web/common.py: use 'Accept:' header to control HTML-vs-text/plain traceback renderings	2009-03-03 21:54:57 -07:00
Brian Warner	53f50fcc78	test/common: rearrange shouldHTTPError	2009-03-03 19:40:59 -07:00
Brian Warner	c4bda3daa3	web: move plural() to common.py	2009-03-03 19:40:19 -07:00
Brian Warner	67571eb033	add more information to NotEnoughSharesError, split out new exceptions for no-servers and no-source-of-ueb-hash	2009-03-03 19:37:15 -07:00
Brian Warner	f15c0be5af	test_web: move shouldHTTPError into common.WebErrorMixin	2009-03-03 16:56:20 -07:00
Brian Warner	559c369574	test_web: clean up shouldHTTPError methods	2009-03-03 16:46:04 -07:00
Brian Warner	400c04c19a	immutable checker add-lease: catch remote IndexError here too	2009-02-27 01:17:24 -07:00
Brian Warner	99ebf6eaf5	servermap add-lease: fix the code that's supposed to catch remote IndexErrors, I forgot that they present as ServerFailures instead. This should stop the deluge of Incidents that occur when you do add-lease against 1.3.0 servers	2009-02-27 01:04:26 -07:00
Brian Warner	8c3013c4f7	rrefutil: add check_remote utility function	2009-02-27 00:59:57 -07:00
Brian Warner	1b3e635936	rrefutil: add trap_remote utility and friends	2009-02-27 00:55:24 -07:00
Brian Warner	8251572e01	web: improve layout of storage status with a table	2009-02-26 19:58:38 -07:00
Brian Warner	112dc35563	crawler: add ETA to get_progress()	2009-02-26 19:42:48 -07:00
Alberto Berti	3035dfb8ed	Fix for bug #645, correct path handling logic so that it works from sshfs	2009-02-26 09:02:37 -07:00
Zooko O'Whielacronx	9ab4aa9016	wui: s/Provisioning/Reliability/ ; suggested by Terrell	2009-02-25 21:09:04 -07:00
Brian Warner	7c35f01073	startstop_node.py: when restarting, tolerate a stale pid too	2009-02-25 19:52:45 -07:00
Brian Warner	63b19e567c	web: fix the ERROR: line to work the same in python2.4 and 2.5	2009-02-25 01:46:21 -07:00
Brian Warner	7d4901197a	test_cli/test_web: fix spurious test failure on solaris (maybe python2.4?) due to variations in the way that exceptions are stringified	2009-02-25 00:01:28 -07:00
Brian Warner	a3c1fe35d9	CLI: modify 'tahoe manifest' and 'tahoe deep-check' to report ERROR: properly. For #590.	2009-02-24 23:44:15 -07:00
Brian Warner	fd4ceb6a87	webapi: modify streaming deep-manifest/deep-checker to emit an ERROR: line if they encounter an unrecoverable+untraversable directory. For #590.	2009-02-24 23:13:35 -07:00
Brian Warner	7f8708cb7c	scripts/common: fix alias handling on windows again, emit slightly nicer error message in response to an unknown alias	2009-02-24 22:21:36 -07:00
Brian Warner	fdb0200eca	#165: make 'tahoe restart --force' the default behavior: warn but do not stop if restart is used on something that wasn't a running node, and always try to start it afterwards. This is particularly important for #315 (restart -m), because otherwise a single not-already-running node will prevent all nodes from being restarted, resulting in longer downtime than necessary	2009-02-24 21:42:13 -07:00
Brian Warner	51e3baeefc	test_deepcheck: switch deep-check tests to use no-network too. This cuts the runtime down by about 50%	2009-02-24 21:04:57 -07:00
Brian Warner	6cf4096156	test_deepcheck: convert MutableChecker to no-network GridTest	2009-02-24 20:00:10 -07:00
Brian Warner	2e7f64d392	tests/no_network: move GET into the GridTestMixin class	2009-02-24 18:33:00 -07:00
Brian Warner	6f981e4eab	common_web.py: oops, add .fields to the other FakeRequest	2009-02-24 18:04:59 -07:00
Brian Warner	376eab0c04	test_cli: exercise the recent tolerate-'c:\dir\file.txt' fix in scripts/common, recorded in a separate match to make it easier to merge the fix to prod	2009-02-24 17:56:20 -07:00
Brian Warner	a0aea06a9e	scripts/common: on windows, tolerate paths like 'c:\dir\file.txt', by treating single-letter aliases on windows/cygwin as non-aliases	2009-02-24 17:55:22 -07:00
Brian Warner	8d8250e44a	test/common_web.py: add a .fields attribute to our FakeRequest, since we support versions of Nevow that are old enough to not do it themselves	2009-02-24 17:20:50 -07:00
Alberto Berti	e1c925b373	Two small fixes on documentation for cli backup command.	2009-02-24 16:36:34 -07:00
Brian Warner	2810de32b1	test_web: add (disabled) test to see what happens when deep-check encounters an unrecoverable directory. We still need code changes to improve this behavior.	2009-02-24 15:40:17 -07:00
Alberto Berti	7e8958671b	Add elapsed timestamp to cli backup command final summary.	2009-02-24 11:14:25 -07:00
Brian Warner	7cfbb9c832	test_repairer: change to use faster no_network.GridTestMixin, split Verifier tests into separate cases, refactor judgement funcs into shared methods	2009-02-23 22:15:06 -07:00
Brian Warner	f95e9b5964	immutable/checker.py: trap ShareVersionIncompatible too. Also, use f.check ... instead of examining the value returned by f.trap, because the latter appears to squash exception types down into their base classes (i.e. since ShareVersionIncompatible is a subclass of LayoutInvalid, f.trap(Failure(ShareVersionIncompatible)) == LayoutInvalid). All this resulted in 'incompatible' shares being misclassified as 'corrupt'.	2009-02-23 22:14:05 -07:00
Brian Warner	9af9d8ae35	immutable/layout.py: wrap to 80 cols, no functional changes	2009-02-23 18:58:37 -07:00
Brian Warner	2be729b1e4	test_repairer: change Repairer to use much-faster no_network.GridTestMixin. As a side-effect, fix what I think was a bug: some of the assert-minimal-effort-expended checks were mixing write counts and allocate counts	2009-02-23 17:42:27 -07:00
Brian Warner	cc3c1ae8df	test/no_network.py: add a basic stats provider	2009-02-23 17:39:37 -07:00
Brian Warner	f35c659b2f	tests: stop using setUpClass/tearDownClass, since they've been deprecated in Twisted-8.2.0	2009-02-23 14:43:12 -07:00
Brian Warner	bc91689f8e	test_checker: improve test coverage for checker results	2009-02-23 14:19:43 -07:00
Alberto Berti	d9649052d4	Fixed tests again so they will pass on windows.	2009-02-22 18:35:02 -07:00
Brian Warner	15ee186499	scripts: stop using RuntimeError, for #639	2009-02-22 17:31:06 -07:00
Brian Warner	c23d051d3f	mutable/publish: stop using RuntimeError, for #639	2009-02-22 17:30:56 -07:00
Brian Warner	c4f8005867	remove more RuntimeError from unit tests, for #639	2009-02-22 17:28:55 -07:00
Brian Warner	5c3d7d8e73	stop using RuntimeError in unit tests, for #639	2009-02-22 17:27:22 -07:00
Brian Warner	d95b01a214	ftpd/sftpd: stop using RuntimeError, for #639	2009-02-22 17:24:26 -07:00
Alberto Berti	b945c275c2	Added tests for the cse when listdir is an iterator	2009-02-22 16:43:56 -07:00
Alberto Berti	cd0bd97954	Fixed tests so that they pass also on buildbots.	2009-02-22 16:43:11 -07:00
Alberto Berti	b2f8fe04c6	Use failUnlessEqual instead of failUnless(a == b)	2009-02-22 16:42:14 -07:00
Alberto Berti	89a20fe4dd	Better implementation of filtering algorithm.	2009-02-22 16:40:49 -07:00
Alberto Berti	2fbca1596b	Removed '.hgrags' from vcs excludes	2009-02-22 16:39:46 -07:00
Alberto Berti	5d629d0050	Added tests for the --exclude* options of backup command.	2009-02-22 10:51:06 -07:00
Alberto Berti	5d57da93fd	Added --exclude, --exclude-from and --exclude-vcs options to backup command. ... It is still impossible to specify absolute exclusion path, only relative. I must check with tar or rsync how they allow them to be specified.	2009-02-22 11:08:29 -07:00
Alberto Berti	c54fdd61b0	Raise a more explanatory exception for errors encountered during backup processing.	2009-02-22 11:02:52 -07:00
Alberto Berti	ff6a77cc6b	Added tests for the fixed alias related command's synopsis	2009-02-22 10:37:32 -07:00
Alberto Berti	ffab4002aa	Add missing synopsis and descriptions for alias commands.	2009-02-20 18:31:06 -07:00
Brian Warner	c85eccbf64	test_web.py: increase test coverage of web.status.plural()	2009-02-21 18:01:16 -07:00
Brian Warner	77f3b83d68	crawler: fix performance problems: only save state once per timeslice (not after every bucket), don't start the crawler until 5 minutes after node startup	2009-02-21 14:56:49 -07:00
Brian Warner	a04d3b8fe8	test_system: oops, don't assume that all files in storage/ are in a deep storage/shares/prefix/si/shnum path, since now the crawler pickle has a short path	2009-02-21 00:17:10 -07:00
Brian Warner	b9c4f4bdf6	crawler: tolerate low-resolution system clocks (i.e. windows)	2009-02-21 00:15:33 -07:00
Brian Warner	106d31b112	BucketCountingCrawler: store just the count, not cycle+count, since it's too easy to make usage mistakes otherwise	2009-02-20 21:58:31 -07:00
Brian Warner	3c6471c717	test_storage: startService the server, as is now the standard practice	2009-02-20 21:57:55 -07:00
Brian Warner	f934289d2d	crawler: load state from the pickle in init, rather than waiting until startService, so get_state() can be called early	2009-02-20 21:57:20 -07:00
Brian Warner	1077826357	BucketCountingCrawler: rename status and state keys to use 'bucket' instead of 'share', because the former is more accurate	2009-02-20 21:46:06 -07:00
Brian Warner	d2d297f12f	storage: also report space-free-for-root and space-free-for-nonroot, since that helps users understand the space-left-for-tahoe number better	2009-02-20 21:28:56 -07:00
Brian Warner	b3cd4952bd	storage: add bucket-counting share crawler, add its output (number of files+directories maintained by a storage server) and status to the webapi /storage page	2009-02-20 21:04:08 -07:00
Brian Warner	d14f00c537	storage: move si_b2a/si_a2b/storage_index_to_dir out of server.py and into common.py	2009-02-20 21:03:09 -07:00
Brian Warner	73e05bf967	crawler: add get_progress, clean up get_state	2009-02-20 18:27:43 -07:00
Brian Warner	2e45619844	web/storage: make sure we can handle platforms without os.statvfs too	2009-02-20 16:03:53 -07:00
Brian Warner	c6a061e600	crawler: provide for one-shot crawlers, which stop after their first full cycle, for share-upgraders and database-populaters	2009-02-20 15:19:11 -07:00
Brian Warner	00677ff9a5	web: add Storage status page, improve tests	2009-02-20 14:29:26 -07:00
Brian Warner	ff6907a557	storage: include reserved_space in stats	2009-02-20 14:29:20 -07:00
Brian Warner	4b8cc42f18	web/check_results: sort share identifiers in the sharemap display	2009-02-20 12:29:22 -07:00
Brian Warner	c6c30ac5d4	webapi: pass client through constructor arguments, remove IClient, should make it easier to test web renderers in isolation	2009-02-20 12:15:54 -07:00
Brian Warner	e357b8a679	test/no_network: do startService on the storage servers, make it easier to customize the storage servers	2009-02-19 20:22:54 -07:00
Brian Warner	ef4ff21ae7	crawler: modify API to support upcoming bucket-counting crawler	2009-02-19 19:31:42 -07:00
Brian Warner	9bc08158c6	test_backupdb: make the not-a-database file larger, since the older sqlite-2.3.2 on OS-X is easily fooled	2009-02-19 18:04:09 -07:00
Brian Warner	26fbe9df95	web/reliability: add parameter descriptions, adapted from a patch from Terrell Russell.	2009-02-19 16:29:18 -07:00
Brian Warner	2f751332e8	test_crawler: hush pyflakes	2009-02-19 14:23:40 -07:00
Brian Warner	ff9811211d	test_crawler: disable the percentage-of-cpu-used test, since it is too unreliable on our slow buildslaves. But leave the code in place for developers to run by hand.	2009-02-19 14:16:54 -07:00
Brian Warner	790a10d1b2	reliability.py: fix the numpy conversion, it was completely broken. Thanks to Terrell Russell for the help.	2009-02-19 13:55:15 -07:00
Brian Warner	268cd538e2	reliability: switch to NumPy, since Numeric is deprecated	2009-02-19 01:44:35 -07:00
Brian Warner	f10aa59e5b	test_crawler: don't require >=1 cycle on cygwin	2009-02-19 00:58:18 -07:00
Brian Warner	dfd72c6483	crawler: use fileutil.move_info_place in preference to our own version	2009-02-18 23:13:42 -07:00
Brian Warner	89f041ac83	fileutil: add move_into_place(), to perform the standard unix trick of atomically replacing a file, with a fallback for windows	2009-02-18 23:13:10 -07:00
Brian Warner	b949ea4f32	crawler: fix problems on windows and our slow cygwin slave	2009-02-18 22:24:31 -07:00
Brian Warner	193889f793	#633: first version of a rate-limited interruptable share-crawler	2009-02-18 21:46:33 -07:00
Brian Warner	4646451de6	change StorageServer to take nodeid in the constructor, instead of assigning it later, since it's cleaner and because the original problem (Tubs not being ready until later) went away	2009-02-18 16:23:01 -07:00
Brian Warner	b398a69f14	test_system: split off checker tests to test_deepcheck.py, this file is too big	2009-02-18 15:42:34 -07:00
Brian Warner	ef53da2b12	break storage.py into smaller pieces in storage/*.py . No behavioral changes.	2009-02-18 14:46:55 -07:00
Brian Warner	a0c5f92cbd	immutable/layout: minor change to repr name	2009-02-18 14:46:48 -07:00
Brian Warner	2346d8621d	interfaces.py: allow add/renew/cancel-lease to return Any, so that 1.3.1 clients (the first to use these calls) can tolerate future storage servers which might return something other than None	2009-02-18 13:29:03 -07:00
Brian Warner	bce4a5385b	add --add-lease to 'tahoe check', 'tahoe deep-check', and webapi.	2009-02-17 19:32:43 -07:00
Brian Warner	e9563ebc02	change RIStorageServer.remote_add_lease to exit silently in case of no-such-bucket, instead of raising IndexError, because that makes the upcoming --add-lease feature faster and less noisy	2009-02-17 19:30:53 -07:00
Brian Warner	fde2289e7b	CLI #590: convert 'tahoe deep-check' to streaming form, improve display, add tests	2009-02-17 17:15:11 -07:00
Brian Warner	0e78b2587c	interfaces.py: document behavior of add_lease/renew_lease/cancel_lease, before I change it	2009-02-17 13:48:09 -07:00
Brian Warner	d52eadfc58	test_backupdb: improve error messages if the test fails	2009-02-17 11:08:38 -07:00
Brian Warner	476a5c8fac	webapi #590: add streaming deep-check. Still need a CLI tool to use it.	2009-02-16 23:35:53 -07:00
Brian Warner	8579e25059	test_web.Grid: change the CHECK() function to make it easier to test t= values with hyphens in them	2009-02-16 23:00:34 -07:00
Brian Warner	522641bb32	test_web: improve checker-results coverage with a no-network -based test, enhance no-network harness to assist, fix some bugs in web/check_results.py that were exposed	2009-02-16 22:12:42 -07:00
Brian Warner	059f24adca	web: fix handling of reliability page when Numeric is not available	2009-02-16 19:56:58 -07:00
Brian Warner	f2f8ede11f	test/no_network: update comments with setup timing: no_network takes 50ms, SystemTestMixin takes 2s (on my laptop)	2009-02-16 18:06:43 -07:00
Brian Warner	040cb39613	test_upload: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary	2009-02-16 17:44:57 -07:00
Brian Warner	911abcc34b	test_download: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary	2009-02-16 17:36:58 -07:00
Brian Warner	e9427ea777	test_dirnode.py: convert Deleter to new no-network gridtest	2009-02-16 17:23:48 -07:00
Brian Warner	267d91f797	test_cli.py: modify to use the new 'no-network' gridtest instead of SystemTestMixin, which speeds it up from 73s to 43s on my system	2009-02-16 17:20:05 -07:00
Brian Warner	191fd8556a	tests: fix no_network framework to work with upload/download and checker	2009-02-16 17:19:47 -07:00
Brian Warner	0409bc5628	client.py: improve docstring	2009-02-16 17:15:32 -07:00
Brian Warner	770f976a85	test_cli: add test coverage for help strings	2009-02-16 15:08:33 -07:00
Brian Warner	ebe0f2ac08	test/no_network: new test harness, like system-test but doesn't use the network so it's faster	2009-02-16 14:58:44 -07:00
Brian Warner	8ff76c6269	interfaces.py: minor docstring edit	2009-02-16 14:58:16 -07:00
Brian Warner	df90dd8e73	test_cli: improve test coverage slightly	2009-02-15 21:04:51 -07:00
Brian Warner	3235b9630b	test_util: get almost full test coverage of dictutil, starting with the original pyutil tests as a base. The remaining three uncovered lines involve funny cases of ValueOrderedDict that I can't figure out how to get at	2009-02-15 20:32:10 -07:00
Brian Warner	cc27b218f5	provisioning/reliability: add tests, hush pyflakes, remove dead code, fix web links	2009-02-15 16:24:51 -07:00
Brian Warner	e1380b132b	util/statistics: add tests, fix mean_repair_cost	2009-02-15 16:23:26 -07:00
Brian Warner	812f23585d	test_repairer: hush pyflakes	2009-02-15 16:23:10 -07:00
Brian Warner	e2efd911a3	build a 'reliability' web page, with a simulation of file decay and repair over time	2009-02-13 17:42:34 -07:00
Shawn Willden	51ab76875b	More lossmodel work, on repair.	2009-01-15 20:56:48 -07:00
Shawn Willden	21832280da	Statistics module ... Added a statistics module for calculating various facets of share survival statistics.	2009-01-13 20:12:35 -07:00
Brian Warner	60b06b8d32	test_dirnode: #625 run deep-check on a readonly dirnode too	2009-02-13 14:53:37 -07:00
Brian Warner	4aee51ff6f	figleaf_htmlizer: fix order of summary counters	2009-02-13 09:57:53 -07:00
Brian Warner	aee319e236	figleaf_htmlizer: oops, re-ignore files that aren't under root, like code in auto-built eggs	2009-02-13 00:00:22 -07:00
Zooko O'Whielacronx	14d23d99a5	setup: make the "full version string" be "allmydata-tahoe/1.3.0" instead of "allmydata-tahoe-1.3.0" and the UserAgent string of the cli be "allmydata-tahoe/1.3.0 (tahoe-client)" ... This is webbish. Thanks to kpreid for suggesting it.	2009-02-12 22:37:38 -07:00
Brian Warner	30724593fc	figleaf_htmlizer: emit stats to stdout, so buildbot can see it	2009-02-12 15:10:20 -07:00
Brian Warner	781c271b82	figleaf_htmlizer: render changes in coverage relative to a previous test run using --old-coverage	2009-02-12 15:04:12 -07:00
Zooko O'Whielacronx	d7dbd6675e	immutable repairer: fix DownUpConnector so that it satisfies short reads the were requested after the last write and before the close ... This is probably the cause of the very rare "loss of progress" bug. This is tested by unit tests. A recent patch changed this to errback instead of losing progress, and now this patch is changing it again to return a short read instead of errbacking. Returning a short read is what the uploader (in encode.py) is expecting, when it is reading the last block of the ciphertext, which might be shorter than the other blocks.	2009-02-12 17:04:47 -07:00
Zooko O'Whielacronx	bdb992467c	immutable repairer: add an assertion that a certain value in this tricky function is always what I think it is	2009-02-12 16:31:32 -07:00
Zooko O'Whielacronx	2d45d192d1	immutable repairer: add tests of how it handles short reads	2009-02-12 16:48:53 -07:00
Zooko O'Whielacronx	8303f90da0	nodeadmin: node stops itself if a hotline file hasn't been touched in 60 seconds now, instead of in 40 seconds ... A test failed on draco (MacPPC) because it took 49 seconds to get around to running the test, and the node had already stopped itself when the hotline file was 40 seconds old.	2009-02-12 15:21:31 -07:00
Zooko O'Whielacronx	76d7cc4404	immutable repairer: errback any pending readers of DownUpConnectorwhen it runs out of bytes, and test that fact	2009-02-11 20:11:29 -07:00
Brian Warner	125bf09528	figleaf_htmlizer: more rearranging, behavior should still be unchanged	2009-02-11 20:05:15 -07:00
Brian Warner	e02ecec1ad	figleaf_htmlizer: break it up into subfunctions, behavior should still be unchanged	2009-02-11 19:56:07 -07:00
Brian Warner	6bde1b0345	figleaf_htmlizer: rewrite in class form, behavior should be the same as before	2009-02-11 19:40:50 -07:00
Brian Warner	6db747b1f0	figleaf_htmlizer: rewrite with twisted.python.usage, remove logging: should behave the same as before	2009-02-11 19:16:43 -07:00
Brian Warner	f3ed579e74	figleaf_htmlizer: expand tabs, fix to 4-space indents. No functional changes.	2009-02-11 19:05:42 -07:00
Zooko O'Whielacronx	7eb260a9cf	versioning: include an "appname" in the application version string in the versioning protocol, and make that appname be controlled by setup.py ... It is currently hardcoded in setup.py to be 'allmydata-tahoe'. Ticket #556 is to make it configurable by a runtime command-line argument to setup.py: "--appname=foo", but I suddenly wondered if we really wanted that and at the same time realized that we don't need that for tahoe-1.3.0 release, so this patch just hardcodes it in setup.py. setup.py inspects a file named 'src/allmydata/_appname.py' and assert that it contains the string "__appname__ = 'allmydata-tahoe'", and creates it if it isn't already present. src/allmydata/__init__.py import _appname and reads __appname__ from it. The rest of the Python code imports allmydata and inspects "allmydata.__appname__", although actually every use it uses "allmydata.__full_version__" instead, where "allmydata.__full_version__" is created in src/allmydata/__init__.py to be: __full_version__ = __appname + '-' + str(__version__). All the code that emits an "application version string" when describing what version of a protocol it supports (introducer server, storage server, upload helper), or when describing itself in general (introducer client), usese allmydata.__full_version__. This fixes ticket #556 at least well enough for tahoe-1.3.0 release.	2009-02-11 17:18:16 -07:00
Brian Warner	23fdd4f48e	backupdb: cosmetic: capitalize the no-pysqlite instructions properly. Thanks to Terrell Russell for the catch.	2009-02-11 15:28:30 -07:00
Brian Warner	5f288842d5	test_repairer: disable repair-from-corruption tests until other things are improved well enough to make it useful	2009-02-11 15:01:59 -07:00
Zooko O'Whielacronx	079c1851a9	trivial: whitespace	2009-02-11 11:19:35 -07:00
Brian Warner	7d907f35aa	storage: make add-lease work, change default ownernum=1 since 0 is reserved to mean 'no lease here'	2009-02-10 23:39:38 -07:00
Brian Warner	13a3ef5ec1	#620: storage: allow mutable shares to be deleted, with a writev where new_length=0	2009-02-10 23:37:56 -07:00
Brian Warner	6a5f28f47d	test_cli.Backup: insert some stalls to make sure two successive backups get distinct timestamps, avoiding intermittent failures	2009-02-10 20:37:09 -07:00
Brian Warner	f7a263eb0b	#619: make 'tahoe backup' complain and refuse to run if sqlite is unavailable and --no-backupdb is not passed	2009-02-10 18:49:10 -07:00
Brian Warner	3b1c8ceb86	tests: increase the default timeout for SystemTestMixin -based tests to 300 seconds, since our slower buildslaves sometimes take longer than the default 120s	2009-02-10 14:44:12 -07:00
Zooko O'Whielacronx	ef1bfdd2bf	immutable: repairer: add a simple test to exercise the "leftover" code path, fix the bug (and rename the variable "leftover" to "extra")	2009-02-10 12:12:45 -07:00
Zooko O'Whielacronx	75e4e67ed7	immutable: tighten preconditions -- you can write empty strings or read zero bytes, and add the first simple unit test of DownUpConnector	2009-02-10 00:56:47 -07:00
Zooko O'Whielacronx	d436c6235d	immutable: tests: the real WRITE_LEEWAY is 35 (it was a mistake to move it from 10 to 35 earlier -- I had seen a failure in which it took 35 times as many writes as I thought were optimal, but I misread and thought it took only 20 times as many)	2009-02-09 23:53:48 -07:00
Zooko O'Whielacronx	c59940852b	immutable: defensive programming: assert that the encrypted readable gave you no more than the number of bytes you asked for ... (There is a bug in the current DownUpConnector which can cause it to give more bytes than you asked for on one request, and then less on the next, effectively shifting some of the bytes to an earlier request, but I think this bug never gets triggered in practice.)	2009-02-09 23:46:05 -07:00
Zooko O'Whielacronx	65c12b24b8	immutable: tests: sigh, raise, again the limit of how many extra writes you can do and still pass this test ... Obviously requiring the code under test to perform within some limit isn't very meaningful if we raise the limit whenever the test goes outside of it. But I still don't want to remove the test code which measures how many writes (and, elsewhere, how many reads) a client does in order to fulfill these duties. Let this number -- now 20 -- stand as an approximation of the inefficiency of our code divided by my mental model of how many operations are actually optimal for these duties.	2009-02-09 20:09:31 -07:00
Zooko O'Whielacronx	5738d94ccd	immutable: tests: assert that verifier gives a clean bill of health after corruption and repair (the previous patch mistakenly did this only after deletion and repair), and also test whether deleting seven other shares and then downloading works. Also count the number of shares stored in the local filesystem.	2009-02-09 20:08:41 -07:00
Zooko O'Whielacronx	bbf48f4a88	immutable: test: add a test after attempting to repair from corruption: does a full verify run give the file a clean bill of health? If not, the you haven't successfully repaired it. ... This will make the repairer tests more consistent -- less accidentally passing due to getting lucky.	2009-02-09 19:01:49 -07:00
Zooko O'Whielacronx	4091f73f34	immutable: tests: put shares back to their pristine condition in between each test of corrupting-and-repairing them ... This is important, because if the repairer doesn't completely repair all kinds of corruption (as the current one doesn't), then the successive tests get messed up by assuming that the shares were uncorrupted when the test first set about to corrupt them.	2009-02-09 18:29:56 -07:00
Brian Warner	a9a3b509df	upload: add a think-of-the-compatibility note to UploadResults	2009-02-09 14:50:04 -07:00
Brian Warner	a5ab6c060d	helper #609: uploading client should ignore old helper's UploadResults, which were in a different format	2009-02-09 14:45:43 -07:00
Brian Warner	b197c5bb8c	test_runner: skip all spawnProcess-using tests on cygwin, since spawnProcess just hangs forever	2009-02-09 02:34:00 -07:00
Brian Warner	19902a0f16	test_runner.py: revert the CreateNode section to using runner() inline, rather than spawning a process, to get more precise coverage	2009-02-09 02:26:17 -07:00
Brian Warner	0b29aae7cf	storage #596: announce 'tolerates-immutable-read-overrun' to the version announcement, to indicate that a read() on an immutable share where offset+length is beyond the end of the file will return a truncated string instead of raising an exception	2009-02-08 19:56:02 -07:00
Brian Warner	89683a3906	test_upload: add test of maximum-immutable-share-size, to complete the last item of #538	2009-02-08 19:41:27 -07:00
Brian Warner	d8e00c37b5	test_cli.Backup: capture stderr when sqlite is unavailable	2009-02-07 15:14:40 -07:00
Brian Warner	213ef3d64b	test_repairer: wrap comments to 80cols, my laptop does not have a wide screen. No functional changes.	2009-02-07 14:06:26 -07:00
Brian Warner	38ee95fec4	immutable/checker: wrap comments to 80cols, my laptop does not have a wide screen. No functional changes.	2009-02-07 14:04:39 -07:00
Brian Warner	50886ea494	test/common.py: in share-layout-reading code, use '>L' consistently, since '>l' doesn't specify the signedness and the windows tests appear to be failing with an endianness-like problem (version==0x01000000). Also use binary mode when editing sharefiles	2009-02-07 13:38:17 -07:00
Brian Warner	1fd8f023ab	test_cli: increase timeout on test_backup, since our dapper buildslave is really slow	2009-02-06 02:17:53 -07:00
Brian Warner	5bc0ef7ffe	backupdb.py: catch OperationalError on duplicate-insert too, since pysqlite2 on dapper raises it instead of IntegrityError	2009-02-06 01:34:01 -07:00
Brian Warner	9933aa0d49	test_backupdb.py: reset the check-timers after one step, otherwise a slow host can false-fail	2009-02-06 01:33:05 -07:00
Brian Warner	814fec5495	test_cli.backup: oops, fix test to work even when sqlite is unavailable	2009-02-05 22:10:42 -07:00
Brian Warner	ca32db5b39	#598: add cli+backupdb tests, improve user display, update docs, move docs out of proposed/	2009-02-05 22:07:01 -07:00
Brian Warner	add0e1f2f8	#598: add backupdb to 'tahoe backup' command, enable it by default	2009-02-05 19:56:40 -07:00
Brian Warner	4be2e11d11	add sqlite-based backupdb, for #598 and others (including 'tahoe cp'). Not enabled yet.	2009-02-05 18:17:56 -07:00
Zooko O'Whielacronx	9ba5f9347d	setup: add a case to execute "python .../twistd.py" if "twistd" is not found	2009-02-04 18:06:20 -07:00
Brian Warner	4e572a5f24	webapi: add verifycap (spelled 'verify_url') to the t=json output on files and directories. Closes #559.	2009-02-03 19:22:48 -07:00
Brian Warner	0716fbcd16	CLI: tahoe_check: stop escaping the JSON output when using --raw	2009-02-02 21:12:32 -07:00
Brian Warner	cfce8b5eab	#598: first cut of 'tahoe backup' command: no backupdb, but yes shared-unchanged-directories and Archives/TIMESTAMP and Latest/	2009-02-02 21:09:02 -07:00
Brian Warner	91dacc3dbc	CLI: move node-url -should-end-in-slash code into a common place, so other tools can rely upon it later	2009-02-02 21:08:56 -07:00
Brian Warner	a7de479767	tahoe_ls: CLI command should return rc=0, not None	2009-02-02 21:07:20 -07:00
Zooko O'Whielacronx	e08e8d3f41	storage: disable test_large_share again: my linux laptop has less than 4 GiB free	2009-01-30 22:16:49 -07:00
Brian Warner	529a8e9fdb	web/directory: add a link from readwrite directories to a read-only version, and fix the 'SI=xxx' header to actually use the storage index, not the writekey	2009-01-30 19:32:05 -07:00
Brian Warner	b77a017872	uri: add abbrev_si() method, which returns the abbreviated storage index	2009-01-30 19:31:10 -07:00
Zooko O'Whielacronx	f0682cf43e	setup: more verbose assertion failure in test_runner	2009-01-29 10:49:06 -07:00
Zooko O'Whielacronx	204629be43	storage: enable the test of a share larger than 2 GiB; this will probably be too expensive on Mac OS X, but I think it won't be on all the other platforms ; I will probably set it to SkipTest if the sys.platform is Mac after seeing the results of this buildbot run	2009-01-28 16:33:12 -07:00
Brian Warner	1427db1c88	CLI: fix examples in tahoe put --help	2009-01-27 15:39:09 -07:00
Zooko O'Whielacronx	8d7fe2adb3	trivial: remove unused imports noticed by pyflakes	2009-01-27 15:11:48 -07:00
Zooko O'Whielacronx	64d12a9402	setup: fix test_runner to invoke bin/tahoe.exe instead of bin/tahoe if on Windows (this is what happens when a user invokes bin/tahoe on Windows)	2009-01-27 14:37:17 -07:00
Zooko O'Whielacronx	1ae08acc8f	setup: fix test_runner to assert that lines which are output to stderr must end with a punctuation mark (apparently re.search("x$", "x\r\n") does not match. :-()	2009-01-27 14:35:05 -07:00
Zooko O'Whielacronx	3cd4722c9b	setup: fix test_runner.RunNode.test_baddir -- it was left incomplete and broken by a recent patch	2009-01-27 14:32:45 -07:00
Zooko O'Whielacronx	688bb7a453	setup: go ahead and check for noise in test_client_no_noise	2009-01-26 17:46:16 -07:00
Zooko O'Whielacronx	8d2a3e78ca	setup: add a test for a warning message from importing nevow, marked as TODO, comment-out the assertion of no-noise inside other test_runner tests	2009-01-26 17:30:46 -07:00
Brian Warner	9d6534d78b	make streaming-manifest stop doing work after the HTTP connection is dropped	2009-01-23 19:39:08 -07:00
Zooko O'Whielacronx	3e4450cf3f	setup: find a "bin/tahoe" executable to test based on allmydata.__file__ instead of based on the CWD ... This means that the tests still work if you are executing them from a CWD other than the src dir -- *if* the "bin/tahoe" is found at os.path.dirname(os.path.dirname(os.path.dirname(allmydata.__file__))). If no file is found at that location, then just skip the tests of executing the "tahoe" executable, because we don't want to accidentally run those tests against an executable from a different version of tahoe.	2009-01-23 18:34:37 -07:00
Brian Warner	1de4110b35	#509: remove non-streaming 'tahoe manifest' CLI form	2009-01-23 17:00:02 -07:00
Brian Warner	6a04103cc2	#509 CLI: add 'tahoe manifest --stream'	2009-01-23 16:33:21 -07:00
Brian Warner	5a0e3f3a40	#509: test_system.py: add test for streamed-manifest	2009-01-23 16:32:47 -07:00
Brian Warner	9ade197c3b	test_system: rearrange DeepCheckWebGood to make it easier to add CLI tests	2009-01-23 16:13:06 -07:00
Brian Warner	6ba154851f	CLI: remove the '-v' alias for --version-and-path, to free up '-v' for more traditional uses (like --verbose)	2009-01-23 15:21:50 -07:00
Brian Warner	26260374e9	#590: add webish t=stream-manifest	2009-01-22 22:01:36 -07:00
Brian Warner	72adeccf2d	dirnode: add get_repair_cap()	2009-01-22 21:44:49 -07:00
Brian Warner	94ab90273d	dirnode.deep_traverse: fix docstring to describe the correct return value	2009-01-22 21:39:50 -07:00
Brian Warner	d8b3505cf5	filenode: add get_repair_cap(), which uses the read-write filecap for immutable files, and the verifycap for immutable files	2009-01-22 21:38:36 -07:00
Zooko O'Whielacronx	b77c89ac60	setup: add test that the tests are testing the right source code ... This is a test of #145, and I think that now the code passes this test.	2009-01-22 15:52:40 -07:00
Zooko O'Whielacronx	967102e1b9	trivial: removed unused imports noticed by pyflakes	2009-01-22 15:52:13 -07:00
Zooko O'Whielacronx	210da5cd9a	setup: change test_runner to invoke "bin/tahoe" in a subprocess instead of executing runner.runner() ... This is necessary because loading allmydata code now depends on PYTHONPATH manipulation which is done in the "bin/tahoe" script. Unfortunately it makes test_runner slower since it launches and waits for many subprocesses.	2009-01-22 15:38:18 -07:00