2726 Commits

Author	SHA1	Message	Date
Brian Warner	e357b8a679	test/no_network: do startService on the storage servers, make it easier to customize the storage servers	2009-02-19 20:22:54 -07:00
Brian Warner	ef4ff21ae7	crawler: modify API to support upcoming bucket-counting crawler	2009-02-19 19:31:42 -07:00
Brian Warner	9bc08158c6	test_backupdb: make the not-a-database file larger, since the older sqlite-2.3.2 on OS-X is easily fooled	2009-02-19 18:04:09 -07:00
Brian Warner	26fbe9df95	web/reliability: add parameter descriptions, adapted from a patch from Terrell Russell.	2009-02-19 16:29:18 -07:00
Brian Warner	2f751332e8	test_crawler: hush pyflakes	2009-02-19 14:23:40 -07:00
Brian Warner	ff9811211d	test_crawler: disable the percentage-of-cpu-used test, since it is too unreliable on our slow buildslaves. But leave the code in place for developers to run by hand.	2009-02-19 14:16:54 -07:00
Brian Warner	790a10d1b2	reliability.py: fix the numpy conversion, it was completely broken. Thanks to Terrell Russell for the help.	2009-02-19 13:55:15 -07:00
Brian Warner	268cd538e2	reliability: switch to NumPy, since Numeric is deprecated	2009-02-19 01:44:35 -07:00
Brian Warner	f10aa59e5b	test_crawler: don't require >=1 cycle on cygwin	2009-02-19 00:58:18 -07:00
Brian Warner	dfd72c6483	crawler: use fileutil.move_info_place in preference to our own version	2009-02-18 23:13:42 -07:00
Brian Warner	89f041ac83	fileutil: add move_into_place(), to perform the standard unix trick of atomically replacing a file, with a fallback for windows	2009-02-18 23:13:10 -07:00
Brian Warner	b949ea4f32	crawler: fix problems on windows and our slow cygwin slave	2009-02-18 22:24:31 -07:00
Brian Warner	193889f793	#633: first version of a rate-limited interruptable share-crawler	2009-02-18 21:46:33 -07:00
Brian Warner	4646451de6	change StorageServer to take nodeid in the constructor, instead of assigning it later, since it's cleaner and because the original problem (Tubs not being ready until later) went away	2009-02-18 16:23:01 -07:00
Brian Warner	b398a69f14	test_system: split off checker tests to test_deepcheck.py, this file is too big	2009-02-18 15:42:34 -07:00
Brian Warner	ef53da2b12	break storage.py into smaller pieces in storage/*.py . No behavioral changes.	2009-02-18 14:46:55 -07:00
Brian Warner	a0c5f92cbd	immutable/layout: minor change to repr name	2009-02-18 14:46:48 -07:00
Brian Warner	2346d8621d	interfaces.py: allow add/renew/cancel-lease to return Any, so that 1.3.1 clients (the first to use these calls) can tolerate future storage servers which might return something other than None	2009-02-18 13:29:03 -07:00
Brian Warner	bce4a5385b	add --add-lease to 'tahoe check', 'tahoe deep-check', and webapi.	2009-02-17 19:32:43 -07:00
Brian Warner	e9563ebc02	change RIStorageServer.remote_add_lease to exit silently in case of no-such-bucket, instead of raising IndexError, because that makes the upcoming --add-lease feature faster and less noisy	2009-02-17 19:30:53 -07:00
Brian Warner	fde2289e7b	CLI #590: convert 'tahoe deep-check' to streaming form, improve display, add tests	2009-02-17 17:15:11 -07:00
Brian Warner	0e78b2587c	interfaces.py: document behavior of add_lease/renew_lease/cancel_lease, before I change it	2009-02-17 13:48:09 -07:00
Brian Warner	d52eadfc58	test_backupdb: improve error messages if the test fails	2009-02-17 11:08:38 -07:00
Brian Warner	476a5c8fac	webapi #590: add streaming deep-check. Still need a CLI tool to use it.	2009-02-16 23:35:53 -07:00
Brian Warner	8579e25059	test_web.Grid: change the CHECK() function to make it easier to test t= values with hyphens in them	2009-02-16 23:00:34 -07:00
Brian Warner	522641bb32	test_web: improve checker-results coverage with a no-network -based test, enhance no-network harness to assist, fix some bugs in web/check_results.py that were exposed	2009-02-16 22:12:42 -07:00
Brian Warner	059f24adca	web: fix handling of reliability page when Numeric is not available	2009-02-16 19:56:58 -07:00
Brian Warner	f2f8ede11f	test/no_network: update comments with setup timing: no_network takes 50ms, SystemTestMixin takes 2s (on my laptop)	2009-02-16 18:06:43 -07:00
Brian Warner	040cb39613	test_upload: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary	2009-02-16 17:44:57 -07:00
Brian Warner	911abcc34b	test_download: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary	2009-02-16 17:36:58 -07:00
Brian Warner	e9427ea777	test_dirnode.py: convert Deleter to new no-network gridtest	2009-02-16 17:23:48 -07:00
Brian Warner	267d91f797	test_cli.py: modify to use the new 'no-network' gridtest instead of SystemTestMixin, which speeds it up from 73s to 43s on my system	2009-02-16 17:20:05 -07:00
Brian Warner	191fd8556a	tests: fix no_network framework to work with upload/download and checker	2009-02-16 17:19:47 -07:00
Brian Warner	0409bc5628	client.py: improve docstring	2009-02-16 17:15:32 -07:00
Brian Warner	770f976a85	test_cli: add test coverage for help strings	2009-02-16 15:08:33 -07:00
Brian Warner	ebe0f2ac08	test/no_network: new test harness, like system-test but doesn't use the network so it's faster	2009-02-16 14:58:44 -07:00
Brian Warner	8ff76c6269	interfaces.py: minor docstring edit	2009-02-16 14:58:16 -07:00
Brian Warner	df90dd8e73	test_cli: improve test coverage slightly	2009-02-15 21:04:51 -07:00
Brian Warner	3235b9630b	test_util: get almost full test coverage of dictutil, starting with the original pyutil tests as a base. The remaining three uncovered lines involve funny cases of ValueOrderedDict that I can't figure out how to get at	2009-02-15 20:32:10 -07:00
Brian Warner	cc27b218f5	provisioning/reliability: add tests, hush pyflakes, remove dead code, fix web links	2009-02-15 16:24:51 -07:00
Brian Warner	e1380b132b	util/statistics: add tests, fix mean_repair_cost	2009-02-15 16:23:26 -07:00
Brian Warner	812f23585d	test_repairer: hush pyflakes	2009-02-15 16:23:10 -07:00
Brian Warner	e2efd911a3	build a 'reliability' web page, with a simulation of file decay and repair over time	2009-02-13 17:42:34 -07:00
Shawn Willden	51ab76875b	More lossmodel work, on repair.	2009-01-15 20:56:48 -07:00
Shawn Willden	21832280da	Statistics module ... Added a statistics module for calculating various facets of share survival statistics.	2009-01-13 20:12:35 -07:00
Brian Warner	60b06b8d32	test_dirnode: #625 run deep-check on a readonly dirnode too	2009-02-13 14:53:37 -07:00
Brian Warner	4aee51ff6f	figleaf_htmlizer: fix order of summary counters	2009-02-13 09:57:53 -07:00
Brian Warner	aee319e236	figleaf_htmlizer: oops, re-ignore files that aren't under root, like code in auto-built eggs	2009-02-13 00:00:22 -07:00
Zooko O'Whielacronx	14d23d99a5	setup: make the "full version string" be "allmydata-tahoe/1.3.0" instead of "allmydata-tahoe-1.3.0" and the UserAgent string of the cli be "allmydata-tahoe/1.3.0 (tahoe-client)" ... This is webbish. Thanks to kpreid for suggesting it.	2009-02-12 22:37:38 -07:00
Brian Warner	30724593fc	figleaf_htmlizer: emit stats to stdout, so buildbot can see it	2009-02-12 15:10:20 -07:00
Brian Warner	781c271b82	figleaf_htmlizer: render changes in coverage relative to a previous test run using --old-coverage	2009-02-12 15:04:12 -07:00
Zooko O'Whielacronx	d7dbd6675e	immutable repairer: fix DownUpConnector so that it satisfies short reads the were requested after the last write and before the close ... This is probably the cause of the very rare "loss of progress" bug. This is tested by unit tests. A recent patch changed this to errback instead of losing progress, and now this patch is changing it again to return a short read instead of errbacking. Returning a short read is what the uploader (in encode.py) is expecting, when it is reading the last block of the ciphertext, which might be shorter than the other blocks.	2009-02-12 17:04:47 -07:00
Zooko O'Whielacronx	bdb992467c	immutable repairer: add an assertion that a certain value in this tricky function is always what I think it is	2009-02-12 16:31:32 -07:00
Zooko O'Whielacronx	2d45d192d1	immutable repairer: add tests of how it handles short reads	2009-02-12 16:48:53 -07:00
Zooko O'Whielacronx	8303f90da0	nodeadmin: node stops itself if a hotline file hasn't been touched in 60 seconds now, instead of in 40 seconds ... A test failed on draco (MacPPC) because it took 49 seconds to get around to running the test, and the node had already stopped itself when the hotline file was 40 seconds old.	2009-02-12 15:21:31 -07:00
Zooko O'Whielacronx	76d7cc4404	immutable repairer: errback any pending readers of DownUpConnectorwhen it runs out of bytes, and test that fact	2009-02-11 20:11:29 -07:00
Brian Warner	125bf09528	figleaf_htmlizer: more rearranging, behavior should still be unchanged	2009-02-11 20:05:15 -07:00
Brian Warner	e02ecec1ad	figleaf_htmlizer: break it up into subfunctions, behavior should still be unchanged	2009-02-11 19:56:07 -07:00
Brian Warner	6bde1b0345	figleaf_htmlizer: rewrite in class form, behavior should be the same as before	2009-02-11 19:40:50 -07:00
Brian Warner	6db747b1f0	figleaf_htmlizer: rewrite with twisted.python.usage, remove logging: should behave the same as before	2009-02-11 19:16:43 -07:00
Brian Warner	f3ed579e74	figleaf_htmlizer: expand tabs, fix to 4-space indents. No functional changes.	2009-02-11 19:05:42 -07:00
Zooko O'Whielacronx	7eb260a9cf	versioning: include an "appname" in the application version string in the versioning protocol, and make that appname be controlled by setup.py ... It is currently hardcoded in setup.py to be 'allmydata-tahoe'. Ticket #556 is to make it configurable by a runtime command-line argument to setup.py: "--appname=foo", but I suddenly wondered if we really wanted that and at the same time realized that we don't need that for tahoe-1.3.0 release, so this patch just hardcodes it in setup.py. setup.py inspects a file named 'src/allmydata/_appname.py' and assert that it contains the string "__appname__ = 'allmydata-tahoe'", and creates it if it isn't already present. src/allmydata/__init__.py import _appname and reads __appname__ from it. The rest of the Python code imports allmydata and inspects "allmydata.__appname__", although actually every use it uses "allmydata.__full_version__" instead, where "allmydata.__full_version__" is created in src/allmydata/__init__.py to be: __full_version__ = __appname + '-' + str(__version__). All the code that emits an "application version string" when describing what version of a protocol it supports (introducer server, storage server, upload helper), or when describing itself in general (introducer client), usese allmydata.__full_version__. This fixes ticket #556 at least well enough for tahoe-1.3.0 release.	2009-02-11 17:18:16 -07:00
Brian Warner	23fdd4f48e	backupdb: cosmetic: capitalize the no-pysqlite instructions properly. Thanks to Terrell Russell for the catch.	2009-02-11 15:28:30 -07:00
Brian Warner	5f288842d5	test_repairer: disable repair-from-corruption tests until other things are improved well enough to make it useful	2009-02-11 15:01:59 -07:00
Zooko O'Whielacronx	079c1851a9	trivial: whitespace	2009-02-11 11:19:35 -07:00
Brian Warner	7d907f35aa	storage: make add-lease work, change default ownernum=1 since 0 is reserved to mean 'no lease here'	2009-02-10 23:39:38 -07:00
Brian Warner	13a3ef5ec1	#620: storage: allow mutable shares to be deleted, with a writev where new_length=0	2009-02-10 23:37:56 -07:00
Brian Warner	6a5f28f47d	test_cli.Backup: insert some stalls to make sure two successive backups get distinct timestamps, avoiding intermittent failures	2009-02-10 20:37:09 -07:00
Brian Warner	f7a263eb0b	#619: make 'tahoe backup' complain and refuse to run if sqlite is unavailable and --no-backupdb is not passed	2009-02-10 18:49:10 -07:00
Brian Warner	3b1c8ceb86	tests: increase the default timeout for SystemTestMixin -based tests to 300 seconds, since our slower buildslaves sometimes take longer than the default 120s	2009-02-10 14:44:12 -07:00
Zooko O'Whielacronx	ef1bfdd2bf	immutable: repairer: add a simple test to exercise the "leftover" code path, fix the bug (and rename the variable "leftover" to "extra")	2009-02-10 12:12:45 -07:00
Zooko O'Whielacronx	75e4e67ed7	immutable: tighten preconditions -- you can write empty strings or read zero bytes, and add the first simple unit test of DownUpConnector	2009-02-10 00:56:47 -07:00
Zooko O'Whielacronx	d436c6235d	immutable: tests: the real WRITE_LEEWAY is 35 (it was a mistake to move it from 10 to 35 earlier -- I had seen a failure in which it took 35 times as many writes as I thought were optimal, but I misread and thought it took only 20 times as many)	2009-02-09 23:53:48 -07:00
Zooko O'Whielacronx	c59940852b	immutable: defensive programming: assert that the encrypted readable gave you no more than the number of bytes you asked for ... (There is a bug in the current DownUpConnector which can cause it to give more bytes than you asked for on one request, and then less on the next, effectively shifting some of the bytes to an earlier request, but I think this bug never gets triggered in practice.)	2009-02-09 23:46:05 -07:00
Zooko O'Whielacronx	65c12b24b8	immutable: tests: sigh, raise, again the limit of how many extra writes you can do and still pass this test ... Obviously requiring the code under test to perform within some limit isn't very meaningful if we raise the limit whenever the test goes outside of it. But I still don't want to remove the test code which measures how many writes (and, elsewhere, how many reads) a client does in order to fulfill these duties. Let this number -- now 20 -- stand as an approximation of the inefficiency of our code divided by my mental model of how many operations are actually optimal for these duties.	2009-02-09 20:09:31 -07:00
Zooko O'Whielacronx	5738d94ccd	immutable: tests: assert that verifier gives a clean bill of health after corruption and repair (the previous patch mistakenly did this only after deletion and repair), and also test whether deleting seven other shares and then downloading works. Also count the number of shares stored in the local filesystem.	2009-02-09 20:08:41 -07:00
Zooko O'Whielacronx	bbf48f4a88	immutable: test: add a test after attempting to repair from corruption: does a full verify run give the file a clean bill of health? If not, the you haven't successfully repaired it. ... This will make the repairer tests more consistent -- less accidentally passing due to getting lucky.	2009-02-09 19:01:49 -07:00
Zooko O'Whielacronx	4091f73f34	immutable: tests: put shares back to their pristine condition in between each test of corrupting-and-repairing them ... This is important, because if the repairer doesn't completely repair all kinds of corruption (as the current one doesn't), then the successive tests get messed up by assuming that the shares were uncorrupted when the test first set about to corrupt them.	2009-02-09 18:29:56 -07:00
Brian Warner	a9a3b509df	upload: add a think-of-the-compatibility note to UploadResults	2009-02-09 14:50:04 -07:00
Brian Warner	a5ab6c060d	helper #609: uploading client should ignore old helper's UploadResults, which were in a different format	2009-02-09 14:45:43 -07:00
Brian Warner	b197c5bb8c	test_runner: skip all spawnProcess-using tests on cygwin, since spawnProcess just hangs forever	2009-02-09 02:34:00 -07:00
Brian Warner	19902a0f16	test_runner.py: revert the CreateNode section to using runner() inline, rather than spawning a process, to get more precise coverage	2009-02-09 02:26:17 -07:00
Brian Warner	0b29aae7cf	storage #596: announce 'tolerates-immutable-read-overrun' to the version announcement, to indicate that a read() on an immutable share where offset+length is beyond the end of the file will return a truncated string instead of raising an exception	2009-02-08 19:56:02 -07:00
Brian Warner	89683a3906	test_upload: add test of maximum-immutable-share-size, to complete the last item of #538	2009-02-08 19:41:27 -07:00
Brian Warner	d8e00c37b5	test_cli.Backup: capture stderr when sqlite is unavailable	2009-02-07 15:14:40 -07:00
Brian Warner	213ef3d64b	test_repairer: wrap comments to 80cols, my laptop does not have a wide screen. No functional changes.	2009-02-07 14:06:26 -07:00
Brian Warner	38ee95fec4	immutable/checker: wrap comments to 80cols, my laptop does not have a wide screen. No functional changes.	2009-02-07 14:04:39 -07:00
Brian Warner	50886ea494	test/common.py: in share-layout-reading code, use '>L' consistently, since '>l' doesn't specify the signedness and the windows tests appear to be failing with an endianness-like problem (version==0x01000000). Also use binary mode when editing sharefiles	2009-02-07 13:38:17 -07:00
Brian Warner	1fd8f023ab	test_cli: increase timeout on test_backup, since our dapper buildslave is really slow	2009-02-06 02:17:53 -07:00
Brian Warner	5bc0ef7ffe	backupdb.py: catch OperationalError on duplicate-insert too, since pysqlite2 on dapper raises it instead of IntegrityError	2009-02-06 01:34:01 -07:00
Brian Warner	9933aa0d49	test_backupdb.py: reset the check-timers after one step, otherwise a slow host can false-fail	2009-02-06 01:33:05 -07:00
Brian Warner	814fec5495	test_cli.backup: oops, fix test to work even when sqlite is unavailable	2009-02-05 22:10:42 -07:00
Brian Warner	ca32db5b39	#598: add cli+backupdb tests, improve user display, update docs, move docs out of proposed/	2009-02-05 22:07:01 -07:00
Brian Warner	add0e1f2f8	#598: add backupdb to 'tahoe backup' command, enable it by default	2009-02-05 19:56:40 -07:00
Brian Warner	4be2e11d11	add sqlite-based backupdb, for #598 and others (including 'tahoe cp'). Not enabled yet.	2009-02-05 18:17:56 -07:00
Zooko O'Whielacronx	9ba5f9347d	setup: add a case to execute "python .../twistd.py" if "twistd" is not found	2009-02-04 18:06:20 -07:00
Brian Warner	4e572a5f24	webapi: add verifycap (spelled 'verify_url') to the t=json output on files and directories. Closes #559.	2009-02-03 19:22:48 -07:00
Brian Warner	0716fbcd16	CLI: tahoe_check: stop escaping the JSON output when using --raw	2009-02-02 21:12:32 -07:00
Brian Warner	cfce8b5eab	#598: first cut of 'tahoe backup' command: no backupdb, but yes shared-unchanged-directories and Archives/TIMESTAMP and Latest/	2009-02-02 21:09:02 -07:00
Brian Warner	91dacc3dbc	CLI: move node-url -should-end-in-slash code into a common place, so other tools can rely upon it later	2009-02-02 21:08:56 -07:00
Brian Warner	a7de479767	tahoe_ls: CLI command should return rc=0, not None	2009-02-02 21:07:20 -07:00
Zooko O'Whielacronx	e08e8d3f41	storage: disable test_large_share again: my linux laptop has less than 4 GiB free	2009-01-30 22:16:49 -07:00
Brian Warner	529a8e9fdb	web/directory: add a link from readwrite directories to a read-only version, and fix the 'SI=xxx' header to actually use the storage index, not the writekey	2009-01-30 19:32:05 -07:00
Brian Warner	b77a017872	uri: add abbrev_si() method, which returns the abbreviated storage index	2009-01-30 19:31:10 -07:00
Zooko O'Whielacronx	f0682cf43e	setup: more verbose assertion failure in test_runner	2009-01-29 10:49:06 -07:00
Zooko O'Whielacronx	204629be43	storage: enable the test of a share larger than 2 GiB; this will probably be too expensive on Mac OS X, but I think it won't be on all the other platforms ; I will probably set it to SkipTest if the sys.platform is Mac after seeing the results of this buildbot run	2009-01-28 16:33:12 -07:00
Brian Warner	1427db1c88	CLI: fix examples in tahoe put --help	2009-01-27 15:39:09 -07:00
Zooko O'Whielacronx	8d7fe2adb3	trivial: remove unused imports noticed by pyflakes	2009-01-27 15:11:48 -07:00
Zooko O'Whielacronx	64d12a9402	setup: fix test_runner to invoke bin/tahoe.exe instead of bin/tahoe if on Windows (this is what happens when a user invokes bin/tahoe on Windows)	2009-01-27 14:37:17 -07:00
Zooko O'Whielacronx	1ae08acc8f	setup: fix test_runner to assert that lines which are output to stderr must end with a punctuation mark (apparently re.search("x$", "x\r\n") does not match. :-()	2009-01-27 14:35:05 -07:00
Zooko O'Whielacronx	3cd4722c9b	setup: fix test_runner.RunNode.test_baddir -- it was left incomplete and broken by a recent patch	2009-01-27 14:32:45 -07:00
Zooko O'Whielacronx	688bb7a453	setup: go ahead and check for noise in test_client_no_noise	2009-01-26 17:46:16 -07:00
Zooko O'Whielacronx	8d2a3e78ca	setup: add a test for a warning message from importing nevow, marked as TODO, comment-out the assertion of no-noise inside other test_runner tests	2009-01-26 17:30:46 -07:00
Brian Warner	9d6534d78b	make streaming-manifest stop doing work after the HTTP connection is dropped	2009-01-23 19:39:08 -07:00
Zooko O'Whielacronx	3e4450cf3f	setup: find a "bin/tahoe" executable to test based on allmydata.__file__ instead of based on the CWD ... This means that the tests still work if you are executing them from a CWD other than the src dir -- *if* the "bin/tahoe" is found at os.path.dirname(os.path.dirname(os.path.dirname(allmydata.__file__))). If no file is found at that location, then just skip the tests of executing the "tahoe" executable, because we don't want to accidentally run those tests against an executable from a different version of tahoe.	2009-01-23 18:34:37 -07:00
Brian Warner	1de4110b35	#509: remove non-streaming 'tahoe manifest' CLI form	2009-01-23 17:00:02 -07:00
Brian Warner	6a04103cc2	#509 CLI: add 'tahoe manifest --stream'	2009-01-23 16:33:21 -07:00
Brian Warner	5a0e3f3a40	#509: test_system.py: add test for streamed-manifest	2009-01-23 16:32:47 -07:00
Brian Warner	9ade197c3b	test_system: rearrange DeepCheckWebGood to make it easier to add CLI tests	2009-01-23 16:13:06 -07:00
Brian Warner	6ba154851f	CLI: remove the '-v' alias for --version-and-path, to free up '-v' for more traditional uses (like --verbose)	2009-01-23 15:21:50 -07:00
Brian Warner	26260374e9	#590: add webish t=stream-manifest	2009-01-22 22:01:36 -07:00
Brian Warner	72adeccf2d	dirnode: add get_repair_cap()	2009-01-22 21:44:49 -07:00
Brian Warner	94ab90273d	dirnode.deep_traverse: fix docstring to describe the correct return value	2009-01-22 21:39:50 -07:00
Brian Warner	d8b3505cf5	filenode: add get_repair_cap(), which uses the read-write filecap for immutable files, and the verifycap for immutable files	2009-01-22 21:38:36 -07:00
Zooko O'Whielacronx	b77c89ac60	setup: add test that the tests are testing the right source code ... This is a test of #145, and I think that now the code passes this test.	2009-01-22 15:52:40 -07:00
Zooko O'Whielacronx	967102e1b9	trivial: removed unused imports noticed by pyflakes	2009-01-22 15:52:13 -07:00
Zooko O'Whielacronx	210da5cd9a	setup: change test_runner to invoke "bin/tahoe" in a subprocess instead of executing runner.runner() ... This is necessary because loading allmydata code now depends on PYTHONPATH manipulation which is done in the "bin/tahoe" script. Unfortunately it makes test_runner slower since it launches and waits for many subprocesses.	2009-01-22 15:38:18 -07:00
Zooko O'Whielacronx	fafdd2ce59	setup: fix "tahoe start" to work on Windows even when a Tahoe base dir hasn't been configured in the Windows registry	2009-01-21 12:47:20 -07:00
Zooko O'Whielacronx	7d91dc5efe	rollback the feature of making "ambient upload authority" configurable ... This reverses some, but not all, of the changes that were committed in the following set of patches. rolling back: Sun Jan 18 09:54:30 MST 2009 toby.murray * add 'web.ambient_upload_authority' as a paramater to tahoe.cfg M ./src/allmydata/client.py -1 +3 M ./src/allmydata/test/common.py -7 +9 A ./src/allmydata/test/test_ambient_upload_authority.py M ./src/allmydata/web/root.py +12 M ./src/allmydata/webish.py -1 +4 Sun Jan 18 09:56:08 MST 2009 zooko@zooko.com * trivial: whitespace I ran emacs's "M-x whitespace-cleanup" on the files that Toby's recent patch had touched that had trailing whitespace on some lines. M ./src/allmydata/test/test_ambient_upload_authority.py -9 +8 M ./src/allmydata/web/root.py -2 +1 M ./src/allmydata/webish.py -2 +1 Mon Jan 19 14:16:19 MST 2009 zooko@zooko.com * trivial: remove unused import noticed by pyflakes M ./src/allmydata/test/test_ambient_upload_authority.py -1 Mon Jan 19 21:38:35 MST 2009 toby.murray * doc: describe web.ambient_upload_authority M ./docs/configuration.txt +14 M ./docs/frontends/webapi.txt +11 Mon Jan 19 21:38:57 MST 2009 zooko@zooko.com * doc: add Toby Murray to the CREDITS M ./CREDITS +4	2009-01-20 20:47:35 -07:00
Zooko O'Whielacronx	6bbbb375e1	setup: fix test_system to require tahoe under its package==distribution name "allmydata-tahoe" instead of its module name "allmydata"	2009-01-20 12:38:09 -07:00
Zooko O'Whielacronx	845b21a5ad	trivial: remove unused import noticed by pyflakes	2009-01-19 15:16:19 -07:00
Zooko O'Whielacronx	f4f69f4a95	setup: refactor versions-and-paths and use pkg_resources to find them ... Using pkg_resources is probably better if it works -- zope.interface doesn't have a __version__ attribute that we can query, but pkg_resources knows zope.interface's version number, for one thing. This code falls back to the old way -- looking at the __version__ attributes and __file__ attributes -- if the pkg_resources way doesn't answer. Note that this patch also changes the capitalization of "Nevow", "Twisted", and "pyOpenSSL", and the spelling of "allmydata-tahoe". These changes are not frivolous: they are reflecting the fact that we are naming Python packages (technically called Python "distributions") instead of Python modules (technically and confusingly called Python "packages") here. The package ("distribution") is named "allmydata-tahoe". The module ("package") is named "allmydata".	2009-01-19 15:04:35 -07:00
Zooko O'Whielacronx	4771e1c9d5	trivial: whitespace ... Ran "M-x whitespace-cleanup" on files that Toby's recent patch touched, even though they didn't have trailing whitespace.	2009-01-18 10:58:15 -07:00
Zooko O'Whielacronx	ce7c448bc0	trivial: whitespace ... I ran emacs's "M-x whitespace-cleanup" on the files that Toby's recent patch had touched that had trailing whitespace on some lines.	2009-01-18 10:56:08 -07:00
Zooko O'Whielacronx	9247dc70e5	trivial: whitespace	2009-01-18 10:54:58 -07:00
toby.murray	66f83c7356	add 'web.ambient_upload_authority' as a paramater to tahoe.cfg	2009-01-18 10:54:30 -07:00
cgalvan	7bfaa74ef1	add option to show version and path to the tahoe executable	2009-01-16 12:47:51 -07:00
Brian Warner	26ca53fa3d	node.py: use NODEDIR/tmp for the 'tempfile' module's temporary directory, so webapi upload tempfiles are put there instead of /tmp . You can set it to something else by setting [node]tempdir in tahoe.cfg	2009-01-14 20:00:15 -07:00
Brian Warner	e9d4334159	web/operations: undo the disable-ophandle-expiration change that inadvertently got included in zooko's recent 'rename wapi.txt to webapi.txt' patch, causing test failures	2009-01-14 19:14:59 -07:00
Zooko O'Whielacronx	f9ee60a88e	cli: tests: skip symlink test if there is no os.symlink	2009-01-14 18:10:10 -07:00
Brian Warner	2fe099a0b3	mutable: move recent operation history management code (MutableWatcher) into history.py, have History provide stats	2009-01-14 17:36:20 -07:00
Brian Warner	aa50c30aa2	download: tiny cleanup of history code	2009-01-14 16:41:51 -07:00
Brian Warner	10268a4f7f	upload: move upload history into History object	2009-01-14 16:41:06 -07:00
Brian Warner	3920e6d1e7	immutable/download.py move recent-downloads history out of Downloader and into a separate class. upload/etc will follow soon.	2009-01-14 16:14:24 -07:00
Zooko O'Whielacronx	0a2895e17a	docs: rename wapi.txt to webapi.txt ... Because Brian argues that the file contains a description of the wui as well as of the wapi, and because the name "webapi.txt" might be more obvious to the untrained eye.	2009-01-14 13:53:48 -07:00
Brian Warner	cc50e2f4aa	upload: use WriteBucketProxy_v2 when uploading a large file (with shares larger than 4GiB). This finally closes #346. I think we can now handle immutable files up to 48EiB.	2009-01-12 20:14:42 -07:00
Brian Warner	bf56e2bb51	deep-check-and-repair: improve results and their HTML representation	2009-01-12 18:56:19 -07:00
Brian Warner	8f2ec16b2c	test_repairer.py: hush pyflakes: remove duplicate/shadowed function name, by using the earlier definition (which is identical)	2009-01-12 15:45:09 -07:00
Brian Warner	fe362c0021	hush pyflakes by removing unused imports	2009-01-12 15:41:20 -07:00
Zooko O'Whielacronx	25063688b4	immutable repairer ... This implements an immutable repairer by marrying a CiphertextDownloader to a CHKUploader. It extends the IDownloadTarget interface so that the downloader can provide some metadata that the uploader requires. The processing is incremental -- it uploads the first segments before it finishes downloading the whole file. This is necessary so that you can repair large files without running out of RAM or using a temporary file on the repairer. It requires only a verifycap, not a readcap. That is: it doesn't need or use the decryption key, only the integrity check codes. There are several tests marked TODO and several instances of XXX in the source code. I intend to open tickets to document further improvements to functionality and testing, but the current version is probably good enough for Tahoe-1.3.0.	2009-01-12 11:00:22 -07:00
Zooko O'Whielacronx	e449052a17	util: dictutil: add DictOfSets.union(key, values) and DictOfSets.update(otherdictofsets)	2009-01-12 10:55:39 -07:00
Zooko O'Whielacronx	8dd3b3185f	immutable: separate tests of immutable upload/download from tests of immutable checking/repair	2009-01-10 15:07:39 -07:00
Zooko O'Whielacronx	b496eba072	trivial: minor changes to in-line comments -- mark plaintext-hash-tree as obsolete	2009-01-10 14:56:01 -07:00
Zooko O'Whielacronx	38bb3e464b	immutable: make the web display of upload results more human-friendly, like they were before my recent change to the meaning of the "sharemap"	2009-01-10 14:02:09 -07:00
Zooko O'Whielacronx	0f9c11cfde	immutable: fix edit-o in interfaces.py documentation introduced in recent patch	2009-01-10 12:54:08 -07:00
Zooko O'Whielacronx	6e3396fb88	immutable: redefine the "sharemap" member of the upload results to be a map from shnum to set of serverids ... It used to be a map from shnum to a string saying "placed this share on XYZ server". The new definition is more in keeping with the "sharemap" object that results from immutable file checking and repair, and it is more useful to the repairer, which is a consumer of immutable upload results.	2009-01-10 11:46:23 -07:00
Zooko O'Whielacronx	ef60e85ec6	naming: finish renaming "CheckerResults" to "CheckResults"	2009-01-09 18:00:52 -07:00
Brian Warner	167742c2b3	storage.py : replace 4294967295 with 2**32-1: python does constant folding, I measured this statement as taking 50ns, versus the 400ns for the call to min(), or the 9us required for the 'assert not os.path.exists' syscall	2009-01-09 19:52:22 -07:00
Brian Warner	c7cd3f38e7	storage.py: announce a maximum-immutable-share-size based upon a 'df' of the disk. Fixes #569, and this should be the last requirement for #346 (remove 12GiB filesize limit)	2009-01-09 19:37:36 -07:00
Brian Warner	45fc3819a2	test_helper: hush pyflakes by avoiding use of 'uri' as a variable, since it shadows an import of the same name	2009-01-08 20:59:41 -07:00
Brian Warner	f8de336039	immutable/checker: include a summary (with 'Healthy' or 'Not Healthy' and a count of shares) in the checker results	2009-01-08 20:01:45 -07:00
Brian Warner	7ee336b274	webapi/deep-manifest t=JSON: don't return the (large) manifest/SI/verifycap lists unless the operation has completed, to avoid the considerable CPU+memory cost of creating the JSON (for 330k dirnodes, it could take two minutes to generate 275MB of JSON). They must be paid eventually, but not on every poll	2009-01-08 19:59:32 -07:00
Brian Warner	39a089dc7e	dirnode deep-traversal: remove use of Limiter, stick with strict depth-first-traversal, to reduce memory usage during very large (300k+ dirnode) traversals	2009-01-08 19:41:16 -07:00
Zooko O'Whielacronx	ade6a4fa74	immutable: add a monitor API to CiphertextDownloader with which to tell it to stop its work	2009-01-08 14:42:15 -07:00
Zooko O'Whielacronx	157e365d2b	naming: Rename a few things which I touched or changed in the recent patch to download-without-decrypting. ... Rename "downloadable" to "target". Rename "u" to "v" in FileDownloader.__init__(). Rename "_uri" to "_verifycap" in FileDownloader. Rename "_downloadable" to "_target" in FileDownloader. Rename "FileDownloader" to "CiphertextDownloader".	2009-01-08 12:13:07 -07:00
Zooko O'Whielacronx	600196f571	immutable: refactor download to do only download-and-decode, not decryption ... FileDownloader takes a verify cap and produces ciphertext, instead of taking a read cap and producing plaintext. FileDownloader does all integrity checking including the mandatory ciphertext hash tree and the optional ciphertext flat hash, rather than expecting its target to do some of that checking. Rename immutable.download.Output to immutable.download.DecryptingOutput. An instance of DecryptingOutput can be passed to FileDownloader to use as the latter's target. Text pushed to the DecryptingOutput is decrypted and then pushed to *its* target. DecryptingOutput satisfies the IConsumer interface, and if its target also satisfies IConsumer, then it forwards and pause/unpause signals to its producer (which is the FileDownloader). This patch also changes some logging code to use the new logging mixin class. Check integrity of a segment and decrypt the segment one block-sized buffer at a time instead of copying the buffers together into one segment-sized buffer (reduces peak memory usage, I think, and is probably a tad faster/less CPU, depending on your encoding parameters). Refactor FileDownloader so that processing of segments and of tail-segment share as much code is possible. FileDownloader and FileNode take caps as instances of URI (Python objects), not as strings.	2009-01-08 11:53:49 -07:00
Zooko O'Whielacronx	9bba578776	trivial: tiny changes to test code	2009-01-08 11:20:48 -07:00
Zooko O'Whielacronx	ecabcc674c	immutable: Make more parts of download use logging mixins and know what their "parent msg id" is.	2009-01-08 11:25:30 -07:00
Zooko O'Whielacronx	2a443cd049	trivial: M-x whitespace-cleanup on src/immutable/download.py	2009-01-08 10:49:01 -07:00
Zooko O'Whielacronx	7d15928faa	immutable: ValidatedExtendedURIProxy computes and stores the tail data size as a convenience to its caller. ... The "tail data size" is how many of the bytes of the tail segment are data (as opposed to padding).	2009-01-08 10:41:39 -07:00
Zooko O'Whielacronx	2e762f39f6	immutable: define a new interface IImmutableFileURI and declare that CHKFileURI and LiteralFileURI provide it	2009-01-07 12:24:51 -07:00
Zooko O'Whielacronx	c01cfc0035	util: log: allow empty msgs (because downloader is using the "format" alternative with no "msg" argument)	2009-01-07 11:54:11 -07:00
Larry Hosken	3267984fa9	'tahoe cp -r', upon encountering a dangling symlink, would assert out. ... This was somewhat sad; the assertion didn't say what path caused the error, what went wrong. So... silently skip over things that are neither dirs nor files.	2009-01-07 23:51:14 -07:00
Zooko O'Whielacronx	83b97ee79f	immutable: fix error in validation of ciphertext hash tree and add test for that code ... pyflakes pointed out to me that I had committed some code that is untested, since it uses an undefined name. This patch exercises that code -- the validation of the ciphertext hash tree -- by corrupting some of the share files in a very specific way, and also fixes the bug.	2009-01-07 23:40:12 -07:00
Zooko O'Whielacronx	6011f4522f	immutable: do not catch arbitrary exceptions/failures from the attempt to get a crypttext hash tree -- catch only ServerFailure, IntegrityCheckReject, LayoutInvalid, ShareVersionIncompatible, and DeadReferenceError ... Once again I inserted a bug into the code, and once again it was hidden by something catching arbitrary exception/failure and assuming that it means the server failed to provide valid data.	2009-01-07 22:25:51 -07:00
Zooko O'Whielacronx	e598ca2f3f	download: make sure you really get all the crypttext hashes ... We were not making sure that we really got all the crypttext hashes during download. If a server were to return less than the complete set of crypttext hashes, then our subsequent attempt to verify the correctness of the ciphertext would fail. (And it wouldn't be obvious without very careful debugging why it had failed.) This patch makes it so that you keep trying to get ciphertext hashes until you have a full set or you run out of servers to ask.	2009-01-07 20:26:38 -07:00
Zooko O'Whielacronx	2122ba78b3	util: deferredutil: undo my recent patch to use our own implementation of gatherResults ... It seems to cause lots of failures on some builders.	2009-01-07 11:00:05 -07:00
Zooko O'Whielacronx	79535e4f5b	util: deferredutil: implement our own gatherResults instead of using Twisted's ... Because we want to maintain backwards compatibility to Twisted 2.4.0.	2009-01-07 10:32:07 -07:00
Zooko O'Whielacronx	78306cf7fa	trivial: M-x whitespace-cleanup	2009-01-07 10:25:28 -07:00
Zooko O'Whielacronx	a54eba1d80	util: deferredutil: add basic test for deferredutil.gatherResults ... Also I checked and Twisted 2.4.0 supports .subFailure and the other parts of the API that we require.	2009-01-07 08:13:42 -07:00
Zooko O'Whielacronx	d5a6eed407	trivial: fix redefinition of name "log" in imports (pyflakes)	2009-01-06 22:08:29 -07:00
Zooko O'Whielacronx	c85f75bb08	immutable: refactor uploader to do just encoding-and-uploading, not encryption ... This makes Uploader take an EncryptedUploadable object instead of an Uploadable object. I also changed it to return a verify cap instead of a tuple of the bits of data that one finds in a verify cap. This will facilitate hooking together an Uploader and a Downloader to make a Repairer. Also move offloaded.py into src/allmydata/immutable/.	2009-01-06 21:48:22 -07:00
Zooko O'Whielacronx	81add135dc	trivial: whitespace and docstring tidyups	2009-01-06 21:41:04 -07:00
Brian Warner	ebe767bccf	storage.py: explain what this large and hard-to-recognize 4294967295 number is	2009-01-06 13:57:21 -07:00
Zooko O'Whielacronx	5e6f90a015	rename "checker results" to "check results", because it is more parallel to "check-and-repair results"	2009-01-06 13:37:03 -07:00
Zooko O'Whielacronx	4ada923e2b	immutable: tests: verifier doesn't always catch corrupted share hashes ... Maybe it already got one of the corrupted hashes from a different server and it doesn't double-check that the hash from every server is correct. Or another problem. But in any case I'm marking this as TODO because an even better (more picky) verifier is less urgent than repairer.	2009-01-06 13:04:49 -07:00
Zooko O'Whielacronx	3bc4b015c8	immutable: fix the writing of share data size into share file in case the share file is used by a < v1.3.0 storage server ... Brian noticed that the constant was wrong, and in fixing that I noticed that we should be saturating instead of modding. This code would never matter unless a server downgraded or a share migrated from Tahoe >= v1.3.0 to Tahoe < v1.3.0. Even in that case, this bug would never matter unless the share size were exactly 4,294,967,296 bytes long. Brian, for good reason, wanted this to be spelled "2**32" instead of "4294967296", but I couldn't stand to see a couple of more Python bytecodes interpreted in the middle of a core, frequent operation on the server like immutable share creation.	2009-01-06 12:24:04 -07:00
Zooko O'Whielacronx	8e92dd1231	trivial: whitespace cleanup	2009-01-06 11:20:58 -07:00
Zooko O'Whielacronx	d4a5f9131b	util: base32: require str-not-unicode inputs -- effectively rolls back [3306] and [3307]	2009-01-06 10:41:22 -07:00
Zooko O'Whielacronx	c35a6ee3a2	trivial: fix a bunch of pyflakes complaints	2009-01-06 08:00:54 -07:00
Zooko O'Whielacronx	821e64203c	cli: make startstop_node wait 40 seconds instead of 20 for a process to go away after we signalled it to go away, before emitting a warning ... Because the unit tests on the VirtualZooko? buildslave failed when it took 31 seconds for a process to go away. Perhaps getting warning message after only 5 seconds instead of 40 seconds is desirable, and we should change the unit tests and set this back to 5, but I don't know exactly how to change the unit tests. Perhaps match this particular warning message about the shutdown taking a while and allow the code under test to pass if the only stderr that it emits is this warning.	2009-01-06 07:51:06 -07:00
Zooko O'Whielacronx	6a12f316a4	immutable: new checker and verifier ... New checker and verifier use the new download class. They are robust against various sorts of failures or corruption. They return detailed results explaining what they learned about your immutable files. Some grotesque sorts of corruption are not properly handled yet, and those ones are marked as TODO or commented-out in the unit tests. There is also a repairer module in this patch with the beginnings of a repairer in it. That repairer is mostly just the interface to the outside world -- the core operation of actually reconstructing the missing data blocks and uploading them is not in there yet. This patch also refactors the unit tests in test_immutable so that the handling of each kind of corruption is reported as passing or failing separately, can be separately TODO'ified, etc. The unit tests are also improved in various ways to require more of the code under test or to stop requiring unreasonable things of it. :-)	2009-01-05 18:28:18 -07:00
Zooko O'Whielacronx	4921a9f243	trivial: fix inline comment in test code	2009-01-05 17:53:42 -07:00
Zooko O'Whielacronx	206ab2b44d	immutable: handle another form of share corruption with LayoutInvalid exception instead of AssertionError	2009-01-05 17:46:45 -07:00
Zooko O'Whielacronx	c84bb795f3	trivial: remove unused import (pyflakes)	2009-01-05 17:31:20 -07:00
Zooko O'Whielacronx	67d72aadd3	immutable: skip the test of large files, because that is too hard on the host if it doesn't efficiently handle sparse files	2009-01-05 17:07:27 -07:00
Zooko O'Whielacronx	f4fab23bf6	immutable: raise a LayoutInvalid exception instead of an AssertionError if the share is corrupted so that the sharehashtree is the wrong size	2009-01-05 14:01:14 -07:00
Zooko O'Whielacronx	98b28c1d5e	immutable: stop reading past the end of the sharefile in the process of optimizing download -- Tahoe storage servers < 1.3.0 return an error if you read past the end of the share file	2009-01-05 13:40:57 -07:00
Zooko O'Whielacronx	8a840469c3	immutable: tidy up the notification of waiters for ReadBucketProxy	2009-01-05 13:35:22 -07:00
Zooko O'Whielacronx	778167c2b1	immutable: refactor downloader to be more reusable for checker/verifier/repairer (and better) ... The code for validating the share hash tree and the block hash tree has been rewritten to make sure it handles all cases, to share metadata about the file (such as the share hash tree, block hash trees, and UEB) among different share downloads, and not to require hashes to be stored on the server unnecessarily, such as the roots of the block hash trees (not needed since they are also the leaves of the share hash tree), and the root of the share hash tree (not needed since it is also included in the UEB). It also passes the latest tests including handling corrupted shares well. ValidatedReadBucketProxy takes a share_hash_tree argument to its constructor, which is a reference to a share hash tree shared by all ValidatedReadBucketProxies for that immutable file download. ValidatedReadBucketProxy requires the block_size and share_size to be provided in its constructor, and it then uses those to compute the offsets and lengths of blocks when it needs them, instead of reading those values out of the share. The user of ValidatedReadBucketProxy therefore has to have first used a ValidatedExtendedURIProxy to compute those two values from the validated contents of the URI. This is pleasingly simplifies safety analysis: the client knows which span of bytes corresponds to a given block from the validated URI data, rather than from the unvalidated data stored on the storage server. It also simplifies unit testing of verifier/repairer, because now it doesn't care about the contents of the "share size" and "block size" fields in the share. It does not relieve the need for share data v2 layout, because we still need to store and retrieve the offsets of the fields which come after the share data, therefore we still need to use share data v2 with its 8-byte fields if we want to store share data larger than about 2^32. Specify which subset of the block hashes and share hashes you need while downloading a particular share. In the future this will hopefully be used to fetch only a subset, for network efficiency, but currently all of them are fetched, regardless of which subset you specify. ReadBucketProxy hides the question of whether it has "started" or not (sent a request to the server to get metadata) from its user. Download is optimized to do as few roundtrips and as few requests as possible, hopefully speeding up download a bit.	2009-01-05 09:51:45 -07:00
Zooko O'Whielacronx	5d5e89d96d	util: add gatherResults which is a deferred-list-like thing that doesn't wrap failures in a FirstError	2009-01-04 10:52:02 -07:00
Zooko O'Whielacronx	38f293a913	immutable: fix think-o in previous patch which caused all reads to return "", and also optimize by not opening the file when the answer is going to be ""	2009-01-03 14:02:45 -07:00
Zooko O'Whielacronx	53b28c1650	immutable: when storage server reads from immutable share, don't try to read past the end of the file (Python allocates space according to the amount of data requested, so if there is corruption and that number is huge it will do a huge memory allocation)	2009-01-03 13:22:22 -07:00
Zooko O'Whielacronx	de4269a643	immutable: mark a failing download test as "todo", because I think it is revealing a limitation of the current downloader's handling of corrupted shares	2009-01-03 13:00:03 -07:00
Zooko O'Whielacronx	8f5cc24948	trivial: remove unused import (pyflakes)	2009-01-03 12:22:15 -07:00
Zooko O'Whielacronx	5954ab456d	immutable: fix test for truncated reads of URI extension block size	2009-01-03 11:44:27 -07:00
Zooko O'Whielacronx	7adf905b9f	immutable: further loosen the performance-regression test to allow up to 45 reads ... This does raise the question of if there is any point to this test, since I apparently don't know what the answer *should* be, and whenever one of the buildbots fails then I redefine success. But, I'm about to commit a bunch of patches to implement checker, verifier, and repairer as well as to refactor downloader, and I would really like to know if these patches *increase* the number of reads required even higher than it currently is.	2009-01-03 11:41:09 -07:00
Zooko O'Whielacronx	2788c80496	trivial: another place where I accidentally committed a note-to-self about the lease fields in the server-side share file	2009-01-03 11:29:41 -07:00
Zooko O'Whielacronx	54787771c3	immutable: fix detection of truncated shares to take into account the fieldsize -- either 4 or 8	2009-01-02 18:57:45 -07:00
Zooko O'Whielacronx	21e0ff97f2	immutable: raise LayoutInvalid instead of struct.error when a share is truncated ... To fix this error from the Windows buildslave: [ERROR]: allmydata.test.test_immutable.Test.test_download_from_only_3_remaining_shares Traceback (most recent call last): File "C:\Documents and Settings\buildslave\windows-native-tahoe\windows\build\src\allmydata\immutable\download.py", line 135, in _bad raise NotEnoughSharesError("ran out of peers, last error was %s" % (f,)) allmydata.interfaces.NotEnoughSharesError: ran out of peers, last error was [Failure instance: Traceback: <class 'struct.error'>: unpack requires a string argument of length 4 c:\documents and settings\buildslave\windows-native-tahoe\windows\build\support\lib\site-packages\foolscap-0.3.2-py2.5.egg\foolscap\call.py:667:_done c:\documents and settings\buildslave\windows-native-tahoe\windows\build\support\lib\site-packages\foolscap-0.3.2-py2.5.egg\foolscap\call.py:53:complete c:\Python25\lib\site-packages\twisted\internet\defer.py:239:callback c:\Python25\lib\site-packages\twisted\internet\defer.py:304:_startRunCallbacks --- <exception caught here> --- c:\Python25\lib\site-packages\twisted\internet\defer.py:317:_runCallbacks C:\Documents and Settings\buildslave\windows-native-tahoe\windows\build\src\allmydata\immutable\layout.py:374:_got_length C:\Python25\lib\struct.py:87:unpack ] ===============================================================================	2009-01-02 18:48:06 -07:00
Zooko O'Whielacronx	5a21638853	immutable: whoops, it actually takes up to 39 reads sometimes to download a corrupted file	2009-01-02 17:43:02 -07:00
Zooko O'Whielacronx	e26cec2502	immutable: add more detailed tests of download, including testing the count of how many reads different sorts of downloads take	2009-01-02 16:54:59 -07:00
Zooko O'Whielacronx	cc70c163ba	trivial: a few improvements to in-line doc and code, and renaming of test/test_immutable_checker.py to test/test_immutable.py ... That file currently tests checker and verifier and repairer, and will soon also test downloader.	2009-01-02 16:49:41 -07:00
Zooko O'Whielacronx	a52b5542e9	immutable: fix name change from BadOrMissingShareHash to BadOrMissingHash ... One of the instances of the name accidentally didn't get changed, and pyflakes noticed. The new downloader/checker/verifier/repairer unit tests would also have noticed, but those tests haven't been rolled into a patch and applied to this repo yet...	2009-01-02 13:27:09 -07:00
Zooko O'Whielacronx	c72be1c553	trivial: remove unused import -- thanks, pyflakes	2009-01-02 13:21:28 -07:00
Zooko O'Whielacronx	d8c9c3dc99	immutable: download.py: Raise the appropriate type of exception to indicate the cause of failure, e.g. BadOrMissingHash, ServerFailure, IntegrityCheckReject (which is a supertype of BadOrMissingHash). This helps users (such as verifier/repairer) catch certain classes of reasons for "why did this download not work". The tests of verifier/repairer test this code and rely on this code.	2009-01-02 12:58:58 -07:00
Zooko O'Whielacronx	fa5c1d8326	immutable: ReadBucketProxy defines classes of exception: LayoutInvalid and its two subtypes RidiculouslyLargeURIExtensionBlock and ShareVersionIncompatible. This helps users (such as verifier/repairer) catch certain classes of reasons for "why did this download not work". This code gets exercised by the verifier/repairer unit tests, which corrupt the shares on disk in order to trigger problems like these.	2009-01-02 12:15:54 -07:00
Zooko O'Whielacronx	0ee027c180	immutable: ValidatedExtendedURIProxy computes and stores block_size and share_size for the convenience of its users	2009-01-02 11:43:17 -07:00
Zooko O'Whielacronx	fe707630af	doc: remove notes to self that I accidentally included in a recent patch	2009-01-01 22:14:57 -07:00
Zooko O'Whielacronx	63efa1665b	immutable: make the test of large files more likely to work by requesting to allocate space for only one huge share, not three	2008-12-31 15:59:42 -07:00
Zooko O'Whielacronx	b0b9e8c1da	trivial: "M-x whitespace-cleanup", and also remove an unused variable	2008-12-31 15:42:33 -07:00
Zooko O'Whielacronx	6c4019ec33	immutable: storage servers accept any size shares now ... Nathan Wilcox observed that the storage server can rely on the size of the share file combined with the count of leases to unambiguously identify the location of the leases. This means that it can hold any size share data, even though the field nominally used to hold the size of the share data is only 32 bits wide. With this patch, the storage server still writes the "size of the share data" field (just in case the server gets downgraded to an earlier version which requires that field, or the share file gets moved to another server which is of an earlier vintage), but it doesn't use it. Also, with this patch, the server no longer rejects requests to write shares which are >= 2^32 bytes in size, and it no longer rejects attempts to read such shares. This fixes http://allmydata.org/trac/tahoe/ticket/346 (increase share-size field to 8 bytes, remove 12GiB filesize limit), although there remains open a question of how clients know that a given server can handle large shares (by using the new versioning scheme, probably). Note that share size is also limited by another factor -- how big of a file we can store on the local filesystem on the server. Currently allmydata.com typically uses ext3 and I think we typically have block size = 4 KiB, which means that the largest file is about 2 TiB. Also, the hard drives themselves are only 1 TB, so the largest share is definitely slightly less than 1 TB, which means (when K == 3), the largest file is less than 3 TB. This patch also refactors the creation of new sharefiles so that only a single fopen() is used. This patch also helps with the unit-testing of repairer, since formerly it was unclear what repairer should expect to find if the "share data size" field was corrupted (some corruptions would have no effect, others would cause failure to download). Now it is clear that repairer is not required to notice if this field is corrupted since it has no effect on download. :-)	2008-12-31 15:42:26 -07:00
Zooko O'Whielacronx	0687f692b0	trivial: "M-x whitespace-cleanup" on immutable/layout.py	2008-12-31 15:07:02 -07:00
Zooko O'Whielacronx	b8904e023c	trivial: remove unused import -- thanks, pyflakes	2008-12-31 15:25:56 -07:00
Zooko O'Whielacronx	c00d20361f	rrefutil: generically wrap any errback from callRemote() in a ServerFailure instance ... This facilitates client code to easily catch ServerFailures without also catching exceptions arising from client-side code. See also: http://foolscap.lothar.com/trac/ticket/105 # make it easy to distinguish server-side failures/exceptions from client-side	2008-12-31 14:28:30 -07:00
Zooko O'Whielacronx	3a47031a51	immutable: more detailed tests for checker/verifier/repairer ... There are a lot of different ways that a share could be corrupted, or that attempting to download it might fail. These tests attempt to exercise many of those ways and require the checker/verifier/repairer to handle each kind of failure well.	2008-12-31 14:18:38 -07:00
Zooko O'Whielacronx	8d83699783	cli: make startstop_node wait 20 seconds instead of 5 for a process to go away after we signalled it to go away ... Because the unit tests on the VirtualZooko buildslave failed when it took 16 seconds for a process to go away. Perhaps getting notification after only 5 seconds instead of 20 seconds is desirable, and we should change the unit tests and set this back to 5, but I don't know exactly how to change the unit tests. Perhaps match this particular warning message about the shutdown taking a while and allow the code under test to pass if the only stderr that it emits is this warning.	2008-12-30 01:20:22 -07:00
Zooko O'Whielacronx	872e4fc84d	doc: sundry amendments to docs and in-line code comments	2008-12-28 16:59:54 -07:00
Zooko O'Whielacronx	9f117dbe8f	cli: mark unicode filenames as unsupported -- see #534 for details	2008-12-24 13:28:02 -07:00
Zooko O'Whielacronx	883e51b02d	cli: undo the effects of [http://allmydata.org/trac/tahoe/changeset/20081222235453-92b7f-f841e18afb94e1fd95e6dafb799a3d876dd85c69] ... We're just going to mark unicode in the cli as unsupported for tahoe-lafs-1.3.0. Unicode filenames on the command-line do actually work for some platforms and probably only if the platform encoding is utf-8, but I'm not sure, and in any case for it to be marked as "supported" it would have to work on all platforms, be thoroughly tested, and also we would have to understand why it worked. :-)	2008-12-24 09:53:17 -07:00
Zooko O'Whielacronx	25d962c67d	test: extend timeout on the hotline file that prevents the client from stopping itself ... The 20-second timeout was apparently tripped on my Powerbook G4 "draco".	2008-12-21 21:06:29 -07:00
Zooko O'Whielacronx	36df38e58b	cli: decode all cli arguments, assuming that they are utf-8 encoded ... Also encode all args to urllib as utf-8 because urllib doesn't handle unicode objects. I'm not sure if it is appropriate to *assume* utf-8 encoding of cli args. Perhaps the Right thing to do is to detect the platform encoding. Any ideas? This patch is mostly due to François Deppierraz.	2008-12-22 17:54:53 -07:00
Zooko O'Whielacronx	ce5effbedf	util/base32: the identity trans table needn't have any contents -- we are using string.translate solely to delete known chars	2008-12-22 17:48:08 -07:00
Zooko O'Whielacronx	3a26a4bd58	util/base32: allow unicode inputs to a2b() or could_be_base32_encoded(), and encode them with utf-8 before processing them	2008-12-22 17:47:13 -07:00
Zooko O'Whielacronx	dde2376c4e	util/base32: loosen the precondition forbidding unicode and requiring str -- now it requires either unicode or str ... Hopefully this will make it so that tests pass with François Deppierraz's patch to fix the tahoe cli's handling of unicode argument.	2008-12-22 16:22:37 -07:00
Zooko O'Whielacronx	c54783f5e1	immutable: don't catch all exception when downloading, catch only DeadReferenceError and IntegrityCheckReject	2008-12-21 17:41:35 -07:00
Zooko O'Whielacronx	ad58f8b693	immutable: invent download.BadOrMissingHashError which is raised if either hashtree.BadHashError, hashtree.NotEnoughHashesError, and which is a subclass of IntegrityCheckReject	2008-12-21 17:41:30 -07:00
Zooko O'Whielacronx	f1fbd4feae	dirnode: don't check MAC on entries in dirnodes ... In an ancient version of directories, we needed a MAC on each entry. In modern times, the entire dirnode comes with a digital signature, so the MAC on each entry is redundant. With this patch, we no longer check those MACs when reading directories, but we still produce them so that older readers will accept directories that we write.	2008-12-21 17:35:18 -07:00
Zooko O'Whielacronx	8b7ce325d7	immutable, checker, and tests: improve docstrings, assertions, tests ... No functional changes, but remove unused code, improve or fix docstrings, etc.	2008-12-21 15:07:52 -07:00
Zooko O'Whielacronx	c79230c854	cli: if response code from wapi server is not 200 then stop instead of proceeding ... Also, include the data that failed to json parse in an exception raised by the json parser.	2008-12-20 07:49:18 -07:00
Zooko O'Whielacronx	ec86563326	immutable: when downloading an immutable file, use primary shares if they are available ... Primary shares require no erasure decoding so the more primary shares you have, the less CPU is used.	2008-12-20 07:14:56 -07:00
Zooko O'Whielacronx	a71a68b31e	trivial: remove unused import (thanks, pyflakes)	2008-12-19 13:46:29 -07:00
Zooko O'Whielacronx	471e1f1b9b	try to tidy up uri-as-string vs. uri-as-object ... I get confused about whether a given argument or return value is a uri-as-string or uri-as-object. This patch adds a lot of assertions that it is one or the other, and also changes CheckerResults to take objects not strings. In the future, I hope that we generally use Python objects except when importing into or exporting from the Python interpreter e.g. over the wire, the UI, or a stored file.	2008-12-19 08:39:24 -07:00
Zooko O'Whielacronx	7b285ebcb1	immutable: remove the last bits of code (only test code or unused code) which did something with plaintext hashes or plaintext hash trees	2008-12-19 08:18:07 -07:00
Zooko O'Whielacronx	d67a3fe4b1	immutable: use new logging mixins to simplify logging	2008-12-16 18:04:50 -07:00
Zooko O'Whielacronx	d511941136	immutable: refactor ReadBucketProxy a little	2008-12-16 17:53:25 -07:00
Zooko O'Whielacronx	82ee44ed5b	debug: pass empty optional arguments to ReadBucketProxy ... because those arguments are about to become non-optional (for other code than test/debug code)	2008-12-16 17:51:45 -07:00
Zooko O'Whielacronx	8e48a0eeef	uri: generalize regexp that recognizes tahoe URLs to work for any host and port	2008-12-16 17:49:30 -07:00
Zooko O'Whielacronx	2b2c2a566b	util: logging: refactor some common logging behavior into mixins	2008-12-16 17:38:07 -07:00
Zooko O'Whielacronx	5fb44a3e79	pyutil: assertutil: copy in simplified assertutil from pyutil	2008-12-16 17:37:45 -07:00
Zooko O'Whielacronx	e29e4c04d9	pyutil: assertutil: simplify handling of exception during formatting of precondition message, and reduce dependency to just the Python Standard Library's logging module	2008-12-10 07:10:57 -07:00
Zooko O'Whielacronx	1d84606d52	client: add get_servers()	2008-12-08 17:04:00 -07:00
Brian Warner	278c47b9bd	mutable publish: if we are surprised by shares that match what we would have written anyways, don't be surprised. This should fix one of the two #546 problems, in which we re-use a server and forget that we already sent them a share.	2008-12-09 22:44:49 -07:00
Zooko O'Whielacronx	db566db31a	immutable: remove unused code to produce plaintext hashes	2008-12-09 16:45:46 -07:00
Zooko O'Whielacronx	c3edae5158	finish renaming 'subshare' to 'block' in immutable/encode.py and in docs/	2008-12-09 16:33:18 -07:00
Zooko O'Whielacronx	09b7151b6f	introducer: fix bug in recent simplification caught by Brian's sharp code-reviewing eye	2008-12-08 17:16:34 -07:00
Zooko O'Whielacronx	a84581fcff	introducer: simplify get_permuted_peers() implementation and add get_peers()	2008-12-08 16:57:25 -07:00
Zooko O'Whielacronx	c456ff8591	rename "get_verifier()" to "get_verify_cap()"	2008-12-08 12:44:11 -07:00
Zooko O'Whielacronx	16b20794dd	mutable: merge renaming with test patches	2008-12-07 08:45:19 -07:00
Zooko O'Whielacronx	b58875fe43	mutable: rename mutable/node.py to mutable/filenode.py and mutable/repair.py to mutable/repairer.py ... To be more consistent with the immutable layout that I am working on.	2008-12-07 08:20:08 -07:00
Brian Warner	735b3493f1	web/directory.py: really really fix #553. Unfortunately it's tricky to simulate the behavior of a brower's relative-url handling in a unit test.	2008-12-05 23:14:12 -07:00
francois	db7ad6da12	filenode.py: Fix partial HTTP Range header handling according to RFC2616 ... Tahoe webapi was failing on HTTP request containing a partial Range header. This change allows movies players like mplayer to seek in movie files stored in tahoe. Associated tests for GET and HEAD methods are also included	2008-11-18 07:41:35 -07:00
Brian Warner	ffb5985146	mutable.modify(): after UCWE, publish even if the second invocation of the modifier didn't modify anything. For #551.	2008-12-05 22:49:23 -07:00
Brian Warner	7a0afb59a4	dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.	2008-12-05 22:08:37 -07:00
Brian Warner	fb9af2c7a0	MutableFileNode.modify: pass first_time= and servermap= to the modifier callback	2008-12-05 22:07:10 -07:00
Zooko O'Whielacronx	21d88a0cba	web: fix more info links again ... Really, *really* closes #553.	2008-12-05 15:39:39 -07:00
Zooko O'Whielacronx	dd834cf7e9	web: fix moreinfo link	2008-12-05 15:29:39 -07:00
Zooko O'Whielacronx	93cd0f58a9	web: "More Info" link describes the same file that the "file" link points to, rather than to the file under the same name in this directory ... It's a subtle but real difference. Fixes #553 -- "More Info" link should point to a file/dir, not a dir+childname .	2008-12-05 15:05:02 -07:00
Zooko O'Whielacronx	60bbc46a53	minor: fix unused imports -- thanks, pyflakes	2008-12-05 13:07:23 -07:00
Zooko O'Whielacronx	b315619d6b	download: refactor handling of URI Extension Block and crypttext hash tree, simplify things ... Refactor into a class the logic of asking each server in turn until one of them gives an answer that validates. It is called ValidatedThingObtainer. Refactor the downloading and verification of the URI Extension Block into a class named ValidatedExtendedURIProxy. The new logic of validating UEBs is minimalist: it doesn't require the UEB to contain any unncessary information, but of course it still accepts such information for backwards compatibility (so that this new download code is able to download files uploaded with old, and for that matter with current, upload code). The new logic of validating UEBs follows the practice of doing all validation up front. This practice advises one to isolate the validation of incoming data into one place, so that all of the rest of the code can assume only valid data. If any redundant information is present in the UEB+URI, the new code cross-checks and asserts that it is all fully consistent. This closes some issues where the uploader could have uploaded inconsistent redundant data, which would probably have caused the old downloader to simply reject that download after getting a Python exception, but perhaps could have caused greater harm to the old downloader. I removed the notion of selecting an erasure codec from codec.py based on the string that was passed in the UEB. Currently "crs" is the only such string that works, so "_assert(codec_name == 'crs')" is simpler and more explicit. This is also in keeping with the "validate up front" strategy -- now if someone sets a different string than "crs" in their UEB, the downloader will reject the download in the "validate this UEB" function instead of in a separate "select the codec instance" function. I removed the code to check plaintext hashes and plaintext Merkle Trees. Uploaders do not produce this information any more (since it potentially exposes confidential information about the file), and the unit tests for it were disabled. The downloader before this patch would check that plaintext hash or plaintext merkle tree if they were present, but not complain if they were absent. The new downloader in this patch complains if they are present and doesn't check them. (We might in the future re-introduce such hashes over the plaintext, but encrypt the hashes which are stored in the UEB to preserve confidentiality. This would be a double- check on the correctness of our own source code -- the current Merkle Tree over the ciphertext is already sufficient to guarantee the integrity of the download unless there is a bug in our Merkle Tree or AES implementation.) This patch increases the lines-of-code count by 8 (from 17,770 to 17,778), and reduces the uncovered-by-tests lines-of-code count by 24 (from 1408 to 1384). Those numbers would be more meaningful if we omitted src/allmydata/util/ from the test-coverage statistics.	2008-12-05 08:17:54 -07:00
Brian Warner	a0b5b5ab2b	test_web: add get_permuted_peers, to unbreak recent checker_results change	2008-12-05 02:12:10 -07:00
Brian Warner	42f8aece05	web checker_results: include a table of servers in permuted order, so you can see the places where new servers have been inserted	2008-12-05 02:03:09 -07:00
Brian Warner	e942ab141b	test_system.py: assert less about the stats we get, since shares (and thus allocate() calls) are distributed randomly	2008-12-04 17:27:04 -07:00
Brian Warner	7c4edac582	stats: don't return booleans: it violates the schema. Add a test.	2008-12-04 15:01:24 -07:00
Brian Warner	7cfc74bcc9	test_system.py: don't ask the stats-gatherer to poll: it tolerates failures, so it isn't really giving us enough test coverage. Removing the call will make it more clear that we need to improve the tests later	2008-12-04 15:00:53 -07:00
Peter Secor	a8e515a8bb	confwiz.py - removing hardcoded version number	2008-12-02 20:38:31 -07:00
Brian Warner	a2ce315074	CLI: check for pre-existing aliases in 'tahoe create-alias' and 'tahoe add-alias'	2008-12-02 20:20:22 -07:00
Brian Warner	cdbb72d5ff	test_cli: pass rc out of do_cli() too	2008-12-02 20:08:28 -07:00
Brian Warner	6958b7fa90	test_storage.py: more windows-vs-readonly-storage fixes	2008-12-02 19:41:02 -07:00
Brian Warner	fe9c3e9529	storage.py: oops, fix windows again, readonly_storage wasn't getting picked up properly	2008-12-02 19:03:17 -07:00
Brian Warner	d3783475b4	test_download.py: remove extra base32 import	2008-12-02 18:31:26 -07:00
Brian Warner	b997e6ca7c	test_download: test both mutable and immutable pre-generated shares	2008-12-02 18:30:07 -07:00
Brian Warner	2d70f7fc0b	test_download.py: added 'known-answer-tests', to make sure current code can download a file that was created by earlier code	2008-12-02 18:22:08 -07:00
Brian Warner	d295a7afb9	storage.py: unbreak readonly_storage=True on windows	2008-12-01 19:49:46 -07:00
Brian Warner	71c62f8fcb	#542 'tahoe create-key-generator': fix the .tac file this creates to be compatible with modern code, add a test	2008-12-01 17:47:21 -07:00
Brian Warner	c5222558f4	storage.py: fix minor typo in comment	2008-12-01 17:25:40 -07:00
Brian Warner	cfba882b30	storage: replace sizelimit with reserved_space, make the stats 'disk_avail' number incorporate this reservation	2008-12-01 17:24:21 -07:00
Brian Warner	39a41f1d85	util/abbreviate: add abbreviated-size parser	2008-12-01 17:24:12 -07:00
Zooko O'Whielacronx	fe6abac87b	wui/wapi: change the default port number from 8123 to 3456 to avoid conflict with TorButton ... See ticket #536 for details.	2008-11-25 17:57:37 -07:00
Brian Warner	7ea0a1316a	mutable publish: reinstate the foolscap-reference-token-bug workaround, both for the original reasons and because of an apparent new foolscap bug that's triggered by reference tokens. See #541 for details.	2008-11-25 14:27:35 -07:00
Zooko O'Whielacronx	51c7580fc8	setup: fix missing import -- thanks, pyflakes	2008-11-25 09:55:28 -07:00
Zooko O'Whielacronx	de9edde8f5	setup: correctly detect Arch Linux in platform description	2008-11-25 09:51:18 -07:00
Brian Warner	bc53c24003	dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.	2008-11-24 14:40:46 -07:00
Brian Warner	2308440fb6	control.py: use get_buckets() instead of get_version() to measure ping time, because the latter changed recently	2008-11-22 23:13:23 -07:00
Brian Warner	3e25efc010	upload: when using a Helper, insist that it provide protocols/helper/v1 . Related to #538.	2008-11-21 20:29:32 -07:00
Brian Warner	0fab511be5	upload: don't use servers which can't support the share size we need. This ought to avoid #439 problems. Some day we'll have a storage server which advertises support for a larger share size. No tests yet.	2008-11-21 20:28:12 -07:00
Brian Warner	bf06492a90	#538: fetch version and attach to the rref. Make IntroducerClient demand v1 support.	2008-11-21 20:07:27 -07:00
Brian Warner	0eb6b324a4	#538: add remote_get_version() to four main Referenceable objects: Introducer Service, Storage Server, Helper, CHK Upload Helper. Remove unused storage-server get_versions().	2008-11-21 17:43:52 -07:00
Brian Warner	4ada26101b	test_dirnode: add an explainError call	2008-11-19 16:02:12 -07:00
Brian Warner	b84c2c6541	manifest: add storage-index strings to the json results	2008-11-19 16:00:27 -07:00
Brian Warner	815e0673e6	manifest: include stats in results. webapi is unchanged.	2008-11-19 15:03:47 -07:00
Brian Warner	6f6556da22	mutable: respect the new tahoe.cfg 'shares.needed' and 'shares.total' settings	2008-11-19 14:05:01 -07:00
Brian Warner	82fe7ba360	oops, update tests to match 'tahoe stats' change	2008-11-18 20:32:59 -07:00
Brian Warner	677928a1da	cli: tahoe stats: abbreviate total sizes too	2008-11-18 20:28:16 -07:00
Brian Warner	9871f3f1b7	cli: 'tahoe stats': add abbreviated size to the histogram. Not sure this actually improves things.	2008-11-18 20:17:36 -07:00
Brian Warner	7391f63fc1	util/abbreviate: little utility to abbreviate seconds and bytes	2008-11-18 20:11:42 -07:00
Brian Warner	0cdf526b50	cli: add 'tahoe check' and 'tahoe deep-check' commands, with primitive reporting code	2008-11-18 19:12:10 -07:00
Brian Warner	9c866ada71	cli: factor out slow-http-operation to a separate module	2008-11-18 19:11:13 -07:00
Brian Warner	5c3e153d0e	cli: tahoe stats/manifest: change --verbose to --raw, since I want -v for --verify for check/deep-check/repair	2008-11-18 18:36:08 -07:00
Brian Warner	d657d22c0c	test_system: make 'where' strings more helpful, to track down test failures better	2008-11-18 18:29:50 -07:00
Brian Warner	7932fadb5e	webapi: add 'summary' string to checker results JSON	2008-11-18 18:28:26 -07:00
Brian Warner	994d97c644	webapi: introducer stats: add 'announcement_distinct_hosts' to the t=json form, to show how many distinct hosts are providing e.g. storage services	2008-11-18 15:30:15 -07:00
Brian Warner	918970b63a	'tahoe create-key-generator': fix help text	2008-11-18 01:47:58 -07:00
Brian Warner	8473a96ada	#330: convert stats-gatherer into a .tac file service, add 'tahoe create-stats-gatherer'	2008-11-18 01:46:20 -07:00
Brian Warner	cae54ab118	tahoe.cfg: add controls for k and N (and shares-of-happiness)	2008-11-18 00:29:44 -07:00
Brian Warner	ead0e4d6ca	cli: add tests for 'tahoe stats --verbose'	2008-11-17 22:11:14 -07:00
Brian Warner	94de73ddc0	cli: add --verbose to 'tahoe manifest', to show the raw JSON data	2008-11-17 22:02:19 -07:00
Brian Warner	d6a67cd566	dirnode manifest/stats: process more than one LIT file per tree; we were accidentally ignoring all but the first	2008-11-14 22:50:49 -07:00
Brian Warner	fc7cd23bd1	limiter.py: fix stack blowout by inserting an eventual-send between _done and maybe_start_task. This was causing failures during a 'tahoe manifest' of a large set of directories	2008-11-14 21:11:44 -07:00
francois	1a3a464534	test_cli.py: Ensure that we can read our uploaded files back	2008-11-14 07:44:58 -07:00
francois	32b5bc6330	test_cli.py: use str objects instead of unicode ones ... This will hopefully fix failing tests with LC_ALL=C	2008-11-14 07:41:37 -07:00
Brian Warner	5a60086dbc	CLI: add 'tahoe stats', to run start-deep-stats and print the results	2008-11-13 19:43:50 -07:00
Brian Warner	9c9994300e	test_system.py: fix new 'tahoe manifest' tests to not break on windows, by providing --node-directory instead of --node-url	2008-11-13 15:27:48 -07:00
francois	5c0c5bfc81	test for bug #534, unicode filenames ... This test assure that uploading a file whose name contains unicode character doesn't prevent further uploads in the same directory.	2008-11-13 05:19:51 -07:00
francois	c1f639d230	Fix an filename encoding issue with "tahoe cp"	2008-11-11 14:08:03 -07:00
Brian Warner	0d5cc38a35	web/info.py: use 128-bit ophandles instead of 64-bit	2008-11-12 20:18:42 -07:00
Brian Warner	0d93d6244e	CLI: add 'tahoe manifest', which takes a directory and returns a list of things you can reach from it	2008-11-12 20:17:25 -07:00
Brian Warner	c9175de0f2	create_node.py: also remove now-unused import of pkg_resources	2008-11-12 18:47:16 -07:00
Brian Warner	9976bd439a	tahoe.cfg: add tub.location, to override the location hints we include in our FURL. This replaces advertised_ip_addresses, which doesn't remain useful enough to retain it. Helps with #517 (Tor).	2008-11-12 18:44:58 -07:00
Zooko O'Whielacronx	74ce1cdc00	setup: remove pkg_resources.require() from create_node.py and add it to runner.py ... Brian correctly points out that the latter is an entry point.	2008-11-12 15:25:03 -07:00
Zooko O'Whielacronx	14444dbcc0	tests: fix comment	1970-01-05 04:10:55 -07:00
Zooko O'Whielacronx	3a0dc8e4ec	tests: add tahoe_cp to the list of scripts that we don't actually have tests for yet	1970-01-05 04:00:58 -07:00
Zooko O'Whielacronx	1aed9fcfa1	setup: the .tac files created by create_node.py call pkg_resources.require() so that they can load tahoe and twisted packages which were installed with setuptools multi-version mode ... Also the create_node.py script itself uses pkg_resources.require() for the same reason.	1970-01-01 17:50:05 -07:00
Brian Warner	68f366b584	web/info: don't let an unrecoverable file break the page (show ? instead of a size)	2008-11-06 22:51:17 -07:00
Brian Warner	dfa2408157	checker: add is_recoverable() to checker results, make our stub immutable-verifier not throw an exception on unrecoverable files, add tests	2008-11-06 22:35:47 -07:00
Brian Warner	b7f7b2f43e	monitor: update interface definition: get_status() can return a Failure	2008-11-06 21:54:52 -07:00
Brian Warner	d7f1f9fd63	web/operations.py: if the operation failed, render the Failure	2008-11-06 21:53:09 -07:00
Peter Secor	ccd029bf15	undoing test change for native_client.php	2008-11-06 16:03:10 -07:00
Brian Warner	926f3958a2	client.py: oops, update FTP/SFTP config names to match current docs	2008-11-05 19:34:42 -07:00
Brian Warner	7c4856c222	SFTP/FTP: merge user/account code, merge docs	2008-11-05 19:25:58 -07:00
Brian Warner	c489a05235	ftp/sftp: move to a new frontends/ directory in preparation for factoring out password-auth component	2008-11-05 14:07:33 -07:00
Brian Warner	4e68faebb4	sftpd: minor debug-logging tweak	2008-11-05 13:45:11 -07:00
Peter Secor	510cb6d8e6	confwiz.py - trying out a new configuration site	2008-11-04 19:18:30 -07:00
Brian Warner	3fc69308eb	ftpd: include an (unused) avatar logout callback	2008-11-04 18:01:04 -07:00
Brian Warner	9f908de9e2	#531: implement an SFTP frontend. Mostly works, still lots of debug messages. Still needs tests and auth-by-pubkey in accounts.file	2008-11-04 18:00:22 -07:00
Zooko O'Whielacronx	eae5403e5a	shebang: replace "/usr/bin/python" with "/usr/bin/env python"	2008-11-04 18:03:06 -07:00
Zooko O'Whielacronx	9c35ca76a4	util: copy in pyutil.fileutil.ReopenableNamedTemporaryFile	2008-11-04 17:47:15 -07:00
Brian Warner	6fa41e738b	immutable: tolerate filenode.read() with a size= that's too big, rather than hanging	2008-11-04 15:29:19 -07:00
Zooko O'Whielacronx	cef24792e7	util: copy in nummedobj from pyutil	2008-11-04 13:55:50 -07:00
Zooko O'Whielacronx	fb85a6280f	util: copy in dictutil from pyutil	2008-11-04 13:53:27 -07:00
Brian Warner	a2e195df3e	debug catalog-shares: tolerate even more errors on bad files/directories	2008-10-30 14:54:47 -07:00
Brian Warner	ba019bfd3a	#527: expire the cached files that are used to support Range: headers, every hour, when the file is unused and older than an hour	2008-10-30 13:39:09 -07:00
Brian Warner	c205a54965	util/cachedir.py: add a cache-directory manager class, which expires+deletes unused files after a while	2008-10-30 13:01:20 -07:00
Brian Warner	1419db6ac7	test_cli: try to fix windows again	2008-10-30 12:32:04 -07:00
Brian Warner	d97c34d622	debug/test_cli: fix error handling for catalog-shares, to make the test stop failing on windows	2008-10-30 12:06:51 -07:00
Brian Warner	b1db6d9ff2	web: add 'Repair' button to checker results when they indicate unhealthyness. Also add the object's uri to the CheckerResults instance.	2008-10-29 18:09:17 -07:00
Brian Warner	6a7d5b893b	create_node.py: add 'web.static = public_html' to the initial tahoe.cfg	2008-10-29 17:13:36 -07:00
Brian Warner	d3f2df00be	webapi: serve the /static URL tree from /public_html (configurable)	2008-10-29 15:34:31 -07:00
Brian Warner	a9101112a4	catalog-shares command: tolerate errors, log them to stderr, handle v2-immutable shares	2008-10-29 15:10:10 -07:00
Brian Warner	186b64b633	test_web.py: one more line of test coverage	2008-10-28 22:00:15 -07:00
Brian Warner	0350994133	test_web: improve test coverage of PUT DIRURL t=uri replace=false	2008-10-28 21:57:44 -07:00
Brian Warner	a5aebf0cbd	web: test (and fix) PUT DIRURL t=uri, which replaces a directory in-place with some other cap	2008-10-28 21:54:46 -07:00
Brian Warner	e1d9169ba3	web/directory.py: slight shuffle to improve test coverage	2008-10-28 21:54:06 -07:00
Brian Warner	248033d78c	test_client.py: improve test coverage a bit	2008-10-28 21:43:35 -07:00
Brian Warner	67ae5c1b2f	node.py: remove unused old_log() function	2008-10-28 21:35:58 -07:00
Brian Warner	ba26239879	node.py: remove support for the old BASEDIR/authorized_keys.PORT file	2008-10-28 21:34:20 -07:00
Brian Warner	b73c380cdb	move testutil into test/common_util.py, since it doesn't count as 'code under test' for our pyflakes numbers	2008-10-28 21:28:31 -07:00
Brian Warner	294e3fb682	util: move PollMixin to a separate file (pollmixin.py), so testutil can be moved into test/	2008-10-28 21:15:48 -07:00
Brian Warner	3dd9b79e3f	control.py: removed unused testutil.PollMixin	2008-10-28 21:03:59 -07:00
Brian Warner	4c2235fcef	web/filenode: oops, fix test failures, not everything has a storage index	2008-10-28 18:17:20 -07:00
Brian Warner	6e1db34817	web/filenode: add Accept-Ranges and ETag (for immutable files) headers to GET responses	2008-10-28 18:01:03 -07:00
Brian Warner	b1ca238176	#527: respond to GETs with early ranges quickly, without waiting for the whole file to download. Fixes the alacrity problems with the earlier code. Still needs cache expiration.	2008-10-28 17:56:18 -07:00
Brian Warner	37e3d8e47c	#527: support HTTP 'Range:' requests, using a cachefile. Adds filenode.read(consumer, offset, size) method. Still needs: cache expiration, reduced alacrity.	2008-10-28 13:41:04 -07:00
Brian Warner	2d4e895096	iputil.py: avoid a DNS lookup at startup (which may timeout tests when run on a partially-offline host) by using 198.41.0.4 instead of A.ROOT-SERVERS.NET	2008-10-28 13:36:46 -07:00
Brian Warner	914655c52b	interfaces.py: promote immutable.encode.NotEnoughSharesError.. it isn't just for immutable files any more	2008-10-27 13:34:49 -07:00
Brian Warner	4b48d94c52	interfaces.IMutableFileNode.download_best_version(): fix return value	2008-10-27 13:20:46 -07:00
Brian Warner	fca158e83a	dirnode lookup: use distinct NoSuchChildError instead of the generic KeyError when a child can't be found	2008-10-27 13:15:25 -07:00
Brian Warner	5fd6633289	storage: don't use colons in the corruption-advisory filename, since windows can't tolerate them	2008-10-25 19:46:33 -07:00
Brian Warner	9f21f7cf65	mutable: call remove_advise_corrupt_share when we see share corruption in mapupdate/download/check, tolerate servers that do not implement it	2008-10-24 13:21:28 -07:00
Brian Warner	db37c14ab7	storage: add remote_advise_corrupt_share, for clients to tell storage servers about share corruption that they've discovered. The server logs the report.	2008-10-24 11:52:48 -07:00
Brian Warner	6dbef907ac	mutable/servermap.py: fix needs_merge(), it was incorrectly claiming that mixed shares with distinct seqnums needed a merge, causing repair(force=False) to fail	2008-10-23 21:00:24 -07:00
Brian Warner	0383ec5a49	test_web.test_POST_DIRURL_deepcheck: confirm that /operations/HANDLE/ works with or without the slash	2008-10-23 19:17:59 -07:00
Brian Warner	750ab8fa59	web/checker_results.py: remove dead code	2008-10-23 17:17:17 -07:00
Brian Warner	cadb045288	test_web: more test coverage	2008-10-23 17:11:18 -07:00
Brian Warner	a1cfac89f9	webapi: fix t=rename from==to, it used to delete the file	2008-10-23 16:32:36 -07:00
Brian Warner	d4b4cd8ab8	test_system: update test to match web checker results	2008-10-23 16:32:02 -07:00
Brian Warner	066aedcfa1	webapi deep-check: show the root as <root>, rather than an empty path string	2008-10-23 16:03:59 -07:00
Brian Warner	54c0ffd214	mutable/checker: announce the mapupdate op on the 'recent uploads+downloads' page	2008-10-23 16:03:19 -07:00
Brian Warner	7338440d86	scripts/create_node.py: remove empty-string defaults for --introducer= and --nickname=	2008-10-23 16:02:35 -07:00
Brian Warner	c455d52453	deep-check: add webapi links to detailed per-file/dir results	2008-10-23 16:00:31 -07:00
Brian Warner	819d6b3d03	interface.py: fix typo	2008-10-23 15:59:36 -07:00
Brian Warner	76b7b9d835	webapi: make the /operations/ 't=status' qualifier optional, remove it from examples	2008-10-23 15:56:58 -07:00
Brian Warner	4025f29b8a	web/info.py: fix 'Check This Object' link, for files it was checking the parent directory by mistake	2008-10-22 10:10:56 -07:00
Brian Warner	0129e9931f	#514: add meta-refresh=60 tag to t=status page for incomplete operations	2008-10-22 09:48:42 -07:00
Brian Warner	2ee6356c89	test_dirnode.py: oops, missed a Monitor(), unbreak tests	2008-10-22 01:50:54 -07:00
Brian Warner	1566a9474c	immutable/filenode.py: add TODO note about the #514 monitor to check(), rather than going through the checker/verifier code and adding it, since Zooko is currently working on that code	2008-10-22 01:42:37 -07:00
Brian Warner	977c6ac510	more #514: pass a Monitor to all checker operations, make mutable-checker honor the cancel flag	2008-10-22 01:38:18 -07:00
Brian Warner	8178b10ef1	dirnode.py: check for cancel during deep-traverse operations, and don't initiate any new ones if we've been cancelled. Gets us closer to #514.	2008-10-22 00:55:52 -07:00
Brian Warner	34ab4e3de3	more #514 log-webop status/cancel: add handle-expiration, test coverage	2008-10-21 22:13:54 -07:00
Brian Warner	0f7c1fd6fd	#514: improve test coverage	2008-10-21 17:52:56 -07:00
Brian Warner	ad3d9207a9	Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514.	2008-10-21 17:03:07 -07:00
Brian Warner	0832334c0d	storage.py: assert that immutable share size will fit in the 4-byte v1 container (see #346). The struct module in py2.4 raises an error on overflow, but py2.5 merely emits a warning	2008-10-20 10:22:08 -07:00
robk-tahoe	4785299980	gui/macapp: minor bugfixes ... though it seemed to work before the 'fstype' passed to fuse of 'allmydata' was today throwing errors that len(fstype) must be at most 7. fixed a typo in changes to 'mount_filesystem()' args bumped the delay between mounting a filesystem and 'open'ing it in Finder to 4s, as it seems to take a little longer to mount now the client and server fuse processes need to coordinate.	2008-10-16 09:30:52 -07:00
robk-tahoe	183dd587d5	gui/macapp: slew of code cleanup; unmount filesystems on quit ... a handful of code cleanup, renaming and refactoring. basically consolidating 'application logic' (mount/unmount fs) into the 'MacGuiApp' class (the wx.App) and cleaning up various scoping things around that. renamed all references to 'app' to refer more clearly to the 'AppContainer' or to the guiapp. globally renamed basedir -> nodedir also made the guiapp keep a note of each filesystem it mounts, and unmount them upon 'quit' so as to cleanup the user's environment before the tahoe node vanishes from out underneath the orphaned tahoe fuse processes	2008-09-25 16:32:35 -07:00
robk-tahoe	9b10f46374	gui/macapp: make submenu of aliases for 'webopen' ... this changes the 'open webroot' menu item to be a submenu listing all aliases defined in ~/.tahoe. Note that the dock menu does not support submenus, so it only offers a single 'open webroot' option for the default tahoe: alias. I had trouble with this at first and concluded that the submenus didn't work, and made it a distinct 'WebUI' menu in it's own right. on further inspection, there are still problems but they seem to be something like once the dock menu has been used, sometimes the app's main menubar menus will cease to function, and this happens regardless of whether submenus or plain simple menus are used. I have no idea what the peoblem is, but it's not submenu specific.	2008-09-25 09:39:19 -07:00
Zooko O'Whielacronx	57d9f6beb6	repairer: fix flaw in testutil.flip_one_bit() that Brian pointed out	2008-10-16 12:48:48 -07:00
Zooko O'Whielacronx	8a6d1e5da6	repairer: test all different kinds of corruption that can happen to share files on disk	2008-10-14 16:09:20 -07:00
Brian Warner	b0a00e0f33	util/time_format.py: accept space separator, add unit tests	2008-10-13 15:52:58 -07:00
Brian Warner	46657b797f	test_storage: use different filenames, poor stupid windows	2008-10-09 19:11:39 -07:00
Brian Warner	fd32d72b20	scripts/debug.py: emit the immutable-share version number, tolerate v2	2008-10-09 18:34:22 -07:00
Brian Warner	c289a9c31b	storage.py: improve some precondition() error messages	2008-10-09 18:14:25 -07:00
Brian Warner	7031a69bee	storage: introduce v2 immutable shares, with 8-byte offsets fields, to remove two of the three size limitations in #346. This code handles v2 shares but does not generate them. We'll make a release with this v2-tolerance, wait a while, then make a second release that actually generates v2 shares, to avoid compatibility problems.	2008-10-09 18:13:27 -07:00
Brian Warner	a0250e320b	debug.py: oops, add missing import for ReadBucketProxy	2008-10-09 17:29:22 -07:00
Brian Warner	288d55825c	storage: split WriteBucketProxy and ReadBucketProxy out into immutable/layout.py . No behavioral changes.	2008-10-09 17:08:00 -07:00
Zooko O'Whielacronx	86e22b8add	interfaces: loosen a few max-size constraints which would limit us to a mere 1.09 TB maximum file size ... These constraints were originally intended to protect against attacks on the storage server protocol layer which exhaust memory in the peer. However, defending against that sort of DoS is hard -- probably it isn't completely achieved -- and it costs development time to think about it, and it sometimes imposes limits on legitimate users which we don't necessarily want to impose. So, for now we forget about limiting the amount of RAM that a foolscap peer can cause you to start using.	2008-10-09 12:13:57 -07:00
Brian Warner	93725a6935	util/limiter: add a repr	2008-10-07 13:19:45 -07:00
Brian Warner	9d4749d546	dirnode.build_manifest: include node.list in the limiter, that's the most important thing to slow down	2008-10-07 13:19:29 -07:00
Brian Warner	cd236efc68	web/directory: t=manifest output=html: make the caps into clickable hrefs	2008-10-07 13:18:45 -07:00
Brian Warner	f7edbc1536	web/directory: factor out the get_root function	2008-10-07 13:17:42 -07:00
Brian Warner	53c9867c5c	web/directory.py: remove unused imports	2008-10-07 12:48:20 -07:00
Brian Warner	2edfe12b88	test_web: deep-size is more variable than I thought, so assert less	2008-10-06 22:11:47 -07:00
Brian Warner	3ffaded809	web: change t=manifest to return a list of (path,read/writecap) tuples, instead of a list of verifycaps. Add output=html,text,json.	2008-10-06 21:36:18 -07:00
Brian Warner	8e6d122ecf	web: rewrite t=deep-size in terms of deep-stats, update test to match inclusion of directory sizes	2008-10-06 21:35:39 -07:00
Brian Warner	18a1070e66	ftpd: hush pyflakes	2008-10-06 18:45:13 -07:00
Brian Warner	701e5cd1da	ftpd: make sure we're using a patched/fixed Twisted, to avoid confusion later	2008-10-06 18:14:11 -07:00
Brian Warner	05a8360177	ftp: change the twisted hack necessary for async-write-close, to one more agreeable to the twisted-dev folks, add a copy of the necessary patch to docs/ftp.txt	2008-10-06 18:06:05 -07:00
Brian Warner	70ae1bfc8e	ftpd: remove debug messages	2008-10-06 16:16:20 -07:00
Brian Warner	bf3d09d430	ftpd: add native_client.php -based HTTP authentication scheme	2008-10-06 16:15:11 -07:00
Brian Warner	daf4af6699	ftpd: add ftp.accounts checker, remove InMemoryPasswordChecker	2008-10-06 15:51:24 -07:00
Brian Warner	d90a3ed7f8	test_system: add test coverage for immutable download.ConsumerAdapter, remove debug messages	2008-10-06 15:50:37 -07:00
Brian Warner	bc237b3956	ftp server: initial implementation. Still needs unit tests, custom Twisted patches. For #512	2008-10-06 12:52:36 -07:00
Brian Warner	934dc6aa77	test_cli.py: remove unused imports	2008-10-06 17:42:04 -07:00
Brian Warner	dbc2337639	CLI: remove 'tahoe admin generate-keypair', since the pycryptopp ecdsa API is about to change incompatibly. We'll undo this once pycryptopp is updated	2008-10-06 17:23:20 -07:00
Brian Warner	09341a969a	dirnode: fix my remarkably-consistent 'metdadata' typo	2008-10-02 18:08:45 -07:00
Brian Warner	41bacca3f1	interfaces: fix minor typo	2008-10-02 17:52:49 -07:00
Brian Warner	d0bdf9a611	dirnode: add get_child_and_metadata_at_path	2008-10-02 17:52:03 -07:00
Brian Warner	9c505e49c2	stop using 'as' as an identifier: as with 'with', 'as' has become a reserved word in python 2.6	2008-10-02 17:27:49 -07:00
Brian Warner	034364c80b	scripts/admin: split up generate_keypair code so that unit tests can use it more easily	2008-10-01 16:52:38 -07:00
Brian Warner	cd26f58305	#518: replace various BASEDIR/* config files with a single BASEDIR/tahoe.cfg, with backwards-compatibility of course	2008-09-30 16:21:49 -07:00
Brian Warner	935d185696	tolerate simplejson-2.0.0 and newer, which frequently return bytestrings instead of unicode objects. Closes #523	2008-09-30 15:21:06 -07:00
Brian Warner	78d58875a8	test_node: improve coverage of advertised_ip_addresses a bit	2008-09-29 23:08:16 -07:00
Brian Warner	1e3500ae5f	testutil.PollMixin: set default timeout (to 100s), emit a more helpful error when the timeout is hit	2008-09-29 22:23:09 -07:00
Zooko O'Whielacronx	2e8b89e9e4	repair: fix test to map from storage index to directory structure properly (thanks, cygwin buildbot, for being so kloodgey that you won't accept random binary filenames and thus making me notice this bug)	2008-09-26 15:49:13 -07:00
Zooko O'Whielacronx	cd0af638f3	repairer: assert that the test code isn't accidentally allowing the repairer code which is being tested to do impossible things	2008-09-26 15:23:53 -07:00
Zooko O'Whielacronx	ebee8ee9f4	repairer: enhance the repairer tests ... Make sure the file can actually be downloaded afterward, that it used one of the deleted and then repaired shares to do so, and that it repairs from multiple deletions at once (without using more than a reasonable amount of calls to storage server allocate).	2008-09-26 10:47:19 -07:00
Brian Warner	98c8e25709	netstring: add required_trailer= argument	2008-09-26 09:57:54 -07:00
Brian Warner	27a30b90d7	test_netstring.py: move netstring tests to a separate file	2008-09-26 09:55:26 -07:00
Brian Warner	e8cf581e3f	move netstring() and split_netstring() into a separate util.netstring module	2008-09-25 21:38:24 -07:00
Zooko O'Whielacronx	39fa9018ef	repairer: remove a test that doesn't apply to the repair-from-corruption case	2008-09-25 15:09:54 -07:00
Zooko O'Whielacronx	73f92ea571	repairer: add a test that repairer fixes corrupted shares (in addition to the test that it fixes deleted shares)	2008-09-25 15:07:12 -07:00
Zooko O'Whielacronx	a3b3d2b7bc	repairer: fix swapped docstrings; thanks Brian	2008-09-25 11:24:36 -07:00
Zooko O'Whielacronx	30111811e3	trivial: remove unused imports; thanks, pyflakes	2008-09-25 11:04:22 -07:00
Zooko O'Whielacronx	a363994c4c	trivial: remove unused imports -- thanks, pyflakes	2008-09-25 10:34:53 -07:00
Zooko O'Whielacronx	1e8d37cc2d	repairer: add basic test of repairer, move tests of immutable checker/repairer from test_system to test_immutable_checker, remove obsolete test helper code from test_filenode ... Hm... "Checker" ought to be renamed to "CheckerRepairer" or "Repairer" at some point...	2008-09-25 10:16:53 -07:00
Zooko O'Whielacronx	39f305e44f	setup: remove a few minimal unit tests from test_filenode which have been obviated by much better tests in test_mutable and test_system	2008-09-25 09:15:44 -07:00
robk-tahoe	1fd085ced7	gui/macapp: rough cut of ui tweaks; configurability, auto-mount ... chatting with peter, two things the mac gui needed were the ability to mount the 'allmydata drive' automatically upon launching the app, and open the Finder to reveal it. (also a request to hide the debug 'open webroot' stuff) this (somewhat rough) patch implements all the above as default behaviour it also contains a quick configuration mechanism for the gui - rather than a preferences gui, running with a more 'tahoe' styled mechanism, the contents of a few optional files can modify the default behaviour, specifically file in ~/.tahoe/gui.conf control behaviour as follows: auto-mount (bool): if set (the default) then the mac app will, upon launch automatically mount the 'tahoe:' alias with the display name 'Allmydata' using a mountpoint of ~/.tahoe/mnt/__auto__ auto-open (bool): if set (the default) then upon mounting a file system (including the auto-mount if set) finder will be opened to the mountpoint of the filesystem, which essentially reveals the newly mounted drive in a Finder window show-webopen (bool): if set (false by default) then the 'open webroot' action will be made available in both the dock and file menus of the app daemon-timout (int): sets the daemon-timeout option passed into tahoe fuse when a filesystem is mounted. this defaults to 5 min files of type (int) much, naturally contain a parsable int representation. files of type (bool) are considered true if their (case-insensitive) contents are any of ['y', 'yes', 'true', 'on', '1'] and considered false otherwise.	2008-09-25 07:12:24 -07:00
robk-tahoe	63ba26a445	gui/macapp: improve 'about' box ... adds exactly 1 metric dollop of professionalism to the previously rather amateurish looking about box.	2008-09-25 06:54:15 -07:00
robk-tahoe	90e3f37173	manhole: be more tolerant of authorized_keys. files in .tahoe ... both peter and I independently tried to do the same thing to eliminate the authorized_keys file which was causing problems with the broken mac build (c.f. #522) namely mv authorized_keys.8223{,.bak} but the node is, ahem, let's say 'intolerant' of the trailing .bak - rather than disable the manhole as one might expect, it instead causes the node to explode on startup. this patch makes it skip over anything that doesn't pass the 'parse this trailing stuff as an int' test.	2008-09-24 20:11:49 -07:00
robk-tahoe	4d9f499de1	macapp: changes to support aliases, updated tahoefuse command line options ... the tahoefuse command line options changed to support the runtests harness, and as part of that gained support for named aliases via --alias this changes the mac app's invocation of tahoefuse to match that, and also changes the gui to present the list of defined aliases as valid mounts this replaces the previous logic which examined the ~/.tahoe/private directory looking for files ending in '.cap' - an ad-hoc alias mechanism. if a file is found matching ~/.tahoe/private/ALIASNAME.icns then that will still be passed to tahoefuse as the icon to display for that filesystem. if no such file is found, the allmydata icon will be used by default. the '-olocal' option is passed to tahoefuse. this is potentially contentious. specifically this is telling the OS that this is a 'local' filesystem, which is intended to be used to locally attached devices. however leopard (OSX 10.5) will only display non-local filesystems in the Finder's side bar if they are of fs types specifically known by Finder to be network file systems (nfs, cifs, webdav, afp) hence the -olocal flag is the only way on leopard to cause finder to display the mounted filesystem in the sidebar, but it displays as a 'device'. there is a potential (i.e. the fuse docs carry warnings) that this may cause vague and unspecified undesirable behaviour. (c.f. http://code.google.com/p/macfuse/wiki/FAQ specifically Q4.3 and Q4.1)	2008-09-24 18:01:28 -07:00
robk-tahoe	ed9873c2a9	CLI: rework webopen, and moreover its tests w.r.t. path handling ... in the recent reconciliation of webopen patches, I wound up adjusting webopen to 'pass through' the state of the trailing slash on the given argument to the resultant url passed to the browser. this change removes the requirement that arguments must be directories, and allows webopen to be used with files. it also broke the tests that assumed that webopen would always normalise the url to have a trailing slash. in fixing the tests, I realised that, IMHO, there's something deeply awry with the way tahoe handles paths; specifically in the combination of '/' being the name of the root path within an alias, but a leading slash on paths, e.g. 'alias:/path', is catagorically incorrect. i.e. 'tahoe:' == 'tahoe:/' == '/' but 'tahoe:/foo' is an invalid path, and must be 'tahoe:foo' I wound up making the internals of webopen simply spot a 'path' of '/' and smash it to '', which 'fixes' webopen to match the behaviour of tahoe's path handling elsewhere, but that special case sort of points to the weirdness. (fwiw, I personally found the fact that the leading / in a path was disallowed to be weird - I'm just used to seeing paths qualified by the leading / I guess - so in a debate about normalising path handling I'd vote to include the /)	2008-09-24 09:45:23 -07:00
robk-tahoe	2c9a854f10	CLI: reconcile webopen changes ... I think this is largely attributable to a cleanup patch I'd made which never got committed upstream somehow, but at any rate various conflicting changes to webopen had been made. This cleans up the conflicts therein, and hopefully brings 'tahoe webopen' in line with other cli commands.	2008-09-24 08:20:02 -07:00
robk-tahoe	8f28d64d77	cli: cleanup webopen command ... moved the body of webopen out of cli.py into tahoe_webopen.py made its invocation consistent with the other cli commands, most notably replacing its 'vdrive path' with the same alias parsing, allowing usage such as 'tahoe webopen private:Pictures/xti'	2008-06-18 13:19:40 -07:00
robk-tahoe	0042881736	macapp: changed to remove 'Tahoe' from .app name ... Change the build product from 'Allmydata Tahoe' to 'Allmydata' more inkeeping with the branding of the Allmydata product	2008-06-10 17:31:45 -07:00
Brian Warner	53f53b7c82	add --syslog argument to 'tahoe start' and 'tahoe restart', used to pass --syslog to twistd for non-Tahoe nodes (like cpu-watcher)	2008-09-24 18:03:02 -07:00
Brian Warner	54fce5c066	webapi: survive slashes in filenames better: make t=info and t=delete to work, and let t=rename fix the problem	2008-09-24 13:35:05 -07:00
Zooko O'Whielacronx	7339e09c72	setup: when detecting platform, ask the Python Standard Library's platform.dist() before executing lsb_release, and cache the result in global (module) variables ... This should make it sufficiently fast, while still giving a better answer on Ubuntu than platform.dist() currently does, and also falling back to lsb_release if platform.dist() says that it doesn't know.	2008-09-24 11:09:22 -07:00
Brian Warner	dd9171eb72	node.py: add BASEDIR/keepalive_timeout and BASEDIR/disconnect_timeout, to set/enable the foolscap timers, for #521	2008-09-24 10:51:12 -07:00
Zooko O'Whielacronx	ff08bab0c4	immutable: remove unused imports (thanks, pyflakes)	2008-09-23 12:26:10 -07:00
Zooko O'Whielacronx	fa302544fa	immutable: refactor immutable filenodes and comparison thereof ... * the two kinds of immutable filenode now have a common base class * they store only an instance of their URI, not both an instance and a string * they delegate comparison to that instance	2008-09-23 11:52:49 -07:00
Zooko O'Whielacronx	2ef977d85a	setup: try parsing /etc/lsb-release first, then invoking lsb_release, because the latter takes half-a-second on my workstation, which is too long ... Also because in some cases the former will work and the latter won't. This patch also tightens the regexes so it won't match random junk.	2008-09-23 10:14:31 -07:00
Zooko O'Whielacronx	37dd91238b	setup: fix a cut-and-paste error in the fallback to parsing /etc/lsb-release	2008-09-23 09:55:51 -07:00
Zooko O'Whielacronx	8a35d2ed83	setup: if executing lsb_release doesn't work, fall back to parsing /etc/lsb-release before falling back to platform.dist() ... An explanatio of why we do it this way is in the docstring.	2008-09-23 09:28:58 -07:00
Zooko O'Whielacronx	e2b1ce13f5	setup: if invoking lsb_release doesn't work (which it doesn't on our etch buildslave), then fall back to the Python Standard Library's platform.dist() function	2008-09-23 08:48:20 -07:00
Zooko O'Whielacronx	8874e27da0	setup: fix bug in recent patch to use allmydata.get_package_versions() to tell the foolscap app-version-tracking what's what	2008-09-22 17:13:47 -07:00
Zooko O'Whielacronx	992339b97e	setup: when using the foolscap "what versions are here?" feature, use allmydata.get_package_versions() instead of specifically importing allmydata, pycryptopp, and zfec	2008-09-22 17:03:51 -07:00
Zooko O'Whielacronx	5075c12725	setup: simplify the implementation of allmydata.get_package_versions() and add "platform" which is a human-oriented summary of the underlying operating system and machine	2008-09-22 16:53:54 -07:00
Brian Warner	c6999d219e	use foolscap's new app_versions API, require foolscap-0.3.1	2008-09-20 11:38:53 -07:00
Brian Warner	063f85d157	BASEDIR/nickname is now UTF-8 encoded	2008-09-20 11:37:13 -07:00
Brian Warner	0a59991ce6	various: use util.log.err instead of twisted.log.err, so we get both Incidents and trial-test-flunking	2008-09-20 10:35:45 -07:00
Brian Warner	8854d0c1b5	util.log: send log.err to Twisted too, so that Trial tests are flunked	2008-09-20 10:34:27 -07:00
Brian Warner	0799e09910	test_cli: disable generate-keypair test on OS-X, pycryptopp still has a bug	2008-09-19 12:38:55 -07:00
Brian Warner	6607fdc586	CLI: add 'tahoe admin generate-keypair' command	2008-09-18 17:11:33 -07:00
Brian Warner	99d5a8d8b9	web: add 'more info' pages for files and directories, move URI/checker-buttons/deep-size/etc off to them	2008-09-17 22:00:41 -07:00
Peter Secor	c253655f66	conf_wiz.py - updating version numbers in file, should really get these from a TAG or conf file	2008-09-16 17:45:47 -07:00
Brian Warner	51612cd1be	webish: add an extra newline to JSON output	2008-09-15 13:43:14 -07:00
Brian Warner	f00d1961d2	startstop_node.py: find twistd in our supportlib if we had to build Twisted as a setuptools dependency. This is a form of cgalvan's #505 patch, simplified because now 'setup.py trial' takes care of sys.path and PYTHONPATH	2008-09-11 19:51:38 -07:00
Brian Warner	ca273eeaf1	web: fix output=JSON, add buttons for repair/json to the 'run deep-check' form	2008-09-10 14:11:37 -07:00
Brian Warner	f570ad7ba5	disallow deep-check on non-directories, simplifies the code a bit	2008-09-10 13:44:58 -07:00
Brian Warner	4bb88fd2ee	dirnode: refactor recursive-traversal methods, add stats to deep_check() method results and t=deep-check webapi	2008-09-10 01:45:04 -07:00
Brian Warner	f6eeb3161f	dirnode: cleanup, make get_verifier() always return a URI instance, not a string	2008-09-10 01:37:55 -07:00
Brian Warner	28a9f8f076	test_system: check t=deep-stats too	2008-09-09 23:54:57 -07:00
Brian Warner	12ff8a6eae	test_system: add deep-check-JSON tests, fix a bug	2008-09-09 23:14:16 -07:00
Brian Warner	ce7fcbde36	test_system: oops, re-enable some tests that got bypassed	2008-09-09 23:02:45 -07:00
Brian Warner	ba336aed3e	test_system: add deep-stats test	2008-09-09 22:56:34 -07:00
Brian Warner	80d8f3e862	hush pyflakes	2008-09-09 19:50:17 -07:00
Brian Warner	1d2d6a35a6	checker results: add output=JSON to webapi, add tests, clean up APIs ... to make the internal ones use binary strings (nodeid, storage index) and the web/JSON ones use base32-encoded strings. The immutable verifier is still incomplete (it returns imaginary healty results).	2008-09-09 19:45:17 -07:00
Brian Warner	04513e3ac5	immutable verifier: provide some dummy results so deep-check works, make the tests ignore these results until we finish it off	2008-09-09 18:08:27 -07:00
Brian Warner	7fb3308498	mutable checker: even more tests. Everything in ICheckerResults should be covered now, except for immutable-verify which is incomplete	2008-09-09 17:57:06 -07:00
Brian Warner	84a5778507	checker results: more tests, update interface docs	2008-09-09 17:30:10 -07:00
Brian Warner	9676586732	mutable checker: oops, fix redefinition of 'healthy' (numshares < N, not numshares < k, which is 'recoverable' not 'healthy')	2008-09-09 17:28:53 -07:00
Brian Warner	f895e39d48	checker results: more tests, more results. immutable verifier tests are disabled until they emit more complete results	2008-09-09 17:15:46 -07:00
Brian Warner	90b934eb71	checker: add tests, add stub for immutable check_and_repair	2008-09-09 16:34:49 -07:00
Brian Warner	137750eca6	interfaces.py: minor improvement to IDirectoryNode.set_node	2008-09-09 16:34:16 -07:00
Brian Warner	918b0543b9	test_system: make log() tolerate the format= form	2008-09-07 20:03:36 -07:00
Brian Warner	af2231563e	immutable/checker: make log() tolerate the format= form	2008-09-07 20:03:08 -07:00
Brian Warner	3408d552cd	checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays	2008-09-07 12:44:56 -07:00
Brian Warner	e2c484a761	introducer: add get_nickname_for_peerid	2008-09-05 22:07:00 -07:00
Brian Warner	bab799297f	testutil.PollMixin: use a custom exception (and convert it) to avoid the ugly 'stash' cycle	2008-09-02 20:32:51 -07:00
Zooko O'Whielacronx	037c9157dc	setup: simplify parsing of python version number	2008-08-28 17:00:45 -07:00
Zooko O'Whielacronx	6801d5f572	setup: emit the version of python in the list of versions	2008-08-28 15:04:54 -07:00
Brian Warner	3b06aa6a79	servermap: don't log late arrivals, and don't log DeadReferenceError at log.WEIRD	2008-08-26 17:37:29 -07:00
Brian Warner	1668401c16	mutable: make mutable-repair work for non-verifier runs, add tests	2008-08-26 16:34:54 -07:00
Zooko O'Whielacronx	f7b4c45d46	mutable: remove work-around for a flaw in an older version of foolscap ... We now require "foolscap[secure_connections] >= 0.3.0", per [source:_auto_deps.py].	2008-08-26 08:50:55 -07:00
Brian Warner	a94af879ff	logging: add 'unique-message-ids' (or 'umids') to each WEIRD-or-higher log.msg call, to make it easier to correlate log message with source code	2008-08-25 18:57:59 -07:00
Brian Warner	735aa895b9	logging cleanups: lower DeadReferenceError from WEIRD (which provokes Incidents) to merely UNUSUAL, don't pre-format Failures in others	2008-08-25 17:51:55 -07:00
Zooko O'Whielacronx	def9fc8cf0	checker: make the log() function of SimpleCHKFileVerifier compatible with the log() function of its superclasses and subclasses	2008-08-25 14:44:07 -07:00
Brian Warner	eb695f1c58	mutable/checker: log a WEIRD-level event when we see a hash failure, to trigger an Incident	2008-08-12 20:50:20 -07:00
Brian Warner	97852cd626	immutable checker: add a status_report field	2008-08-12 20:35:30 -07:00
Brian Warner	e82d77c3cf	mutable/servermap: lower the priority of many log messages	2008-08-12 20:35:06 -07:00
Brian Warner	e5252fd5b2	web/deep-check: show the webapi runtime at the bottom of the page	2008-08-12 20:34:26 -07:00
Brian Warner	7a206421b4	CLI: tolerate blank lines in the aliases file	2008-08-12 19:50:50 -07:00
Brian Warner	bcbb51fe4a	test_web: workaround broken HEAD behavior in twisted-2.5.0 and earlier	2008-08-12 19:45:20 -07:00
Brian Warner	6af4619928	test_web: oops, actually use HEAD (instead of GET) in the HEAD test	2008-08-12 19:04:51 -07:00
Brian Warner	4d6b504883	web: use get_size_of_best_version for HEAD requests, provide correct content-type	2008-08-12 19:04:10 -07:00
Brian Warner	d43baa2ad7	mutable: add get_size_of_best_version to the interface, to simplify the web HEAD code, and tests	2008-08-12 19:02:52 -07:00
Brian Warner	014c9b5969	CLI: add 'tahoe debug corrupt-share', and use it for deep-verify tests, and fix non-deep web checker API to pass verify=true into node	2008-08-12 17:05:01 -07:00
Brian Warner	c80e352951	IFilesystemNode: add get_storage_index(), it makes tests easier	2008-08-12 16:14:07 -07:00
Brian Warner	5c0bfd7681	test_system: rename Checker to ImmutableChecker, to make room for a mutable one	2008-08-12 15:59:32 -07:00
Brian Warner	67e7333955	'tahoe debug dump-share': add --offsets, to show section offsets	2008-08-12 14:46:56 -07:00
Brian Warner	022fcf3db8	test_cli: oops, fix tests after recent stdout/stderr cleanup	2008-08-12 14:46:34 -07:00
Brian Warner	acb0297806	scripts/debug: split out dump_immutable_share	2008-08-12 13:55:17 -07:00
Brian Warner	533afd039f	scripts/debug: clean up use of stdout/stderr	2008-08-12 13:52:42 -07:00
Brian Warner	1711b5591f	CLI: move the 'repl' command to 'tahoe debug repl'	2008-08-12 13:40:17 -07:00
Brian Warner	bb33e3e4c2	CLI: move all debug commands (dump-share, dump-cap, find-shares, catalog-shares) into a 'debug' subcommand, and improve --help output	2008-08-12 13:37:32 -07:00
Brian Warner	3cf70697e8	hush a pyflakes warning	2008-08-11 21:24:23 -07:00
Brian Warner	08e229f29a	web/directory: enable verify=true in t=deep-check	2008-08-11 21:24:09 -07:00
Brian Warner	1306b11a76	dirnode: add some deep-check logging	2008-08-11 21:23:38 -07:00
Brian Warner	79576cf75b	checker_results.problems: don't str the whole Failure, just extract the reason string	2008-08-11 21:23:06 -07:00
Brian Warner	d106e411af	checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted	2008-08-11 21:03:26 -07:00
Brian Warner	17d7f7d983	mutable/checker: rearrange a bit, change checker-results to have a status_report string	2008-08-11 20:20:33 -07:00
Brian Warner	a15ae52122	mutable/servermap: add summarize_version	2008-08-11 20:19:30 -07:00
Brian Warner	376df2eb5a	CLI: make 'tahoe webopen' command accept aliases like 'tahoe ls'	2008-08-11 18:20:23 -07:00
Brian Warner	0d5c8468e9	storage: include disk-free information in the stats-gatherer output	2008-08-06 14:06:02 -07:00
Brian Warner	c7c57bd85c	mutable: more repair tests, one with force=True to check out merging	2008-08-06 12:06:07 -07:00
Brian Warner	a68c9364b8	test/common: add ShouldFailMixin	2008-08-06 12:05:52 -07:00
Brian Warner	572115a405	test_mutable: add comment about minimal-bandwidth repairer, comma lack of	2008-08-06 10:38:50 -07:00
Brian Warner	cdbfccd23e	test_mutable: factor out common setup code	2008-08-06 10:38:04 -07:00
Brian Warner	dd6ec73efa	mutable: start adding Repair tests, fix a simple bug	2008-08-05 23:12:39 -07:00
Brian Warner	3ad0168fbf	test_system: factor out find_shares/replace_shares to a common class, so they can be used by other tests	2008-08-05 18:49:58 -07:00
Brian Warner	56943bc5e2	web: add 'report incident' button at the bottom of the welcome page	2008-08-05 12:09:21 -07:00
Brian Warner	45d12baba2	test_cli: more coverage for 'tahoe put' modifying a mutable file in-place, by filename, closes #441	2008-08-04 13:26:43 -07:00
Brian Warner	4fb03c2510	check_grid.py: update to match new CLI: 'put - TARGET' instead of 'put TARGET'	2008-08-01 19:48:56 -07:00
Brian Warner	b9f4006b96	test_cli: remove windows-worrying newlines from test data	2008-08-01 19:47:34 -07:00
Brian Warner	35afe726c1	test_cli.py: factor out CLITestMixin	2008-08-01 19:29:38 -07:00
Brian Warner	f7d2fcc233	CLI: change one-arg forms of 'tahoe put' to make an unlinked file, fix replace-mutable #441	2008-08-01 19:27:29 -07:00
Brian Warner	2eec38279d	CLI: add create-alias command, to merge mkdir and add-alias into a single (secure-from-argv-snooping) step	2008-08-01 19:10:41 -07:00
Brian Warner	dc57ab4d36	test_cli: add system-based tests for PUT, including a mutable put that fails/todo (#441)	2008-08-01 15:10:09 -07:00
Brian Warner	071cbec865	tests: simplify CLI tests that use stdin, now that runner supports it	2008-08-01 15:05:14 -07:00
Brian Warner	f181a0458a	CLI: simplify argument-passing, use options= for everthing, including stdout	2008-08-01 11:46:24 -07:00
Zooko O'Whielacronx	6d3ec390b0	tests: add test that verifier notices any (randomly chosen) bit flipped in the verifiable part of any (randomly chosen) share ... The currently verifier doesn't (usually) pass this randomized test, hence the TODO.	2008-07-30 17:20:15 -07:00
Zooko O'Whielacronx	86160c3fc9	tests: test that checker doesn't cause reads on the storage servers ... It would still pass the test if it noticed a corrupted share. (It won't notice, of course.) But it is required to do its work without causing storage servers to read blocks from the filesystem.	2008-07-30 16:54:20 -07:00
Zooko O'Whielacronx	29255568df	storage: make storage servers declare oldest supported version == 1.0, and storage clients declare oldest supported version == 1.0 ... See comments in patch for intended semantics.	2008-07-30 15:51:07 -07:00
Zooko O'Whielacronx	dbd1c2397e	tests: use the handy dandy TestCase.mktemp() function from trial to give unique and nicely named directories for each testcase	2008-07-30 15:49:20 -07:00
Zooko O'Whielacronx	6aabaa0797	tests: don't use SignalMixin ... It seems like we no longer need it, and it screws up something internal in trial which causes trial's TestCase.mktemp() method to exhibit wrong behavior (always using a certain test method name instead of using the current test method name), and I wish to use TestCase.mktemp(). Of course, it is possible that the buildbot is about to tell me that we do still require SignalMixin on some of our platforms...	2008-07-30 15:35:36 -07:00
Zooko O'Whielacronx	56282613c5	tests: add test_system.Checker which tests basic checking (without verification) functionality	2008-07-28 16:43:17 -07:00
Zooko O'Whielacronx	9d14341c7d	test: add testutil.flip_one_bit which flips a randomly chosen bit of the input string	2008-07-28 16:42:17 -07:00
Zooko O'Whielacronx	fe522e31a4	tests: make it so that you can use common.py's SystemTestMixin.set_up_nodes() more than once with the same introducer	2008-07-28 16:40:29 -07:00
Brian Warner	cd4d3cdeea	download.py: set up self._paused before registering the producer, since they might call pauseProducing right away	2008-07-28 14:57:31 -07:00
Brian Warner	9e22353773	test/common.py: use pre-computed Tub certificates for the system-test mixin, to speed such tests up by maybe 15%. The goal is to encourage more full-grid tests.	2008-07-28 12:44:21 -07:00
Brian Warner	914783acdf	web: add /status/?t=json, with active upload/download ops. Addresses #493.	2008-07-25 17:41:10 -07:00
Brian Warner	f8221c2c66	web: make t=json stats pages use text/plain, instead of leaving it at text/html	2008-07-25 17:24:27 -07:00
Brian Warner	48c64b7a29	test_system.py: factor SystemTestMixin out of SystemTest	2008-07-25 15:33:49 -07:00
Brian Warner	10bbc740e9	test_system.py: modify system-test setup code in preparation for merge with common.SystemTestMixin	2008-07-25 15:29:31 -07:00
Brian Warner	766cdd9c52	test_system.py: move SystemTestMixin out into common.py, where further improvements will occur	2008-07-25 15:17:58 -07:00
Brian Warner	e317fcc08c	test_system.py: create SystemTestMixin, with less cruft, for faster system-like tests	2008-07-25 15:13:00 -07:00
Brian Warner	afda2a43e4	storage: remove update_write_enabler method, it won't serve the desired purpose, and I have a better scheme in mind. See #489 for details	2008-07-21 17:28:28 -07:00
Zooko O'Whielacronx	9461887e0a	immutable file download: make the ciphertext hash tree mandatory ... This fixes #491 (URIs do not refer to unique files in Allmydata Tahoe). Fortunately all of the versions of Tahoe currently in use are already producing this ciphertext hash tree when uploading, so there is no backwards-compatibility problem with having the downloader require it to be present.	2008-07-21 09:31:02 -07:00
Brian Warner	edec917060	test_web: test that save=true filename=unicode doesn't crash	2008-07-18 18:58:57 -07:00
Brian Warner	572c848d98	web: for GET save=true, don't interpret the filename= arg with any character set, just copy the bytes back into the Content-Disposition header. This seems to make it maximally compatible with Firefox and IE7	2008-07-18 18:06:50 -07:00
Brian Warner	879fefe5f3	first pass at a mutable repairer. not tested at all yet, but of course all existing tests pass	2008-07-17 21:09:23 -07:00
Brian Warner	3e95681bad	interfaces: add IRepairable	2008-07-17 17:32:17 -07:00
Brian Warner	923c9c242a	oops, fix import/pyflakes problems	2008-07-17 17:06:20 -07:00
Brian Warner	67db0a4967	deep-check: add webapi, add 'DEEP-CHECK' button to wui, add tests, rearrange checker API a bit	2008-07-17 16:47:09 -07:00
Brian Warner	69156aeb28	dirnode deep-check: add tests of cycles, fix failures	2008-07-17 14:37:04 -07:00
Brian Warner	acf3180fac	dirnode deep-check: rearrange traversal approach, simplify code a bit	2008-07-17 14:25:04 -07:00
Brian Warner	9289433ba3	first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests	2008-07-16 18:20:57 -07:00
Brian Warner	7b9fff388d	immutable download: remove dead LiteralDownloader, now that we use filenodes for download	2008-07-16 16:31:47 -07:00
Brian Warner	3e9322bcb6	checker: re-enable checker web results (although they just say 'Healthy' right now)	2008-07-16 15:42:56 -07:00
Brian Warner	7394607141	move encode/upload/download/checker.py into a new immutable/ directory. No behavior changes expected.	2008-07-16 13:14:39 -07:00
Brian Warner	94e619c1f6	overhaul checker invocation ... Removed the Checker service, removed checker results storage (both in-memory and the tiny stub of sqlite-based storage). Added ICheckable, all check/verify is now done by calling the check() method on filenodes and dirnodes (immutable files, literal files, mutable files, and directory instances). Checker results are returned in a Results instance, with an html() method for display. Checker results have been temporarily removed from the wui directory listing until we make some other fixes. Also fixed client.create_node_from_uri() to create LiteralFileNodes properly, since they have different checking behavior. Previously we were creating full FileNodes with LIT uris inside, which were downloadable but not checkable.	2008-07-15 17:23:25 -07:00
Brian Warner	fd465b4aaf	download: fix stopProducing failure ('self._paused_at not defined'), add tests	2008-07-14 15:25:21 -07:00
Brian Warner	c46f5bc634	storage: rename the latency key names so they sort properly	2008-07-11 21:51:02 -07:00
Brian Warner	bffe14ef3e	add 1%,10% percentiles to the storage server latency output	2008-07-11 21:34:36 -07:00
Brian Warner	60725ed065	storage: add add_lease/update_write_enabler to remote API, revamp lease handling	2008-07-09 18:06:55 -07:00
Brian Warner	de8ce0cb5b	welcome page: add link to statistics page	2008-07-09 17:37:22 -07:00
Brian Warner	a7aa6f8686	implement a mutable checker+verifier. No repair yet. Part of #205.	2008-07-07 17:36:00 -07:00
Brian Warner	a0de01e971	servermap.shares_available: report N too, not just k	2008-07-07 15:06:15 -07:00
Brian Warner	45405d85c4	interfaces: add verify= and repair= args to check()	2008-07-07 14:37:36 -07:00
Brian Warner	9b6f23ac05	'tahoe dump-share': show verify-cap too	2008-07-07 14:11:02 -07:00
Brian Warner	2074c92dd1	test_mutable.py: add tests for no-servers conditions, closes #463.	2008-07-07 12:18:10 -07:00
Brian Warner	bbc63e5d3f	web/root.py: fix minor typo	2008-07-07 00:18:16 -07:00
Brian Warner	91c7e0f689	mutable: if there are no servers available, return an error, rather than hanging forever	2008-07-07 00:17:41 -07:00
Brian Warner	7dc8ff0263	re-enable incident-reporting, but disable it in unit tests, because they take 150% longer, and the leftover trailing timers cause errors	2008-07-06 23:49:08 -07:00
Brian Warner	13d0ba72a4	disable incident reporting: unit tests are failing because of the leftover post-incident timer	2008-07-02 18:24:00 -07:00
Brian Warner	d58591ceba	start using Foolscap's 'incident-logging' feature, which requires foolscap-0.2.9	2008-07-02 17:40:29 -07:00
Brian Warner	dfe235bcba	test_storage: oops, update the test to match the leave-incoming/ change	2008-06-26 11:36:17 -07:00
Brian Warner	02ce46cfb4	catalog-shares: ignore incoming/	2008-06-26 11:23:44 -07:00
Brian Warner	edabd2f004	storage.py: leave the storage/shares/incoming/ directory in place when the bucket is closed	2008-06-26 11:07:57 -07:00
Brian Warner	af4e4dc847	storage: record latency stats in a flat dict, not nested, to conform to RIStatsProvider	2008-06-24 17:21:18 -07:00
Brian Warner	dba0ad3d28	test_mutable.py: add more tests of post-mapupdate corruption, to support #474 testing	2008-06-24 11:08:10 -07:00
Brian Warner	4e5b9ee63e	introducer: move the relevant interfaces out to introducer/interfaces.py	2008-06-18 17:04:41 -07:00
Brian Warner	fa453c9b5e	introducer: add old (V1) introducer code, add test framework for compatibility testing	2008-06-18 16:58:34 -07:00
Brian Warner	9ed9eb2048	check_grid.py: make it work, move node start/stop responsibility from Makefile to script	2008-06-19 12:39:52 -07:00
Brian Warner	50d945ea8f	start work on 'check-grid' target, for use in an automated prodnet tester. Just a docstring so far.	2008-06-18 12:57:47 -07:00
Brian Warner	28f4652b96	break introducer up into separate modules in the new allmydata.introducer package	2008-06-18 12:24:16 -07:00
Brian Warner	5bdff74e5b	web: stop using absolute links (or url.here) in forms and pages, since they break behind proxies. Partially addresses #461	2008-06-17 19:49:40 -07:00
Brian Warner	c09c342718	test_storage: yet more coverage	2008-06-17 17:44:10 -07:00
Brian Warner	6b55b8b022	test_storage.py: improve test coverage	2008-06-17 17:01:42 -07:00
Brian Warner	8e52b3d97d	storage.ReadBucketProxy: avoid double-start, this only affected tests	2008-06-17 17:01:29 -07:00
Brian Warner	b62b3974d3	storage: remove unused method	2008-06-17 11:36:32 -07:00
Brian Warner	2154b5751b	test_storage: add coverage for discard_storage	2008-06-16 17:52:40 -07:00
Brian Warner	75e662cc46	test_storage: add coverage for readonly_storage	2008-06-16 17:52:13 -07:00
Brian Warner	166e68f2a3	web /statistics : show 0 instead of None for mutable files	2008-06-16 16:37:04 -07:00
Brian Warner	1ce3b77dde	storage: improve stats, make them accessible via webport /statistics	2008-06-16 16:35:59 -07:00
Brian Warner	6b7ff02e36	storage: measure latency-per-operation, calculate mean/median/percentiles	2008-06-16 15:21:55 -07:00
robk-tahoe	212c19fc45	webish: re-fix the addSlash issue in directory rendering ... having fixed the relevant unit test not to assert incorrect behaviour of the node when rendering directories, it's now safe to fix the latter.	2008-06-11 15:35:27 -07:00
robk-tahoe	b803158de7	test_web: fix webapi test around redirection issues ... this fixes the test_web test test_POST_upload_mutable which chdir's into a /foo subdirectory, but then later asserts that the web ui redirects the user back to /foo, which should really be /foo/ since it's a directory.	2008-06-11 15:19:17 -07:00
Brian Warner	4c681847f8	directory.py: revert that addSlash, it causes a test to fail and we don't understand it yet, and we don't need the fix quite yet	2008-06-11 12:59:28 -07:00
robk-tahoe	021a9e370f	webish: fix directory url redirection ... when rendering a directory in webish, if the url did not correctly include a trailing slash '/' then the browser will be redirected to a url which does. this causes any relative links (e.g. the 'other representations' which are links to files relative to the current directory) to be correctly intrpreted as relative to the directory in question by the browser. uses twisted's addSlash mechanism, which is designed for this purpose :-)	2008-06-11 12:46:28 -07:00
Zooko O'Whielacronx	65618ba5a2	wapi/wui: add a trailing slash to the targets of hyperlinks of children of the current directory when those targets are directories ... This is intended to fix #458 '"other representations" broken in webish ui'.	2008-06-10 19:59:38 -07:00
Brian Warner	baf3b6868e	storage.py: remove unused import	2008-06-10 13:05:44 -07:00
Brian Warner	bc04b8528a	upload: abort the bucket upon any write error, and do it with callRemoteOnly to avoid double errors	2008-06-10 11:55:28 -07:00
Brian Warner	814922a9a1	storage: ignore shares in incoming/, to make clients use other servers during simultaneous uploads	2008-06-10 11:53:10 -07:00
Brian Warner	0aa6d24d62	web: add test for unicode POST when the name comes from name=, not the filename attribute	2008-06-03 17:09:39 -07:00
Brian Warner	9f5075e83d	web: make nickname more visible in the welcome page, closes #361	2008-06-03 15:02:10 -07:00
Brian Warner	9f59ecafbb	webish: make /cap/ equivalent to /uri/, accepting both with the same meanings. Closes #428	2008-06-03 14:34:00 -07:00
Brian Warner	32c89a8d59	web: transform FileTooLargeError into a friendlier '413 Request Entity Too Large' error	2008-06-03 00:03:16 -07:00
Brian Warner	f4496bd553	mutable/publish.py: raise FileTooLargeError instead of an ugly assertion when the SDMF restrictions are exceeded	2008-06-03 00:02:10 -07:00
Brian Warner	5289064dcf	move FileTooLargeError out to a common location	2008-06-03 00:01:15 -07:00
Brian Warner	beb502ac18	CLI: make 'tahoe webopen' use the 'tahoe:' alias properly, instead of the old --dir-cap option	2008-06-02 17:55:54 -07:00
Brian Warner	3e8d58767f	CLI: improve docs w.r.t. aliases, add examples to 'tahoe put' and 'tahoe get' help output. Addresses part of #431	2008-06-02 17:54:56 -07:00
Brian Warner	8c37b8e3af	Don't allow uploads of large files (about 12GiB or larger), since they're doomed to be corrupted. Closes #439	2008-06-02 16:57:01 -07:00
Brian Warner	b6c2f80170	test_system: even more 'cp -r' coverage	2008-05-21 18:40:49 -07:00
Brian Warner	4824d058cc	test_system.py: improve 'cp -r' coverage: exercise copy from tahoe to local disk	2008-05-21 18:36:25 -07:00
Brian Warner	0f8f213baf	CLI: remove the '-r' shortcut for --dir-cap, to make it available for cp -r	2008-05-21 17:55:32 -07:00
Brian Warner	457248f023	tahoe_cp: rewrite, make --recursive work	2008-05-21 17:35:21 -07:00
Brian Warner	15fbe05092	tahoe_ls: improve error message when the target is missing	2008-05-21 17:34:52 -07:00
Brian Warner	282e22abcf	scripts/common_http.py: add comment about why we need .seek	2008-05-21 17:31:19 -07:00
Brian Warner	7ae432abf8	tahoe_cp.py: fix pyflakes complaint	2008-05-21 12:09:13 -07:00
Brian Warner	7afd869854	cli: initial implementation of 'cp -r', probably doesn't work yet	2008-05-21 11:49:22 -07:00
Brian Warner	fe44b8fb4b	util.dictutil: move DictOfSets out to a separate utility module	2008-05-21 09:43:49 -07:00
Brian Warner	0df663b7ae	CLI: implement the easy part of cp (no -r, only two arguments)	2008-05-20 16:56:03 -07:00
Brian Warner	3ac4a734e5	web t=json: add 'mutable' key to the information dict	2008-05-20 15:40:49 -07:00
Brian Warner	9c992c0d06	test_web: improve test coverage of web.common utility code	2008-05-20 15:21:46 -07:00
Brian Warner	8dded7819c	web: fix JSON output for mutable files	2008-05-20 15:14:19 -07:00
Brian Warner	1d8e452184	CLI: add 'list-aliases', factor out get_aliases	2008-05-20 14:36:04 -07:00
Brian Warner	1236bc3408	CLI: add 'ln', just like move but without the delete	2008-05-20 13:30:31 -07:00
Brian Warner	f09f92c371	test_system.py: remove dead code	2008-05-20 13:20:48 -07:00
Brian Warner	3803d591cf	tahoe_ls: improve output formatting	2008-05-20 13:07:50 -07:00
Brian Warner	abe4addbdb	CLI mv: if we can't created the new child, don't delete the old one	2008-05-20 12:49:47 -07:00
Brian Warner	a8ed3208ba	CLI ls: add --readonly-uri to display readonly URIs for all children	2008-05-20 12:49:11 -07:00
Brian Warner	e889463f0c	CLI: add put --mutable, enhance ls to show mutable vs immutable as rw/r-	2008-05-20 12:36:55 -07:00
Brian Warner	e323019708	web: handle PUT mutable=true properly	2008-05-20 12:36:02 -07:00
Brian Warner	d72f75019f	test_web: add HEAD coverage	2008-05-20 11:47:43 -07:00
Brian Warner	b0e306a753	web: more mutable-file coverage	2008-05-20 11:35:47 -07:00
Brian Warner	6b51841ec8	web: even more test coverage	2008-05-20 11:33:14 -07:00
Brian Warner	2eadabbf82	web: improve test coverage, remove some dead code	2008-05-20 11:13:12 -07:00
Brian Warner	88457fbc32	test_web/test_system: improve test coverage	2008-05-19 23:28:52 -07:00
Brian Warner	506ce2dfd0	test_system: fix windows test failure, open files with binary mode	2008-05-19 20:24:12 -07:00
Brian Warner	8e92dfcb50	overhaul CLI: not quite complete but it works a lot better than it used to. The new scheme uses 'tahoe add-alias' and rsync/scp-style 'alias:foo/bar.txt' arguments	2008-05-19 19:28:50 -07:00
Brian Warner	9662e6d986	test/common: oops, forgot the FakeMutableFileNode.get_readonly fix	2008-05-19 18:52:19 -07:00
Brian Warner	ef929e51f9	web: improve test coverage	2008-05-19 18:38:39 -07:00
Brian Warner	fe8fdfac64	web/filenode: remove unreachable code	2008-05-19 18:38:28 -07:00
Brian Warner	daebb42060	web/directory: fix rw_uri output in t=JSON to reflect mutable files properly	2008-05-19 18:37:28 -07:00
Brian Warner	d0685d42ff	web: factor out identical renderHTTP methods	2008-05-19 15:19:25 -07:00
Brian Warner	242b98f750	webish: censor URLs that start with /file and /named too	2008-05-19 14:34:13 -07:00
Brian Warner	fdb4149bad	test_web: remove leftover import to hush pyflakes	2008-05-19 14:28:39 -07:00
Brian Warner	b1d2922ea3	test_system: match webapi change, making new files now returns 201 Created, not 200 OK	2008-05-19 14:09:31 -07:00
Brian Warner	40172047a0	filenode: add is_mutable to non-MutableFileNode classes	2008-05-19 13:03:00 -07:00
Brian Warner	f9cd30d9bc	webish: complete rewrite, break into smaller pieces, auto-create directories, improve error handling	2008-05-19 12:57:04 -07:00
Brian Warner	b8c0217ad5	test_web: implement API changes from the recent webapi.txt overhaul	2008-05-19 12:56:02 -07:00
Brian Warner	869e1ed41a	test_web.py: minor cleanups, improved error reporting	2008-05-19 12:33:39 -07:00
Brian Warner	3360fd24fc	test_web.py: localdir=/localfile= is going away, so remove the tests that exercise it	2008-05-19 12:32:09 -07:00
Brian Warner	b2f8c22fb8	debug dump-share: print size of encprivkey and signature too	2008-05-19 14:24:41 -07:00
Brian Warner	154dca10a0	servermap.py: add size_of_version(), to retrieve the size of a mutable file version	2008-05-19 14:16:02 -07:00
Brian Warner	87c1e8e066	dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child	2008-05-16 16:09:47 -07:00
Brian Warner	1121df6664	testutil.shouldFail: mention the 'which' string in substring failures too	2008-05-16 16:08:38 -07:00
Brian Warner	304abfee32	webish: add /file links, change directory page to use them. This fixes filenames in wget. Closes #221.	2008-05-14 14:32:21 -07:00
Brian Warner	5098297a2b	introweb.py: tolerate foolscap>=0.2.6, which changed the internals of .locationHints	2008-05-14 13:55:47 -07:00
Brian Warner	ff0b9e2549	test_mutable.py: test replacing a file that has one new outlier share present: closes #272	2008-05-14 13:10:41 -07:00
Brian Warner	26187bfc81	use a weakref cache in the client to manage singleton filenodes/dirnodes, to avoid autocollision. Should close #391.	2008-05-08 18:02:55 -07:00
Brian Warner	8f4e409677	check_memory: oops, fix HTTP-based modes: trailing slash in node.url vs no slash	2008-05-08 18:01:31 -07:00
Brian Warner	7323ced8e4	test_dirnode: improve coverage of not-mutable-error a bit	2008-05-08 16:53:35 -07:00
Brian Warner	411f578c19	check_memory: don't use fixed webports, they cause occasional test failures	2008-05-08 16:49:27 -07:00
Brian Warner	fabdc28c06	deep-stats: add file-size histogram	2008-05-08 16:19:42 -07:00
Brian Warner	a1ea3d9b37	test_dirnode: the size of the directories varies, tolerate it as low as 500 bytes	2008-05-08 13:42:42 -07:00
Brian Warner	e6ae7a2c60	dirnode: refactor deep-stats a bit	2008-05-08 13:33:07 -07:00
Brian Warner	6c00a70dbc	dirnode: add a deep_stats(), like deep-size but with more information. webish adds t=deeps-size too.	2008-05-08 13:21:14 -07:00
Brian Warner	462ef2a0ac	run a stats provider even if there's no gatherer, since the HTTP /statistics page is then useful. Only run the once-per-second load-monitor if there is a gatherer configured	2008-05-08 11:37:30 -07:00
Brian Warner	3cb361e233	dirnode: use the concurrency limiter in t=manifest and t=deep-size, allow 10 retrievals in parallel	2008-05-07 18:36:37 -07:00
Brian Warner	d76cf6f748	limiter: fix bug that ignores the limit= argument	2008-05-07 18:35:53 -07:00
Brian Warner	15a9f3d01b	test_util: improve ConcurrencyLimiter tests	2008-05-07 17:00:47 -07:00
Brian Warner	c7e441309d	add a basic concurrency limiter utility	2008-05-07 16:53:30 -07:00
Brian Warner	1e4b904b1a	web status: abbreviate file size in the upload+download summary page	2008-05-07 13:53:58 -07:00
Zooko O'Whielacronx	c1184f51e4	docs: fix a few stale comments in code	2008-05-07 08:39:03 -07:00
Brian Warner	fc774803ce	test_stats.py: improve test coverage	2008-04-30 11:52:31 -07:00
Brian Warner	a5a7ba24ef	stats: add tests for CPUUsageMonitor, modify it a bit to facilitate testing	2008-04-30 11:39:13 -07:00
Zooko O'Whielacronx	fc271a0ee9	setup: don't catch ImportError when importing _auto_deps in allmydata/__init__.py ... Nowadays pkg_resources is a runtime requirement, and if there is something screwed up in the installation, we want an explicit ImportError exception as early as possible.	2008-04-30 13:22:04 -07:00
Brian Warner	88951548ee	test_mutable: update notify_publish() to match new signature	2008-04-29 18:24:57 -07:00
Brian Warner	4b3adb3fcf	mutable stats: track mutable bytes published too	2008-04-29 18:20:05 -07:00
Brian Warner	3be921174b	stats: add CPU-percentage monitor, with 1min/5min/15min moving-window averages, using time.clock()	2008-04-29 18:12:53 -07:00
Zooko O'Whielacronx	f842656ec2	wui: reorganize the welcome.xhtml page ... Jake Edge tried Tahoe out and didn't notice the /status page. Hopefully with this new organization people like he will see that link more easily. This also addresses drewp's suggestion that the controls appear above the list of servers instead of below. (I think that was his suggestion.) I also reordered the controls.	2008-04-29 15:10:14 -07:00
Brian Warner	351a7f4632	offloaded: add 'resumes' counter to stats: how many upload requests are resuming an interrupted upload?	2008-04-28 13:27:55 -07:00
ben	c85869fc08	support freebsd 6	2008-04-28 00:41:40 -07:00
Brian Warner	186492e620	introducer: only record one announcement per (tubid,service) tuple. Fixes #343.	2008-04-23 15:05:39 -07:00
Brian Warner	55dfb697a4	client: don't start the IntroducerClient until the Tub is ready, otherwise we will sometimes connect to the introducer (or other clients) before we've done Tub.setLocation, which loses some information on the introducer status page	2008-04-23 14:52:34 -07:00
Brian Warner	a557d44877	test_mutable: test that all servers refusing our share means a publish fails	2008-04-22 18:53:20 -07:00
Brian Warner	13eb00145e	mutable: improve test coverage in Retrieve, when shares change after mapupdate	2008-04-22 17:25:14 -07:00
Brian Warner	acdd98432d	hush pyflakes warning about code that got moved in the recent StallMixin refactoring	2008-04-22 17:14:26 -07:00
Brian Warner	a7632a345e	mutable/servermap: improve test coverage	2008-04-22 16:47:52 -07:00
Brian Warner	81ab1ec785	testutil: factor stall() out into a common location	2008-04-22 16:47:15 -07:00
Brian Warner	57d5341715	test_introducer.py: don't log nodeids as binary goop	2008-04-22 12:54:16 -07:00
Brian Warner	9ad6ed3777	web/status: remove trailing whitespace	2008-04-22 11:51:14 -07:00
Brian Warner	e6074f5dfc	mutable: test write failures, uncoordinated write detection	2008-04-22 11:49:53 -07:00
Brian Warner	7e1f9761e8	mutable: remove some dead code, rearrange use of populate_pubkey	2008-04-21 17:51:22 -07:00
Brian Warner	db4ddfc1bd	mutable: improve testing to exercise the non-cached case	2008-04-21 17:50:50 -07:00
Brian Warner	f39f8ae032	check_speed: rearrange a bit to make it easy to disable certain portions	2008-04-21 17:29:59 -07:00
Brian Warner	8377244c46	mutable status: make the 'total' timing value optional: unit tests fail otherwise	2008-04-21 17:29:12 -07:00
Brian Warner	6af124dc3e	mutable read: enable the cache (written during mapupdate, read during retrieve). This speeds up small-file reads by about 30% over a link with an average 25ms RTT	2008-04-21 17:27:50 -07:00
Brian Warner	e8b0f3d3cb	mutable status: use google-chart-API to draw server response times for servermap update	2008-04-21 16:16:55 -07:00
Brian Warner	a04c2866b1	mutable: improve test coverage slightly	2008-04-21 15:10:50 -07:00
Brian Warner	f16f48a909	download status: add time spent paused by the client (when we're serving over a slow HTTP link)	2008-04-21 12:19:17 -07:00
Zooko O'Whielacronx	78829dcf08	key_generator: make default key size be a constructor argument instead of a class variable, pass default key size of 522 (the smallest that we can do) in unit tests to make them faster	2008-04-22 12:28:18 -07:00
Brian Warner	c375b1503a	mutable/servermap: improve test coverage	2008-04-18 20:13:28 -07:00
Brian Warner	09dcfeae22	mutable: improve test coverage, fix bug in privkey fetching, add .finished to stats, remove dead code	2008-04-18 19:55:12 -07:00
Brian Warner	fc7b75e55f	mutable.node: avoid reentrancy problems in Deferred code on twisted2.5, by adding an eventual-send call	2008-04-18 00:43:29 -07:00
Zooko O'Whielacronx	a504f3395f	let MutableFileNode.__repr__() return something useful even if it isn't not initialize yet ... This is because I sometimes log or examine a repr of an object from a function which is being called from that object's own __init__()... (I'm committing this patch in order to test our patch management infrastructure.)	2008-04-17 21:32:38 -07:00
Brian Warner	d7b82f73c5	dirnode: return to 'delete fails if the child wasn't actually there' semantics, to make tests pass. There's a switch to enable/disable this	2008-04-17 20:06:06 -07:00
Brian Warner	d5b182c183	test_mutable: hush pyflakes	2008-04-17 19:57:55 -07:00
Brian Warner	1bff08a7df	dirnode: update to use MutableFileNode.modify	2008-04-17 19:57:04 -07:00
Brian Warner	b6b3b6b1bd	testutil: minor reformatting	2008-04-17 19:56:45 -07:00
Brian Warner	945003ea0f	test_mutable: factor out ShouldFailMixin	2008-04-17 19:55:51 -07:00
Brian Warner	004d3c14bf	mutable: implement MutableFileNode.modify, plus tests	2008-04-17 19:12:42 -07:00
Brian Warner	a379690b04	mutable: replace MutableFileNode API, update tests. Changed all callers to use overwrite(), but that will change soon	2008-04-17 17:51:38 -07:00
Brian Warner	157073d8d8	mutable WIP: merge in patches from current trunk	2008-04-17 13:09:22 -07:00
Brian Warner	ee218c262d	mutable WIP: use fireOnOneErrback when using a DeferredList	2008-04-17 13:11:48 -07:00
Brian Warner	e1838ba217	mutable WIP: clean up status handling, shrink the code a lot, improve test coverage	2008-04-17 13:02:22 -07:00
Brian Warner	a1670497a8	mutable WIP: add servermap update status pages	2008-04-16 19:05:41 -07:00
Brian Warner	842b1f1a70	mutable WIP: oops, fix test_mutable	2008-04-16 18:06:54 -07:00
Brian Warner	f35f20e10b	mutable WIP: publish status doesn't know its size early enough to update the stats_provider	2008-04-16 17:55:17 -07:00