2854 Commits

Author	SHA1	Message	Date
Brian Warner	ddc9a7ae74	consolidator: re-use more directories, add total directories seen-vs-used counts	2009-03-12 20:48:01 -07:00
Brian Warner	1c24707f19	consolidator: add more verbose traversal of directories	2009-03-12 16:29:00 -07:00
Brian Warner	760688a224	consolidate: create multiple numbered backups of the original Archives directory, not just the first time	2009-03-12 16:04:27 -07:00
Brian Warner	c4c6a62954	add 'tahoe debug consolidate' command, to merge directories created by repeated 'tahoe cp -r' or the allmydata win32 backup tool, into the form that would have been created by 'tahoe backup'.	2009-03-12 13:56:06 -07:00
Brian Warner	1ccd426a34	expirer: fix prediction math, thanks to Zandr for the catch	2009-03-09 13:42:17 -07:00
Brian Warner	a68ad06254	storage.expirer: exercise the last missing line of webstatus code	2009-03-08 20:38:28 -07:00
Brian Warner	6d6049430c	expirer: tolerate corrupt shares, add them to the state and history for future examination	2009-03-08 20:08:40 -07:00
Brian Warner	1a98521c3d	storage/immutable: raise a specific error upon seeing a bad version number, instead of using assert. Also wrap to 80cols.	2009-03-08 20:07:32 -07:00
Brian Warner	02b40ec499	storage: add test coverage for BucketReader.__repr__	2009-03-08 20:05:27 -07:00
Brian Warner	6d7319c588	storage/mutable: raise a specific error upon seeing bad magic, instead of using assert	2009-03-08 19:02:01 -07:00
Brian Warner	5675b4e7e0	expirer: make web display a bit more consistent	2009-03-07 16:14:42 -07:00
Brian Warner	2c2c3c2295	test_cli.py: assert that 'ls' on an unrecoverable file now gives a better error message	2009-03-07 05:08:15 -07:00
Brian Warner	67d7600781	test_web: test behavior of broken-dirnode GET, both html and json	2009-03-07 04:57:07 -07:00
Brian Warner	9c7fafd5af	test_cli: use explicit (and stable) testdirs, instead of using self.mktemp	2009-03-07 03:04:28 -07:00
Brian Warner	091cfd04ca	test_storage: solaris doesn't appear to give the same block count as other platforms, so don't assert as much about 'diskbytes' recovered	2009-03-07 02:45:18 -07:00
Brian Warner	0dee2a6036	storage: add a lease-checker-and-expirer crawler, plus web status page. ... This walks slowly through all shares, examining their leases, deciding which are still valid and which have expired. Once enabled, it will then remove the expired leases, and delete shares which no longer have any valid leases. Note that there is not yet a tahoe.cfg option to enable lease-deletion: the current code is read-only. A subsequent patch will add a tahoe.cfg knob to control this, as well as docs. Some other minor items included in this patch: tahoe debug dump-share has a new --leases-only flag storage sharefile/leaseinfo code is cleaned up storage web status page (/storage) has more info, more tests coverage space-left measurement on OS-X should be more accurate (it was off by 2048x) (use stat .f_frsize instead of f_bsize)	2009-03-06 22:45:17 -07:00
Brian Warner	c5ecb7b3ba	test_cli: validate non-HTML error response of 'tahoe get' on an unrecoverable file	2009-03-03 22:11:46 -07:00
Brian Warner	f42e3bb107	web: full patch for HTML-vs-plaintext traceback renderings, improve test coverage of exception rendering	2009-03-03 21:56:30 -07:00
Brian Warner	53f50fcc78	test/common: rearrange shouldHTTPError	2009-03-03 19:40:59 -07:00
Brian Warner	67571eb033	add more information to NotEnoughSharesError, split out new exceptions for no-servers and no-source-of-ueb-hash	2009-03-03 19:37:15 -07:00
Brian Warner	f15c0be5af	test_web: move shouldHTTPError into common.WebErrorMixin	2009-03-03 16:56:20 -07:00
Brian Warner	559c369574	test_web: clean up shouldHTTPError methods	2009-03-03 16:46:04 -07:00
Brian Warner	8c3013c4f7	rrefutil: add check_remote utility function	2009-02-27 00:59:57 -07:00
Brian Warner	1b3e635936	rrefutil: add trap_remote utility and friends	2009-02-27 00:55:24 -07:00
Brian Warner	8251572e01	web: improve layout of storage status with a table	2009-02-26 19:58:38 -07:00
Brian Warner	112dc35563	crawler: add ETA to get_progress()	2009-02-26 19:42:48 -07:00
Brian Warner	63b19e567c	web: fix the ERROR: line to work the same in python2.4 and 2.5	2009-02-25 01:46:21 -07:00
Brian Warner	7d4901197a	test_cli/test_web: fix spurious test failure on solaris (maybe python2.4?) due to variations in the way that exceptions are stringified	2009-02-25 00:01:28 -07:00
Brian Warner	a3c1fe35d9	CLI: modify 'tahoe manifest' and 'tahoe deep-check' to report ERROR: properly. For #590.	2009-02-24 23:44:15 -07:00
Brian Warner	fd4ceb6a87	webapi: modify streaming deep-manifest/deep-checker to emit an ERROR: line if they encounter an unrecoverable+untraversable directory. For #590.	2009-02-24 23:13:35 -07:00
Brian Warner	7f8708cb7c	scripts/common: fix alias handling on windows again, emit slightly nicer error message in response to an unknown alias	2009-02-24 22:21:36 -07:00
Brian Warner	51e3baeefc	test_deepcheck: switch deep-check tests to use no-network too. This cuts the runtime down by about 50%	2009-02-24 21:04:57 -07:00
Brian Warner	6cf4096156	test_deepcheck: convert MutableChecker to no-network GridTest	2009-02-24 20:00:10 -07:00
Brian Warner	2e7f64d392	tests/no_network: move GET into the GridTestMixin class	2009-02-24 18:33:00 -07:00
Brian Warner	6f981e4eab	common_web.py: oops, add .fields to the other FakeRequest	2009-02-24 18:04:59 -07:00
Brian Warner	376eab0c04	test_cli: exercise the recent tolerate-'c:\dir\file.txt' fix in scripts/common, recorded in a separate match to make it easier to merge the fix to prod	2009-02-24 17:56:20 -07:00
Brian Warner	8d8250e44a	test/common_web.py: add a .fields attribute to our FakeRequest, since we support versions of Nevow that are old enough to not do it themselves	2009-02-24 17:20:50 -07:00
Brian Warner	2810de32b1	test_web: add (disabled) test to see what happens when deep-check encounters an unrecoverable directory. We still need code changes to improve this behavior.	2009-02-24 15:40:17 -07:00
Brian Warner	7cfbb9c832	test_repairer: change to use faster no_network.GridTestMixin, split Verifier tests into separate cases, refactor judgement funcs into shared methods	2009-02-23 22:15:06 -07:00
Brian Warner	2be729b1e4	test_repairer: change Repairer to use much-faster no_network.GridTestMixin. As a side-effect, fix what I think was a bug: some of the assert-minimal-effort-expended checks were mixing write counts and allocate counts	2009-02-23 17:42:27 -07:00
Brian Warner	cc3c1ae8df	test/no_network.py: add a basic stats provider	2009-02-23 17:39:37 -07:00
Brian Warner	f35c659b2f	tests: stop using setUpClass/tearDownClass, since they've been deprecated in Twisted-8.2.0	2009-02-23 14:43:12 -07:00
Brian Warner	bc91689f8e	test_checker: improve test coverage for checker results	2009-02-23 14:19:43 -07:00
Alberto Berti	d9649052d4	Fixed tests again so they will pass on windows.	2009-02-22 18:35:02 -07:00
Brian Warner	c4f8005867	remove more RuntimeError from unit tests, for #639	2009-02-22 17:28:55 -07:00
Brian Warner	5c3d7d8e73	stop using RuntimeError in unit tests, for #639	2009-02-22 17:27:22 -07:00
Alberto Berti	b945c275c2	Added tests for the cse when listdir is an iterator	2009-02-22 16:43:56 -07:00
Alberto Berti	cd0bd97954	Fixed tests so that they pass also on buildbots.	2009-02-22 16:43:11 -07:00
Alberto Berti	b2f8fe04c6	Use failUnlessEqual instead of failUnless(a == b)	2009-02-22 16:42:14 -07:00
Alberto Berti	5d629d0050	Added tests for the --exclude* options of backup command.	2009-02-22 10:51:06 -07:00
Alberto Berti	ff6a77cc6b	Added tests for the fixed alias related command's synopsis	2009-02-22 10:37:32 -07:00
Brian Warner	c85eccbf64	test_web.py: increase test coverage of web.status.plural()	2009-02-21 18:01:16 -07:00
Brian Warner	77f3b83d68	crawler: fix performance problems: only save state once per timeslice (not after every bucket), don't start the crawler until 5 minutes after node startup	2009-02-21 14:56:49 -07:00
Brian Warner	a04d3b8fe8	test_system: oops, don't assume that all files in storage/ are in a deep storage/shares/prefix/si/shnum path, since now the crawler pickle has a short path	2009-02-21 00:17:10 -07:00
Brian Warner	3c6471c717	test_storage: startService the server, as is now the standard practice	2009-02-20 21:57:55 -07:00
Brian Warner	1077826357	BucketCountingCrawler: rename status and state keys to use 'bucket' instead of 'share', because the former is more accurate	2009-02-20 21:46:06 -07:00
Brian Warner	b3cd4952bd	storage: add bucket-counting share crawler, add its output (number of files+directories maintained by a storage server) and status to the webapi /storage page	2009-02-20 21:04:08 -07:00
Brian Warner	73e05bf967	crawler: add get_progress, clean up get_state	2009-02-20 18:27:43 -07:00
Brian Warner	2e45619844	web/storage: make sure we can handle platforms without os.statvfs too	2009-02-20 16:03:53 -07:00
Brian Warner	c6a061e600	crawler: provide for one-shot crawlers, which stop after their first full cycle, for share-upgraders and database-populaters	2009-02-20 15:19:11 -07:00
Brian Warner	00677ff9a5	web: add Storage status page, improve tests	2009-02-20 14:29:26 -07:00
Brian Warner	c6c30ac5d4	webapi: pass client through constructor arguments, remove IClient, should make it easier to test web renderers in isolation	2009-02-20 12:15:54 -07:00
Brian Warner	e357b8a679	test/no_network: do startService on the storage servers, make it easier to customize the storage servers	2009-02-19 20:22:54 -07:00
Brian Warner	ef4ff21ae7	crawler: modify API to support upcoming bucket-counting crawler	2009-02-19 19:31:42 -07:00
Brian Warner	9bc08158c6	test_backupdb: make the not-a-database file larger, since the older sqlite-2.3.2 on OS-X is easily fooled	2009-02-19 18:04:09 -07:00
Brian Warner	2f751332e8	test_crawler: hush pyflakes	2009-02-19 14:23:40 -07:00
Brian Warner	ff9811211d	test_crawler: disable the percentage-of-cpu-used test, since it is too unreliable on our slow buildslaves. But leave the code in place for developers to run by hand.	2009-02-19 14:16:54 -07:00
Brian Warner	790a10d1b2	reliability.py: fix the numpy conversion, it was completely broken. Thanks to Terrell Russell for the help.	2009-02-19 13:55:15 -07:00
Brian Warner	268cd538e2	reliability: switch to NumPy, since Numeric is deprecated	2009-02-19 01:44:35 -07:00
Brian Warner	f10aa59e5b	test_crawler: don't require >=1 cycle on cygwin	2009-02-19 00:58:18 -07:00
Brian Warner	b949ea4f32	crawler: fix problems on windows and our slow cygwin slave	2009-02-18 22:24:31 -07:00
Brian Warner	193889f793	#633: first version of a rate-limited interruptable share-crawler	2009-02-18 21:46:33 -07:00
Brian Warner	4646451de6	change StorageServer to take nodeid in the constructor, instead of assigning it later, since it's cleaner and because the original problem (Tubs not being ready until later) went away	2009-02-18 16:23:01 -07:00
Brian Warner	b398a69f14	test_system: split off checker tests to test_deepcheck.py, this file is too big	2009-02-18 15:42:34 -07:00
Brian Warner	ef53da2b12	break storage.py into smaller pieces in storage/*.py . No behavioral changes.	2009-02-18 14:46:55 -07:00
Brian Warner	bce4a5385b	add --add-lease to 'tahoe check', 'tahoe deep-check', and webapi.	2009-02-17 19:32:43 -07:00
Brian Warner	e9563ebc02	change RIStorageServer.remote_add_lease to exit silently in case of no-such-bucket, instead of raising IndexError, because that makes the upcoming --add-lease feature faster and less noisy	2009-02-17 19:30:53 -07:00
Brian Warner	fde2289e7b	CLI #590: convert 'tahoe deep-check' to streaming form, improve display, add tests	2009-02-17 17:15:11 -07:00
Brian Warner	d52eadfc58	test_backupdb: improve error messages if the test fails	2009-02-17 11:08:38 -07:00
Brian Warner	476a5c8fac	webapi #590: add streaming deep-check. Still need a CLI tool to use it.	2009-02-16 23:35:53 -07:00
Brian Warner	8579e25059	test_web.Grid: change the CHECK() function to make it easier to test t= values with hyphens in them	2009-02-16 23:00:34 -07:00
Brian Warner	522641bb32	test_web: improve checker-results coverage with a no-network -based test, enhance no-network harness to assist, fix some bugs in web/check_results.py that were exposed	2009-02-16 22:12:42 -07:00
Brian Warner	f2f8ede11f	test/no_network: update comments with setup timing: no_network takes 50ms, SystemTestMixin takes 2s (on my laptop)	2009-02-16 18:06:43 -07:00
Brian Warner	040cb39613	test_upload: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary	2009-02-16 17:44:57 -07:00
Brian Warner	911abcc34b	test_download: rewrite in terms of no-network GridTestMixin, improve no_network.py as necessary	2009-02-16 17:36:58 -07:00
Brian Warner	e9427ea777	test_dirnode.py: convert Deleter to new no-network gridtest	2009-02-16 17:23:48 -07:00
Brian Warner	267d91f797	test_cli.py: modify to use the new 'no-network' gridtest instead of SystemTestMixin, which speeds it up from 73s to 43s on my system	2009-02-16 17:20:05 -07:00
Brian Warner	191fd8556a	tests: fix no_network framework to work with upload/download and checker	2009-02-16 17:19:47 -07:00
Brian Warner	770f976a85	test_cli: add test coverage for help strings	2009-02-16 15:08:33 -07:00
Brian Warner	ebe0f2ac08	test/no_network: new test harness, like system-test but doesn't use the network so it's faster	2009-02-16 14:58:44 -07:00
Brian Warner	df90dd8e73	test_cli: improve test coverage slightly	2009-02-15 21:04:51 -07:00
Brian Warner	3235b9630b	test_util: get almost full test coverage of dictutil, starting with the original pyutil tests as a base. The remaining three uncovered lines involve funny cases of ValueOrderedDict that I can't figure out how to get at	2009-02-15 20:32:10 -07:00
Brian Warner	cc27b218f5	provisioning/reliability: add tests, hush pyflakes, remove dead code, fix web links	2009-02-15 16:24:51 -07:00
Brian Warner	e1380b132b	util/statistics: add tests, fix mean_repair_cost	2009-02-15 16:23:26 -07:00
Brian Warner	812f23585d	test_repairer: hush pyflakes	2009-02-15 16:23:10 -07:00
Shawn Willden	21832280da	Statistics module ... Added a statistics module for calculating various facets of share survival statistics.	2009-01-13 20:12:35 -07:00
Brian Warner	60b06b8d32	test_dirnode: #625 run deep-check on a readonly dirnode too	2009-02-13 14:53:37 -07:00
Zooko O'Whielacronx	2d45d192d1	immutable repairer: add tests of how it handles short reads	2009-02-12 16:48:53 -07:00
Zooko O'Whielacronx	8303f90da0	nodeadmin: node stops itself if a hotline file hasn't been touched in 60 seconds now, instead of in 40 seconds ... A test failed on draco (MacPPC) because it took 49 seconds to get around to running the test, and the node had already stopped itself when the hotline file was 40 seconds old.	2009-02-12 15:21:31 -07:00
Zooko O'Whielacronx	76d7cc4404	immutable repairer: errback any pending readers of DownUpConnectorwhen it runs out of bytes, and test that fact	2009-02-11 20:11:29 -07:00
Zooko O'Whielacronx	7eb260a9cf	versioning: include an "appname" in the application version string in the versioning protocol, and make that appname be controlled by setup.py ... It is currently hardcoded in setup.py to be 'allmydata-tahoe'. Ticket #556 is to make it configurable by a runtime command-line argument to setup.py: "--appname=foo", but I suddenly wondered if we really wanted that and at the same time realized that we don't need that for tahoe-1.3.0 release, so this patch just hardcodes it in setup.py. setup.py inspects a file named 'src/allmydata/_appname.py' and assert that it contains the string "__appname__ = 'allmydata-tahoe'", and creates it if it isn't already present. src/allmydata/__init__.py import _appname and reads __appname__ from it. The rest of the Python code imports allmydata and inspects "allmydata.__appname__", although actually every use it uses "allmydata.__full_version__" instead, where "allmydata.__full_version__" is created in src/allmydata/__init__.py to be: __full_version__ = __appname + '-' + str(__version__). All the code that emits an "application version string" when describing what version of a protocol it supports (introducer server, storage server, upload helper), or when describing itself in general (introducer client), usese allmydata.__full_version__. This fixes ticket #556 at least well enough for tahoe-1.3.0 release.	2009-02-11 17:18:16 -07:00
Brian Warner	5f288842d5	test_repairer: disable repair-from-corruption tests until other things are improved well enough to make it useful	2009-02-11 15:01:59 -07:00
Zooko O'Whielacronx	079c1851a9	trivial: whitespace	2009-02-11 11:19:35 -07:00
Brian Warner	7d907f35aa	storage: make add-lease work, change default ownernum=1 since 0 is reserved to mean 'no lease here'	2009-02-10 23:39:38 -07:00
Brian Warner	13a3ef5ec1	#620: storage: allow mutable shares to be deleted, with a writev where new_length=0	2009-02-10 23:37:56 -07:00
Brian Warner	6a5f28f47d	test_cli.Backup: insert some stalls to make sure two successive backups get distinct timestamps, avoiding intermittent failures	2009-02-10 20:37:09 -07:00
Brian Warner	f7a263eb0b	#619: make 'tahoe backup' complain and refuse to run if sqlite is unavailable and --no-backupdb is not passed	2009-02-10 18:49:10 -07:00
Brian Warner	3b1c8ceb86	tests: increase the default timeout for SystemTestMixin -based tests to 300 seconds, since our slower buildslaves sometimes take longer than the default 120s	2009-02-10 14:44:12 -07:00
Zooko O'Whielacronx	ef1bfdd2bf	immutable: repairer: add a simple test to exercise the "leftover" code path, fix the bug (and rename the variable "leftover" to "extra")	2009-02-10 12:12:45 -07:00
Zooko O'Whielacronx	75e4e67ed7	immutable: tighten preconditions -- you can write empty strings or read zero bytes, and add the first simple unit test of DownUpConnector	2009-02-10 00:56:47 -07:00
Zooko O'Whielacronx	d436c6235d	immutable: tests: the real WRITE_LEEWAY is 35 (it was a mistake to move it from 10 to 35 earlier -- I had seen a failure in which it took 35 times as many writes as I thought were optimal, but I misread and thought it took only 20 times as many)	2009-02-09 23:53:48 -07:00
Zooko O'Whielacronx	65c12b24b8	immutable: tests: sigh, raise, again the limit of how many extra writes you can do and still pass this test ... Obviously requiring the code under test to perform within some limit isn't very meaningful if we raise the limit whenever the test goes outside of it. But I still don't want to remove the test code which measures how many writes (and, elsewhere, how many reads) a client does in order to fulfill these duties. Let this number -- now 20 -- stand as an approximation of the inefficiency of our code divided by my mental model of how many operations are actually optimal for these duties.	2009-02-09 20:09:31 -07:00
Zooko O'Whielacronx	5738d94ccd	immutable: tests: assert that verifier gives a clean bill of health after corruption and repair (the previous patch mistakenly did this only after deletion and repair), and also test whether deleting seven other shares and then downloading works. Also count the number of shares stored in the local filesystem.	2009-02-09 20:08:41 -07:00
Zooko O'Whielacronx	bbf48f4a88	immutable: test: add a test after attempting to repair from corruption: does a full verify run give the file a clean bill of health? If not, the you haven't successfully repaired it. ... This will make the repairer tests more consistent -- less accidentally passing due to getting lucky.	2009-02-09 19:01:49 -07:00
Zooko O'Whielacronx	4091f73f34	immutable: tests: put shares back to their pristine condition in between each test of corrupting-and-repairing them ... This is important, because if the repairer doesn't completely repair all kinds of corruption (as the current one doesn't), then the successive tests get messed up by assuming that the shares were uncorrupted when the test first set about to corrupt them.	2009-02-09 18:29:56 -07:00
Brian Warner	b197c5bb8c	test_runner: skip all spawnProcess-using tests on cygwin, since spawnProcess just hangs forever	2009-02-09 02:34:00 -07:00
Brian Warner	19902a0f16	test_runner.py: revert the CreateNode section to using runner() inline, rather than spawning a process, to get more precise coverage	2009-02-09 02:26:17 -07:00
Brian Warner	89683a3906	test_upload: add test of maximum-immutable-share-size, to complete the last item of #538	2009-02-08 19:41:27 -07:00
Brian Warner	d8e00c37b5	test_cli.Backup: capture stderr when sqlite is unavailable	2009-02-07 15:14:40 -07:00
Brian Warner	213ef3d64b	test_repairer: wrap comments to 80cols, my laptop does not have a wide screen. No functional changes.	2009-02-07 14:06:26 -07:00
Brian Warner	50886ea494	test/common.py: in share-layout-reading code, use '>L' consistently, since '>l' doesn't specify the signedness and the windows tests appear to be failing with an endianness-like problem (version==0x01000000). Also use binary mode when editing sharefiles	2009-02-07 13:38:17 -07:00
Brian Warner	1fd8f023ab	test_cli: increase timeout on test_backup, since our dapper buildslave is really slow	2009-02-06 02:17:53 -07:00
Brian Warner	9933aa0d49	test_backupdb.py: reset the check-timers after one step, otherwise a slow host can false-fail	2009-02-06 01:33:05 -07:00
Brian Warner	814fec5495	test_cli.backup: oops, fix test to work even when sqlite is unavailable	2009-02-05 22:10:42 -07:00
Brian Warner	ca32db5b39	#598: add cli+backupdb tests, improve user display, update docs, move docs out of proposed/	2009-02-05 22:07:01 -07:00
Brian Warner	4be2e11d11	add sqlite-based backupdb, for #598 and others (including 'tahoe cp'). Not enabled yet.	2009-02-05 18:17:56 -07:00
Brian Warner	4e572a5f24	webapi: add verifycap (spelled 'verify_url') to the t=json output on files and directories. Closes #559.	2009-02-03 19:22:48 -07:00
Brian Warner	cfce8b5eab	#598: first cut of 'tahoe backup' command: no backupdb, but yes shared-unchanged-directories and Archives/TIMESTAMP and Latest/	2009-02-02 21:09:02 -07:00
Zooko O'Whielacronx	e08e8d3f41	storage: disable test_large_share again: my linux laptop has less than 4 GiB free	2009-01-30 22:16:49 -07:00
Zooko O'Whielacronx	f0682cf43e	setup: more verbose assertion failure in test_runner	2009-01-29 10:49:06 -07:00
Zooko O'Whielacronx	204629be43	storage: enable the test of a share larger than 2 GiB; this will probably be too expensive on Mac OS X, but I think it won't be on all the other platforms ; I will probably set it to SkipTest if the sys.platform is Mac after seeing the results of this buildbot run	2009-01-28 16:33:12 -07:00
Zooko O'Whielacronx	8d7fe2adb3	trivial: remove unused imports noticed by pyflakes	2009-01-27 15:11:48 -07:00
Zooko O'Whielacronx	64d12a9402	setup: fix test_runner to invoke bin/tahoe.exe instead of bin/tahoe if on Windows (this is what happens when a user invokes bin/tahoe on Windows)	2009-01-27 14:37:17 -07:00
Zooko O'Whielacronx	1ae08acc8f	setup: fix test_runner to assert that lines which are output to stderr must end with a punctuation mark (apparently re.search("x$", "x\r\n") does not match. :-()	2009-01-27 14:35:05 -07:00
Zooko O'Whielacronx	3cd4722c9b	setup: fix test_runner.RunNode.test_baddir -- it was left incomplete and broken by a recent patch	2009-01-27 14:32:45 -07:00
Zooko O'Whielacronx	688bb7a453	setup: go ahead and check for noise in test_client_no_noise	2009-01-26 17:46:16 -07:00
Zooko O'Whielacronx	8d2a3e78ca	setup: add a test for a warning message from importing nevow, marked as TODO, comment-out the assertion of no-noise inside other test_runner tests	2009-01-26 17:30:46 -07:00
Zooko O'Whielacronx	3e4450cf3f	setup: find a "bin/tahoe" executable to test based on allmydata.__file__ instead of based on the CWD ... This means that the tests still work if you are executing them from a CWD other than the src dir -- *if* the "bin/tahoe" is found at os.path.dirname(os.path.dirname(os.path.dirname(allmydata.__file__))). If no file is found at that location, then just skip the tests of executing the "tahoe" executable, because we don't want to accidentally run those tests against an executable from a different version of tahoe.	2009-01-23 18:34:37 -07:00
Brian Warner	1de4110b35	#509: remove non-streaming 'tahoe manifest' CLI form	2009-01-23 17:00:02 -07:00
Brian Warner	6a04103cc2	#509 CLI: add 'tahoe manifest --stream'	2009-01-23 16:33:21 -07:00
Brian Warner	5a0e3f3a40	#509: test_system.py: add test for streamed-manifest	2009-01-23 16:32:47 -07:00
Brian Warner	9ade197c3b	test_system: rearrange DeepCheckWebGood to make it easier to add CLI tests	2009-01-23 16:13:06 -07:00
Brian Warner	26260374e9	#590: add webish t=stream-manifest	2009-01-22 22:01:36 -07:00
Brian Warner	72adeccf2d	dirnode: add get_repair_cap()	2009-01-22 21:44:49 -07:00
Brian Warner	d8b3505cf5	filenode: add get_repair_cap(), which uses the read-write filecap for immutable files, and the verifycap for immutable files	2009-01-22 21:38:36 -07:00
Zooko O'Whielacronx	b77c89ac60	setup: add test that the tests are testing the right source code ... This is a test of #145, and I think that now the code passes this test.	2009-01-22 15:52:40 -07:00
Zooko O'Whielacronx	967102e1b9	trivial: removed unused imports noticed by pyflakes	2009-01-22 15:52:13 -07:00
Zooko O'Whielacronx	210da5cd9a	setup: change test_runner to invoke "bin/tahoe" in a subprocess instead of executing runner.runner() ... This is necessary because loading allmydata code now depends on PYTHONPATH manipulation which is done in the "bin/tahoe" script. Unfortunately it makes test_runner slower since it launches and waits for many subprocesses.	2009-01-22 15:38:18 -07:00
Zooko O'Whielacronx	7d91dc5efe	rollback the feature of making "ambient upload authority" configurable ... This reverses some, but not all, of the changes that were committed in the following set of patches. rolling back: Sun Jan 18 09:54:30 MST 2009 toby.murray * add 'web.ambient_upload_authority' as a paramater to tahoe.cfg M ./src/allmydata/client.py -1 +3 M ./src/allmydata/test/common.py -7 +9 A ./src/allmydata/test/test_ambient_upload_authority.py M ./src/allmydata/web/root.py +12 M ./src/allmydata/webish.py -1 +4 Sun Jan 18 09:56:08 MST 2009 zooko@zooko.com * trivial: whitespace I ran emacs's "M-x whitespace-cleanup" on the files that Toby's recent patch had touched that had trailing whitespace on some lines. M ./src/allmydata/test/test_ambient_upload_authority.py -9 +8 M ./src/allmydata/web/root.py -2 +1 M ./src/allmydata/webish.py -2 +1 Mon Jan 19 14:16:19 MST 2009 zooko@zooko.com * trivial: remove unused import noticed by pyflakes M ./src/allmydata/test/test_ambient_upload_authority.py -1 Mon Jan 19 21:38:35 MST 2009 toby.murray * doc: describe web.ambient_upload_authority M ./docs/configuration.txt +14 M ./docs/frontends/webapi.txt +11 Mon Jan 19 21:38:57 MST 2009 zooko@zooko.com * doc: add Toby Murray to the CREDITS M ./CREDITS +4	2009-01-20 20:47:35 -07:00
Zooko O'Whielacronx	6bbbb375e1	setup: fix test_system to require tahoe under its package==distribution name "allmydata-tahoe" instead of its module name "allmydata"	2009-01-20 12:38:09 -07:00
Zooko O'Whielacronx	845b21a5ad	trivial: remove unused import noticed by pyflakes	2009-01-19 15:16:19 -07:00
Zooko O'Whielacronx	4771e1c9d5	trivial: whitespace ... Ran "M-x whitespace-cleanup" on files that Toby's recent patch touched, even though they didn't have trailing whitespace.	2009-01-18 10:58:15 -07:00
Zooko O'Whielacronx	ce7c448bc0	trivial: whitespace ... I ran emacs's "M-x whitespace-cleanup" on the files that Toby's recent patch had touched that had trailing whitespace on some lines.	2009-01-18 10:56:08 -07:00
toby.murray	66f83c7356	add 'web.ambient_upload_authority' as a paramater to tahoe.cfg	2009-01-18 10:54:30 -07:00
Zooko O'Whielacronx	f9ee60a88e	cli: tests: skip symlink test if there is no os.symlink	2009-01-14 18:10:10 -07:00
Brian Warner	2fe099a0b3	mutable: move recent operation history management code (MutableWatcher) into history.py, have History provide stats	2009-01-14 17:36:20 -07:00
Brian Warner	cc50e2f4aa	upload: use WriteBucketProxy_v2 when uploading a large file (with shares larger than 4GiB). This finally closes #346. I think we can now handle immutable files up to 48EiB.	2009-01-12 20:14:42 -07:00
Brian Warner	8f2ec16b2c	test_repairer.py: hush pyflakes: remove duplicate/shadowed function name, by using the earlier definition (which is identical)	2009-01-12 15:45:09 -07:00
Brian Warner	fe362c0021	hush pyflakes by removing unused imports	2009-01-12 15:41:20 -07:00
Zooko O'Whielacronx	25063688b4	immutable repairer ... This implements an immutable repairer by marrying a CiphertextDownloader to a CHKUploader. It extends the IDownloadTarget interface so that the downloader can provide some metadata that the uploader requires. The processing is incremental -- it uploads the first segments before it finishes downloading the whole file. This is necessary so that you can repair large files without running out of RAM or using a temporary file on the repairer. It requires only a verifycap, not a readcap. That is: it doesn't need or use the decryption key, only the integrity check codes. There are several tests marked TODO and several instances of XXX in the source code. I intend to open tickets to document further improvements to functionality and testing, but the current version is probably good enough for Tahoe-1.3.0.	2009-01-12 11:00:22 -07:00
Zooko O'Whielacronx	8dd3b3185f	immutable: separate tests of immutable upload/download from tests of immutable checking/repair	2009-01-10 15:07:39 -07:00
Zooko O'Whielacronx	ef60e85ec6	naming: finish renaming "CheckerResults" to "CheckResults"	2009-01-09 18:00:52 -07:00
Brian Warner	45fc3819a2	test_helper: hush pyflakes by avoiding use of 'uri' as a variable, since it shadows an import of the same name	2009-01-08 20:59:41 -07:00
Zooko O'Whielacronx	ade6a4fa74	immutable: add a monitor API to CiphertextDownloader with which to tell it to stop its work	2009-01-08 14:42:15 -07:00
Zooko O'Whielacronx	157e365d2b	naming: Rename a few things which I touched or changed in the recent patch to download-without-decrypting. ... Rename "downloadable" to "target". Rename "u" to "v" in FileDownloader.__init__(). Rename "_uri" to "_verifycap" in FileDownloader. Rename "_downloadable" to "_target" in FileDownloader. Rename "FileDownloader" to "CiphertextDownloader".	2009-01-08 12:13:07 -07:00
Zooko O'Whielacronx	600196f571	immutable: refactor download to do only download-and-decode, not decryption ... FileDownloader takes a verify cap and produces ciphertext, instead of taking a read cap and producing plaintext. FileDownloader does all integrity checking including the mandatory ciphertext hash tree and the optional ciphertext flat hash, rather than expecting its target to do some of that checking. Rename immutable.download.Output to immutable.download.DecryptingOutput. An instance of DecryptingOutput can be passed to FileDownloader to use as the latter's target. Text pushed to the DecryptingOutput is decrypted and then pushed to *its* target. DecryptingOutput satisfies the IConsumer interface, and if its target also satisfies IConsumer, then it forwards and pause/unpause signals to its producer (which is the FileDownloader). This patch also changes some logging code to use the new logging mixin class. Check integrity of a segment and decrypt the segment one block-sized buffer at a time instead of copying the buffers together into one segment-sized buffer (reduces peak memory usage, I think, and is probably a tad faster/less CPU, depending on your encoding parameters). Refactor FileDownloader so that processing of segments and of tail-segment share as much code is possible. FileDownloader and FileNode take caps as instances of URI (Python objects), not as strings.	2009-01-08 11:53:49 -07:00
Zooko O'Whielacronx	9bba578776	trivial: tiny changes to test code	2009-01-08 11:20:48 -07:00
Zooko O'Whielacronx	ecabcc674c	immutable: Make more parts of download use logging mixins and know what their "parent msg id" is.	2009-01-08 11:25:30 -07:00
Larry Hosken	3267984fa9	'tahoe cp -r', upon encountering a dangling symlink, would assert out. ... This was somewhat sad; the assertion didn't say what path caused the error, what went wrong. So... silently skip over things that are neither dirs nor files.	2009-01-07 23:51:14 -07:00
Zooko O'Whielacronx	83b97ee79f	immutable: fix error in validation of ciphertext hash tree and add test for that code ... pyflakes pointed out to me that I had committed some code that is untested, since it uses an undefined name. This patch exercises that code -- the validation of the ciphertext hash tree -- by corrupting some of the share files in a very specific way, and also fixes the bug.	2009-01-07 23:40:12 -07:00
Zooko O'Whielacronx	a54eba1d80	util: deferredutil: add basic test for deferredutil.gatherResults ... Also I checked and Twisted 2.4.0 supports .subFailure and the other parts of the API that we require.	2009-01-07 08:13:42 -07:00
Zooko O'Whielacronx	c85f75bb08	immutable: refactor uploader to do just encoding-and-uploading, not encryption ... This makes Uploader take an EncryptedUploadable object instead of an Uploadable object. I also changed it to return a verify cap instead of a tuple of the bits of data that one finds in a verify cap. This will facilitate hooking together an Uploader and a Downloader to make a Repairer. Also move offloaded.py into src/allmydata/immutable/.	2009-01-06 21:48:22 -07:00
Zooko O'Whielacronx	81add135dc	trivial: whitespace and docstring tidyups	2009-01-06 21:41:04 -07:00
Zooko O'Whielacronx	5e6f90a015	rename "checker results" to "check results", because it is more parallel to "check-and-repair results"	2009-01-06 13:37:03 -07:00
Zooko O'Whielacronx	4ada923e2b	immutable: tests: verifier doesn't always catch corrupted share hashes ... Maybe it already got one of the corrupted hashes from a different server and it doesn't double-check that the hash from every server is correct. Or another problem. But in any case I'm marking this as TODO because an even better (more picky) verifier is less urgent than repairer.	2009-01-06 13:04:49 -07:00
Zooko O'Whielacronx	6a12f316a4	immutable: new checker and verifier ... New checker and verifier use the new download class. They are robust against various sorts of failures or corruption. They return detailed results explaining what they learned about your immutable files. Some grotesque sorts of corruption are not properly handled yet, and those ones are marked as TODO or commented-out in the unit tests. There is also a repairer module in this patch with the beginnings of a repairer in it. That repairer is mostly just the interface to the outside world -- the core operation of actually reconstructing the missing data blocks and uploading them is not in there yet. This patch also refactors the unit tests in test_immutable so that the handling of each kind of corruption is reported as passing or failing separately, can be separately TODO'ified, etc. The unit tests are also improved in various ways to require more of the code under test or to stop requiring unreasonable things of it. :-)	2009-01-05 18:28:18 -07:00
Zooko O'Whielacronx	4921a9f243	trivial: fix inline comment in test code	2009-01-05 17:53:42 -07:00
Zooko O'Whielacronx	67d72aadd3	immutable: skip the test of large files, because that is too hard on the host if it doesn't efficiently handle sparse files	2009-01-05 17:07:27 -07:00
Zooko O'Whielacronx	98b28c1d5e	immutable: stop reading past the end of the sharefile in the process of optimizing download -- Tahoe storage servers < 1.3.0 return an error if you read past the end of the share file	2009-01-05 13:40:57 -07:00
Zooko O'Whielacronx	778167c2b1	immutable: refactor downloader to be more reusable for checker/verifier/repairer (and better) ... The code for validating the share hash tree and the block hash tree has been rewritten to make sure it handles all cases, to share metadata about the file (such as the share hash tree, block hash trees, and UEB) among different share downloads, and not to require hashes to be stored on the server unnecessarily, such as the roots of the block hash trees (not needed since they are also the leaves of the share hash tree), and the root of the share hash tree (not needed since it is also included in the UEB). It also passes the latest tests including handling corrupted shares well. ValidatedReadBucketProxy takes a share_hash_tree argument to its constructor, which is a reference to a share hash tree shared by all ValidatedReadBucketProxies for that immutable file download. ValidatedReadBucketProxy requires the block_size and share_size to be provided in its constructor, and it then uses those to compute the offsets and lengths of blocks when it needs them, instead of reading those values out of the share. The user of ValidatedReadBucketProxy therefore has to have first used a ValidatedExtendedURIProxy to compute those two values from the validated contents of the URI. This is pleasingly simplifies safety analysis: the client knows which span of bytes corresponds to a given block from the validated URI data, rather than from the unvalidated data stored on the storage server. It also simplifies unit testing of verifier/repairer, because now it doesn't care about the contents of the "share size" and "block size" fields in the share. It does not relieve the need for share data v2 layout, because we still need to store and retrieve the offsets of the fields which come after the share data, therefore we still need to use share data v2 with its 8-byte fields if we want to store share data larger than about 2^32. Specify which subset of the block hashes and share hashes you need while downloading a particular share. In the future this will hopefully be used to fetch only a subset, for network efficiency, but currently all of them are fetched, regardless of which subset you specify. ReadBucketProxy hides the question of whether it has "started" or not (sent a request to the server to get metadata) from its user. Download is optimized to do as few roundtrips and as few requests as possible, hopefully speeding up download a bit.	2009-01-05 09:51:45 -07:00
Zooko O'Whielacronx	de4269a643	immutable: mark a failing download test as "todo", because I think it is revealing a limitation of the current downloader's handling of corrupted shares	2009-01-03 13:00:03 -07:00
Zooko O'Whielacronx	5954ab456d	immutable: fix test for truncated reads of URI extension block size	2009-01-03 11:44:27 -07:00
Zooko O'Whielacronx	7adf905b9f	immutable: further loosen the performance-regression test to allow up to 45 reads ... This does raise the question of if there is any point to this test, since I apparently don't know what the answer *should* be, and whenever one of the buildbots fails then I redefine success. But, I'm about to commit a bunch of patches to implement checker, verifier, and repairer as well as to refactor downloader, and I would really like to know if these patches *increase* the number of reads required even higher than it currently is.	2009-01-03 11:41:09 -07:00
Zooko O'Whielacronx	5a21638853	immutable: whoops, it actually takes up to 39 reads sometimes to download a corrupted file	2009-01-02 17:43:02 -07:00
Zooko O'Whielacronx	e26cec2502	immutable: add more detailed tests of download, including testing the count of how many reads different sorts of downloads take	2009-01-02 16:54:59 -07:00
Zooko O'Whielacronx	cc70c163ba	trivial: a few improvements to in-line doc and code, and renaming of test/test_immutable_checker.py to test/test_immutable.py ... That file currently tests checker and verifier and repairer, and will soon also test downloader.	2009-01-02 16:49:41 -07:00
Zooko O'Whielacronx	63efa1665b	immutable: make the test of large files more likely to work by requesting to allocate space for only one huge share, not three	2008-12-31 15:59:42 -07:00
Zooko O'Whielacronx	b0b9e8c1da	trivial: "M-x whitespace-cleanup", and also remove an unused variable	2008-12-31 15:42:33 -07:00
Zooko O'Whielacronx	6c4019ec33	immutable: storage servers accept any size shares now ... Nathan Wilcox observed that the storage server can rely on the size of the share file combined with the count of leases to unambiguously identify the location of the leases. This means that it can hold any size share data, even though the field nominally used to hold the size of the share data is only 32 bits wide. With this patch, the storage server still writes the "size of the share data" field (just in case the server gets downgraded to an earlier version which requires that field, or the share file gets moved to another server which is of an earlier vintage), but it doesn't use it. Also, with this patch, the server no longer rejects requests to write shares which are >= 2^32 bytes in size, and it no longer rejects attempts to read such shares. This fixes http://allmydata.org/trac/tahoe/ticket/346 (increase share-size field to 8 bytes, remove 12GiB filesize limit), although there remains open a question of how clients know that a given server can handle large shares (by using the new versioning scheme, probably). Note that share size is also limited by another factor -- how big of a file we can store on the local filesystem on the server. Currently allmydata.com typically uses ext3 and I think we typically have block size = 4 KiB, which means that the largest file is about 2 TiB. Also, the hard drives themselves are only 1 TB, so the largest share is definitely slightly less than 1 TB, which means (when K == 3), the largest file is less than 3 TB. This patch also refactors the creation of new sharefiles so that only a single fopen() is used. This patch also helps with the unit-testing of repairer, since formerly it was unclear what repairer should expect to find if the "share data size" field was corrupted (some corruptions would have no effect, others would cause failure to download). Now it is clear that repairer is not required to notice if this field is corrupted since it has no effect on download. :-)	2008-12-31 15:42:26 -07:00
Zooko O'Whielacronx	3a47031a51	immutable: more detailed tests for checker/verifier/repairer ... There are a lot of different ways that a share could be corrupted, or that attempting to download it might fail. These tests attempt to exercise many of those ways and require the checker/verifier/repairer to handle each kind of failure well.	2008-12-31 14:18:38 -07:00
Zooko O'Whielacronx	872e4fc84d	doc: sundry amendments to docs and in-line code comments	2008-12-28 16:59:54 -07:00
Zooko O'Whielacronx	9f117dbe8f	cli: mark unicode filenames as unsupported -- see #534 for details	2008-12-24 13:28:02 -07:00
Zooko O'Whielacronx	25d962c67d	test: extend timeout on the hotline file that prevents the client from stopping itself ... The 20-second timeout was apparently tripped on my Powerbook G4 "draco".	2008-12-21 21:06:29 -07:00
Zooko O'Whielacronx	f1fbd4feae	dirnode: don't check MAC on entries in dirnodes ... In an ancient version of directories, we needed a MAC on each entry. In modern times, the entire dirnode comes with a digital signature, so the MAC on each entry is redundant. With this patch, we no longer check those MACs when reading directories, but we still produce them so that older readers will accept directories that we write.	2008-12-21 17:35:18 -07:00
Zooko O'Whielacronx	8b7ce325d7	immutable, checker, and tests: improve docstrings, assertions, tests ... No functional changes, but remove unused code, improve or fix docstrings, etc.	2008-12-21 15:07:52 -07:00
Zooko O'Whielacronx	471e1f1b9b	try to tidy up uri-as-string vs. uri-as-object ... I get confused about whether a given argument or return value is a uri-as-string or uri-as-object. This patch adds a lot of assertions that it is one or the other, and also changes CheckerResults to take objects not strings. In the future, I hope that we generally use Python objects except when importing into or exporting from the Python interpreter e.g. over the wire, the UI, or a stored file.	2008-12-19 08:39:24 -07:00
Zooko O'Whielacronx	7b285ebcb1	immutable: remove the last bits of code (only test code or unused code) which did something with plaintext hashes or plaintext hash trees	2008-12-19 08:18:07 -07:00
Zooko O'Whielacronx	82ee44ed5b	debug: pass empty optional arguments to ReadBucketProxy ... because those arguments are about to become non-optional (for other code than test/debug code)	2008-12-16 17:51:45 -07:00
Brian Warner	278c47b9bd	mutable publish: if we are surprised by shares that match what we would have written anyways, don't be surprised. This should fix one of the two #546 problems, in which we re-use a server and forget that we already sent them a share.	2008-12-09 22:44:49 -07:00
Zooko O'Whielacronx	c456ff8591	rename "get_verifier()" to "get_verify_cap()"	2008-12-08 12:44:11 -07:00
Zooko O'Whielacronx	16b20794dd	mutable: merge renaming with test patches	2008-12-07 08:45:19 -07:00
Zooko O'Whielacronx	b58875fe43	mutable: rename mutable/node.py to mutable/filenode.py and mutable/repair.py to mutable/repairer.py ... To be more consistent with the immutable layout that I am working on.	2008-12-07 08:20:08 -07:00
francois	db7ad6da12	filenode.py: Fix partial HTTP Range header handling according to RFC2616 ... Tahoe webapi was failing on HTTP request containing a partial Range header. This change allows movies players like mplayer to seek in movie files stored in tahoe. Associated tests for GET and HEAD methods are also included	2008-11-18 07:41:35 -07:00
Brian Warner	ffb5985146	mutable.modify(): after UCWE, publish even if the second invocation of the modifier didn't modify anything. For #551.	2008-12-05 22:49:23 -07:00
Brian Warner	7a0afb59a4	dirnode.py: dirnode.delete which hits UCWE should not fail with NoSuchChildError. Fixes #550.	2008-12-05 22:08:37 -07:00
Brian Warner	fb9af2c7a0	MutableFileNode.modify: pass first_time= and servermap= to the modifier callback	2008-12-05 22:07:10 -07:00
Zooko O'Whielacronx	60bbc46a53	minor: fix unused imports -- thanks, pyflakes	2008-12-05 13:07:23 -07:00
Zooko O'Whielacronx	b315619d6b	download: refactor handling of URI Extension Block and crypttext hash tree, simplify things ... Refactor into a class the logic of asking each server in turn until one of them gives an answer that validates. It is called ValidatedThingObtainer. Refactor the downloading and verification of the URI Extension Block into a class named ValidatedExtendedURIProxy. The new logic of validating UEBs is minimalist: it doesn't require the UEB to contain any unncessary information, but of course it still accepts such information for backwards compatibility (so that this new download code is able to download files uploaded with old, and for that matter with current, upload code). The new logic of validating UEBs follows the practice of doing all validation up front. This practice advises one to isolate the validation of incoming data into one place, so that all of the rest of the code can assume only valid data. If any redundant information is present in the UEB+URI, the new code cross-checks and asserts that it is all fully consistent. This closes some issues where the uploader could have uploaded inconsistent redundant data, which would probably have caused the old downloader to simply reject that download after getting a Python exception, but perhaps could have caused greater harm to the old downloader. I removed the notion of selecting an erasure codec from codec.py based on the string that was passed in the UEB. Currently "crs" is the only such string that works, so "_assert(codec_name == 'crs')" is simpler and more explicit. This is also in keeping with the "validate up front" strategy -- now if someone sets a different string than "crs" in their UEB, the downloader will reject the download in the "validate this UEB" function instead of in a separate "select the codec instance" function. I removed the code to check plaintext hashes and plaintext Merkle Trees. Uploaders do not produce this information any more (since it potentially exposes confidential information about the file), and the unit tests for it were disabled. The downloader before this patch would check that plaintext hash or plaintext merkle tree if they were present, but not complain if they were absent. The new downloader in this patch complains if they are present and doesn't check them. (We might in the future re-introduce such hashes over the plaintext, but encrypt the hashes which are stored in the UEB to preserve confidentiality. This would be a double- check on the correctness of our own source code -- the current Merkle Tree over the ciphertext is already sufficient to guarantee the integrity of the download unless there is a bug in our Merkle Tree or AES implementation.) This patch increases the lines-of-code count by 8 (from 17,770 to 17,778), and reduces the uncovered-by-tests lines-of-code count by 24 (from 1408 to 1384). Those numbers would be more meaningful if we omitted src/allmydata/util/ from the test-coverage statistics.	2008-12-05 08:17:54 -07:00
Brian Warner	a0b5b5ab2b	test_web: add get_permuted_peers, to unbreak recent checker_results change	2008-12-05 02:12:10 -07:00
Brian Warner	e942ab141b	test_system.py: assert less about the stats we get, since shares (and thus allocate() calls) are distributed randomly	2008-12-04 17:27:04 -07:00
Brian Warner	7c4edac582	stats: don't return booleans: it violates the schema. Add a test.	2008-12-04 15:01:24 -07:00
Brian Warner	7cfc74bcc9	test_system.py: don't ask the stats-gatherer to poll: it tolerates failures, so it isn't really giving us enough test coverage. Removing the call will make it more clear that we need to improve the tests later	2008-12-04 15:00:53 -07:00
Brian Warner	a2ce315074	CLI: check for pre-existing aliases in 'tahoe create-alias' and 'tahoe add-alias'	2008-12-02 20:20:22 -07:00
Brian Warner	cdbb72d5ff	test_cli: pass rc out of do_cli() too	2008-12-02 20:08:28 -07:00
Brian Warner	6958b7fa90	test_storage.py: more windows-vs-readonly-storage fixes	2008-12-02 19:41:02 -07:00
Brian Warner	d3783475b4	test_download.py: remove extra base32 import	2008-12-02 18:31:26 -07:00
Brian Warner	b997e6ca7c	test_download: test both mutable and immutable pre-generated shares	2008-12-02 18:30:07 -07:00
Brian Warner	2d70f7fc0b	test_download.py: added 'known-answer-tests', to make sure current code can download a file that was created by earlier code	2008-12-02 18:22:08 -07:00
Brian Warner	71c62f8fcb	#542 'tahoe create-key-generator': fix the .tac file this creates to be compatible with modern code, add a test	2008-12-01 17:47:21 -07:00
Brian Warner	cfba882b30	storage: replace sizelimit with reserved_space, make the stats 'disk_avail' number incorporate this reservation	2008-12-01 17:24:21 -07:00
Brian Warner	39a41f1d85	util/abbreviate: add abbreviated-size parser	2008-12-01 17:24:12 -07:00
Zooko O'Whielacronx	fe6abac87b	wui/wapi: change the default port number from 8123 to 3456 to avoid conflict with TorButton ... See ticket #536 for details.	2008-11-25 17:57:37 -07:00
Brian Warner	bc53c24003	dirnode manifest: add verifycaps, both to internal API and to webapi. This will give the manual-GC tools more to work with, so they can estimate how much space will be freed.	2008-11-24 14:40:46 -07:00
Brian Warner	0fab511be5	upload: don't use servers which can't support the share size we need. This ought to avoid #439 problems. Some day we'll have a storage server which advertises support for a larger share size. No tests yet.	2008-11-21 20:28:12 -07:00
Brian Warner	bf06492a90	#538: fetch version and attach to the rref. Make IntroducerClient demand v1 support.	2008-11-21 20:07:27 -07:00
Brian Warner	0eb6b324a4	#538: add remote_get_version() to four main Referenceable objects: Introducer Service, Storage Server, Helper, CHK Upload Helper. Remove unused storage-server get_versions().	2008-11-21 17:43:52 -07:00
Brian Warner	4ada26101b	test_dirnode: add an explainError call	2008-11-19 16:02:12 -07:00
Brian Warner	b84c2c6541	manifest: add storage-index strings to the json results	2008-11-19 16:00:27 -07:00
Brian Warner	815e0673e6	manifest: include stats in results. webapi is unchanged.	2008-11-19 15:03:47 -07:00
Brian Warner	6f6556da22	mutable: respect the new tahoe.cfg 'shares.needed' and 'shares.total' settings	2008-11-19 14:05:01 -07:00
Brian Warner	82fe7ba360	oops, update tests to match 'tahoe stats' change	2008-11-18 20:32:59 -07:00
Brian Warner	7391f63fc1	util/abbreviate: little utility to abbreviate seconds and bytes	2008-11-18 20:11:42 -07:00
Brian Warner	5c3e153d0e	cli: tahoe stats/manifest: change --verbose to --raw, since I want -v for --verify for check/deep-check/repair	2008-11-18 18:36:08 -07:00
Brian Warner	d657d22c0c	test_system: make 'where' strings more helpful, to track down test failures better	2008-11-18 18:29:50 -07:00
Brian Warner	7932fadb5e	webapi: add 'summary' string to checker results JSON	2008-11-18 18:28:26 -07:00
Brian Warner	994d97c644	webapi: introducer stats: add 'announcement_distinct_hosts' to the t=json form, to show how many distinct hosts are providing e.g. storage services	2008-11-18 15:30:15 -07:00
Brian Warner	8473a96ada	#330: convert stats-gatherer into a .tac file service, add 'tahoe create-stats-gatherer'	2008-11-18 01:46:20 -07:00
Brian Warner	cae54ab118	tahoe.cfg: add controls for k and N (and shares-of-happiness)	2008-11-18 00:29:44 -07:00
Brian Warner	ead0e4d6ca	cli: add tests for 'tahoe stats --verbose'	2008-11-17 22:11:14 -07:00
Brian Warner	d6a67cd566	dirnode manifest/stats: process more than one LIT file per tree; we were accidentally ignoring all but the first	2008-11-14 22:50:49 -07:00
francois	1a3a464534	test_cli.py: Ensure that we can read our uploaded files back	2008-11-14 07:44:58 -07:00
francois	32b5bc6330	test_cli.py: use str objects instead of unicode ones ... This will hopefully fix failing tests with LC_ALL=C	2008-11-14 07:41:37 -07:00
Brian Warner	5a60086dbc	CLI: add 'tahoe stats', to run start-deep-stats and print the results	2008-11-13 19:43:50 -07:00
Brian Warner	9c9994300e	test_system.py: fix new 'tahoe manifest' tests to not break on windows, by providing --node-directory instead of --node-url	2008-11-13 15:27:48 -07:00
francois	5c0c5bfc81	test for bug #534, unicode filenames ... This test assure that uploading a file whose name contains unicode character doesn't prevent further uploads in the same directory.	2008-11-13 05:19:51 -07:00
Brian Warner	0d93d6244e	CLI: add 'tahoe manifest', which takes a directory and returns a list of things you can reach from it	2008-11-12 20:17:25 -07:00
Brian Warner	9976bd439a	tahoe.cfg: add tub.location, to override the location hints we include in our FURL. This replaces advertised_ip_addresses, which doesn't remain useful enough to retain it. Helps with #517 (Tor).	2008-11-12 18:44:58 -07:00
Zooko O'Whielacronx	14444dbcc0	tests: fix comment	1970-01-05 04:10:55 -07:00
Zooko O'Whielacronx	3a0dc8e4ec	tests: add tahoe_cp to the list of scripts that we don't actually have tests for yet	1970-01-05 04:00:58 -07:00
Brian Warner	dfa2408157	checker: add is_recoverable() to checker results, make our stub immutable-verifier not throw an exception on unrecoverable files, add tests	2008-11-06 22:35:47 -07:00
Zooko O'Whielacronx	eae5403e5a	shebang: replace "/usr/bin/python" with "/usr/bin/env python"	2008-11-04 18:03:06 -07:00
Brian Warner	6fa41e738b	immutable: tolerate filenode.read() with a size= that's too big, rather than hanging	2008-11-04 15:29:19 -07:00
Brian Warner	a2e195df3e	debug catalog-shares: tolerate even more errors on bad files/directories	2008-10-30 14:54:47 -07:00
Brian Warner	ba019bfd3a	#527: expire the cached files that are used to support Range: headers, every hour, when the file is unused and older than an hour	2008-10-30 13:39:09 -07:00
Brian Warner	c205a54965	util/cachedir.py: add a cache-directory manager class, which expires+deletes unused files after a while	2008-10-30 13:01:20 -07:00
Brian Warner	1419db6ac7	test_cli: try to fix windows again	2008-10-30 12:32:04 -07:00
Brian Warner	d97c34d622	debug/test_cli: fix error handling for catalog-shares, to make the test stop failing on windows	2008-10-30 12:06:51 -07:00
Brian Warner	b1db6d9ff2	web: add 'Repair' button to checker results when they indicate unhealthyness. Also add the object's uri to the CheckerResults instance.	2008-10-29 18:09:17 -07:00
Brian Warner	d3f2df00be	webapi: serve the /static URL tree from /public_html (configurable)	2008-10-29 15:34:31 -07:00
Brian Warner	a9101112a4	catalog-shares command: tolerate errors, log them to stderr, handle v2-immutable shares	2008-10-29 15:10:10 -07:00
Brian Warner	186b64b633	test_web.py: one more line of test coverage	2008-10-28 22:00:15 -07:00
Brian Warner	0350994133	test_web: improve test coverage of PUT DIRURL t=uri replace=false	2008-10-28 21:57:44 -07:00
Brian Warner	a5aebf0cbd	web: test (and fix) PUT DIRURL t=uri, which replaces a directory in-place with some other cap	2008-10-28 21:54:46 -07:00
Brian Warner	248033d78c	test_client.py: improve test coverage a bit	2008-10-28 21:43:35 -07:00
Brian Warner	b73c380cdb	move testutil into test/common_util.py, since it doesn't count as 'code under test' for our pyflakes numbers	2008-10-28 21:28:31 -07:00
Brian Warner	294e3fb682	util: move PollMixin to a separate file (pollmixin.py), so testutil can be moved into test/	2008-10-28 21:15:48 -07:00
Brian Warner	4c2235fcef	web/filenode: oops, fix test failures, not everything has a storage index	2008-10-28 18:17:20 -07:00
Brian Warner	b1ca238176	#527: respond to GETs with early ranges quickly, without waiting for the whole file to download. Fixes the alacrity problems with the earlier code. Still needs cache expiration.	2008-10-28 17:56:18 -07:00
Brian Warner	37e3d8e47c	#527: support HTTP 'Range:' requests, using a cachefile. Adds filenode.read(consumer, offset, size) method. Still needs: cache expiration, reduced alacrity.	2008-10-28 13:41:04 -07:00
Brian Warner	914655c52b	interfaces.py: promote immutable.encode.NotEnoughSharesError.. it isn't just for immutable files any more	2008-10-27 13:34:49 -07:00
Brian Warner	fca158e83a	dirnode lookup: use distinct NoSuchChildError instead of the generic KeyError when a child can't be found	2008-10-27 13:15:25 -07:00
Brian Warner	9f21f7cf65	mutable: call remove_advise_corrupt_share when we see share corruption in mapupdate/download/check, tolerate servers that do not implement it	2008-10-24 13:21:28 -07:00
Brian Warner	db37c14ab7	storage: add remote_advise_corrupt_share, for clients to tell storage servers about share corruption that they've discovered. The server logs the report.	2008-10-24 11:52:48 -07:00
Brian Warner	6dbef907ac	mutable/servermap.py: fix needs_merge(), it was incorrectly claiming that mixed shares with distinct seqnums needed a merge, causing repair(force=False) to fail	2008-10-23 21:00:24 -07:00
Brian Warner	0383ec5a49	test_web.test_POST_DIRURL_deepcheck: confirm that /operations/HANDLE/ works with or without the slash	2008-10-23 19:17:59 -07:00
Brian Warner	cadb045288	test_web: more test coverage	2008-10-23 17:11:18 -07:00
Brian Warner	a1cfac89f9	webapi: fix t=rename from==to, it used to delete the file	2008-10-23 16:32:36 -07:00
Brian Warner	d4b4cd8ab8	test_system: update test to match web checker results	2008-10-23 16:32:02 -07:00
Brian Warner	c455d52453	deep-check: add webapi links to detailed per-file/dir results	2008-10-23 16:00:31 -07:00
Brian Warner	76b7b9d835	webapi: make the /operations/ 't=status' qualifier optional, remove it from examples	2008-10-23 15:56:58 -07:00
Brian Warner	2ee6356c89	test_dirnode.py: oops, missed a Monitor(), unbreak tests	2008-10-22 01:50:54 -07:00
Brian Warner	977c6ac510	more #514: pass a Monitor to all checker operations, make mutable-checker honor the cancel flag	2008-10-22 01:38:18 -07:00
Brian Warner	8178b10ef1	dirnode.py: check for cancel during deep-traverse operations, and don't initiate any new ones if we've been cancelled. Gets us closer to #514.	2008-10-22 00:55:52 -07:00
Brian Warner	34ab4e3de3	more #514 log-webop status/cancel: add handle-expiration, test coverage	2008-10-21 22:13:54 -07:00
Brian Warner	0f7c1fd6fd	#514: improve test coverage	2008-10-21 17:52:56 -07:00
Brian Warner	ad3d9207a9	Change deep-size/stats/check/manifest to a start+poll model instead of a single long-running synchronous operation. No cancel or handle-expiration yet. #514.	2008-10-21 17:03:07 -07:00
Zooko O'Whielacronx	8a6d1e5da6	repairer: test all different kinds of corruption that can happen to share files on disk	2008-10-14 16:09:20 -07:00
Brian Warner	b0a00e0f33	util/time_format.py: accept space separator, add unit tests	2008-10-13 15:52:58 -07:00
Brian Warner	46657b797f	test_storage: use different filenames, poor stupid windows	2008-10-09 19:11:39 -07:00
Brian Warner	7031a69bee	storage: introduce v2 immutable shares, with 8-byte offsets fields, to remove two of the three size limitations in #346. This code handles v2 shares but does not generate them. We'll make a release with this v2-tolerance, wait a while, then make a second release that actually generates v2 shares, to avoid compatibility problems.	2008-10-09 18:13:27 -07:00
Brian Warner	288d55825c	storage: split WriteBucketProxy and ReadBucketProxy out into immutable/layout.py . No behavioral changes.	2008-10-09 17:08:00 -07:00
Brian Warner	2edfe12b88	test_web: deep-size is more variable than I thought, so assert less	2008-10-06 22:11:47 -07:00
Brian Warner	3ffaded809	web: change t=manifest to return a list of (path,read/writecap) tuples, instead of a list of verifycaps. Add output=html,text,json.	2008-10-06 21:36:18 -07:00
Brian Warner	8e6d122ecf	web: rewrite t=deep-size in terms of deep-stats, update test to match inclusion of directory sizes	2008-10-06 21:35:39 -07:00
Brian Warner	d90a3ed7f8	test_system: add test coverage for immutable download.ConsumerAdapter, remove debug messages	2008-10-06 15:50:37 -07:00
Brian Warner	934dc6aa77	test_cli.py: remove unused imports	2008-10-06 17:42:04 -07:00
Brian Warner	dbc2337639	CLI: remove 'tahoe admin generate-keypair', since the pycryptopp ecdsa API is about to change incompatibly. We'll undo this once pycryptopp is updated	2008-10-06 17:23:20 -07:00
Brian Warner	09341a969a	dirnode: fix my remarkably-consistent 'metdadata' typo	2008-10-02 18:08:45 -07:00
Brian Warner	d0bdf9a611	dirnode: add get_child_and_metadata_at_path	2008-10-02 17:52:03 -07:00
Brian Warner	9c505e49c2	stop using 'as' as an identifier: as with 'with', 'as' has become a reserved word in python 2.6	2008-10-02 17:27:49 -07:00
Brian Warner	935d185696	tolerate simplejson-2.0.0 and newer, which frequently return bytestrings instead of unicode objects. Closes #523	2008-09-30 15:21:06 -07:00
Brian Warner	78d58875a8	test_node: improve coverage of advertised_ip_addresses a bit	2008-09-29 23:08:16 -07:00
Zooko O'Whielacronx	2e8b89e9e4	repair: fix test to map from storage index to directory structure properly (thanks, cygwin buildbot, for being so kloodgey that you won't accept random binary filenames and thus making me notice this bug)	2008-09-26 15:49:13 -07:00
Zooko O'Whielacronx	cd0af638f3	repairer: assert that the test code isn't accidentally allowing the repairer code which is being tested to do impossible things	2008-09-26 15:23:53 -07:00
Zooko O'Whielacronx	ebee8ee9f4	repairer: enhance the repairer tests ... Make sure the file can actually be downloaded afterward, that it used one of the deleted and then repaired shares to do so, and that it repairs from multiple deletions at once (without using more than a reasonable amount of calls to storage server allocate).	2008-09-26 10:47:19 -07:00
Brian Warner	98c8e25709	netstring: add required_trailer= argument	2008-09-26 09:57:54 -07:00
Brian Warner	27a30b90d7	test_netstring.py: move netstring tests to a separate file	2008-09-26 09:55:26 -07:00
Zooko O'Whielacronx	39fa9018ef	repairer: remove a test that doesn't apply to the repair-from-corruption case	2008-09-25 15:09:54 -07:00
Zooko O'Whielacronx	73f92ea571	repairer: add a test that repairer fixes corrupted shares (in addition to the test that it fixes deleted shares)	2008-09-25 15:07:12 -07:00
Zooko O'Whielacronx	a3b3d2b7bc	repairer: fix swapped docstrings; thanks Brian	2008-09-25 11:24:36 -07:00
Zooko O'Whielacronx	30111811e3	trivial: remove unused imports; thanks, pyflakes	2008-09-25 11:04:22 -07:00
Zooko O'Whielacronx	a363994c4c	trivial: remove unused imports -- thanks, pyflakes	2008-09-25 10:34:53 -07:00
Zooko O'Whielacronx	1e8d37cc2d	repairer: add basic test of repairer, move tests of immutable checker/repairer from test_system to test_immutable_checker, remove obsolete test helper code from test_filenode ... Hm... "Checker" ought to be renamed to "CheckerRepairer" or "Repairer" at some point...	2008-09-25 10:16:53 -07:00
Zooko O'Whielacronx	39f305e44f	setup: remove a few minimal unit tests from test_filenode which have been obviated by much better tests in test_mutable and test_system	2008-09-25 09:15:44 -07:00
robk-tahoe	ed9873c2a9	CLI: rework webopen, and moreover its tests w.r.t. path handling ... in the recent reconciliation of webopen patches, I wound up adjusting webopen to 'pass through' the state of the trailing slash on the given argument to the resultant url passed to the browser. this change removes the requirement that arguments must be directories, and allows webopen to be used with files. it also broke the tests that assumed that webopen would always normalise the url to have a trailing slash. in fixing the tests, I realised that, IMHO, there's something deeply awry with the way tahoe handles paths; specifically in the combination of '/' being the name of the root path within an alias, but a leading slash on paths, e.g. 'alias:/path', is catagorically incorrect. i.e. 'tahoe:' == 'tahoe:/' == '/' but 'tahoe:/foo' is an invalid path, and must be 'tahoe:foo' I wound up making the internals of webopen simply spot a 'path' of '/' and smash it to '', which 'fixes' webopen to match the behaviour of tahoe's path handling elsewhere, but that special case sort of points to the weirdness. (fwiw, I personally found the fact that the leading / in a path was disallowed to be weird - I'm just used to seeing paths qualified by the leading / I guess - so in a debate about normalising path handling I'd vote to include the /)	2008-09-24 09:45:23 -07:00
Brian Warner	54fce5c066	webapi: survive slashes in filenames better: make t=info and t=delete to work, and let t=rename fix the problem	2008-09-24 13:35:05 -07:00
Brian Warner	dd9171eb72	node.py: add BASEDIR/keepalive_timeout and BASEDIR/disconnect_timeout, to set/enable the foolscap timers, for #521	2008-09-24 10:51:12 -07:00
Brian Warner	0799e09910	test_cli: disable generate-keypair test on OS-X, pycryptopp still has a bug	2008-09-19 12:38:55 -07:00
Brian Warner	6607fdc586	CLI: add 'tahoe admin generate-keypair' command	2008-09-18 17:11:33 -07:00
Brian Warner	99d5a8d8b9	web: add 'more info' pages for files and directories, move URI/checker-buttons/deep-size/etc off to them	2008-09-17 22:00:41 -07:00
Brian Warner	f570ad7ba5	disallow deep-check on non-directories, simplifies the code a bit	2008-09-10 13:44:58 -07:00
Brian Warner	4bb88fd2ee	dirnode: refactor recursive-traversal methods, add stats to deep_check() method results and t=deep-check webapi	2008-09-10 01:45:04 -07:00
Brian Warner	f6eeb3161f	dirnode: cleanup, make get_verifier() always return a URI instance, not a string	2008-09-10 01:37:55 -07:00
Brian Warner	28a9f8f076	test_system: check t=deep-stats too	2008-09-09 23:54:57 -07:00
Brian Warner	12ff8a6eae	test_system: add deep-check-JSON tests, fix a bug	2008-09-09 23:14:16 -07:00
Brian Warner	ce7fcbde36	test_system: oops, re-enable some tests that got bypassed	2008-09-09 23:02:45 -07:00
Brian Warner	ba336aed3e	test_system: add deep-stats test	2008-09-09 22:56:34 -07:00
Brian Warner	80d8f3e862	hush pyflakes	2008-09-09 19:50:17 -07:00
Brian Warner	1d2d6a35a6	checker results: add output=JSON to webapi, add tests, clean up APIs ... to make the internal ones use binary strings (nodeid, storage index) and the web/JSON ones use base32-encoded strings. The immutable verifier is still incomplete (it returns imaginary healty results).	2008-09-09 19:45:17 -07:00
Brian Warner	04513e3ac5	immutable verifier: provide some dummy results so deep-check works, make the tests ignore these results until we finish it off	2008-09-09 18:08:27 -07:00
Brian Warner	7fb3308498	mutable checker: even more tests. Everything in ICheckerResults should be covered now, except for immutable-verify which is incomplete	2008-09-09 17:57:06 -07:00
Brian Warner	84a5778507	checker results: more tests, update interface docs	2008-09-09 17:30:10 -07:00
Brian Warner	f895e39d48	checker results: more tests, more results. immutable verifier tests are disabled until they emit more complete results	2008-09-09 17:15:46 -07:00
Brian Warner	90b934eb71	checker: add tests, add stub for immutable check_and_repair	2008-09-09 16:34:49 -07:00
Brian Warner	918b0543b9	test_system: make log() tolerate the format= form	2008-09-07 20:03:36 -07:00
Brian Warner	3408d552cd	checker: overhaul checker results, split check/check_and_repair into separate methods, improve web displays	2008-09-07 12:44:56 -07:00
Brian Warner	e2c484a761	introducer: add get_nickname_for_peerid	2008-09-05 22:07:00 -07:00
Brian Warner	1668401c16	mutable: make mutable-repair work for non-verifier runs, add tests	2008-08-26 16:34:54 -07:00
Brian Warner	bcbb51fe4a	test_web: workaround broken HEAD behavior in twisted-2.5.0 and earlier	2008-08-12 19:45:20 -07:00
Brian Warner	6af4619928	test_web: oops, actually use HEAD (instead of GET) in the HEAD test	2008-08-12 19:04:51 -07:00
Brian Warner	d43baa2ad7	mutable: add get_size_of_best_version to the interface, to simplify the web HEAD code, and tests	2008-08-12 19:02:52 -07:00
Brian Warner	014c9b5969	CLI: add 'tahoe debug corrupt-share', and use it for deep-verify tests, and fix non-deep web checker API to pass verify=true into node	2008-08-12 17:05:01 -07:00
Brian Warner	c80e352951	IFilesystemNode: add get_storage_index(), it makes tests easier	2008-08-12 16:14:07 -07:00
Brian Warner	5c0bfd7681	test_system: rename Checker to ImmutableChecker, to make room for a mutable one	2008-08-12 15:59:32 -07:00
Brian Warner	67e7333955	'tahoe debug dump-share': add --offsets, to show section offsets	2008-08-12 14:46:56 -07:00
Brian Warner	022fcf3db8	test_cli: oops, fix tests after recent stdout/stderr cleanup	2008-08-12 14:46:34 -07:00
Brian Warner	bb33e3e4c2	CLI: move all debug commands (dump-share, dump-cap, find-shares, catalog-shares) into a 'debug' subcommand, and improve --help output	2008-08-12 13:37:32 -07:00
Brian Warner	d106e411af	checker: add information to results, add some deep-check tests, fix a bug in which unhealthy files were not counted	2008-08-11 21:03:26 -07:00
Brian Warner	376df2eb5a	CLI: make 'tahoe webopen' command accept aliases like 'tahoe ls'	2008-08-11 18:20:23 -07:00
Brian Warner	c7c57bd85c	mutable: more repair tests, one with force=True to check out merging	2008-08-06 12:06:07 -07:00
Brian Warner	a68c9364b8	test/common: add ShouldFailMixin	2008-08-06 12:05:52 -07:00
Brian Warner	572115a405	test_mutable: add comment about minimal-bandwidth repairer, comma lack of	2008-08-06 10:38:50 -07:00
Brian Warner	cdbfccd23e	test_mutable: factor out common setup code	2008-08-06 10:38:04 -07:00
Brian Warner	dd6ec73efa	mutable: start adding Repair tests, fix a simple bug	2008-08-05 23:12:39 -07:00
Brian Warner	3ad0168fbf	test_system: factor out find_shares/replace_shares to a common class, so they can be used by other tests	2008-08-05 18:49:58 -07:00
Brian Warner	45d12baba2	test_cli: more coverage for 'tahoe put' modifying a mutable file in-place, by filename, closes #441	2008-08-04 13:26:43 -07:00
Brian Warner	4fb03c2510	check_grid.py: update to match new CLI: 'put - TARGET' instead of 'put TARGET'	2008-08-01 19:48:56 -07:00
Brian Warner	b9f4006b96	test_cli: remove windows-worrying newlines from test data	2008-08-01 19:47:34 -07:00
Brian Warner	35afe726c1	test_cli.py: factor out CLITestMixin	2008-08-01 19:29:38 -07:00
Brian Warner	f7d2fcc233	CLI: change one-arg forms of 'tahoe put' to make an unlinked file, fix replace-mutable #441	2008-08-01 19:27:29 -07:00
Brian Warner	2eec38279d	CLI: add create-alias command, to merge mkdir and add-alias into a single (secure-from-argv-snooping) step	2008-08-01 19:10:41 -07:00
Brian Warner	dc57ab4d36	test_cli: add system-based tests for PUT, including a mutable put that fails/todo (#441)	2008-08-01 15:10:09 -07:00
Brian Warner	071cbec865	tests: simplify CLI tests that use stdin, now that runner supports it	2008-08-01 15:05:14 -07:00
Brian Warner	f181a0458a	CLI: simplify argument-passing, use options= for everthing, including stdout	2008-08-01 11:46:24 -07:00
Zooko O'Whielacronx	6d3ec390b0	tests: add test that verifier notices any (randomly chosen) bit flipped in the verifiable part of any (randomly chosen) share ... The currently verifier doesn't (usually) pass this randomized test, hence the TODO.	2008-07-30 17:20:15 -07:00
Zooko O'Whielacronx	86160c3fc9	tests: test that checker doesn't cause reads on the storage servers ... It would still pass the test if it noticed a corrupted share. (It won't notice, of course.) But it is required to do its work without causing storage servers to read blocks from the filesystem.	2008-07-30 16:54:20 -07:00
Zooko O'Whielacronx	dbd1c2397e	tests: use the handy dandy TestCase.mktemp() function from trial to give unique and nicely named directories for each testcase	2008-07-30 15:49:20 -07:00
Zooko O'Whielacronx	6aabaa0797	tests: don't use SignalMixin ... It seems like we no longer need it, and it screws up something internal in trial which causes trial's TestCase.mktemp() method to exhibit wrong behavior (always using a certain test method name instead of using the current test method name), and I wish to use TestCase.mktemp(). Of course, it is possible that the buildbot is about to tell me that we do still require SignalMixin on some of our platforms...	2008-07-30 15:35:36 -07:00
Zooko O'Whielacronx	56282613c5	tests: add test_system.Checker which tests basic checking (without verification) functionality	2008-07-28 16:43:17 -07:00
Zooko O'Whielacronx	fe522e31a4	tests: make it so that you can use common.py's SystemTestMixin.set_up_nodes() more than once with the same introducer	2008-07-28 16:40:29 -07:00
Brian Warner	9e22353773	test/common.py: use pre-computed Tub certificates for the system-test mixin, to speed such tests up by maybe 15%. The goal is to encourage more full-grid tests.	2008-07-28 12:44:21 -07:00
Brian Warner	914783acdf	web: add /status/?t=json, with active upload/download ops. Addresses #493.	2008-07-25 17:41:10 -07:00
Brian Warner	48c64b7a29	test_system.py: factor SystemTestMixin out of SystemTest	2008-07-25 15:33:49 -07:00
Brian Warner	10bbc740e9	test_system.py: modify system-test setup code in preparation for merge with common.SystemTestMixin	2008-07-25 15:29:31 -07:00
Brian Warner	766cdd9c52	test_system.py: move SystemTestMixin out into common.py, where further improvements will occur	2008-07-25 15:17:58 -07:00
Brian Warner	e317fcc08c	test_system.py: create SystemTestMixin, with less cruft, for faster system-like tests	2008-07-25 15:13:00 -07:00
Brian Warner	afda2a43e4	storage: remove update_write_enabler method, it won't serve the desired purpose, and I have a better scheme in mind. See #489 for details	2008-07-21 17:28:28 -07:00
Zooko O'Whielacronx	9461887e0a	immutable file download: make the ciphertext hash tree mandatory ... This fixes #491 (URIs do not refer to unique files in Allmydata Tahoe). Fortunately all of the versions of Tahoe currently in use are already producing this ciphertext hash tree when uploading, so there is no backwards-compatibility problem with having the downloader require it to be present.	2008-07-21 09:31:02 -07:00
Brian Warner	edec917060	test_web: test that save=true filename=unicode doesn't crash	2008-07-18 18:58:57 -07:00
Brian Warner	879fefe5f3	first pass at a mutable repairer. not tested at all yet, but of course all existing tests pass	2008-07-17 21:09:23 -07:00
Brian Warner	923c9c242a	oops, fix import/pyflakes problems	2008-07-17 17:06:20 -07:00
Brian Warner	67db0a4967	deep-check: add webapi, add 'DEEP-CHECK' button to wui, add tests, rearrange checker API a bit	2008-07-17 16:47:09 -07:00
Brian Warner	69156aeb28	dirnode deep-check: add tests of cycles, fix failures	2008-07-17 14:37:04 -07:00
Brian Warner	9289433ba3	first pass at deep-checker, no webapi yet, probably big problems with it, only minimal tests	2008-07-16 18:20:57 -07:00
Brian Warner	3e9322bcb6	checker: re-enable checker web results (although they just say 'Healthy' right now)	2008-07-16 15:42:56 -07:00
Brian Warner	7394607141	move encode/upload/download/checker.py into a new immutable/ directory. No behavior changes expected.	2008-07-16 13:14:39 -07:00
Brian Warner	94e619c1f6	overhaul checker invocation ... Removed the Checker service, removed checker results storage (both in-memory and the tiny stub of sqlite-based storage). Added ICheckable, all check/verify is now done by calling the check() method on filenodes and dirnodes (immutable files, literal files, mutable files, and directory instances). Checker results are returned in a Results instance, with an html() method for display. Checker results have been temporarily removed from the wui directory listing until we make some other fixes. Also fixed client.create_node_from_uri() to create LiteralFileNodes properly, since they have different checking behavior. Previously we were creating full FileNodes with LIT uris inside, which were downloadable but not checkable.	2008-07-15 17:23:25 -07:00
Brian Warner	fd465b4aaf	download: fix stopProducing failure ('self._paused_at not defined'), add tests	2008-07-14 15:25:21 -07:00
Brian Warner	c46f5bc634	storage: rename the latency key names so they sort properly	2008-07-11 21:51:02 -07:00
Brian Warner	60725ed065	storage: add add_lease/update_write_enabler to remote API, revamp lease handling	2008-07-09 18:06:55 -07:00
Brian Warner	a7aa6f8686	implement a mutable checker+verifier. No repair yet. Part of #205.	2008-07-07 17:36:00 -07:00
Brian Warner	a0de01e971	servermap.shares_available: report N too, not just k	2008-07-07 15:06:15 -07:00
Brian Warner	9b6f23ac05	'tahoe dump-share': show verify-cap too	2008-07-07 14:11:02 -07:00
Brian Warner	2074c92dd1	test_mutable.py: add tests for no-servers conditions, closes #463.	2008-07-07 12:18:10 -07:00
Brian Warner	7dc8ff0263	re-enable incident-reporting, but disable it in unit tests, because they take 150% longer, and the leftover trailing timers cause errors	2008-07-06 23:49:08 -07:00
Brian Warner	dfe235bcba	test_storage: oops, update the test to match the leave-incoming/ change	2008-06-26 11:36:17 -07:00
Brian Warner	dba0ad3d28	test_mutable.py: add more tests of post-mapupdate corruption, to support #474 testing	2008-06-24 11:08:10 -07:00
Brian Warner	fa453c9b5e	introducer: add old (V1) introducer code, add test framework for compatibility testing	2008-06-18 16:58:34 -07:00
Brian Warner	9ed9eb2048	check_grid.py: make it work, move node start/stop responsibility from Makefile to script	2008-06-19 12:39:52 -07:00
Brian Warner	50d945ea8f	start work on 'check-grid' target, for use in an automated prodnet tester. Just a docstring so far.	2008-06-18 12:57:47 -07:00
Brian Warner	28f4652b96	break introducer up into separate modules in the new allmydata.introducer package	2008-06-18 12:24:16 -07:00
Brian Warner	5bdff74e5b	web: stop using absolute links (or url.here) in forms and pages, since they break behind proxies. Partially addresses #461	2008-06-17 19:49:40 -07:00
Brian Warner	c09c342718	test_storage: yet more coverage	2008-06-17 17:44:10 -07:00
Brian Warner	6b55b8b022	test_storage.py: improve test coverage	2008-06-17 17:01:42 -07:00
Brian Warner	2154b5751b	test_storage: add coverage for discard_storage	2008-06-16 17:52:40 -07:00
Brian Warner	75e662cc46	test_storage: add coverage for readonly_storage	2008-06-16 17:52:13 -07:00
Brian Warner	1ce3b77dde	storage: improve stats, make them accessible via webport /statistics	2008-06-16 16:35:59 -07:00
Brian Warner	6b7ff02e36	storage: measure latency-per-operation, calculate mean/median/percentiles	2008-06-16 15:21:55 -07:00
robk-tahoe	b803158de7	test_web: fix webapi test around redirection issues ... this fixes the test_web test test_POST_upload_mutable which chdir's into a /foo subdirectory, but then later asserts that the web ui redirects the user back to /foo, which should really be /foo/ since it's a directory.	2008-06-11 15:19:17 -07:00
Brian Warner	814922a9a1	storage: ignore shares in incoming/, to make clients use other servers during simultaneous uploads	2008-06-10 11:53:10 -07:00
Brian Warner	0aa6d24d62	web: add test for unicode POST when the name comes from name=, not the filename attribute	2008-06-03 17:09:39 -07:00
Brian Warner	9f5075e83d	web: make nickname more visible in the welcome page, closes #361	2008-06-03 15:02:10 -07:00
Brian Warner	32c89a8d59	web: transform FileTooLargeError into a friendlier '413 Request Entity Too Large' error	2008-06-03 00:03:16 -07:00
Brian Warner	f4496bd553	mutable/publish.py: raise FileTooLargeError instead of an ugly assertion when the SDMF restrictions are exceeded	2008-06-03 00:02:10 -07:00
Brian Warner	5289064dcf	move FileTooLargeError out to a common location	2008-06-03 00:01:15 -07:00
Brian Warner	8c37b8e3af	Don't allow uploads of large files (about 12GiB or larger), since they're doomed to be corrupted. Closes #439	2008-06-02 16:57:01 -07:00
Brian Warner	b6c2f80170	test_system: even more 'cp -r' coverage	2008-05-21 18:40:49 -07:00
Brian Warner	4824d058cc	test_system.py: improve 'cp -r' coverage: exercise copy from tahoe to local disk	2008-05-21 18:36:25 -07:00
Brian Warner	0f8f213baf	CLI: remove the '-r' shortcut for --dir-cap, to make it available for cp -r	2008-05-21 17:55:32 -07:00
Brian Warner	457248f023	tahoe_cp: rewrite, make --recursive work	2008-05-21 17:35:21 -07:00
Brian Warner	15fbe05092	tahoe_ls: improve error message when the target is missing	2008-05-21 17:34:52 -07:00
Brian Warner	7afd869854	cli: initial implementation of 'cp -r', probably doesn't work yet	2008-05-21 11:49:22 -07:00
Brian Warner	0df663b7ae	CLI: implement the easy part of cp (no -r, only two arguments)	2008-05-20 16:56:03 -07:00
Brian Warner	3ac4a734e5	web t=json: add 'mutable' key to the information dict	2008-05-20 15:40:49 -07:00
Brian Warner	9c992c0d06	test_web: improve test coverage of web.common utility code	2008-05-20 15:21:46 -07:00
Brian Warner	8dded7819c	web: fix JSON output for mutable files	2008-05-20 15:14:19 -07:00
Brian Warner	1d8e452184	CLI: add 'list-aliases', factor out get_aliases	2008-05-20 14:36:04 -07:00
Brian Warner	1236bc3408	CLI: add 'ln', just like move but without the delete	2008-05-20 13:30:31 -07:00
Brian Warner	f09f92c371	test_system.py: remove dead code	2008-05-20 13:20:48 -07:00
Brian Warner	a8ed3208ba	CLI ls: add --readonly-uri to display readonly URIs for all children	2008-05-20 12:49:11 -07:00
Brian Warner	e889463f0c	CLI: add put --mutable, enhance ls to show mutable vs immutable as rw/r-	2008-05-20 12:36:55 -07:00
Brian Warner	e323019708	web: handle PUT mutable=true properly	2008-05-20 12:36:02 -07:00
Brian Warner	d72f75019f	test_web: add HEAD coverage	2008-05-20 11:47:43 -07:00
Brian Warner	b0e306a753	web: more mutable-file coverage	2008-05-20 11:35:47 -07:00
Brian Warner	6b51841ec8	web: even more test coverage	2008-05-20 11:33:14 -07:00
Brian Warner	2eadabbf82	web: improve test coverage, remove some dead code	2008-05-20 11:13:12 -07:00
Brian Warner	88457fbc32	test_web/test_system: improve test coverage	2008-05-19 23:28:52 -07:00
Brian Warner	506ce2dfd0	test_system: fix windows test failure, open files with binary mode	2008-05-19 20:24:12 -07:00
Brian Warner	8e92dfcb50	overhaul CLI: not quite complete but it works a lot better than it used to. The new scheme uses 'tahoe add-alias' and rsync/scp-style 'alias:foo/bar.txt' arguments	2008-05-19 19:28:50 -07:00
Brian Warner	9662e6d986	test/common: oops, forgot the FakeMutableFileNode.get_readonly fix	2008-05-19 18:52:19 -07:00
Brian Warner	ef929e51f9	web: improve test coverage	2008-05-19 18:38:39 -07:00
Brian Warner	daebb42060	web/directory: fix rw_uri output in t=JSON to reflect mutable files properly	2008-05-19 18:37:28 -07:00
Brian Warner	fdb4149bad	test_web: remove leftover import to hush pyflakes	2008-05-19 14:28:39 -07:00
Brian Warner	b1d2922ea3	test_system: match webapi change, making new files now returns 201 Created, not 200 OK	2008-05-19 14:09:31 -07:00
Brian Warner	40172047a0	filenode: add is_mutable to non-MutableFileNode classes	2008-05-19 13:03:00 -07:00
Brian Warner	b8c0217ad5	test_web: implement API changes from the recent webapi.txt overhaul	2008-05-19 12:56:02 -07:00
Brian Warner	869e1ed41a	test_web.py: minor cleanups, improved error reporting	2008-05-19 12:33:39 -07:00
Brian Warner	3360fd24fc	test_web.py: localdir=/localfile= is going away, so remove the tests that exercise it	2008-05-19 12:32:09 -07:00
Brian Warner	87c1e8e066	dirnode: add overwrite= to most API calls, defaulting to True. When False, this raises ExistingChildError rather than overwriting an existing child	2008-05-16 16:09:47 -07:00
Brian Warner	304abfee32	webish: add /file links, change directory page to use them. This fixes filenames in wget. Closes #221.	2008-05-14 14:32:21 -07:00
Brian Warner	ff0b9e2549	test_mutable.py: test replacing a file that has one new outlier share present: closes #272	2008-05-14 13:10:41 -07:00
Brian Warner	26187bfc81	use a weakref cache in the client to manage singleton filenodes/dirnodes, to avoid autocollision. Should close #391.	2008-05-08 18:02:55 -07:00
Brian Warner	8f4e409677	check_memory: oops, fix HTTP-based modes: trailing slash in node.url vs no slash	2008-05-08 18:01:31 -07:00
Brian Warner	7323ced8e4	test_dirnode: improve coverage of not-mutable-error a bit	2008-05-08 16:53:35 -07:00
Brian Warner	411f578c19	check_memory: don't use fixed webports, they cause occasional test failures	2008-05-08 16:49:27 -07:00
Brian Warner	fabdc28c06	deep-stats: add file-size histogram	2008-05-08 16:19:42 -07:00
Brian Warner	a1ea3d9b37	test_dirnode: the size of the directories varies, tolerate it as low as 500 bytes	2008-05-08 13:42:42 -07:00
Brian Warner	e6ae7a2c60	dirnode: refactor deep-stats a bit	2008-05-08 13:33:07 -07:00
Brian Warner	6c00a70dbc	dirnode: add a deep_stats(), like deep-size but with more information. webish adds t=deeps-size too.	2008-05-08 13:21:14 -07:00
Brian Warner	15a9f3d01b	test_util: improve ConcurrencyLimiter tests	2008-05-07 17:00:47 -07:00
Brian Warner	c7e441309d	add a basic concurrency limiter utility	2008-05-07 16:53:30 -07:00
Brian Warner	fc774803ce	test_stats.py: improve test coverage	2008-04-30 11:52:31 -07:00
Brian Warner	a5a7ba24ef	stats: add tests for CPUUsageMonitor, modify it a bit to facilitate testing	2008-04-30 11:39:13 -07:00
Brian Warner	88951548ee	test_mutable: update notify_publish() to match new signature	2008-04-29 18:24:57 -07:00
Brian Warner	186492e620	introducer: only record one announcement per (tubid,service) tuple. Fixes #343.	2008-04-23 15:05:39 -07:00
Brian Warner	a557d44877	test_mutable: test that all servers refusing our share means a publish fails	2008-04-22 18:53:20 -07:00
Brian Warner	13eb00145e	mutable: improve test coverage in Retrieve, when shares change after mapupdate	2008-04-22 17:25:14 -07:00
Brian Warner	acdd98432d	hush pyflakes warning about code that got moved in the recent StallMixin refactoring	2008-04-22 17:14:26 -07:00
Brian Warner	a7632a345e	mutable/servermap: improve test coverage	2008-04-22 16:47:52 -07:00
Brian Warner	81ab1ec785	testutil: factor stall() out into a common location	2008-04-22 16:47:15 -07:00
Brian Warner	57d5341715	test_introducer.py: don't log nodeids as binary goop	2008-04-22 12:54:16 -07:00
Brian Warner	e6074f5dfc	mutable: test write failures, uncoordinated write detection	2008-04-22 11:49:53 -07:00
Brian Warner	db4ddfc1bd	mutable: improve testing to exercise the non-cached case	2008-04-21 17:50:50 -07:00
Brian Warner	f39f8ae032	check_speed: rearrange a bit to make it easy to disable certain portions	2008-04-21 17:29:59 -07:00
Brian Warner	a04c2866b1	mutable: improve test coverage slightly	2008-04-21 15:10:50 -07:00
Zooko O'Whielacronx	78829dcf08	key_generator: make default key size be a constructor argument instead of a class variable, pass default key size of 522 (the smallest that we can do) in unit tests to make them faster	2008-04-22 12:28:18 -07:00
Brian Warner	c375b1503a	mutable/servermap: improve test coverage	2008-04-18 20:13:28 -07:00
Brian Warner	09dcfeae22	mutable: improve test coverage, fix bug in privkey fetching, add .finished to stats, remove dead code	2008-04-18 19:55:12 -07:00
Brian Warner	d7b82f73c5	dirnode: return to 'delete fails if the child wasn't actually there' semantics, to make tests pass. There's a switch to enable/disable this	2008-04-17 20:06:06 -07:00
Brian Warner	d5b182c183	test_mutable: hush pyflakes	2008-04-17 19:57:55 -07:00
Brian Warner	1bff08a7df	dirnode: update to use MutableFileNode.modify	2008-04-17 19:57:04 -07:00
Brian Warner	945003ea0f	test_mutable: factor out ShouldFailMixin	2008-04-17 19:55:51 -07:00
Brian Warner	004d3c14bf	mutable: implement MutableFileNode.modify, plus tests	2008-04-17 19:12:42 -07:00
Brian Warner	a379690b04	mutable: replace MutableFileNode API, update tests. Changed all callers to use overwrite(), but that will change soon	2008-04-17 17:51:38 -07:00
Brian Warner	157073d8d8	mutable WIP: merge in patches from current trunk	2008-04-17 13:09:22 -07:00
Brian Warner	e1838ba217	mutable WIP: clean up status handling, shrink the code a lot, improve test coverage	2008-04-17 13:02:22 -07:00
Brian Warner	a1670497a8	mutable WIP: add servermap update status pages	2008-04-16 19:05:41 -07:00
Brian Warner	842b1f1a70	mutable WIP: oops, fix test_mutable	2008-04-16 18:06:54 -07:00
Brian Warner	749c42fa2c	mutable WIP: re-enable publish/retrieve status	2008-04-16 17:49:06 -07:00
Brian Warner	4f07d96667	mutable WIP: improve logging a bit	2008-04-16 15:22:30 -07:00
Brian Warner	c0c8d72b44	mutable WIP: rewrite ServerMap data structure, add tests	2008-04-16 14:49:47 -07:00
Brian Warner	1b4b4cbd4a	mutable WIP: rename NotEnoughPeersError to NotEnoughSharesError	2008-04-15 16:08:32 -07:00
Brian Warner	cf84bae850	mutable WIP: merge conflicts in test_system.py	2008-04-15 15:57:28 -07:00
Brian Warner	d4230d1781	mutable WIP: split mutable.py into separate files. All tests pass.	2008-04-11 14:31:16 -07:00
Brian Warner	0515b1099c	mutable WIP: all tests pass, but publish/retrieve status is still stubbed out	2008-04-10 19:26:35 -07:00
Brian Warner	418407ee5c	mutable: WIP. make Publish work, remove some test scaffolding. test_system still fails.	2008-04-10 18:44:06 -07:00
Brian Warner	2c939bfdd3	mutable.py: checkpointing #303 work: retrieve does what I want, now starting in on publish	2008-04-04 17:09:26 -07:00
Brian Warner	68527b25fc	client: add 'node.uptime' to the stats we collect	2008-04-17 11:13:39 -07:00
Brian Warner	944f8babec	test_mutable.py: remove spurious Retrieve during a publish test	2008-04-15 11:20:38 -07:00
Brian Warner	f153dafaa0	web: return a proper error upon POST with a bad t= value	2008-04-15 11:11:29 -07:00
Brian Warner	6de2a0794f	web: don't break status page when there is no helper running	2008-04-14 23:42:20 -07:00
Brian Warner	a2dd4d14a6	helper status: include percentage fetched+pushed, add helper-uploads to the upload/download list	2008-04-14 18:36:27 -07:00
Brian Warner	86578bc3cd	webish: fix 'not running helper' status indicator on welcome page, add tests	2008-04-14 16:28:11 -07:00
Brian Warner	5b8320442a	stats: add /statistics web page to show them, add tests	2008-04-14 14:17:08 -07:00
Brian Warner	500934b72f	helper stats: fix the /helper_status page, the recent conflict merging missed some uses. Added tests, updated the munin plugins to match	2008-04-14 13:18:53 -07:00
robk-tahoe	27ac71458f	key_generator: remove poll timeouts from test ... the timeouts on uses of 'poll' were there purely to make sure a test doesn't poll indefinitely. however having such timeouts makes tests susceptible to premature timeouts under high load, or on slow machines. (e.g. cygwin slaves running in virtual machines on loaded hosts) purportedly trial by default applies a timeout to tests to prevent them hanging out indefinitely, so these poll timeouts are redundant and cause intermittent failures on slow hosts. hence they're more bother than they're worth, and should be culled.	2008-04-08 11:06:58 -07:00
robk-tahoe	c838cfef44	key_generator: up timeouts on keygen test ... in both these cases, the timeout only serves to abort a stuck test, and the key_generator should respond more quickly, but seeing test failures in buildbot on some platforms suggests that the test is too susceptible to timing issues on loaded buildslaves.	2008-04-03 16:26:24 -07:00
robk-tahoe	1ae2c39862	key_generator: service related cleanups, incorporation into system test ... this cleans up KeyGenerator to be a service (a subservice of the KeyGeneratorService as instantiated by the key-generator.tac app) this means that the timer which replenishes the keypool will be shutdown cleanly when the service is stopped. adds checks on the key_generator service and client into the system test 'test_mutable' such that one of the nodes (clients[3]) uses the key_generator service, and checks that mutable file creation in that node, via a variety of means, are all consuming keys from the key_generator.	2008-04-03 15:57:07 -07:00
robk-tahoe	ccda06b061	key_generator: added a unit test ... implemented a unit test of basic KeyGenService functionality, fixed a bug in the timing of pool refreshes	2008-04-03 13:01:43 -07:00
Brian Warner	06dae8fd67	trial_figleaf.py: make our figleaf code compatible with both Twisted-8.x and Twisted-2.5.x	2008-04-02 17:48:55 -07:00
robk-tahoe	5578559b85	added offloaded key generation ... this adds a new service to pre-generate RSA key pairs. This allows the expensive (i.e. slow) key generation to be placed into a process outside the node, so that the node's reactor will not block when it needs a key pair, but instead can retrieve them from a pool of already generated key pairs in the key-generator service. it adds a tahoe create-key-generator command which initialises an empty dir with a tahoe-key-generator.tac file which can then be run via twistd. it stashes its .pem and portnum for furl stability and writes the furl of the key gen service to key_generator.furl, also printing it to stdout. by placing a key_generator.furl file into the nodes config directory (e.g. ~/.tahoe) a node will attempt to connect to such a service, and will use that when creating mutable files (i.e. directories) whenever possible. if the keygen service is unavailable, it will perform the key generation locally instead, as before.	2008-04-01 18:45:13 -07:00
Brian Warner	36f5c025a6	helper: add stats for the gatherer, show some on the webish welcome page	2008-03-27 15:55:32 -07:00
Brian Warner	9b3a32d0b3	add GET /uri/URI/?t=deep-size, to compute the total size of immutable files reachable from a given directory	2008-03-27 11:33:42 -07:00
Brian Warner	4531d1e953	introweb.py: add ?t=json, to provide machine-readable subscriber counts	2008-03-25 12:56:12 -07:00
Brian Warner	1e097766c9	disable plaintext hashes in shares, but leave a switch to turn it back on	2008-03-24 13:39:51 -07:00
Brian Warner	7b21054c33	UNDO: upload: stop putting plaintext and ciphertext hashes in shares. ... This removes the guess-partial-information attack vector, and reduces the amount of overhead that we consume with each file. It also introduces a forwards-compability break: older versions of the code (before the previous download-time "make hashes optional" patch) will be unable to read files uploaded by this version, as they will complain about the missing hashes. This patch is experimental, and is being pushed into trunk to obtain test coverage. We may undo it before releasing 1.0.	2008-03-23 15:35:54 -07:00
Zooko O'Whielacronx	dfdbb95648	fix check-memory to use new upload API (which requires a "convergence" argument), and change it to measure convergence instead of random-key, since convergence is the use case we care about more	2008-03-24 15:28:04 -07:00
Zooko O'Whielacronx	fc3bd0c987	use added secret to protect convergent encryption ... Now upload or encode methods take a required argument named "convergence" which can be either None, indicating no convergent encryption at all, or a string, which is the "added secret" to be mixed in to the content hash key. If you want traditional convergent encryption behavior, set the added secret to be the empty string. This patch also renames "content hash key" to "convergent encryption" in a argument names and variable names. (A different and larger renaming is needed in order to clarify that Tahoe supports immutable files which are not encrypted content-hash-key a.k.a. convergent encryption.) This patch also changes a few unit tests to use non-convergent encryption, because it doesn't matter for what they are testing and non-convergent encryption is slightly faster.	2008-03-24 09:46:06 -07:00
Brian Warner	7996131a0a	upload: stop putting plaintext and ciphertext hashes in shares. ... This removes the guess-partial-information attack vector, and reduces the amount of overhead that we consume with each file. It also introduces a forwards-compability break: older versions of the code (before the previous download-time "make hashes optional" patch) will be unable to read files uploaded by this version, as they will complain about the missing hashes. This patch is experimental, and is being pushed into trunk to obtain test coverage. We may undo it before releasing 1.0.	2008-03-23 15:35:54 -07:00
Brian Warner	553367d567	download: make plaintext and ciphertext hashes in the UEB optional. ... Removing the plaintext hashes can help with the guess-partial-information attack. This does not affect compatibility, but if and when we actually remove any hashes from the share, that will introduce a forwards-compatibility break: tahoe-0.9 will not be able to read such files.	2008-03-23 14:46:49 -07:00
robk-tahoe	e75c417bb5	webish mkdir-p: added unit test ... added a test for the simple mkdir-p hack I added yesterday checks that mkdir-p can create a directory hierarchy, and that resubmitting a request for the same path yields the existing dir's uri	2008-03-18 13:11:08 -07:00
Brian Warner	2ef70ab814	mutable.py: split replace() into update() and overwrite(). Addresses #328.	2008-03-12 18:00:43 -07:00
Brian Warner	89be2e1bea	introweb: combine announcement and subscriber information to show version+nickname for each client	2008-03-11 19:21:29 -07:00
Brian Warner	810ba68343	add a webserver for the Introducer, showing service announcements and subscriber lists	2008-03-11 17:36:25 -07:00
Brian Warner	c727348d85	mutable: tolerate multiple encodings, using whichever version is recoverable first. Closes #312	2008-03-11 00:26:00 -07:00
Brian Warner	791482cf8d	test_mutable: improve multiple-encodings test coverage	2008-03-10 23:47:35 -07:00
Brian Warner	9ca55b8b79	test_mutable: make test-multiple-encodings work	2008-03-10 23:16:28 -07:00
Brian Warner	57bd23f35f	test_mutable: more test coverage, building up a framework to cause reads to occur in a specific order	2008-03-10 22:15:43 -07:00
Brian Warner	11445b5109	test_mutable: exercise short reads too	2008-03-10 18:08:23 -07:00
Brian Warner	be5a6147b4	test_mutable: test all hash-failure cases except a corrupted encrypted private key	2008-03-10 17:46:52 -07:00
Brian Warner	647734cd3b	test_mutable: add Roundtrip test, suitable for new share-mangling tests	2008-03-10 16:14:08 -07:00
Brian Warner	9e460cd22e	mutable: cosmetic changes	2008-03-10 15:44:05 -07:00
Brian Warner	d6cde55c63	mutable: minor refactoring of _do_read, to make other tests easier	2008-03-10 15:42:56 -07:00
Brian Warner	4fa622f9b1	webish: this file is too big, start breaking it into pieces, beginning with status	2008-03-05 14:59:56 -07:00
Brian Warner	7d9bfbcabe	mutable: oops, .download *is* in use, by the speedtest. Restore it and add a test.	2008-03-04 15:11:40 -07:00
Brian Warner	4daa817502	test_util.py: get 100% test coverage for hashutil.py	2008-03-04 14:42:25 -07:00
Brian Warner	95a085763f	uri.py: get 100% test coverage, fix a few bugs in the process	2008-03-04 14:27:45 -07:00
Brian Warner	e6489b1ef2	test_filenode.py : improve coverage of mutable filenode, fix a bug in __hash__	2008-03-04 14:01:28 -07:00
Brian Warner	f6ca62df31	test_system: improve test coverage of publish/retrieve status	2008-03-04 01:24:35 -07:00
Brian Warner	68fbd89e66	webish: add primitive publish/retrieve status pages	2008-03-04 01:07:44 -07:00
Brian Warner	7e159feb27	stats: make StatsGatherer happy about sharing a process with other services, add one during system test to get some test coverage	2008-03-03 23:55:58 -07:00
Brian Warner	abb51e70f3	test_web: improve upload/download status coverage	2008-03-03 21:56:23 -07:00
Brian Warner	3ecb483e35	test_system: add test coverage for download-status and upload-status	2008-03-03 21:37:17 -07:00
Brian Warner	886ef22335	webish: download-results: add server_problems	2008-03-03 20:30:35 -07:00
Brian Warner	c8e24f0904	webish: make upload timings visible on the recent uploads/downloads status page	2008-03-03 14:48:52 -07:00
Brian Warner	436baa1b19	webish: add per-file upload/download status pages	2008-02-29 23:03:00 -07:00
Brian Warner	1a7651ce82	retain 10 most recent upload/download status objects, show them in /status . Prep for showing individual status objects	2008-02-29 22:19:03 -07:00
Zooko O'Whielacronx	99f006c584	wapi: add POST /uri/$DIRECTORY?t=set_children ... Unfinished bits: doc in webapi.txt, test handling of badly formed JSON, return reasonable HTTP response, examination of the effect of this patch on code coverage -- but I'm committing it anyway because MikeB can use it and I'm being called to dinner...	2008-02-29 18:40:27 -07:00
Zooko O'Whielacronx	4287079455	test: update todo string in test_nevow	2008-02-29 13:10:08 -07:00
Brian Warner	301dd3d489	webish status: distinguish active uploads/downloads from recent ones	2008-02-26 15:35:28 -07:00
Brian Warner	1af3bfb1d4	test_runner.py: test launching an introducer too	2008-02-18 00:28:56 -07:00
Brian Warner	6ff98a3c0b	'tahoe start': allow this to start arbitrary .tac file -bearing directories (like the log gatherer)	2008-02-18 00:28:17 -07:00
Brian Warner	c3a1491cf4	test_system.py: improve coverage of webish.py	2008-02-15 04:02:50 -07:00
Brian Warner	72874390fe	BIG COMPATIBILITY BREAK: update hash tags, switch to SHA-256d everywhere	2008-02-14 19:58:01 -07:00
Zooko O'Whielacronx	fc0d637523	docs: update install and usage docs, improve cli "usage" output, make new example directories, add unit test that fails code which prints out sentences that don't end with punctuation marks	2008-02-15 13:11:02 -07:00
Zooko O'Whielacronx	5942fca1e1	add unit test asserting that allmydata.util.base32 matches rfc 3548	2008-02-14 20:20:19 -07:00
Zooko O'Whielacronx	3eb95c1fad	merge patch which adds base32 to test_util with patch which removes bencode from test_util	2008-02-14 19:35:32 -07:00
Brian Warner	09bfa3bea2	remove bencode, yay	2008-02-14 18:06:41 -07:00
Brian Warner	7927495cbe	unicode handling: declare dirnodes to contain unicode child names, update webish to match	2008-02-14 15:45:56 -07:00
Zooko O'Whielacronx	7c6de95bc6	switch from base62 to base32 for storage indices, switch from z-base-32 to rfc 3548 base-32 for everything, separate out base32 encoding from idlib	2008-02-14 19:27:47 -07:00
Brian Warner	a050204833	add test coverage for the /stats web page	2008-02-13 13:57:39 -07:00
Brian Warner	871e7756f7	test_web: remove test_GET_DIRURL_large, it failed to exercise the problem in #237 and consumes a disproportionate amount of time	2008-02-13 13:08:26 -07:00
Zooko O'Whielacronx	e89edf8803	fix a few unused imports and suchlike, discovered by pyflakes	2008-02-13 07:38:08 -07:00
Brian Warner	a7867aeea2	introducer: remove encoding-parameter config, for now	2008-02-12 18:59:54 -07:00
Zooko O'Whielacronx	3f8df27063	use base62 encoding for storage indexes, on disk and in verifier caps, and in logging and diagnostic tools ... base62 encoding fits more information into alphanumeric chars while avoiding the troublesome non-alphanumeric chars of base64 encoding. In particular, this allows us to work around the ext3 "32,000 entries in a directory" limit while retaining the convenient property that the intermediate directory names are leading prefixes of the storage index file names.	2008-02-12 20:48:37 -07:00
Brian Warner	a70fba4ef1	webish: add edge metadata to t=json output, including timestamps	2008-02-11 19:14:10 -07:00
Brian Warner	ca27b8e5ad	add 'tahoe catalog-shares' tool, to make a one-line summary of each share file. This can help do cross-server correlation of sharefiles, looking for anomalies	2008-02-11 18:17:01 -07:00
Brian Warner	5f50543c3b	test_dirnode.py: improve error messages in case timestamp tests fail	2008-02-11 17:37:51 -07:00
Brian Warner	09cf1e2a8a	test_dirnode.py: simplejson-1.7.1 incorrectly rounds floats to two decimal places. Don't let this bug flunk the timestamp test.	2008-02-11 17:35:17 -07:00
Brian Warner	7123c9d875	test_system.py: refactor bounce_client, probably make it stop failing on cygwin	2008-02-11 15:26:58 -07:00
Brian Warner	3a1db45579	test_dirnode.py: assert that we update mtime and preserve ctime	2008-02-11 15:12:55 -07:00
Brian Warner	e6ddd9f3da	dirnode.py: add metadata= to add_file(), add tests	2008-02-11 14:53:28 -07:00
Brian Warner	b90eaf75da	test_dirnode.py: add diag output to test-ctime/mtime tests	2008-02-11 14:13:07 -07:00
Brian Warner	622c477e31	dirnode: add ctime/mtime to metadata, update metadata-modifying APIs. Needs more testing and sanity checking.	2008-02-08 18:43:47 -07:00
Brian Warner	873dee53e9	test_system: remove the hackish debug_interrupt= attribute magic used to exercise interrupted-upload resumption, instead just make the Uploadable bounce the helper halfway through the upload	2008-02-07 20:15:37 -07:00
Brian Warner	3bd79917b2	test_system.py: remove that ugly debug_stash_RemoteencryptedUploadable hack, now that UploadResults give us a better approach	2008-02-07 17:27:30 -07:00
Brian Warner	96ca6c2033	test_upload.py: hush pyflakes	2008-02-06 20:04:31 -07:00
Brian Warner	f78c80d7aa	test_upload.py: add test to exercise CHK hashing variations	2008-02-06 20:03:35 -07:00
Brian Warner	02e7c56563	test_cli: oops, need to update this when the CHK hash changes	2008-02-06 19:58:53 -07:00
Brian Warner	da9610e90a	change encryption-key hash to include encoding parameters. This is a minor compatibility break: CHK files encoded (with convergence) before and after this will have different keys and ciphertexts. Also switched to SHA-256d for both the data-to-key hash and the key-to-storageindex hash	2008-02-06 19:50:47 -07:00
Brian Warner	9f9a458249	hashutil: add tagged_hash_256d and tagged_hasher_256d	2008-02-06 19:36:43 -07:00
Brian Warner	81c5ceae16	upload: rework passing of default encoding parameters: move more responsibility into BaseUploadable	2008-02-06 18:39:03 -07:00
Brian Warner	75e4332462	offloaded: oops, need more tricks to make the unit tests pass	2008-02-06 17:51:11 -07:00
Brian Warner	d87bfa9eb2	test_util: add full coverage for allmydata.util.deferredutil	2008-02-06 16:41:04 -07:00
Brian Warner	4d4073fb16	test_system: deferred-handling logic in test_upload_and_download was broken	2008-02-06 14:05:11 -07:00
Brian Warner	2dcac796e9	add 'tahoe find-shares' command, to locate share files on a local node's disk	2008-02-06 13:19:51 -07:00
Brian Warner	e92a2b5ab2	dump-cap: include UEB_hash in output	2008-02-06 12:48:19 -07:00
Brian Warner	6e0d3059a3	webish: add when_done= to POST /uri?t=upload . I did not add a 'recent uploads' section to the welcome page, but I think the new upload-results page provides the desired data	2008-02-06 02:38:16 -07:00
Brian Warner	124fb5ecdf	add upload-results timing info for helper uploads. This changes the Helper protocol, and introduces a compatibility break	2008-02-06 01:52:25 -07:00
Brian Warner	1e4504463c	webish: make POST /uri?t=upload deposit you on an 'Upload Results' page	2008-02-05 23:01:37 -07:00
Brian Warner	27f46f2aaf	test_web.py: hush pyflakes	2008-02-05 22:37:51 -07:00
Brian Warner	23c2c8d660	webish: add PUT /uri?mutable=true	2008-02-05 22:18:02 -07:00
Brian Warner	e5dc9a8486	webish: add POST /uri?t=upload&mutable=true	2008-02-05 22:10:22 -07:00
Brian Warner	6afe50e2aa	webish: add upload/view-uri forms (not associated with any particular directory) to the welcome page. Document POST /uri?t=upload .	2008-02-05 21:44:40 -07:00
Brian Warner	66f33ee504	upload: return an UploadResults instance (with .uri) instead of just a URI	2008-02-05 21:01:38 -07:00
Brian Warner	5103bf8148	storage: change service name from 'storageserver' to 'storage'	2008-02-05 20:28:59 -07:00
Brian Warner	d146ef7e09	webish: add extra introducer data (version, timestamps) to Welcome page	2008-02-05 17:32:27 -07:00
Brian Warner	a7ea39e626	test_introducer.py: increase timeouts on poll() calls	2008-02-05 16:37:58 -07:00
Brian Warner	d98fde952c	introducer: remove remaining bits of 'push-to-myself' flags. The uploading/downloading node is no longer special.	2008-02-05 14:16:01 -07:00
Brian Warner	daecca6589	big introducer refactoring: separate publish+subscribe. Addresses #271.	2008-02-05 13:05:13 -07:00
Brian Warner	3a5ba35215	PollMixin: add timeout= argument, rewrite to avoid tail-recursion problems	2008-02-04 20:35:07 -07:00
Brian Warner	a01f9ce9cc	introducer: allow nodes to refrain from publishing themselves, by passing furl=None. This would be useful for clients who do not run storage servers.	2008-02-01 19:48:38 -07:00
Zooko O'Whielacronx	1d1628e525	rename storage_index_chk_hash() to storage_index_hash() and add TODO about how our use of it now includes keys that are not CHKs	2008-02-01 12:27:37 -07:00
Brian Warner	670933ecee	storage: clean up use of si_s vs si_dir, add test for BadWriterEnabler message, add some logging	2008-01-31 17:48:48 -07:00
Zooko O'Whielacronx	7bf21082f7	remove unused import (thanks, pyflakes)	2008-01-31 17:00:59 -07:00
Zooko O'Whielacronx	79c439d026	storage: make two levels of share directories so as not to exceed certain filesystems's limitations on directory size ... The filesystem which gets my vote for most undeservedly popular is ext3, and it has a hard limit of 32,000 entries in a directory. Many other filesystems (even ones that I like more than I like ext3) have either hard limits or bad performance consequences or weird edge cases when you get too many entries in a single directory. This patch makes it so that there is a layer of intermediate directories between the "shares" directory and the actual storage-index directory (the one whose name contains the entire storage index (z-base-32 encoded) and which contains one or more share files named by their share number). The intermediate directories are named by the first 14 bits of the storage index, which means there are at most 16384 of them. (This also means that the intermediate directory names are not a leading prefix of the storage-index directory names -- to do that would have required us to have intermediate directories limited to either 1024 (2-char), which is too few, or 32768 (3-chars of a full 5 bits each), which would overrun ext3's funny hard limit of 32,000.)) This closes #150, and please see the "convertshares.py" script attached to #150 to convert your old tahoe-0.7.0 storage/shares directory into a new tahoe-0.8.0 storage/shares directory.	2008-01-31 16:26:28 -07:00
Brian Warner	22071c00e0	upload: oops, fix breakage after removing upload_file/upload_data/etc	2008-01-30 19:41:43 -07:00
Brian Warner	909d848071	remove upload.upload_(data,filename,filehandle) convenience functions	2008-01-30 19:03:19 -07:00
Brian Warner	81eeafc574	upload-helper: avoid duplicate uploads: check the grid to see if the file already exists	2008-01-30 18:49:02 -07:00
Zooko O'Whielacronx	ca971559e6	make content-hash-key encryption a parameter of uploading ... fixes #293	2008-01-30 12:24:50 -07:00
Zooko O'Whielacronx	424d338c82	tests: add a test that nevow can find its default css file	2008-01-30 11:41:58 -07:00
Brian Warner	492cb92dc8	speedcheck: track SSK creation time separately	2008-01-29 20:44:32 -07:00
Brian Warner	d9d0920177	webish: link to directory URIs rather than a child path. Addresses #103.	2008-01-29 18:04:32 -07:00
Brian Warner	8063aa8b86	WriteBucketProxy: improve __repr__	2008-01-28 18:53:51 -07:00
Brian Warner	4c5518faef	webish: upload+localdir=missing should give an error	2008-01-28 14:48:06 -07:00
Brian Warner	69a0b5cc00	CHK upload helper: don't let one failed upload prevent us from trying again	2008-01-28 12:58:13 -07:00
Brian Warner	8f1212edac	encode.py: don't allow a shareholder which dies in start() to kill the whole upload	2008-01-28 12:14:48 -07:00
Brian Warner	1ff21d1d64	test_upload.py: implement remote_abort on our fake BucketWriter	2008-01-23 18:07:34 -07:00
robk-tahoe	7c34658438	offloaded: fix failure in unit test on windows ... in trying to test my fix for the failure of the offloaded unit test on windows (by closing the reader before unlinking the encoding file - which, perhaps disturbingly doesn't actually make a difference in my windows environment) I was unable too because the unit test failed every time with a connection lost error. after much more time than I'd like to admit it took, I eventually managed to track that down to a part of the unit test which is supposed to be be dropping a connection. it looks like the exceptions that get thrown on unix, or at least all the specific environments brian tested in, for that dropped connection are different from what is thrown on my box (which is running py2.4 and twisted 2.4.0, for reference) adding ConnectionLost to the list of expected exceptions makes the test pass. though curiously still my test logs a NotEnoughWritersError error, and I'm not currently able to fathom why that exception isn't leading to any overall failure of the unit test itself. for general interest, a large part of the time spent trying to track this down was lost to the state of logging. I added a whole bunch of logging to try and track down where the tests were failing, but then spent a bunch of time searching in vain for that log output. as far as I can tell at this point the unit tests are themselves logging to foolscap's log module, but that isn't being directed anywhere, so all the test's logging is being black holed.	2008-01-17 20:57:29 -07:00
Brian Warner	fd0dc3013c	offloaded: update unit tests: assert that interrupt/resume works, and that the helper deletes tempfiles	2008-01-17 01:18:10 -07:00
Brian Warner	51321944f0	megapatch: overhaul encoding_parameters handling: now it comes from the Uploadable, or the Client. Removed options= too. Also move helper towards resumability.	2008-01-16 03:03:35 -07:00
Brian Warner	821521cc3e	test_storage: fix pyflakes warnings	2008-01-14 21:26:48 -07:00
Brian Warner	9f0ee4b758	test_system: fix pyflakes warnings	2008-01-14 21:26:28 -07:00
Brian Warner	168a8c3b73	offloaded: improve logging, pass through options, get ready for testing interrupted uploads. test_system: add (disabled) interrupted-upload test	2008-01-14 21:24:26 -07:00
Brian Warner	a6ca98ac53	upload: add Encoder.abort(), to abandon the upload in progress. Add some debug hooks to enable unit tests.	2008-01-14 21:22:55 -07:00
Brian Warner	cb76ed36f9	logging: enable flogging in more places, replace Node.log with flogging	2008-01-14 21:16:58 -07:00
Brian Warner	7ac2b94aba	remove wait_for_numpeers and the when_enough_peers call in mutable.Publish	2008-01-14 14:55:59 -07:00
Brian Warner	222718f13c	test_GET_DIRURL_large: reduce from 400 to 200 children: the test fails to ... fail anyways, and 200 ought to be enough to trigger the problem, so 400 is overkill, and just wastes CPU.	2008-01-14 22:31:41 -07:00
Brian Warner	37b7f136ce	'tahoe dump-cap': accept http:// -prefixed URLs too	2008-01-14 14:12:27 -07:00
Brian Warner	6ca0efeef6	add 'tahoe dump-cap' command, to show storage index, lease secrets, etc	2008-01-14 13:43:25 -07:00
Brian Warner	76ee9cccfe	storage: improve logging a bit	2008-01-14 11:58:58 -07:00
Zooko O'Whielacronx	676ded6a7e	tests: put back skipped and todo tests ... closes #258 -- "put back skipped and todo tests"	2008-01-14 21:02:41 -07:00
Brian Warner	964edadf44	offloaded: add a system test, make it pass. files are now being uploaded through the helper.	2008-01-11 05:42:55 -07:00
Brian Warner	6ac01fde4c	offloaded: more test coverage on client side, change interfaces a bit	2008-01-11 04:53:37 -07:00
Brian Warner	11179bb7d7	test_mutable.py: accomodate changes to mutable.py logging	2008-01-10 22:18:34 -07:00
Zooko O'Whielacronx	69dba9a704	tests: increase the timeout on a test which failed on the overloaded virtual buildslaves	2008-01-11 14:27:54 -07:00
Brian Warner	e825406fc2	offloaded: move interfaces to interfaces.py, start implementing backend	2008-01-09 21:25:47 -07:00
Brian Warner	da7778774b	offloaded: basic test for client-side of AssistedUploader	2008-01-09 20:25:50 -07:00
Brian Warner	0e2ddb00be	test_system: slight refactoring to eventually make it easier to configure some nodes with the output of others	2008-01-09 20:23:54 -07:00
Brian Warner	d4e9e3b9c4	check_speed.py: re-enable 100x200B and 1x100MB CHK tests	2008-01-08 20:59:48 -07:00
Brian Warner	464637605e	test_system: assert that BASEDIR/node.url is created properly	2008-01-07 17:46:22 -07:00
Zooko O'Whielacronx	4223f7216d	rename "dir-uri" to "dir-cap"	2008-01-08 10:41:27 -07:00
Zooko O'Whielacronx	bf14613b6a	suppress the skipped and the todo unit tests in order to make unit test results prettier (the install.html instructs users to run the unit tests)	2008-01-04 00:36:18 -07:00
Zooko O'Whielacronx	32d2cc8aba	fix test_cli to put the root_dir.cap in the private subdir	2008-01-03 17:48:53 -07:00
Zooko O'Whielacronx	23961448da	a few formatting tidy-ups	2008-01-03 17:14:19 -07:00
Brian Warner	acfb11d26f	test_client.py: validate more versioning code	2008-01-03 14:38:24 -07:00
Brian Warner	6a2e5d4aea	test_client.py: assert allmydata.__version__ is not unknown	2008-01-03 14:34:59 -07:00
Zooko O'Whielacronx	5995d11a3c	remove automatic private dir ... * rename my_private_dir.cap to root_dir.cap * move it into the private subdir * change the cmdline argument "--root-uri=[private]" to "--dir-uri=[root]"	2008-01-03 17:02:05 -07:00
Zooko O'Whielacronx	eb373c0001	add human-encodings of caps, refactor encodings of caps, tighten checks, fix unit tests to use values of the right size	2008-01-03 16:55:43 -07:00
Brian Warner	6b81ebfd8e	refactor webish.py slightly, improve the test coverage a bit. One test is marked SKIP pending improvements in webish.py	2007-12-25 03:48:57 -07:00
Brian Warner	fae5de8d0d	test_web.py: fix pyflakes error	2007-12-25 01:56:04 -07:00
Brian Warner	4ba1af37a6	test_web: more coverage of URIPOSTHandler error cases	2007-12-24 22:49:35 -07:00
Brian Warner	7d311b8dd7	test_web: test POST_upload_no_link against /uri, not /uri/	2007-12-24 17:55:38 -07:00
Brian Warner	ae760b3ee4	test_web.py: use /uri?t=mkdir instead of /uri/?t=mkdir, and confirm that the redirection target is correct	2007-12-24 17:46:52 -07:00
Zooko O'Whielacronx	1ac11301a7	setup: finish switching from Tahoe's versions of autoversioning tools to pyutil's versions	2007-12-21 14:42:38 -07:00
Zooko O'Whielacronx	a85a4d71f7	webish: add button to make directories and unit test of it ... Unfortunately although it passes the unit tests, it doesn't work, because the unit tests and the implementation use the "encode params into URL" technique but the button uses the "encode params into request body" technique.	2007-12-20 15:31:12 -07:00
Zooko O'Whielacronx	a7361179aa	make the unit test of the overwrite button more general	2007-12-20 15:23:07 -07:00
Zooko O'Whielacronx	a4bc623fa5	make the wapi/wui [*] support creation of a new directory without there already being an existing directory to link the new one into ... [*] WebAPI/WebUI	2007-12-20 12:58:17 -07:00
Brian Warner	d4283ac1ec	mutable: always include ourselves in the querylist when Publishing, because 0.7.0 usually stores our root dirnode only on ourselves, and our nodeid might not appear in the first N+epsilon peers	2007-12-19 00:06:53 -07:00
Brian Warner	9a8f68c41f	dirnode: add set_uris() and set_nodes() (plural), to set multiple children at once. Use it to set up a new webapi test for issue #237.	2007-12-18 23:30:02 -07:00
Zooko O'Whielacronx	a0d73f8afb	remove some no-longer needed replacements of "/" with "!" in uris	2007-12-19 17:54:40 -07:00
Brian Warner	2db158dd52	more new-pyflakes warnings fixed	2007-12-18 18:51:33 -07:00
Brian Warner	50bc0d2fb3	the new pyflakes is stricter, complaining about function definitions that overshadow earlier definitions or imports. Fix some of its complaints.	2007-12-18 18:47:28 -07:00
Brian Warner	318b63fbc7	hush pyflakes	2007-12-18 18:37:22 -07:00
Brian Warner	01a93dfe8f	check_load: add stats-gathering	2007-12-18 14:07:37 -07:00
Brian Warner	50840b1085	add a simple load-generating tool to do random reads and writes	2007-12-17 21:06:07 -07:00
Brian Warner	b8d77fbb46	move my_private_dir.cap into private/	2007-12-17 20:57:40 -07:00
Zooko O'Whielacronx	2a7f374e45	unit test that POST /uri/?t=mkdir works	2007-12-18 17:47:49 -07:00
Zooko O'Whielacronx	a5a54ac5ca	remove the DirnodeURI foolscap schema and mv those regexes into uri.py ... We currently do not pass dirnode uris over foolscap.	2007-12-18 17:44:24 -07:00
Zooko O'Whielacronx	9848d2043d	make more precise regexp for WriteableSSKFileURI and DirnodeURI and use it in unit tests ... Also allow an optional leading "http://127.0.0.1:8123/uri/". Also fix a few unit tests to generate bogus Dirnode URIs of the modern form instead of the former form.	2007-12-18 13:15:08 -07:00
Zooko O'Whielacronx	ea43753386	fix unit tests to assert that we do *not* link to start.html when there is no private dir, instead of asserting that we *do*	2007-12-17 18:51:16 -07:00
Zooko O'Whielacronx	a983f6d60c	rename "my_private_dir.uri" to "my_private_dir.cap"	2007-12-17 18:35:25 -07:00
Zooko O'Whielacronx	08a64c3a2b	rename "secret" to "lease_secret" and change its size from 16 to 32 bytes	2007-12-17 18:34:11 -07:00
Zooko O'Whielacronx	8c65bdcf9d	put all private state in $BASEDIR/private ... fixes #219 The only part of #219 that this doesn't include is the part about logpublisher, which has been moved out of tahoe into foolscap.	2007-12-17 16:39:54 -07:00
Zooko O'Whielacronx	ae90016845	fix the overwrite button and add unit test for it	2007-12-14 17:52:05 -07:00
Brian Warner	cd59cded91	test_system.py: oops, match change in RIControl	2007-12-14 03:08:16 -07:00
Brian Warner	7924f5ac21	test_web.py: cosmetic, remove leftover trailing space	2007-12-14 02:06:09 -07:00
Brian Warner	f6b2072af1	check-speed: test SSK upload/download speed too. SDMF imposes a limit on the file sizes, no 10MB or 100MB test	2007-12-14 02:05:31 -07:00
Brian Warner	69b65b6b01	remove logpublisher, use the Foolscap version now that this functionality has been moved into Foolscap-0.2.2	2007-12-12 20:31:01 -07:00
Brian Warner	e52e14cc66	remove tests of logging functionality that's been subsumed by foolscap logging	2007-12-12 20:23:53 -07:00
Zooko O'Whielacronx	2915f5b7f4	remove unimplemented and skipped test for feature that we don't plan to implement any time soon (XML-RPC interface)	2007-12-12 20:06:05 -07:00
Zooko O'Whielacronx	1ac09840a4	a few documentation and naming convention updates ... Notable: the argument to make REPORTER has been renamed to TRIALARGS.	2007-12-12 19:34:08 -07:00
Zooko O'Whielacronx	a7f954ac06	editing: change names like "MyThing" to "FakeThing" for fake objects in unit tests	2007-12-11 18:10:29 -07:00
Zooko O'Whielacronx	38d1af7ede	fix unit test to pass forward-slashes to the CLI since it demands that the CLI emit forward-slashes	2007-12-11 20:03:44 -07:00
Brian Warner	2cb05817a6	test_web.py: survive localdir/localfile= names with spaces. Should close #223	2007-12-11 19:47:04 -07:00
Brian Warner	b085097adc	test_system: write test data in 'b' verbatim mode, since on windows the default text-mode is different. Addresses one of the failures in #223	2007-12-11 19:16:33 -07:00
Brian Warner	74820cc3de	webish: append save=true to a GET URL to save-to-disk. Closes #222.	2007-12-11 18:04:44 -07:00
Zooko O'Whielacronx	4210ba2b8a	fix test_introducer to wait for the right things in order to avoid intermittent test failures	2007-12-11 14:08:15 -07:00
Zooko O'Whielacronx	40edccf9c5	fix IntroducerClient.when_enough_peers() ... add IntroducerClient.when_few_enough_peers(), fix and improve test_introducer	2007-12-10 20:22:59 -07:00
Zooko O'Whielacronx	9a30ab2dfd	make check-memory use the new POST /uri?t=upload	2007-12-06 17:28:23 -07:00
Zooko O'Whielacronx	8ad2e85fdf	add POST /uri?t=upload and tests thereof ... Hm... I refactored processing of segments in a way that I marked as "XXX HELP I AM YUCKY", and then I ran out of time for rerefactoring it before I committed. At least all the tests pass.	2007-12-06 17:17:02 -07:00
Brian Warner	51af1fa2e3	webish: fix overwrite form display	2007-12-05 00:15:13 -07:00
Brian Warner	53e865cf49	test_web.py: add coverage for directory listings that include mutable files	2007-12-04 23:57:40 -07:00
Brian Warner	785f21b9bf	test_web.py: add coverage for POST t=check	2007-12-04 23:49:38 -07:00
Brian Warner	56e02b274b	webish: add POST t=mutable, make it replace files in-place, add t=overwrite	2007-12-04 23:42:54 -07:00
Brian Warner	4c79b4cd07	test_web.py: remove last use of fake_* methods, remove dead code	2007-12-04 23:11:00 -07:00
Brian Warner	f190382d5e	refactor web tests, and interfaces.IFileNode	2007-12-04 23:01:37 -07:00
Brian Warner	4b8c2e93b3	checker: improve test coverage a little bit	2007-12-04 18:00:12 -07:00
Brian Warner	bf1b7d78e2	uri.py: improve test coverage	2007-12-04 17:38:31 -07:00
Brian Warner	cf2bd03813	remove most (maybe all?) traces of old Dirnode class. Yay for negative code days.	2007-12-04 17:00:58 -07:00
Brian Warner	1a5257c388	improve test coverage on FileNode.check	2007-12-04 15:55:27 -07:00
Brian Warner	0f5ef5184d	test_dirnode.py: obtain full coverage of dirnode.py	2007-12-04 14:32:04 -07:00
Brian Warner	cca166a4f5	rename dirnode2.py to dirnode.py	2007-12-04 11:45:20 -07:00
Brian Warner	b780127f49	test_mutable: improve test coverage a bit, add test_filenode.py	2007-12-03 21:37:54 -07:00
Brian Warner	955bd5383d	remove PyCrypto, now we only use pycrypto++	2007-12-03 18:10:01 -07:00
Brian Warner	87e6ccbd4a	remove leftover defer.setDebugging(), to speed up tests from 200s to 83s	2007-12-03 17:10:02 -07:00
Zooko O'Whielacronx	3605354a95	fix several bugs and warnings -- thanks, pyflakes	2007-12-03 15:42:35 -07:00
Zooko O'Whielacronx	e28d7c580c	merge patch to integrate decentralized directories with patch "download: use hierarchical logging"	2007-12-03 15:27:21 -07:00
Brian Warner	33a5f8ba6b	more hierarchical logging: download/upload/encode	2007-11-19 19:33:41 -07:00
Brian Warner	869b690378	download: use hierarchical logging	2007-11-19 19:07:10 -07:00
Brian Warner	3d046d6ac7	logging: only test log.err when Twisted is new enough to let us ignore the generated errors	2007-11-19 18:37:00 -07:00
Zooko O'Whielacronx	2c58d3399c	merge patch to integrate decentralized directories with patch to "only test log.err when Twisted is new enough to let us ignore the generated errors"	2007-12-03 15:25:14 -07:00
Brian Warner	cc1612aee5	hierarchical logging: add numbered messages and parent= args	2007-11-19 18:23:18 -07:00
Brian Warner	ce0609d29b	test_client: stall 2.0s between shutdown and restart, trying to fix cygwin	2007-11-18 20:41:26 -07:00
Brian Warner	b29ce1c30a	logpublisher: implement subscribe/publish for log, add a sample client	2007-11-16 20:07:50 -07:00
Brian Warner	8a2736480a	test_mutable: workaround: use more peers to avoid random test failures. ... The underlying issue is recorded in #211: one corrupt share in a query response will cause us to ignore the remaining shares in that response, even if they are good. In our tests (with N=10 but only 5 peers), this can leave us with too few shares to recover the file. The temporary workaround is to use 10 peers, to make sure we never get multiple shares per response. The real fix will be to fix the control flow. This fixes #209.	2007-11-15 14:55:00 -07:00
Zooko O'Whielacronx	8fee58fcba	merge patch to integrate decentralized directories with patch to handle bad hashes	2007-12-03 15:21:14 -07:00
Brian Warner	d6f2dbbac7	mutable: handle bad hashes, improve test coverage, rearrange slightly to facilitate these	2007-11-13 23:08:15 -07:00
Zooko O'Whielacronx	59d6c3c822	decentralized directories: integration and testing ... * use new decentralized directories everywhere instead of old centralized directories * provide UI to them through the web server * provide UI to them through the CLI * update unit tests to simulate decentralized mutable directories in order to test other components that rely on them * remove the notion of a "vdrive server" and a client thereof * remove the notion of a "public vdrive", which was a directory that was centrally published/subscribed automatically by the tahoe node (you can accomplish this manually by making a directory and posting the URL to it on your web site, for example) * add a notion of "wait_for_numpeers" when you need to publish data to peers, which is how many peers should be attached before you start. The default is 1. * add __repr__ for filesystem nodes (note: these reprs contain a few bits of the secret key!) * fix a few bugs where we used to equate "mutable" with "not read-only". Nowadays all directories are mutable, but some might be read-only (to you). * fix a few bugs where code wasn't aware of the new general-purpose metadata dict the comes with each filesystem edge * sundry fixes to unit tests to adjust to the new directories, e.g. don't assume that every share on disk belongs to a chk file.	2007-12-03 14:52:42 -07:00
Zooko O'Whielacronx	7b24eebd0a	unit tests: bump up a timeout which I encountered when running on a very slow machine	2007-11-29 14:47:35 -07:00
Brian Warner	1f22768dc7	webish: add preliminary mutable file support: upload, download, listings, JSON, URI, RO-URI. No replace yet.	2007-11-09 03:54:27 -07:00
Brian Warner	63233ecf37	consolidate dirnode/filenode-creation code into Client	2007-11-09 02:54:51 -07:00
Brian Warner	a4606d6560	test_system: RSA keys are even more variable than I thought, 2044..2049	2007-11-08 05:04:11 -07:00
Brian Warner	2eaa4195cf	mutable: add basic test coverage of new-dirnodes-using-mutable-files	2007-11-08 04:31:00 -07:00
Brian Warner	c538e2b020	mutable: fix multiple-versions-interfering-with-each-other bug. replace() tests now pass.	2007-11-08 04:07:33 -07:00
Brian Warner	0ad7ff48fa	test_system: RSA keys vary in size, expand valid ranges in test	2007-11-08 03:01:13 -07:00
Brian Warner	e7ad7f8791	mutable: grab encprivkey when necessary during publish, fix test_mutable	2007-11-08 02:46:27 -07:00
Brian Warner	09fd2dfb3a	mutable: rearrange order of Publish to allow replace() to work. Doesn't work yet. Also test_mutable is disabled for a while.	2007-11-07 21:01:39 -07:00
Brian Warner	c783128012	mutable: wire in RSA for real, using pycryptopp	2007-11-07 17:51:35 -07:00
Brian Warner	be94960680	mutable: test roundtrip, make it work	2007-11-07 14:19:01 -07:00
Brian Warner	a46e64b0bb	test_system.mutable: make sure we exercise FEC padding	2007-11-06 19:50:33 -07:00
Brian Warner	cc5d35cc07	dump-share: emit SDMF information too	2007-11-06 19:46:31 -07:00
Brian Warner	fdbe692810	add container_size to mutable dump-share output	2007-11-06 19:31:22 -07:00
Brian Warner	ddf4d23bef	test_system.mutable: add test coverage for the 'dump-share' debug command	2007-11-06 19:10:49 -07:00
Brian Warner	bc8605ad39	test_system: add early test for mutable slots, currently publish-only	2007-11-06 18:57:11 -07:00
Brian Warner	530085df4a	test_mutable: remove dead code	2007-11-06 18:56:39 -07:00
Brian Warner	7e43c7b5f9	mutable: fix use of storage API	2007-11-06 18:53:34 -07:00
Brian Warner	c4f7412f1c	stabilize on 20-byte nodeids everywhere, printed with foolscap's base32	2007-11-06 18:49:59 -07:00
Brian Warner	c595f6ccd0	mutable: parameterize Publish/Retrieve classes in MutableFileNode, for tests	2007-11-06 15:18:09 -07:00
Brian Warner	2ed394e471	mutable: move IV into signed prefix, add more retrieval code	2007-11-06 15:04:46 -07:00
Brian Warner	a431411418	test_mutable.py: hush pyflakes	2007-11-06 01:32:39 -07:00
Brian Warner	85a56a3d74	mutable.Publish: rearrange create() to use more of it in unit tests	2007-11-06 00:33:40 -07:00
Brian Warner	8cdda6e199	mutable.Publish: more tests	2007-11-05 22:38:43 -07:00
Brian Warner	281afe7cfc	mutable.Publish: create a dispatch_map for the benefit of recovery code, and pull pack/unpack methods out into functions	2007-11-05 22:14:59 -07:00
Brian Warner	59632c6812	mutable: use proper enable/renew/cancel secrets	2007-11-05 21:51:08 -07:00
Brian Warner	fade06ef4d	mutable: added send-messages-to-peers code, about 70% done. No recovery code yet.	2007-11-05 21:29:47 -07:00
Brian Warner	e08b091d9f	storage: rewrite slot API, now use testv_and_readv_and_writev or readv	2007-11-05 20:17:14 -07:00
Brian Warner	95a3da92fe	test_mutable: hush pyflakes warning	2007-11-05 00:41:53 -07:00
Brian Warner	27b67ef0d6	mutable: add peer-selection to Publish, and some basic unit tests	2007-11-05 00:38:07 -07:00
Brian Warner	8f21424449	storage: add readv_slots: get data from all shares	2007-11-05 00:37:01 -07:00
Brian Warner	6e5b799d46	mutable.py: add share-unpacking code, use it for more tests	2007-11-02 22:28:31 -07:00
Brian Warner	78c45c82d1	mutable: implement filenode share-packing, still pretty rough	2007-11-02 20:51:39 -07:00
Brian Warner	d772ea850d	mutable: split dirnode stuff out to dirnode2.py, will be renamed later	2007-11-01 23:46:47 -07:00
Brian Warner	1158c4aae5	test_client: increase test_reloadable inter-run timeout, cygwin was failing	2007-11-01 19:19:08 -07:00
Brian Warner	f1c3ff62c1	mutable: improve NewDirectoryNode test coverage	2007-11-01 18:35:54 -07:00
Brian Warner	d777283e9e	implement preliminary log publisher/gatherer ... This creates a Referenceable object that will eventually be able to publish log events to a remote subscriber (at present all it can do is provide version information). The FURL for this logport is written to 'logport.furl'. In addition, if a file named 'log_gatherer.furl' is present, the given target will be contacted and offered access to the logport. This can be used by a centralized logging agent to subscribe to logs, e.g. from all the nodes in a centrally-maintained storage grid. (think syslog -r, but with all the security properties of FURLs, and permitting non-printable strings and structured data). Once this framework matures a bit, it will be moved into Foolscap.	2007-11-01 17:29:15 -07:00
Brian Warner	4eab5bb50d	trailing-whitespace eradication, no functional changes	2007-11-01 15:34:12 -07:00
Brian Warner	1889af0e61	trailing-whitespace eradication, no functional changes	2007-11-01 15:25:04 -07:00
Brian Warner	20af973272	trailing-whitespace eradication, no functional changes	2007-11-01 15:25:00 -07:00
Brian Warner	1d8a4cdfe7	mutable: first pass at dirnodes, filenodes, new URIs. Some test coverage. ... The URI typenames need revision, and only a few dirnode methods are implemented. Filenodes are non-functional, but URI/key-management is in place. There are a lot of classes with names like "NewDirectoryNode" that will need to be rename once we decide what (if any) backwards compatibility want to retain.	2007-11-01 15:15:29 -07:00
Brian Warner	bcf84c1238	storage.py: fix tests, timestamps get updated when leases are renewed	2007-10-31 12:31:33 -07:00
Brian Warner	70e7961088	storage.py: more test coverage, make sure leases survive resizing	2007-10-31 12:07:47 -07:00
Brian Warner	948e6b34dd	storage.py: improve test coverage even more	2007-10-31 01:44:01 -07:00
Brian Warner	4bd739435f	storage.py: more mutable-slot coverage, renewing/cancelling leases	2007-10-31 01:31:56 -07:00
Brian Warner	f9a72e0e75	test_node.py: more coverage of Node.log	2007-10-31 00:56:59 -07:00
Brian Warner	22e3487b9e	test_node.py: improve test coverage of node.py	2007-10-31 00:54:42 -07:00
Brian Warner	256ef1bf53	mutable slots: add some test coverage for lease-addition	2007-10-31 00:38:30 -07:00
Brian Warner	68d3d62002	mutable slots: finish up basic coding on server-side containers, add some tests. Remove all caching from MutableShareFile.	2007-10-31 00:10:40 -07:00
Brian Warner	b24c2925e8	checkpointing mutable-file work. Storage layer is 80% in place.	2007-10-30 19:47:36 -07:00
Brian Warner	57f994fb02	checker: remember checker results, but only in ram for now	2007-10-22 17:46:24 -07:00
Brian Warner	e992fed1f5	cli: improve test coverage	2007-10-21 12:33:17 -07:00
Brian Warner	aff059dd4b	checker: return more information per CHK file, including the shareholder list	2007-10-17 02:25:50 -07:00
Brian Warner	d1f13fd117	add an equally-simple file-verifier	2007-10-16 12:25:09 -07:00
Brian Warner	9da1d70676	add a simple checker, for both files and directories	2007-10-15 16:16:39 -07:00
czooko	d589c385e6	make the anchor text in the WUI be descriptive of the target -- not "Click Here!"!	2007-10-22 16:56:30 -07:00
Zooko O'Whielacronx	74f52d79f2	tests: make test_encode specify the erasure coding params it wants instead of expecting the defaults to be what it wants	2007-10-15 20:07:42 -07:00
Brian Warner	07b03bd55b	test_runner.RunNode: pass an explicit webport, to avoid using 8123 (which might be in used by a running node). Closes #175.	2007-10-13 16:06:39 -07:00
Zooko O'Whielacronx	af0edec753	filter out "0.0.0.0" from detected IP addresses	2007-10-13 00:38:16 -07:00
Brian Warner	4361b32f2d	cli: implement 'mv'. Closes #162.	2007-10-11 20:31:48 -07:00
Brian Warner	8985b6565d	cli: add test coverage	2007-10-11 19:20:41 -07:00
Brian Warner	dc573554c8	rename bin/allmydata-tahoe to bin/tahoe. Closes #155.	2007-10-11 03:38:24 -07:00
Brian Warner	ee72a5adfd	rename client.tac to tahoe-client.tac, so that 'ps ax|grep tahoe' works. Closes #156.	2007-10-11 02:48:06 -07:00
Brian Warner	6ed8664c7f	check_memory.py: don't use self.keepalive_file until it's been initialized	2007-10-11 02:20:20 -07:00
Brian Warner	8ce93b5e27	test_cli.py: hush pyflakes with a dummy usage, until we get some real CLI tests	2007-10-11 01:55:29 -07:00
Brian Warner	7fd1964c6f	runner: make most commands use ~/.tahoe by default (create-client, start/stop/restart, all CLI tools, but *not* create-introducer	2007-10-11 01:54:23 -07:00
Brian Warner	a29ab33379	webish: write node.url, for the benefit of CLI tools	2007-10-11 01:38:04 -07:00
Brian Warner	5b28411b04	cli: add --node-directory and --root-uri to all commands	2007-10-11 00:30:36 -07:00
Zooko O'Whielacronx	021b5d4f2a	more information SkipTest for XMLRPC	2007-10-04 11:07:46 -07:00
Brian Warner	3fcee101ee	check_speed: measure RTT, report per-file times as a multiple of RTT	2007-09-26 13:07:33 -07:00
Brian Warner	c301b41f50	control: add measure_peer_response_time(), to estimate RTT for the mesh	2007-09-26 12:21:15 -07:00
Brian Warner	0596b9fe19	test_system.py minor typo	2007-09-26 12:07:37 -07:00
Brian Warner	0cfed96796	test_system.py: add coverage for allmydata.control	2007-09-26 12:06:55 -07:00
Brian Warner	aafccf7008	test_client.py: add TODO to test the hotline-timeout file	2007-09-25 21:03:54 -07:00
Brian Warner	c14349f695	check_speed.py: minor comment	2007-09-25 20:07:03 -07:00
Brian Warner	1691524044	check_speed.py: use more small-file tests to improve accuracy of per-file time	2007-09-25 18:57:36 -07:00
Brian Warner	50415cf9c4	check_memory: more keepalive-file debug messages	2007-09-25 18:47:48 -07:00
Brian Warner	8d7368f748	check_memory: put framework log in a stable place, add debug code to figure ... out why the keepalive-file isn't doing its job	2007-09-25 18:26:54 -07:00
Brian Warner	9b09fd3bf1	test_system.py: do one upload, then test debug scripts, then do other uploads	2007-09-24 18:12:37 -07:00
Zooko O'Whielacronx	a18f7aa6d5	fix test_vdrive (fixes #144) ... It turns out that we actually have *two* files in our storage servers at the time that test_vdrive asserts things about the shares. I suppose that test_vdrive happens to pass on all other operating systems because the filesystem happens to return the right share as the first one in a "listdir()". The fix in this patch is slightly kludgey -- allow either share to pass -- but good enough.	2007-09-22 15:26:27 -07:00
Brian Warner	ff07ce6234	check_speed: add optional 100MB test, if the 10MB test finished fast enough	2007-09-22 00:04:46 -07:00
Brian Warner	1c29065f1c	check_memory: touch the hotline file more often, to avoid accidental timeout	2007-09-20 19:35:31 -07:00
Brian Warner	d58d794757	check_speed: do both upload and download tests	2007-09-20 18:52:44 -07:00
Brian Warner	2632c0ad8c	check_speed: upload multiple files, measure Ax+B	2007-09-20 16:55:33 -07:00
Brian Warner	3774ce59ea	check_memory: fix race condition for startup of in-process server nodes	2007-09-20 15:33:58 -07:00
Brian Warner	77fae0b338	check_memory.py: record initial memory usage (before any connections are made)	2007-09-20 12:36:27 -07:00
Brian Warner	b9d5a4ead4	check_speed.py: run two 1MB uploads and measure the time it takes	2007-09-19 18:40:18 -07:00
Brian Warner	f6be35e122	started framework for an automated speed-checking tool. Doesn't do much yet.	2007-09-19 18:27:47 -07:00
Brian Warner	2b93164a03	test_runner: try harder to work on slow buildslaves and cygwin	2007-09-19 13:56:00 -07:00
Brian Warner	bede39a156	check-memory: add 'receive' mode, for #97 (consumption during share receive	2007-09-19 12:59:32 -07:00
Brian Warner	0cbf2cca08	introducer.py: add test coverage of _disconnected()	2007-09-19 11:50:13 -07:00
Brian Warner	e2f19825d9	startstop_node.py: improve test coverage a little bit	2007-09-19 01:50:27 -07:00
Brian Warner	d31d7ff941	test_system.py: do a large-file test (1.5MB) to trigger pauseProducing	2007-09-19 01:43:44 -07:00
Brian Warner	ee9b7fd678	check_memory: oops, silly bug make 'upload' push to ourselves, raising the usage	2007-09-19 04:14:48 -07:00
Brian Warner	08cc32bb77	check_memory: add download-GET-slow, to simulate memory usage of a node feeding downloaded data via HTTP GET to a slow client	2007-09-18 20:35:27 -07:00
Brian Warner	a5fab9935b	test_runner: better diagnostics in case test_client fails	2007-09-18 20:03:18 -07:00
Brian Warner	2886c5aeae	check_memory: don't accept shares for download/download-GET test, since that hits bug #97	2007-09-18 19:40:29 -07:00
Brian Warner	f6acf5a0c3	check_memory: add download, download-GET	2007-09-18 18:56:05 -07:00
Brian Warner	38946439f3	test_runner.py: fix race conditions in start/stop node, should run on cygwin now	2007-09-18 15:17:26 -07:00
Brian Warner	645927ca73	test_runner.py: skip the start/stop test when sys.platform is win32/cygwin. ... The previous twisted.python.runtime.platformType approach didn't catch cygwin for some reason.	2007-09-17 21:56:08 -07:00
Brian Warner	f72ef4d8f9	test_runner.py: add tests for startstop_node.py	2007-09-17 02:25:31 -07:00
Brian Warner	b7c18d6505	webish: improve test coverage	2007-09-17 01:53:46 -07:00
Brian Warner	79bd7d422d	provisioning.py: get full test coverage	2007-09-17 01:38:54 -07:00
Brian Warner	311ed144f8	uri.py: improve test coverage a bit	2007-09-17 01:09:47 -07:00
Brian Warner	8451b485a4	storage: fill alreadygot= with all known shares for the given storageindex, not just the ones they asked about	2007-09-17 00:48:40 -07:00
Brian Warner	808f851589	upload: make peer-selection a bit more uniform. Closes #132.	2007-09-16 17:08:34 -07:00
Brian Warner	24e6ccddce	peer-selection: if we must loop, send a minimal number of queries (by asking for more than one share per peer on the second pass)	2007-09-16 01:53:00 -07:00
Brian Warner	baa16087cd	upload: switch to Tahoe2, add test for uniform share allocation	2007-09-16 01:25:03 -07:00
Brian Warner	d628d5f503	storage: remove the leftover incoming/XYZ/ directory when we're done with it	2007-09-15 14:34:04 -07:00
Brian Warner	e9d32f14b9	check_memory.py: preserve client.log and stats.out in _test_memory/ ... Put the nodes in _test_memory/test/, which is clobbered on each test. Also kill the client with SIGINT instead of SIGKILL. Also don't daemonize the client, since we're going to kill it at the end of the test anyways: this cleans up shutdown a bit.	2007-09-15 20:53:06 -07:00
Brian Warner	c7986389e5	check_memory.py: have all clients write their logs to _test_memory/client.log instead of a separate file per client	2007-09-15 12:34:05 -07:00
Brian Warner	e7f61a3a6c	check_memory: oops, only bail if the process ended abnormally	2007-09-15 11:47:29 -07:00
Brian Warner	32f25f33e3	check-memory: if the child process fails to start, shut down instead of hanging forever	2007-09-14 20:16:57 -07:00
Brian Warner	df49fa55e0	test_client.py: remove unused 'stat' import	2007-09-13 20:17:42 -07:00
Zooko O'Whielacronx	f443f8adbc	remove test in test_client -- we can't easily assert that files aren't readable by others, on Windows	2007-09-13 20:12:26 -07:00
nejucomo	96f55be844	Add some passing unit tests for testutil.PollMixin.	2007-09-07 16:15:41 -07:00
Brian Warner	3eaa888a0e	provisioning: oops, fix test_provisioning too	2007-09-07 15:01:17 -07:00
Brian Warner	0ae7d31d33	provisioning.py: update test with new encoding-parameters format	2007-09-07 14:45:43 -07:00
Brian Warner	76be4a582c	webish: implement 'PUT /uri?t=mkdir' (to create anonymous dirnodes)	2007-09-05 17:23:06 -07:00
Brian Warner	db3f11648c	webish: implement 'PUT /uri' (to create anonymous files)	2007-09-05 17:12:27 -07:00
Brian Warner	277e720f7c	storage: add version number to share data. Closes #90.	2007-09-04 09:00:24 -07:00
Brian Warner	85f3107b12	storage: handle simultanous uploads: add a lease for the pre-empted client	2007-09-02 14:57:49 -07:00
Brian Warner	dbcabc1142	debug: 'dump-uri-extension' command becomes 'dump-share', add 'dump-share-leases'. Both display leases.	2007-09-02 14:48:20 -07:00
Brian Warner	0fe1205789	storage: replace sqlite with in-share lease records	2007-09-02 14:47:15 -07:00
Brian Warner	c7288286ae	test_uri.py: remove tiny whitespace	2007-08-28 23:40:03 -07:00
wilcoxjg	33bae16ed2	created DirnodeURI schema	2007-08-28 23:28:26 -07:00
Brian Warner	2a63fe8b01	deletion phase3: add a sqlite database to track renew/cancel-lease secrets, implement renew/cancel_lease (but nobody calls them yet). Also, move the shares from BASEDIR/storage/* down to BASEDIR/storage/shares/*	2007-08-27 23:41:40 -07:00
Brian Warner	850bc9da02	deletion phase2b: create renew/cancel secrets for real.	2007-08-27 19:30:26 -07:00
Brian Warner	56afda11d1	deletion phase2a: improve creation of renew/cancel secrets. Still fake though.	2007-08-27 19:00:18 -07:00
Brian Warner	739ae1ccde	deletion phase1: send renew/cancel-lease secrets, but my_secret is fake, and the StorageServer discards them	2007-08-27 17:28:51 -07:00
Brian Warner	ed525f7478	test_web: improve provisioning.py test coverage a bit by using a live web hit	2007-08-27 17:26:39 -07:00
Brian Warner	5946985c07	add a provisioning utility page which shows necessary storage space and transfer rates for grids of various sizes	2007-08-26 23:44:24 -07:00
Brian Warner	3e1c417f81	webish: modify JSON to match zooko's proposed API changes in #118	2007-08-23 13:00:39 -07:00
Brian Warner	f3353ee5e4	web: replace FILE links with /uri -based ones, to prevent an XSS attack against the secret vdrive URI contained in the current URL	2007-08-22 17:35:01 -07:00
Brian Warner	4de5767c98	web: remove /vdrive/private, replace with a start.html file that points at the /uri/PRIVATE_URI, to prevent XSRF attacks	2007-08-22 14:54:34 -07:00
Brian Warner	75056b6222	test_system: weaken the assertion about the 410 error message when downloading a bogus URI, since some systems emit error messages differently	2007-08-16 17:05:26 -07:00
Brian Warner	6b5e05cc67	test_system.py: verify that we can replace files in place	2007-08-16 17:03:50 -07:00
Brian Warner	fe06b3be8b	dirnode: change the defined behavior of RIVirtualDriveServer.set to allow replace-in-place without raising an exception	2007-08-16 17:03:19 -07:00
Brian Warner	be0ab3b1f1	test_system.py: many (failing) web tests were accidentally bypassed, fix those. Add some PUT tests.	2007-08-16 16:49:40 -07:00
Zooko O'Whielacronx	4ed7f0ac2a	fix small bug in unit tests which caused spurious failures on Windows	2007-08-16 14:14:41 -07:00
Zooko O'Whielacronx	094b687d6e	import version class and make-version script from pyutil -- fixes win32 build, improves error handling, and eliminates unused features	2007-08-16 14:09:30 -07:00
Brian Warner	c1da0c11bc	webish: implement replace= for POST commands	2007-08-15 15:21:38 -07:00
Brian Warner	1752c9e29e	webish: implement replace= for PUT commands	2007-08-15 13:22:23 -07:00
Brian Warner	42dcc3088e	IDirectoryNode: add has_child() method	2007-08-15 13:22:01 -07:00
Brian Warner	244471dcc4	check_memory.py: Disable the 100MB test for now: our buildslave can't currently handle it because the testnet/framework processes uses something like 600M of RSS.	2007-08-15 12:55:11 -07:00
Brian Warner	179d5462c0	webish: look for when_done= in POST fields as well as queryargs. Closes #101. ... We need to look in the fields because that's how we build the mkdir/upload forms. Without this, uploading or creating directories would leave us on a page that had just a URI, instead of something actually useful to a human.	2007-08-13 17:45:02 -07:00
Brian Warner	6c6c62037d	fix pyflakes warnings from recent b32decode change	2007-08-12 16:33:51 -07:00
Brian Warner	0332e94656	test_system.py: add coverage for get_permuted_peers()	2007-08-12 16:29:34 -07:00
Zooko O'Whielacronx	42f8e57416	don't over-encode the nodeid many times with ascii-encoding	2007-08-12 10:29:38 -07:00
Brian Warner	2f5a27316f	webish: localfile=/localdir= are now disabled by default, a special switch is required to enable them	2007-08-10 18:21:22 -07:00
Brian Warner	998802fd6d	#96: add flag to enable pushing data to ourselves, defaulting to False	2007-08-09 18:30:24 -07:00
Zooko O'Whielacronx	531109e803	fix test_web refactoring so that the WebMixin class isn't a TestCase ... Thanks, Brian.	2007-08-10 08:40:02 -07:00
Zooko O'Whielacronx	39f218ab90	refactor test_web so that other tests can use the part of test_web that sets up a simple filesystem	2007-08-09 13:08:42 -07:00
Brian Warner	9ac5c14037	introducer: don't log.err() an initial connection failure, since that flunks tests. Use self.log() instead. Also improve test_client.py to always trigger this case, before it was finishing to quickly to always hit the problem.	2007-08-09 12:53:44 -07:00
Brian Warner	84d41adfcc	check_memory.py: include a single 100MB test	2007-08-09 11:30:33 -07:00
Brian Warner	57df32aac5	now that the buildslave is moved to a new machine, enable the 50MB test	2007-08-09 01:32:52 -07:00
Zooko O'Whielacronx	42d3afffa2	refactor test_uri and add a test of the empty file	2007-07-26 10:48:30 -07:00
Brian Warner	8de93635af	test_web.py: oops, fix syntax error caused by removing comments	2007-07-25 03:21:30 -07:00
Brian Warner	6aff425fd2	test_web.py: remove spurious '# YES' lines, now that all the tests pass	2007-07-24 20:16:21 -07:00
Brian Warner	e3eba2d6c5	remove unused imports: hush pyflakes warnings	2007-07-24 18:13:58 -07:00
Zooko O'Whielacronx	32c90c6468	switch from rfc 3548 base-32 to z-base-32 except for tubids/nodeids	2007-07-24 13:46:06 -07:00
Brian Warner	e6e9ddc588	refactor upload/encode, to split encrypt and encode responsibilities	2007-07-23 19:31:53 -07:00
Brian Warner	9c5ab89afe	truncate storage index to 128 bits, since it's derived from a 128 bit AES key	2007-07-22 19:48:44 -07:00
Brian Warner	81a9904455	CHK: remove the storage index from the URI, deriving it from the key instead	2007-07-21 18:23:15 -07:00
Brian Warner	1d9a58977f	uri: implement URI-processing classes, IFileURI/IDirnodeURI, use internally	2007-07-21 15:40:36 -07:00
Brian Warner	e3a57fca98	upload: finish refactoring, all unit tests pass now	2007-07-19 22:53:29 -07:00
Brian Warner	9af506900b	upload: refactor to enable streaming upload. not all tests pass yet	2007-07-19 18:21:44 -07:00
Brian Warner	a7f04616bf	webish: improve test coverage further	2007-07-17 12:28:29 -07:00
Brian Warner	3197e1d39b	webish: improve test coverage of WebDownloadTarget	2007-07-17 12:16:45 -07:00
Brian Warner	d927d371b7	check_memory.py: test POST-based uploads as well as from-disk ones	2007-07-16 20:17:51 -07:00
Brian Warner	0269a6229b	check_memory.py: fix benign-but-noisy vdrive.furl handling bug	2007-07-16 19:34:52 -07:00
Brian Warner	0fd5dcacf6	check_memory: update it, write stats to a file, disable 50MB test for now	2007-07-16 18:08:55 -07:00
robk-org	920fed7f2a	added unit test to webish's rename function ... added unit tests to test various permutations of the rename function, and some sanity checks on the rename-form function. also added a guard to prevent '/' in from_/to_name in rename calls.	2007-07-16 17:05:01 -07:00
Brian Warner	56dcb814a8	webish: test error cases more thoroughly by looking inside the response text	2007-07-16 12:01:19 -07:00
Brian Warner	f207f4a199	webish.py: disallow slashes in POSTed filenames. Closes #75.	2007-07-16 11:53:12 -07:00
Brian Warner	225fbfee57	storage.py: add a test for the next_power_of_k fix I made a few hours ago, basically do an upload with a non-power-of-two number of segments	2007-07-13 22:24:06 -07:00
Brian Warner	6e1ad64c9d	webish.py: test that _get_or_create_directories fix I made a few hours ago	2007-07-13 22:19:16 -07:00
Brian Warner	c6f52e379a	rename storageserver.py to just storage.py, since it has both server and client sides now	2007-07-13 17:25:45 -07:00
Brian Warner	4d868e6649	fix dump-uri-extension	2007-07-13 16:58:08 -07:00
Brian Warner	7589a8ee82	storage: we must truncate short segments. Now most tests pass (except uri_extension)	2007-07-13 16:38:25 -07:00
Brian Warner	1f8e407d9c	more #85 work, system test still fails	2007-07-13 15:09:01 -07:00
Brian Warner	cd8648d39b	storage: use one file per share instead of 7 (#85). work-in-progress, tests still fail	2007-07-13 14:04:49 -07:00
Brian Warner	ba7e14a870	fix several methods to handle LIT URIs correctly, rather than assuming that all filenodes are CHK URIs	2007-07-12 16:17:49 -07:00
Brian Warner	5399395c27	allow the introducer to set default encoding parameters. Closes #84. ... By writing something like "25 75 100" into a file named 'encoding_parameters' in the central Introducer's base directory, all clients which use that introducer will be advised to use 25-out-of-100 encoding for files (i.e. 100 shares will be produced, 25 are required to reconstruct, and the upload process will be happy if it can find homes for at least 75 shares). The default values are "3 7 10". For small meshes, the defaults are probably good, but for larger ones it may be appropriate to increase the number of shares.	2007-07-12 15:33:30 -07:00
Brian Warner	def63d193e	implement URI:LIT, which stores small (<55B) files inside the URI itself. Fixes #81.	2007-07-12 13:22:36 -07:00
Brian Warner	4a16d20d85	runner.py: start to refactor into separate files	2007-07-10 18:41:52 -07:00
Brian Warner	13e84526a1	webapi: normalized API: use t=upload or t=download when providing localdir= or localfile=	2007-07-10 13:24:10 -07:00
Brian Warner	dce1dc2730	storage: wrap buckets in a local proxy ... This will make it easier to change RIBucketWriter in the future to reduce the wire protocol to just open/write(offset,data)/close, and do all the structuring on the client end. The ultimate goal is to store each bucket in a single file, to reduce the considerable filesystem-quantization/inode overhead on the storage servers.	2007-07-08 23:27:46 -07:00
Brian Warner	72fc8c5cb8	web: use real JSON instead of the fake stubs ... Also include the encoder portion of Bob Ippolito's simplejson-1.7.1 as allmydata.util.json_encoder . simplejson is distributed under a more liberal license than Tahoe (looks to be modified BSD), so redistributing it should be ok.	2007-07-08 00:17:11 -07:00
Brian Warner	2d06a66ec1	test_web.test_welcome: give the rooturl a trailing slash, otherwise older versions of twisted complain	2007-07-07 23:22:52 -07:00
Brian Warner	bd8625076a	web: remove t=XML, and other dead code	2007-07-07 22:55:15 -07:00
Brian Warner	159f8bfd50	web: improve test coverage	2007-07-07 22:47:18 -07:00
Brian Warner	e2f99247f2	web: make sure we present read-only directories properly	2007-07-07 22:21:20 -07:00
Brian Warner	62e8528cc6	web: /uri/ must escape slashes, we use bangs for this	2007-07-07 22:06:52 -07:00
Brian Warner	617a85df67	web: remove debug prints	2007-07-07 20:11:30 -07:00
Brian Warner	464f25e5f2	web: more test work, now all tests pass, POST too, only XMLRPC left to implement	2007-07-07 20:06:58 -07:00
Brian Warner	f35c9c6540	test_system.py: change/remove the tests that currently fail due to web changes	2007-07-07 20:06:44 -07:00
Brian Warner	0cd730a7b3	web: more test work, now all tests either pass or are skipped (POST, XMLRPC, and URI/)	2007-07-07 10:34:05 -07:00
Brian Warner	d501984eba	webapi: checkpointing more test progress	2007-07-07 00:16:36 -07:00
Brian Warner	6570253d6b	checkpointing new webapi: not all tests pass yet	2007-07-06 19:43:55 -07:00
Brian Warner	21e12f383d	web: missed a IndexError-to-KeyError conversion	2007-07-06 19:43:03 -07:00
Brian Warner	71c04fc2e7	web: use KeyError (rather than IndexError) to signal a missing child	2007-07-06 19:40:08 -07:00
Brian Warner	9e42dda6a4	add IDirectoryNode.get_child_at_path	2007-07-06 19:38:37 -07:00
Brian Warner	a1c97aa0be	storageserver: include metadata in the sizelimit, at least after the bucket has been closed	2007-07-03 17:38:49 -07:00
Brian Warner	94e6e6160b	activate storage size limits in the client. Closes #34. ... To use this, write a number like 10MB or 5Gb or 5000000000 to a file named 'sizelimit' in the client's base directory. The node will not grant leases for shares that would take it much beyond this many bytes of storage. Note that metadata is not included in the allocation count until a restart, so the actual space consumed may grow beyond the limit if the node is not restarted very frequently and the amount of metadata is significant.	2007-07-03 17:27:07 -07:00
Brian Warner	c80ea7d693	storageserver: implement size limits. No code to enable them yet, though	2007-07-03 17:08:02 -07:00
Brian Warner	9ddb929651	fileutil: add du() function	2007-07-03 15:49:45 -07:00
Brian Warner	622acc690a	webish: improve reporting of web download errors that occur early enough ... If the error occurs before any data has been sent, we can give a sensible error message (code 500, stack trace, etc). This will cover most of the error cases. The ones that aren't covered are when we run out of good peers after successfully decoding the first segment, either because they go away or because their shares are corrupt.	2007-07-03 13:47:37 -07:00
Brian Warner	f15bb302a1	webish.py: handle errors during download better. Addresses #65. ... Previously, exceptions during a web download caused a hang rather than some kind of exception or error message. This patch improves the situation by terminating the HTTP download rather than letting it hang forever. The behavior still isn't ideal, however, because the error can occur too late to abort the HTTP request cleanly (i.e. with an error code). In fact, the Content-Type header and response code have already been set by the time any download errors have been detected, so the browser is committed to displaying an image or whatever (thus any error message we put into the stream is unlikely to be displayed in a meaningful way).	2007-07-03 13:18:14 -07:00
Brian Warner	7e32d3c8af	test_util: improve test coverage of allmydata.util.fileutil	2007-07-03 11:15:05 -07:00
Brian Warner	9d47c2524e	test_system: stall for a second while bouncing the client, it might help windows	2007-06-28 19:20:28 -07:00
Brian Warner	efb99078b6	change IVirtualDrive.get_node_at_path to accept either a list or a single slash-separated string	2007-06-28 17:46:14 -07:00
Brian Warner	5854916299	test_system.py: more vdrive coverage	2007-06-28 15:05:33 -07:00
Brian Warner	d6e732b65f	vdrive: add get_node_at_path(pathlist)	2007-06-28 11:00:03 -07:00
Brian Warner	b06c74c2a1	Add the 'vdrive' service, for clients to access the public/private root dirs. ... These allow client-side code to conveniently retrieve the IDirectoryNode instances for both the global shared public root directory, and the per-user private root directory.	2007-06-27 17:11:06 -07:00
Brian Warner	18ab5ce837	dirnode: add build_manifest() and introduce 'refresh capabilities'	2007-06-26 19:41:20 -07:00
Brian Warner	b11fa20191	merge vdrive.py and filetable.py into a single dirnode.py	2007-06-26 17:16:58 -07:00
Brian Warner	78c2376fa3	more runner.py test coverage: don't bypass argv parsing	2007-06-26 16:51:38 -07:00
Brian Warner	912fc5a77e	runner.py: improve test coverage further: implement --quiet with StringIOs	2007-06-26 16:19:18 -07:00
Brian Warner	de24d3cd94	dump_uri_extension: improve test coverage of runner.py	2007-06-26 15:55:00 -07:00
Brian Warner	a505560bbe	improve test coverage of runner.py	2007-06-26 15:36:46 -07:00
Brian Warner	bc2603c818	vdrive: protect dirnode contents with an HMAC	2007-06-26 12:36:21 -07:00
Brian Warner	4a0682874b	remove old filetree code	2007-06-25 20:34:19 -07:00
Brian Warner	fb02488a8e	vdrive: switch to URI:DIR and URI:DIR-RO, providing transitive readonlyness	2007-06-25 13:23:51 -07:00
Brian Warner	53da82ec04	test_system.py: match change to /global_vdrive URL	2007-06-15 01:38:55 -07:00
Brian Warner	82c38d370a	make new vdrive work, implement convenience wrapper, passes all tests	2007-06-15 00:37:32 -07:00
Brian Warner	5e5347a1bc	test_filetable: fix to match new vdrive approach	2007-06-15 00:01:01 -07:00
Brian Warner	ec77a227be	revamp vdrive: nodes with furls. tests still fail.	2007-06-14 20:14:34 -07:00
Brian Warner	106177a7f2	filetable: switch to new approach with anonymous nodes	2007-06-14 17:24:56 -07:00
Brian Warner	956d5ae256	rename fileid/verifierid to plaintext_hash/crypttext_hash	2007-06-09 20:46:04 -07:00
Brian Warner	c9ef291c02	rename thingA to 'uri extension'	2007-06-08 15:59:16 -07:00
Brian Warner	72bc8627de	consolidate multiple definitions of NotEnoughPeersError	2007-06-07 22:20:55 -07:00
Brian Warner	f62a544b93	remove several leftover defintions of netstring()	2007-06-07 22:13:18 -07:00
Brian Warner	6df5c856ac	test_client.py: improve test coverage a bit	2007-06-07 22:09:02 -07:00
Brian Warner	c049941529	move almost all hashing to SHA256, consolidate into hashutil.py ... The only SHA-1 hash that remains is used in the permutation of nodeids, where we need to decide if we care about performance or long-term security. I suspect that we could use a much weaker hash (and faster) hash for this purpose. In the long run, we'll be doing thousands of such hashes for each file uploaded or downloaded (one per known peer).	2007-06-07 21:47:21 -07:00
Brian Warner	cabba59fe7	test_encode.py: even more testing of merkle trees, getting fairly comprehensive now	2007-06-07 21:24:39 -07:00
Brian Warner	053109b28b	add tests for bad/inconsistent plaintext/crypttext merkle tree hashes	2007-06-07 19:32:29 -07:00
Brian Warner	4f001bedb3	test_encode.py: further refactoring of send_and_recover	2007-06-07 18:36:25 -07:00
Brian Warner	c7160af7ee	test_encode.py: refactor send_and_recover a bit	2007-06-07 18:24:26 -07:00
Brian Warner	8907e04ef5	client.py: allow operation without vdrive.furl, for storage-only no-UI nodes	2007-06-07 17:55:49 -07:00
Brian Warner	e04ff3adac	fetch plaintext/crypttext merkle trees during download, but don't check the segments against them yet	2007-06-07 00:15:41 -07:00
Brian Warner	5cbdc240e2	encode: add plaintext/crypttext merkle trees to the shares, and the thingA block. Still needs tests and download-side verification.	2007-06-06 19:40:20 -07:00
Brian Warner	6bb9debc16	encode: tolerate lost peers, as long as we still get enough shares out. Closes #17.	2007-06-06 10:32:40 -07:00
Brian Warner	3dfd26970b	move validation data to thingA, URI has storage_index plus thingA hash ... This (compatibility-breaking) change moves much of the validation data and encoding parameters out of the URI and into the so-called "thingA" block (which will get a better name as soon as we find one we're comfortable with). The URI retains the "storage_index" (a generalized term for the role that we're currently using the verifierid for, the unique index for each file that gets used by storage servers to decide which shares to return), the decryption key, the needed_shares/total_shares counts (since they affect peer selection), and the hash of the thingA block. This shortens the URI and lets us add more kinds of validation data without growing the URI (like plaintext merkle trees, to enable strong incremental plaintext validation), at the cost of maybe 150 bytes of alacrity. Each storage server holds an identical copy of the thingA block. This is an incompatible change: new messages have been added to the storage server interface, and the URI format has changed drastically.	2007-06-01 18:48:01 -07:00
Brian Warner	edaed9c32c	test_system.py: minor reformatting	2007-05-31 18:31:01 -07:00
Brian Warner	8875adf807	test_system.py: add a log message to help track down the occasional cygwin failure	2007-05-31 12:01:14 -07:00
Brian Warner	ea78b4b605	check_memory: getting closer, now we have memusage numbers for uploads of 10kB and 10MB files	2007-05-29 17:39:39 -07:00
Brian Warner	c405c6117f	check_memory.py: finish the failsafe-shutdown code	2007-05-24 17:34:42 -07:00
Brian Warner	8fc1bf4330	test_node.py: hush pyflakes warnings	2007-05-23 17:55:04 -07:00
Brian Warner	2d21029367	test_introducer_and_vdrive: remove the assumption that startService returns a Deferred	2007-05-23 17:37:20 -07:00
Brian Warner	83daa265ba	test_system.py: check size in web-ui display. closes #37	2007-05-23 17:25:49 -07:00
Zooko O'Whielacronx	44902c5152	add unit test for "advertised_ip_addresses" feature and fix bug in that feature uncovered by this unit test	2007-05-23 15:08:55 -07:00
Zooko O'Whielacronx	3872e94da2	rename all "*PBURL*" to "*FURL*" ... This breaks backwards compatibility with Tahoe v0.2 -- the first public release of Tahoe.	2007-05-22 14:08:30 -07:00
Brian Warner	05163ec8e1	change uri-packer-unpacker to deal with dictionaries, not fragile tuples	2007-05-23 11:18:49 -07:00
Brian Warner	82ba0f8540	UNDO: forget about old peers (closes #26) ... Add a new method to RIIntroducer, to allow the central introducer node to remove peers from the active set after they've gone away. Without this, client nodes accumulate stale peer FURLs forever. This introduces a compatibility break, as old introducers won't know about the 'lost_peers' message, although the errors produced are probably harmless.	2007-05-07 19:10:24 -07:00
Brian Warner	04af4a48b5	forget about old peers (closes #26) ... Add a new method to RIIntroducer, to allow the central introducer node to remove peers from the active set after they've gone away. Without this, client nodes accumulate stale peer FURLs forever. This introduces a compatibility break, as old introducers won't know about the 'lost_peers' message, although the errors produced are probably harmless.	2007-05-07 19:10:24 -07:00
Brian Warner	5b3a1cdc56	test_introducer.py: turn off the defer.setDebugging(True) call that was left over from a troubleshooting session: it slows down the test suite by a factor of 2	2007-05-04 00:45:23 -07:00
Zooko O'Whielacronx	ff08abb67e	bump test timeout even higher ... It does indeed take longer than 2400 seconds to run test_upload_and_download on a virtual windows machine when the underlying real machine is heavily loaded down with filesystem analysis runs...	2007-05-02 15:31:05 -07:00
Zooko O'Whielacronx	e697c84df1	bump timeout up because it timed out even on my super fast MacBook Pro ... Hm. This probably means that it is never going to finish...	2007-04-30 23:16:06 -07:00
Zooko O'Whielacronx	137a427a47	bump some unit tests up to very high timeouts because my poor G4 867 MHz PowerBook, which is busy doing video iChat, takes a long time to run these tests	2007-04-30 21:03:43 -07:00
Brian Warner	461d64e371	test_system: increase timeouts to almost 20 minutes: we have buildslaves that take more than 5 minutes to finish these tests.	2007-04-30 20:14:58 -07:00
Zooko O'Whielacronx	e0a18d12af	globally search and replace "mesh" with "grid" and adjust description of the effect of NAT on the topology	2007-04-30 13:06:09 -07:00
Zooko O'Whielacronx	d19d1058e0	extirpate all references the "queen" and "metatracker" ... This is a potentially disruptive and potentially ugly change to the code base, because I renamed the object that serves in both roles from "Queen" to "IntroducerAndVdrive", which is a bit of an ugly name. However, I think that clarity is important enough in this release to make this change. All unit tests pass. I'm now darcs recording this patch in order to pull it to other machines for more testing.	2007-04-30 09:57:52 -07:00
Brian Warner	5ceeaaea6a	add RIClient.get_versions, in the hopes of enabling backwards-compatibility code in the future	2007-04-26 12:01:25 -07:00
Brian Warner	4b2298937b	use real encryption, generate/store/verify verifierid and fileid	2007-04-25 17:53:10 -07:00
Brian Warner	9251b6e85e	test_system: improve webish.py coverage	2007-04-24 01:41:54 -07:00
Brian Warner	813fdd70da	test_runner.py: improve test coverage a little bit	2007-04-23 21:28:19 -07:00
Brian Warner	d1a848e0a7	runner.py: add --quiet, use it from test cases	2007-04-23 21:17:13 -07:00
Brian Warner	8307aaccb6	testutil: make SignalMixin actually be a mixin (and not inherit from TestCase), use it from all tests that start notes and thus exec ifconfig	2007-04-23 21:15:02 -07:00
Brian Warner	78430d3124	test_upload.py: add some TODO items	2007-04-23 17:30:40 -07:00
Brian Warner	491f96258e	test_runner.py: add some coverage for allmydata.scripts.runner, to create nodes	2007-04-19 18:56:45 -07:00
Brian Warner	49e992b8b6	make test_encode less CPU-intense by using 4-out-of-10 encoding instead of 25-out-of-100	2007-04-19 10:56:15 -07:00
Brian Warner	073333c791	iputil/testutil: fix pyflakes errors/warnings	2007-04-18 18:33:37 -07:00
Brian Warner	2d0e240466	encode: handle uploads of the same file multiple times. Unfortunately we have to do almost as much work the second time around, to compute the full URI	2007-04-18 18:29:10 -07:00
Brian Warner	d1c00bd527	test_system: turn off test_connections, since it is slow and subsumed by the other system tests	2007-04-18 16:06:57 -07:00
Brian Warner	91d72bb504	test_storage: test StorageServer code too: allocation, lookup, multiple-writer behavior	2007-04-18 15:42:34 -07:00
Zooko O'Whielacronx	a154641462	iputil.list_async_addresses now "works" on cygwin	2007-04-18 17:30:08 -07:00
Brian Warner	a4d7607a9e	test_system: minor comment	2007-04-17 20:28:19 -07:00
Brian Warner	008e418523	storageserver: assert that blocks are written in-order, clean up tests a bit	2007-04-17 20:21:05 -07:00
Brian Warner	e040b85f5d	test_storage: add (failing) test of the BucketWriter/BucketReader implementation	2007-04-17 20:03:44 -07:00
Brian Warner	2e15c9aed2	system_test: exercise multiple segments	2007-04-17 13:40:47 -07:00
Brian Warner	a0dc26ee11	test_encode.Roundtrip: cover more combinations of data size relative to segment size and number of block hash tree leaves	2007-04-17 12:57:55 -07:00
Brian Warner	96812507a0	test_encode.Encode: cover more combinations of data size relative to segment size and number of block hash tree leaves	2007-04-17 12:29:56 -07:00
Brian Warner	a05b713076	test_encode: test filesizes which are an exact multiple of the segment size. This test fails right now.	2007-04-16 19:55:03 -07:00
Brian Warner	ff8cb4d32e	encode: make MAX_SEGMENT_SIZE controllable, to support tests which force the use of multiple segments. Also, remove not-very-useful upload-side debug messages	2007-04-16 19:29:57 -07:00
Brian Warner	b9624502c9	download: more test coverage	2007-04-16 17:21:37 -07:00
Brian Warner	2f5fb51848	download: validate handling of missing sharehashes too	2007-04-16 17:15:44 -07:00
Brian Warner	42179e5ae2	download: verify that bad blocks or hashes are caught by the download process	2007-04-16 16:30:21 -07:00
Brian Warner	281a0d376c	test_system: bump up timeout again, the new extra download forms take more time on poor overloaded slave1	2007-04-16 15:53:25 -07:00
Brian Warner	88a7fdcaab	test_iputil: improve error message	2007-04-16 15:05:25 -07:00
Brian Warner	7dabb68a51	download: improve test coverage on our IDownloadTarget classes, make FileHandle return the filehandle when its done so that it is easier to close	2007-04-16 13:07:36 -07:00
Brian Warner	0ecb26ec44	test_hashtree.py: get full coverage for hashtree.py	2007-04-12 19:58:13 -07:00
Brian Warner	30133a7cdf	hash trees: further cleanup, to make sure we're validating the right thing ... hashtree.py: improve the methods available for finding out which hash nodes are needed. Change set_hashes() to require that every hash provided can be validated up to the root. download.py: validate from the top down, including the URI-derived roothash in the share hash tree, and stashing the thus-validated share hash for use in the block hash tree.	2007-04-12 19:41:48 -07:00
Brian Warner	d8215e0c6f	rename chunk.py to hashtree.py	2007-04-12 13:13:25 -07:00
Brian Warner	8f58b30db9	verify hash chains on incoming blocks ... Implement enough of chunk.IncompleteHashTree to be usable. Rearrange download: all block/hash requests now go through a ValidatedBucket instance, which is responsible for retrieving and verifying hashes before providing validated data. Download was changed to use ValidatedBuckets everywhere instead of unwrapped RIBucketReader references.	2007-04-12 13:07:40 -07:00
Brian Warner	64ea644a48	test_util: add more coverage for assertutil.py	2007-04-08 13:02:13 -07:00
Brian Warner	6f4280cc38	test_util: add full coverage for mathutil.py	2007-04-08 12:43:01 -07:00
Brian Warner	fe786350dc	test_system: bump up timeouts for the sake of slow slave1, give each test a separate base directory	2007-04-06 20:34:32 -07:00
Brian Warner	6a6631a92e	test_util.py: sigh, one last minor python-2.5 issue	2007-04-06 17:21:25 -07:00
Brian Warner	736fbe6ea5	test_util.py: fix another minor python-2.5 issue	2007-04-06 17:12:26 -07:00
Brian Warner	e74e370d00	test_util.py: fix problems	2007-04-06 16:36:22 -07:00
Brian Warner	d432099880	more allmydata.util test coverage, rename the test case to be more correct	2007-04-06 16:33:22 -07:00
Brian Warner	1a4cf6f7b7	add test coverage for stuff in allmydata/utils	2007-04-06 16:29:25 -07:00
Brian Warner	654854635a	chunk: add IncompleteHashTree for download purposes, plus tests	2007-04-06 09:09:57 -07:00
Brian Warner	8d2def5b04	encode: clean up some weirdness that was there to make unit tests easier to write	2007-04-05 22:36:18 -07:00
Brian Warner	919ca3e902	rename encode_new.py to encode.py, now that there isn't an old one anymore	2007-04-05 21:17:42 -07:00
Brian Warner	2122fbaca9	tests: clean up tearDown to use flushEventualQueue instead of hacking fixed-time delays	2007-04-04 16:09:13 -07:00
Zooko O'Whielacronx	223bf98992	use the "binary" flag on open() for files that shouldn't have line-endings automatically converted	2007-04-04 16:12:30 -07:00
Brian Warner	8bc248a515	remove util.ring, no longer used. also remove test_ring.py	2007-03-31 20:38:43 -07:00
Brian Warner	7cd9ef3bbf	finish making the new encoder/decoder/upload/download work	2007-03-30 16:50:50 -07:00
Zooko O'Whielacronx	3d694a90f3	add unit tests and fix bugs in upload	2007-03-30 14:54:33 -07:00
Brian Warner	234b2f354e	add new test for doing an encode/decode round trip, and make it almost work	2007-03-30 13:20:01 -07:00
Brian Warner	9a2e0cf28e	switch upload to use encode_new, fix a few things (but not nearly all of them)	2007-03-30 11:53:03 -07:00
Brian Warner	7d7ce7a059	test_encode: make sure encode_new can produce the data it is supposed to	2007-03-30 11:32:57 -07:00
Zooko O'Whielacronx	f4a718c5b6	finish storage server and write new download	2007-03-30 10:52:19 -07:00
Zooko O'Whielacronx	17299fc96e	new upload and storage server	2007-03-29 20:19:52 -07:00
Brian Warner	60bddba827	webish: add 'my nodeid' to the page	2007-03-29 14:31:55 -07:00
Zooko O'Whielacronx	4b4f5bbcba	change #!/usr/bin/python to #!/usr/bin/env python ... Note that using "whatever version of python the name 'python' maps to in the current shell environment" is more error-prone that specifying which python you mean, such as by executing "/usr/bin/python setup.py" instead of executing "./setup.py". When you build tahoe (by running "make") it will make a copy of bin/allmydata-tahoe in instdir/bin/allmydata-tahoe with the shebang line rewritten to execute the specific version of python that was used when building instead of to execute "/usr/bin/env python". However, it seems better that the default for lazy people be "whatever 'python' means currently" instead of "whatever 'python' meant to the manufacturer of your operating system".	2007-03-29 14:01:28 -07:00
Brian Warner	93c4a5ebb0	test_iputil: remove the test that only works on linux, since we're using the cross-unix 'get_local_addresses_async' anyways. This should allow the tests to pass on OS-X	2007-03-29 11:21:17 -07:00
Brian Warner	7802aa5303	hush pyflakes warnings in codec and test_codec	2007-03-28 15:31:51 -07:00
Zooko O'Whielacronx	e4463056f3	fix test_codec and test_upload to handle current API	2007-03-27 22:57:15 -07:00
Brian Warner	611178490a	fix some python2.5 incompatibilities, and remove an old webish display that suggested we might know about peers but not be connected to them	2007-03-27 17:44:49 -07:00
Brian Warner	51ad4418f7	test_introducer: flushEventualQueue at the end of the test run	2007-03-27 17:16:13 -07:00
Brian Warner	25ff9e1f97	complete the Introducer changes, separate out vdrive access, make everything work again	2007-03-27 16:12:11 -07:00
Zooko O'Whielacronx	e48a8b8e66	incomplete work to be finished elsewhere	2007-03-23 16:15:57 -07:00
Brian Warner	1264d77fd5	hush a pyflakes warning	2007-03-22 22:22:00 -07:00
Brian Warner	ca33c8f848	more work on a memory-footprint test program	2007-03-12 16:28:37 -07:00
Brian Warner	1e25c9e897	start work on a memory-measuring test tool	2007-03-08 18:12:24 -07:00
Brian Warner	e08308e019	test_system.py: remove the lowered (20s) timeouts, since some buildslaves require more like 30 or 40 seconds to complete the test	2007-03-08 17:30:09 -07:00
Brian Warner	b9c143b0e8	test_observer: improve test coverage	2007-03-08 16:13:14 -07:00
Brian Warner	78d19c271c	rearrange service startup a bit, now Node.startService() returns a Deferred that fires when the tub is actually ready, and there is also a Node.when_tub_ready() hook. This allows get_local_addresses() to be slow and not break everything. Changed all necessary test cases to accomodate this slow startup.	2007-03-08 15:10:36 -07:00
Brian Warner	2f02659b5a	test_observer.py: remove the code that allows observers to run out-of-order, our use of foolscap.eventual removes this error case	2007-03-08 15:09:29 -07:00
Brian Warner	929d725577	add OneShotObserverList from the amdlib tree	2007-03-08 15:07:38 -07:00
Brian Warner	2c261ce996	change node startup to put all local addresses in the PBURL, including 127.0.0.1. This should facilitate testing on both connected and disconnected systems.	2007-03-07 18:43:17 -07:00
Brian Warner	18325251bf	iputil: add get_local_addresses(), an attempt to enumerate all IPv4 addresses on this host. This is pretty unix-specific for right now (it calls ifconfig)	2007-03-07 18:22:30 -07:00
Brian Warner	9d123605cb	hush pyflakes warnings	2007-02-01 17:13:01 -07:00
Zooko O'Whielacronx	dd4ad3d542	use pyfec instead of py_ecc for erasure coding and update API to codec	2007-02-01 16:07:00 -07:00
Brian Warner	ef73ebaf0a	download: update all users to match Zooko's change to ICodecDecoder.decode (as it now returns a list instead of a single string)	2007-01-24 17:23:22 -07:00
Brian Warner	bbf188d2c6	filetree: make delete() work	2007-01-24 15:10:53 -07:00
Brian Warner	22731125f3	filetree: change the way addpath works, now we add workqueue steps for all involved subtrees at about the same time, rather than letting one step add the next when it runs. Finally add a (passing) test for uploading files to CHK-based directories	2007-01-22 01:06:09 -07:00
Brian Warner	9dc1c0cfc0	filetree: add vdrive upload/download test, change workqueue relative-filename semantics	2007-01-21 16:03:15 -07:00
Brian Warner	81d093b649	move IWorkQueue into allmydata.interfaces, give VirtualDrive an uploader	2007-01-21 15:15:31 -07:00
Brian Warner	430b3a03fc	move upload/download interfaces to allmydata.interfaces, let SubTreeMaker assert IDownloader-ness of its 'downloader' argument	2007-01-21 15:01:34 -07:00
Brian Warner	a8ecaf45b6	test_filetree: more vdrive._get_file_uri() coverage	2007-01-21 13:39:40 -07:00
Brian Warner	124d531b41	filetree: mark leaf nodes by adding is_leaf_subtree(), stop traversing when we hit them, to make vdrive._get_file_uri() work	2007-01-21 13:31:16 -07:00
Brian Warner	9e7dbf20bc	filetree: test NoSuchDirectoryError in vdrive.list()	2007-01-21 04:50:37 -07:00
Brian Warner	7b8c524d7c	more filetree, workqueue-boxes now hold serialized Nodes, move NodeMaker out to a separate module	2007-01-21 04:18:54 -07:00
Brian Warner	324033c9e2	filetree: start testing IVirtualDrive, beginning with list()	2007-01-20 21:14:41 -07:00
Brian Warner	8c7d33f4a2	filetree: put SubTreeMaker and NodeMaker in separate classes	2007-01-20 17:04:56 -07:00
Brian Warner	ce4610c3e6	more filetree work, more tests now pass	2007-01-20 15:50:21 -07:00
Brian Warner	b61a4ff371	more filetree, it's actually starting to make sense now	2007-01-20 14:41:51 -07:00
Brian Warner	3d76250894	filetree: more tests, still very early	2007-01-20 05:13:15 -07:00
Brian Warner	c808d5a5ef	filetree: refactor INode serialization, start on tests	2007-01-20 04:52:53 -07:00
Brian Warner	8921c1b666	snapshot filetree work: fix pyflakes complaints	2007-01-19 02:35:36 -07:00
Brian Warner	ff6b09d973	snapshot filetree work: it's getting close	2007-01-19 02:23:03 -07:00
Brian Warner	56df1a48f4	improve test coverage a bit	2007-01-17 15:34:29 -07:00
Brian Warner	c94098b93a	split filetree_new.py up into smaller pieces, in a new subpackage	2007-01-17 13:54:38 -07:00
Brian Warner	18ec38acf0	hush pyflakes warnings	2007-01-16 21:34:34 -07:00
Brian Warner	4101bcf218	update URI format, include codec name	2007-01-16 21:29:59 -07:00
Brian Warner	56cf2dbd7e	now that foolscap-0.0.7 accepts connect-to-self, allow peers to know about themselves. We now require foolscap-0.0.7	2007-01-16 15:12:49 -07:00
Brian Warner	2150ac0e53	hush pyflakes warnings	2007-01-15 21:31:10 -07:00
Brian Warner	3209fd5e09	rearrange encode/upload, add URIs, switch to ReplicatingEncoder ... Added metadata to the bucket store, which is used to hold the share number (but the bucket doesn't know that, it just gets a string). Modified the codec interfaces a bit. Try to pass around URIs to/from download/upload instead of verifierids. URI format is still in flux. Change the current (primitive) file encoder to use a ReplicatingEncoder because it provides ICodecEncoder. We will be moving to the (less primitive) file encoder (currently in allmydata.encode_new) eventually, but for now this change lets us test out PyRS or zooko's upcoming C-based RS codec in something larger than a single unit test. This primitive file encoder only uses a single segment, and has no merkle trees. Also added allmydata.util.deferredutil for a DeferredList wrapper that errbacks (but only when all component Deferreds have fired) if there were any errors, which unfortunately is not a behavior available from the standard DeferredList.	2007-01-15 21:22:22 -07:00
Brian Warner	78a9e815c5	add simple metadata (a single string) to the storage protocol	2007-01-15 14:01:22 -07:00
Brian Warner	417c17755b	use the word 'codec' for erasure coding, for now. 'encode' is used for file-level segmentation/hashing	2007-01-11 20:51:27 -07:00
Brian Warner	ceda350892	test_system.py: exercise queen.Roster._lost_node too	2007-01-09 19:40:36 -07:00
Brian Warner	aad0a9dfac	workqueue: more tests	2007-01-09 01:58:50 -07:00
Brian Warner	d61c0a6ef6	workqueue: more tests, coverage now at 63.4%, yay	2007-01-09 01:36:12 -07:00
Brian Warner	b641f6cbc7	workqueue: more improvements, more tests	2007-01-08 22:29:42 -07:00
Brian Warner	b9edb02820	workqueue: start adding tests	2007-01-08 21:29:33 -07:00
Brian Warner	2567a26519	rename the new filetable code to 'filetree', since robk astutely pointed out that 'table' is misleading and implies a flat list of files in a single directory	2007-01-08 16:29:14 -07:00
Brian Warner	2566452091	test_encode_share.py: fix some pyflakes warnings	2007-01-05 18:53:22 -07:00
Brian Warner	4ba6f47d2c	add some (disabled) encoder benchmarking code	2007-01-05 18:46:03 -07:00
Brian Warner	42c0d2e336	disable figleaf tracing during py_ecc, since it takes *forever*, especially on the slow buildslave	2007-01-05 18:12:04 -07:00
Brian Warner	409d92e746	only run a single (short) py_ecc test on slave3, since it is so slow the tests timeout	2007-01-05 00:42:52 -07:00
Brian Warner	e1c6ee9dcf	encoding: fix the last py_ecc problem, tests pass now	2007-01-05 00:06:42 -07:00
Brian Warner	c91d14dca8	fix our use of py_ecc (set log2FieldSize=8 explicitly)	2007-01-04 23:50:21 -07:00
Brian Warner	f31fc06d89	establish IEncoder/IDecoder, create suitable interfaces for both the simple replicating encoder and the py_ecc one, add a (failing) unit test for it	2007-01-04 21:52:51 -07:00
Brian Warner	fa11c3f722	more pyflakes cleanups	2007-01-04 18:06:20 -07:00
Brian Warner	f97eec893e	figleaf: move a copy into allmydata.util.figleaf, update Makefile/trial stuff	2007-01-03 21:38:29 -07:00
Brian Warner	e600571f82	checkpoint work-in-progress for WorkQueue, a disk-persistent list of work to be done	2007-01-01 23:47:16 -07:00
Brian Warner	cf53abab42	more filetable_new tests	2006-12-25 00:56:18 -07:00
Brian Warner	70f5c13e26	checkpointing new filetable work.. tests don't pass yet	2006-12-24 12:39:24 -07:00
Brian Warner	eb3b9b16af	encode: start to fix a few problems, still a lot of work left to go	2006-12-14 04:31:17 -07:00
Brian Warner	3e5e2ee9fe	make initial simple encode_new test pass	2006-12-14 04:17:01 -07:00
Brian Warner	3490378551	move all packages into src/, fix allmydata.Crypto build. Now you must perform a 'setup.py build' before using anything, and you must add the build directory (build/lib.linux-i686-2.4) to your PYTHONPATH before doing anything	2006-12-14 03:39:50 -07:00