1474 Commits

Author	SHA1	Message	Date
Zooko O'Whielacronx	23961448da	a few formatting tidy-ups	2008-01-03 17:14:19 -07:00
Brian Warner	acfb11d26f	test_client.py: validate more versioning code	2008-01-03 14:38:24 -07:00
Brian Warner	6a2e5d4aea	test_client.py: assert allmydata.__version__ is not unknown	2008-01-03 14:34:59 -07:00
Zooko O'Whielacronx	5995d11a3c	remove automatic private dir ... * rename my_private_dir.cap to root_dir.cap * move it into the private subdir * change the cmdline argument "--root-uri=[private]" to "--dir-uri=[root]"	2008-01-03 17:02:05 -07:00
Zooko O'Whielacronx	eb373c0001	add human-encodings of caps, refactor encodings of caps, tighten checks, fix unit tests to use values of the right size	2008-01-03 16:55:43 -07:00
Zooko O'Whielacronx	46db7cf1dc	add regexes to idlib to match certain kinds of base32-encoded values	2008-01-03 16:44:46 -07:00
Zooko O'Whielacronx	c066903566	fix idlib.could_be_base32_encoded_l() to accept *only* valid strings	2008-01-03 11:43:11 -07:00
Zooko O'Whielacronx	eae499539d	refactor constructing URI objects from strings and add human-encoding of WriteableSSKFileURI	2008-01-02 22:12:15 -07:00
Brian Warner	6b81ebfd8e	refactor webish.py slightly, improve the test coverage a bit. One test is marked SKIP pending improvements in webish.py	2007-12-25 03:48:57 -07:00
Brian Warner	fae5de8d0d	test_web.py: fix pyflakes error	2007-12-25 01:56:04 -07:00
Brian Warner	0e804da8d9	webish: use get_args() everywhere, put .fields=None in MyRequest	2007-12-25 00:07:04 -07:00
Brian Warner	4ba1af37a6	test_web: more coverage of URIPOSTHandler error cases	2007-12-24 22:49:35 -07:00
Brian Warner	7d311b8dd7	test_web: test POST_upload_no_link against /uri, not /uri/	2007-12-24 17:55:38 -07:00
Brian Warner	ae760b3ee4	test_web.py: use /uri?t=mkdir instead of /uri/?t=mkdir, and confirm that the redirection target is correct	2007-12-24 17:46:52 -07:00
Brian Warner	47761bde93	webish: factor out queryargs-or-formargs extraction, make /uri?t=mkdir work and redirect to a valid url	2007-12-24 17:26:52 -07:00
Brian Warner	4ce196a9c4	node.py: remove the provoke-logport-furlfile-creation hack now that foolscap-0.2.3 does it for us, and add bridge-twisted-logs	2007-12-24 17:24:40 -07:00
Zooko O'Whielacronx	5e8e2faf5d	setup: update doc in __init__.py	2007-12-22 10:46:50 -07:00
Zooko O'Whielacronx	1ac11301a7	setup: finish switching from Tahoe's versions of autoversioning tools to pyutil's versions	2007-12-21 14:42:38 -07:00
Zooko O'Whielacronx	a85a4d71f7	webish: add button to make directories and unit test of it ... Unfortunately although it passes the unit tests, it doesn't work, because the unit tests and the implementation use the "encode params into URL" technique but the button uses the "encode params into request body" technique.	2007-12-20 15:31:12 -07:00
Zooko O'Whielacronx	a7361179aa	make the unit test of the overwrite button more general	2007-12-20 15:23:07 -07:00
Zooko O'Whielacronx	a4bc623fa5	make the wapi/wui [*] support creation of a new directory without there already being an existing directory to link the new one into ... [*] WebAPI/WebUI	2007-12-20 12:58:17 -07:00
Brian Warner	d4283ac1ec	mutable: always include ourselves in the querylist when Publishing, because 0.7.0 usually stores our root dirnode only on ourselves, and our nodeid might not appear in the first N+epsilon peers	2007-12-19 00:06:53 -07:00
Brian Warner	fc3574a654	mutable.py: one more logging fix	2007-12-18 23:42:41 -07:00
Brian Warner	22f9e8c55b	mutable.py: oops, our logging wrappers can't take posargs yet	2007-12-18 23:36:16 -07:00
Brian Warner	c00b88025d	mutable.Publish: improve logging	2007-12-18 23:30:50 -07:00
Brian Warner	9a8f68c41f	dirnode: add set_uris() and set_nodes() (plural), to set multiple children at once. Use it to set up a new webapi test for issue #237.	2007-12-18 23:30:02 -07:00
Brian Warner	2308420fc6	node: provoke foolscap-0.2.2 into saving logport.furl, so we can attach to it with 'flogtool dump'. Also, place it in private/, since logs are considered somewhat private	2007-12-18 23:27:02 -07:00
Zooko O'Whielacronx	05b04d2b90	fix representation of node ids in PeerTracker objects	2007-12-19 17:55:28 -07:00
Zooko O'Whielacronx	a0d73f8afb	remove some no-longer needed replacements of "/" with "!" in uris	2007-12-19 17:54:40 -07:00
Brian Warner	2db158dd52	more new-pyflakes warnings fixed	2007-12-18 18:51:33 -07:00
Brian Warner	50bc0d2fb3	the new pyflakes is stricter, complaining about function definitions that overshadow earlier definitions or imports. Fix some of its complaints.	2007-12-18 18:47:28 -07:00
Brian Warner	318b63fbc7	hush pyflakes	2007-12-18 18:37:22 -07:00
Brian Warner	7fb37209ee	create_node.py: need to create private/ dir in create-client so we can touch the my_private_dir.cap file	2007-12-18 15:42:18 -07:00
Brian Warner	60d6728ab3	oops, touch private/my_private_dir.cap when creating the client	2007-12-18 15:36:59 -07:00
Brian Warner	01a93dfe8f	check_load: add stats-gathering	2007-12-18 14:07:37 -07:00
Brian Warner	50840b1085	add a simple load-generating tool to do random reads and writes	2007-12-17 21:06:07 -07:00
Brian Warner	b8d77fbb46	move my_private_dir.cap into private/	2007-12-17 20:57:40 -07:00
Brian Warner	a592551ccf	my_private_dir.cap: add newline for readability, open with mode 'w' not 'w+' since we don't need to read from this filehandle	2007-12-17 20:48:48 -07:00
Brian Warner	8b9f86fb17	remove the slash-to-bang conversion from CLI tools and webapi.txt	2007-12-17 20:22:26 -07:00
Zooko O'Whielacronx	2a7f374e45	unit test that POST /uri/?t=mkdir works	2007-12-18 17:47:49 -07:00
Zooko O'Whielacronx	a5a54ac5ca	remove the DirnodeURI foolscap schema and mv those regexes into uri.py ... We currently do not pass dirnode uris over foolscap.	2007-12-18 17:44:24 -07:00
Zooko O'Whielacronx	9848d2043d	make more precise regexp for WriteableSSKFileURI and DirnodeURI and use it in unit tests ... Also allow an optional leading "http://127.0.0.1:8123/uri/". Also fix a few unit tests to generate bogus Dirnode URIs of the modern form instead of the former form.	2007-12-18 13:15:08 -07:00
Zooko O'Whielacronx	ea43753386	fix unit tests to assert that we do *not* link to start.html when there is no private dir, instead of asserting that we *do*	2007-12-17 18:51:16 -07:00
Zooko O'Whielacronx	36e8e4e561	do not put a link from the welcoXme page to the start.html if there is not a private directory	2007-12-17 18:35:59 -07:00
Zooko O'Whielacronx	a983f6d60c	rename "my_private_dir.uri" to "my_private_dir.cap"	2007-12-17 18:35:25 -07:00
Zooko O'Whielacronx	08a64c3a2b	rename "secret" to "lease_secret" and change its size from 16 to 32 bytes	2007-12-17 18:34:11 -07:00
Zooko O'Whielacronx	8c65bdcf9d	put all private state in $BASEDIR/private ... fixes #219 The only part of #219 that this doesn't include is the part about logpublisher, which has been moved out of tahoe into foolscap.	2007-12-17 16:39:54 -07:00
Zooko O'Whielacronx	d3f23d4052	we no longer need to replace "/" with "!" in urls	2007-12-14 18:25:50 -07:00
Zooko O'Whielacronx	20f549baa1	tidy-up some comments and logging messages	2007-12-14 18:24:46 -07:00
Zooko O'Whielacronx	ae90016845	fix the overwrite button and add unit test for it	2007-12-14 17:52:05 -07:00
Brian Warner	cd59cded91	test_system.py: oops, match change in RIControl	2007-12-14 03:08:16 -07:00
Brian Warner	7924f5ac21	test_web.py: cosmetic, remove leftover trailing space	2007-12-14 02:06:09 -07:00
Brian Warner	f6b2072af1	check-speed: test SSK upload/download speed too. SDMF imposes a limit on the file sizes, no 10MB or 100MB test	2007-12-14 02:05:31 -07:00
Brian Warner	69b65b6b01	remove logpublisher, use the Foolscap version now that this functionality has been moved into Foolscap-0.2.2	2007-12-12 20:31:01 -07:00
Brian Warner	e52e14cc66	remove tests of logging functionality that's been subsumed by foolscap logging	2007-12-12 20:23:53 -07:00
Zooko O'Whielacronx	2915f5b7f4	remove unimplemented and skipped test for feature that we don't plan to implement any time soon (XML-RPC interface)	2007-12-12 20:06:05 -07:00
Zooko O'Whielacronx	0949638c11	remove unused import: thanks, pyflakes	2007-12-12 20:05:30 -07:00
Zooko O'Whielacronx	caa69b007e	refactor the feature of getting versions of packages, include the version numbers of more of the packages that we use	2007-12-12 19:37:37 -07:00
Zooko O'Whielacronx	1ac09840a4	a few documentation and naming convention updates ... Notable: the argument to make REPORTER has been renamed to TRIALARGS.	2007-12-12 19:34:08 -07:00
Zooko O'Whielacronx	a7f954ac06	editing: change names like "MyThing" to "FakeThing" for fake objects in unit tests	2007-12-11 18:10:29 -07:00
Zooko O'Whielacronx	38d1af7ede	fix unit test to pass forward-slashes to the CLI since it demands that the CLI emit forward-slashes	2007-12-11 20:03:44 -07:00
Brian Warner	2cb05817a6	test_web.py: survive localdir/localfile= names with spaces. Should close #223	2007-12-11 19:47:04 -07:00
Brian Warner	b085097adc	test_system: write test data in 'b' verbatim mode, since on windows the default text-mode is different. Addresses one of the failures in #223	2007-12-11 19:16:33 -07:00
Brian Warner	74820cc3de	webish: append save=true to a GET URL to save-to-disk. Closes #222.	2007-12-11 18:04:44 -07:00
Zooko O'Whielacronx	995cece54a	fix import error, detected thanks to "make test"	2007-12-11 17:23:01 -07:00
Zooko O'Whielacronx	5c3c1fea9a	fix missing import, thanks to pyflakes	2007-12-11 17:00:29 -07:00
Zooko O'Whielacronx	4210ba2b8a	fix test_introducer to wait for the right things in order to avoid intermittent test failures	2007-12-11 14:08:15 -07:00
Zooko O'Whielacronx	40edccf9c5	fix IntroducerClient.when_enough_peers() ... add IntroducerClient.when_few_enough_peers(), fix and improve test_introducer	2007-12-10 20:22:59 -07:00
Brian Warner	406ff52cf9	control.py: hush pyflakes warning	2007-12-06 18:38:42 -07:00
Brian Warner	0dc84963f1	the wait_for_numpeers= argument to client.upload() is optional: make both the code and the Interface reflect this	2007-12-06 18:36:58 -07:00
Zooko O'Whielacronx	192477e4a2	move increase_rlimits() into iputil and make it a no-op on Windows	2007-12-07 08:03:43 -07:00
Zooko O'Whielacronx	9a30ab2dfd	make check-memory use the new POST /uri?t=upload	2007-12-06 17:28:23 -07:00
Zooko O'Whielacronx	4923d8dcb5	make control.get_memory_usage() return all zeroes if reading /proc/mem/whatsit raises an exception	2007-12-06 17:27:40 -07:00
Zooko O'Whielacronx	8ad2e85fdf	add POST /uri?t=upload and tests thereof ... Hm... I refactored processing of segments in a way that I marked as "XXX HELP I AM YUCKY", and then I ran out of time for rerefactoring it before I committed. At least all the tests pass.	2007-12-06 17:17:02 -07:00
Brian Warner	841c1a8509	storage.py: factor out a common compare() routine	2007-12-05 00:20:34 -07:00
Brian Warner	51af1fa2e3	webish: fix overwrite form display	2007-12-05 00:15:13 -07:00
Brian Warner	53e865cf49	test_web.py: add coverage for directory listings that include mutable files	2007-12-04 23:57:40 -07:00
Brian Warner	785f21b9bf	test_web.py: add coverage for POST t=check	2007-12-04 23:49:38 -07:00
Brian Warner	56e02b274b	webish: add POST t=mutable, make it replace files in-place, add t=overwrite	2007-12-04 23:42:54 -07:00
Brian Warner	4c79b4cd07	test_web.py: remove last use of fake_* methods, remove dead code	2007-12-04 23:11:00 -07:00
Brian Warner	f190382d5e	refactor web tests, and interfaces.IFileNode	2007-12-04 23:01:37 -07:00
Brian Warner	4b8c2e93b3	checker: improve test coverage a little bit	2007-12-04 18:00:12 -07:00
Brian Warner	bf1b7d78e2	uri.py: improve test coverage	2007-12-04 17:38:31 -07:00
Brian Warner	cf2bd03813	remove most (maybe all?) traces of old Dirnode class. Yay for negative code days.	2007-12-04 17:00:58 -07:00
Brian Warner	1a5257c388	improve test coverage on FileNode.check	2007-12-04 15:55:27 -07:00
Brian Warner	0f5ef5184d	test_dirnode.py: obtain full coverage of dirnode.py	2007-12-04 14:32:04 -07:00
Brian Warner	cca166a4f5	rename dirnode2.py to dirnode.py	2007-12-04 11:45:20 -07:00
Brian Warner	b780127f49	test_mutable: improve test coverage a bit, add test_filenode.py	2007-12-03 21:37:54 -07:00
Brian Warner	955bd5383d	remove PyCrypto, now we only use pycrypto++	2007-12-03 18:10:01 -07:00
Brian Warner	0bf5a762a9	use AES from pycryptopp instead of pycrypto, also truncate the keys slightly differently	2007-12-03 17:27:46 -07:00
Brian Warner	87e6ccbd4a	remove leftover defer.setDebugging(), to speed up tests from 200s to 83s	2007-12-03 17:10:02 -07:00
Zooko O'Whielacronx	3605354a95	fix several bugs and warnings -- thanks, pyflakes	2007-12-03 15:42:35 -07:00
Zooko O'Whielacronx	184351d1f7	merge patch to integrate decentralized directories with "introducer_and_vdrive.py: use logpublisher too"	2007-12-03 15:29:04 -07:00
Brian Warner	d3e782a226	introducer_and_vdrive.py: use logpublisher too	2007-11-20 14:22:26 -07:00
Brian Warner	1f91e856ef	encode.py: trivial whitespace change	2007-11-19 23:00:29 -07:00
Zooko O'Whielacronx	e28d7c580c	merge patch to integrate decentralized directories with patch "download: use hierarchical logging"	2007-12-03 15:27:21 -07:00
Brian Warner	33a5f8ba6b	more hierarchical logging: download/upload/encode	2007-11-19 19:33:41 -07:00
Brian Warner	869b690378	download: use hierarchical logging	2007-11-19 19:07:10 -07:00
Brian Warner	3d046d6ac7	logging: only test log.err when Twisted is new enough to let us ignore the generated errors	2007-11-19 18:37:00 -07:00
Zooko O'Whielacronx	2c58d3399c	merge patch to integrate decentralized directories with patch to "only test log.err when Twisted is new enough to let us ignore the generated errors"	2007-12-03 15:25:14 -07:00
Brian Warner	cc1612aee5	hierarchical logging: add numbered messages and parent= args	2007-11-19 18:23:18 -07:00
Brian Warner	ce0609d29b	test_client: stall 2.0s between shutdown and restart, trying to fix cygwin	2007-11-18 20:41:26 -07:00
Brian Warner	5e4b63e981	logpublisher: hush pyflakes warning	2007-11-16 20:14:46 -07:00
Brian Warner	b29ce1c30a	logpublisher: implement subscribe/publish for log, add a sample client	2007-11-16 20:07:50 -07:00
Brian Warner	c2765bd8c6	mutable: fix control flow to allow good+bad shares from a peer. Fixes #211.	2007-11-16 17:12:33 -07:00
Brian Warner	2fc5247996	node.py: try to fix rlimit-setting again	2007-11-16 16:12:38 -07:00
Brian Warner	0900f328b7	node.py: try rlimit fix again	2007-11-15 23:09:02 -07:00
Brian Warner	d5c0a85c68	node.py: try to fix RLIMIT_NOFILE on solaris too	2007-11-15 23:06:44 -07:00
Brian Warner	c6df66d736	node.py: raise RLIMIT_NOFILE on bsd/cygwin to more than 256	2007-11-15 22:53:55 -07:00
Brian Warner	8a2736480a	test_mutable: workaround: use more peers to avoid random test failures. ... The underlying issue is recorded in #211: one corrupt share in a query response will cause us to ignore the remaining shares in that response, even if they are good. In our tests (with N=10 but only 5 peers), this can leave us with too few shares to recover the file. The temporary workaround is to use 10 peers, to make sure we never get multiple shares per response. The real fix will be to fix the control flow. This fixes #209.	2007-11-15 14:55:00 -07:00
Brian Warner	8742f2806d	mutable: add more logging to retrieval process	2007-11-15 14:22:56 -07:00
Brian Warner	3dfe8783cb	provisioning: add more options for num-servers and server-dBA	2007-11-15 13:40:49 -07:00
Brian Warner	e3037a7541	mutable: validate share_hash_chain for each inbound share	2007-11-14 15:26:46 -07:00
Zooko O'Whielacronx	8fee58fcba	merge patch to integrate decentralized directories with patch to handle bad hashes	2007-12-03 15:21:14 -07:00
Brian Warner	d6f2dbbac7	mutable: handle bad hashes, improve test coverage, rearrange slightly to facilitate these	2007-11-13 23:08:15 -07:00
Zooko O'Whielacronx	59d6c3c822	decentralized directories: integration and testing ... * use new decentralized directories everywhere instead of old centralized directories * provide UI to them through the web server * provide UI to them through the CLI * update unit tests to simulate decentralized mutable directories in order to test other components that rely on them * remove the notion of a "vdrive server" and a client thereof * remove the notion of a "public vdrive", which was a directory that was centrally published/subscribed automatically by the tahoe node (you can accomplish this manually by making a directory and posting the URL to it on your web site, for example) * add a notion of "wait_for_numpeers" when you need to publish data to peers, which is how many peers should be attached before you start. The default is 1. * add __repr__ for filesystem nodes (note: these reprs contain a few bits of the secret key!) * fix a few bugs where we used to equate "mutable" with "not read-only". Nowadays all directories are mutable, but some might be read-only (to you). * fix a few bugs where code wasn't aware of the new general-purpose metadata dict the comes with each filesystem edge * sundry fixes to unit tests to adjust to the new directories, e.g. don't assume that every share on disk belongs to a chk file.	2007-12-03 14:52:42 -07:00
Zooko O'Whielacronx	7b24eebd0a	unit tests: bump up a timeout which I encountered when running on a very slow machine	2007-11-29 14:47:35 -07:00
Zooko O'Whielacronx	a50cea61bd	remove tahoe_put-web2ish.py ... It isn't currently used, and I don't remember what part of its behavior was so much better than tahoe_put.py, and Brian has subsequently improved tahoe_put.py.	2007-11-19 14:07:28 -07:00
Zooko O'Whielacronx	ae727a550a	IMutableFileNode is a subtype of IFileNode ... I'm not 100% sure that this is correct, but it looks reasonable, it passes unit tests (although note that unit tests are currently not covering the new mutable files very well), and it makes the "view JSON" link on a directory work instead of raising an assertion error.	2007-11-10 16:37:18 -07:00
Zooko O'Whielacronx	35aea346d3	don't provide the "overwrite" button if the file is readonly to you	2007-11-10 16:07:05 -07:00
Zooko O'Whielacronx	85b972a4c3	WUI: hook up an "overwrite" button on mutable files	2007-11-10 10:53:45 -07:00
Zooko O'Whielacronx	97de6a03d2	remove parts of pycrypto that we are no longer going to use: SHA256 and RSA	2007-11-09 18:21:12 -07:00
Zooko O'Whielacronx	f6451deb74	mutable.py: fix padding/shape-of-input-data to zfec	2007-11-09 18:06:25 -07:00
Zooko O'Whielacronx	3c7f96400d	hashutil.py: switch from pycrypto to pycryptopp SHA256	2007-11-09 14:40:13 -07:00
Brian Warner	1f22768dc7	webish: add preliminary mutable file support: upload, download, listings, JSON, URI, RO-URI. No replace yet.	2007-11-09 03:54:27 -07:00
Brian Warner	63233ecf37	consolidate dirnode/filenode-creation code into Client	2007-11-09 02:54:51 -07:00
Brian Warner	61d9ab7690	mutable: cheap padding hack to make zfec tolerate short files	2007-11-08 04:30:37 -07:00
Brian Warner	30a91c84c8	mutable: verify incoming share signatures during Publish, it's not that expensive and it's a good idea	2007-11-08 14:02:36 -07:00
Brian Warner	a4606d6560	test_system: RSA keys are even more variable than I thought, 2044..2049	2007-11-08 05:04:11 -07:00
Brian Warner	2eaa4195cf	mutable: add basic test coverage of new-dirnodes-using-mutable-files	2007-11-08 04:31:00 -07:00
Brian Warner	c538e2b020	mutable: fix multiple-versions-interfering-with-each-other bug. replace() tests now pass.	2007-11-08 04:07:33 -07:00
Brian Warner	0ad7ff48fa	test_system: RSA keys vary in size, expand valid ranges in test	2007-11-08 03:01:13 -07:00
Brian Warner	e7ad7f8791	mutable: grab encprivkey when necessary during publish, fix test_mutable	2007-11-08 02:46:27 -07:00
Brian Warner	09fd2dfb3a	mutable: rearrange order of Publish to allow replace() to work. Doesn't work yet. Also test_mutable is disabled for a while.	2007-11-07 21:01:39 -07:00
Brian Warner	247bfe8d58	mutable: fix usage of NeedMoreDataError	2007-11-07 17:52:09 -07:00
Brian Warner	c783128012	mutable: wire in RSA for real, using pycryptopp	2007-11-07 17:51:35 -07:00
Brian Warner	30e7ca06f5	mutable: make error handling more robust	2007-11-07 17:45:45 -07:00
Zooko O'Whielacronx	f60dc4adfb	UNDO: hashutil: replace pycrypto's SHA256 with pycryptopp's SHA256	2007-11-07 18:02:39 -07:00
Zooko O'Whielacronx	11412da436	hashutil: replace pycrypto's SHA256 with pycryptopp's SHA256	2007-11-07 18:02:39 -07:00
Brian Warner	be94960680	mutable: test roundtrip, make it work	2007-11-07 14:19:01 -07:00
Brian Warner	ba43c033fa	storage.py: add a little logging (disabled)	2007-11-07 14:14:54 -07:00
Zooko O'Whielacronx	3c222bbe26	setup: print out the version number of pycryptopp in "tahoe --version"	2007-11-07 10:11:56 -07:00
Brian Warner	a46e64b0bb	test_system.mutable: make sure we exercise FEC padding	2007-11-06 19:50:33 -07:00
Brian Warner	cc5d35cc07	dump-share: emit SDMF information too	2007-11-06 19:46:31 -07:00
Brian Warner	fdbe692810	add container_size to mutable dump-share output	2007-11-06 19:31:22 -07:00
Brian Warner	8be9ef5149	mutable: stub out pubkey creation until we wire in pycryptopp properly	2007-11-06 19:27:06 -07:00
Brian Warner	ddf4d23bef	test_system.mutable: add test coverage for the 'dump-share' debug command	2007-11-06 19:10:49 -07:00
Brian Warner	bc8605ad39	test_system: add early test for mutable slots, currently publish-only	2007-11-06 18:57:11 -07:00
Brian Warner	530085df4a	test_mutable: remove dead code	2007-11-06 18:56:39 -07:00
Brian Warner	0b2f62466b	debug: add mutable-slot support to 'dump-share' command	2007-11-06 18:55:55 -07:00
Brian Warner	5739126d11	mutable: storage_index is always 16 bytes	2007-11-06 18:54:34 -07:00
Brian Warner	7e43c7b5f9	mutable: fix use of storage API	2007-11-06 18:53:34 -07:00
Brian Warner	c4f7412f1c	stabilize on 20-byte nodeids everywhere, printed with foolscap's base32	2007-11-06 18:49:59 -07:00
Brian Warner	7cf9a13f2a	mutable: rearrange classes, putting MutableFileNode at the bottom	2007-11-06 15:19:48 -07:00
Brian Warner	c595f6ccd0	mutable: parameterize Publish/Retrieve classes in MutableFileNode, for tests	2007-11-06 15:18:09 -07:00
Brian Warner	2ed394e471	mutable: move IV into signed prefix, add more retrieval code	2007-11-06 15:04:46 -07:00
Brian Warner	207888a97b	mutable: get most of the retrieve-side code written. no tests yet.	2007-11-06 03:47:29 -07:00
Brian Warner	a431411418	test_mutable.py: hush pyflakes	2007-11-06 01:32:39 -07:00
Brian Warner	85a56a3d74	mutable.Publish: rearrange create() to use more of it in unit tests	2007-11-06 00:33:40 -07:00
Brian Warner	8cdda6e199	mutable.Publish: more tests	2007-11-05 22:38:43 -07:00
Brian Warner	281afe7cfc	mutable.Publish: create a dispatch_map for the benefit of recovery code, and pull pack/unpack methods out into functions	2007-11-05 22:14:59 -07:00
Brian Warner	59632c6812	mutable: use proper enable/renew/cancel secrets	2007-11-05 21:51:08 -07:00
Brian Warner	fade06ef4d	mutable: added send-messages-to-peers code, about 70% done. No recovery code yet.	2007-11-05 21:29:47 -07:00
Brian Warner	e08b091d9f	storage: rewrite slot API, now use testv_and_readv_and_writev or readv	2007-11-05 20:17:14 -07:00
Brian Warner	95a3da92fe	test_mutable: hush pyflakes warning	2007-11-05 00:41:53 -07:00
Brian Warner	27b67ef0d6	mutable: add peer-selection to Publish, and some basic unit tests	2007-11-05 00:38:07 -07:00
Brian Warner	8f21424449	storage: add readv_slots: get data from all shares	2007-11-05 00:37:01 -07:00
Brian Warner	6137329f05	mutable.py: start writing share-mapping code	2007-11-02 22:59:02 -07:00
Brian Warner	6e5b799d46	mutable.py: add share-unpacking code, use it for more tests	2007-11-02 22:28:31 -07:00
Brian Warner	78c45c82d1	mutable: implement filenode share-packing, still pretty rough	2007-11-02 20:51:39 -07:00
Brian Warner	6510510ea7	mutable.py: sketch out data-structure packing/unpacking methods	2007-11-02 00:03:28 -07:00
Brian Warner	d772ea850d	mutable: split dirnode stuff out to dirnode2.py, will be renamed later	2007-11-01 23:46:47 -07:00
Brian Warner	1158c4aae5	test_client: increase test_reloadable inter-run timeout, cygwin was failing	2007-11-01 19:19:08 -07:00
Brian Warner	f1c3ff62c1	mutable: improve NewDirectoryNode test coverage	2007-11-01 18:35:54 -07:00
Brian Warner	d777283e9e	implement preliminary log publisher/gatherer ... This creates a Referenceable object that will eventually be able to publish log events to a remote subscriber (at present all it can do is provide version information). The FURL for this logport is written to 'logport.furl'. In addition, if a file named 'log_gatherer.furl' is present, the given target will be contacted and offered access to the logport. This can be used by a centralized logging agent to subscribe to logs, e.g. from all the nodes in a centrally-maintained storage grid. (think syslog -r, but with all the security properties of FURLs, and permitting non-printable strings and structured data). Once this framework matures a bit, it will be moved into Foolscap.	2007-11-01 17:29:15 -07:00
Brian Warner	97f4d8c524	Client.tub_ready: upcall to Node	2007-11-01 17:27:12 -07:00
Brian Warner	36921fedb3	mutable: implement most remaining dirnode methods. No tests yet.	2007-11-01 16:57:58 -07:00
Brian Warner	57373c9889	bencode.py: reencode as UTF-8	2007-11-01 15:42:57 -07:00
Brian Warner	4423bc7510	trailing-whitespace eradication, no functional changes	2007-11-01 15:34:35 -07:00
Brian Warner	c63b43bdbc	trailing-whitespace eradication, no functional changes	2007-11-01 15:34:31 -07:00
Brian Warner	f714c5915a	trailing-whitespace eradication, no functional changes	2007-11-01 15:34:28 -07:00
Brian Warner	f425ee3600	trailing-whitespace eradication, no functional changes	2007-11-01 15:34:21 -07:00
Brian Warner	6849647856	trailing-whitespace eradication, no functional changes	2007-11-01 15:34:18 -07:00
Brian Warner	4eab5bb50d	trailing-whitespace eradication, no functional changes	2007-11-01 15:34:12 -07:00
Brian Warner	e304c74f8f	trailing-whitespace eradication, no functional changes	2007-11-01 15:34:07 -07:00
Brian Warner	a037049188	trailing-whitespace eradication, no functional changes	2007-11-01 15:34:01 -07:00
Brian Warner	516ce89a1e	trailing-whitespace eradication, no functional changes	2007-11-01 15:33:47 -07:00
Brian Warner	9cd3eaec1e	trailing-whitespace eradication, no functional changes	2007-11-01 15:33:41 -07:00
Brian Warner	ef546da72b	trailing-whitespace eradication, no functional changes	2007-11-01 15:29:31 -07:00
Brian Warner	8d000aa1ec	trailing-whitespace eradication, no functional changes	2007-11-01 15:29:28 -07:00
Brian Warner	8390d83322	trailing-whitespace eradication, no functional changes	2007-11-01 15:29:17 -07:00
Brian Warner	8465e99fe4	trailing-whitespace eradication, no functional changes	2007-11-01 15:29:12 -07:00
Brian Warner	3425cd9202	trailing-whitespace eradication, no functional changes	2007-11-01 15:29:03 -07:00
Brian Warner	433802f5ca	trailing-whitespace eradication, no functional changes	2007-11-01 15:28:58 -07:00
Brian Warner	bf29cb38ed	trailing-whitespace eradication, no functional changes	2007-11-01 15:28:54 -07:00
Brian Warner	f192ffc18d	trailing-whitespace eradication, no functional changes	2007-11-01 15:28:50 -07:00
Brian Warner	fb59627faa	trailing-whitespace eradication, no functional changes	2007-11-01 15:25:20 -07:00
Brian Warner	5e974ede20	trailing-whitespace eradication, no functional changes	2007-11-01 15:25:16 -07:00
Brian Warner	164da05ead	trailing-whitespace eradication, no functional changes	2007-11-01 15:25:12 -07:00
Brian Warner	284b3e77e0	trailing-whitespace eradication, no functional changes	2007-11-01 15:25:09 -07:00
Brian Warner	1889af0e61	trailing-whitespace eradication, no functional changes	2007-11-01 15:25:04 -07:00
Brian Warner	20af973272	trailing-whitespace eradication, no functional changes	2007-11-01 15:25:00 -07:00
Brian Warner	f4946bc22e	trailing-whitespace eradication, no functional changes	2007-11-01 15:22:57 -07:00
Brian Warner	828e2734e6	trailing-whitespace eradication, no functional changes	2007-11-01 15:22:47 -07:00
Brian Warner	d06d4f302f	trailing-whitespace eradication, no functional changes	2007-11-01 15:22:41 -07:00
Brian Warner	1e76e9267b	trailing-whitespace eradication, no functional changes	2007-11-01 15:22:38 -07:00
Brian Warner	1d8a4cdfe7	mutable: first pass at dirnodes, filenodes, new URIs. Some test coverage. ... The URI typenames need revision, and only a few dirnode methods are implemented. Filenodes are non-functional, but URI/key-management is in place. There are a lot of classes with names like "NewDirectoryNode" that will need to be rename once we decide what (if any) backwards compatibility want to retain.	2007-11-01 15:15:29 -07:00
Brian Warner	fb3eddafdb	move NotMutableError from dirnode.py into interfaces.py	2007-11-01 15:03:07 -07:00
Brian Warner	bcf84c1238	storage.py: fix tests, timestamps get updated when leases are renewed	2007-10-31 12:31:33 -07:00
Brian Warner	70e7961088	storage.py: more test coverage, make sure leases survive resizing	2007-10-31 12:07:47 -07:00
Brian Warner	948e6b34dd	storage.py: improve test coverage even more	2007-10-31 01:44:01 -07:00
Brian Warner	4bd739435f	storage.py: more mutable-slot coverage, renewing/cancelling leases	2007-10-31 01:31:56 -07:00
Brian Warner	f9a72e0e75	test_node.py: more coverage of Node.log	2007-10-31 00:56:59 -07:00
Brian Warner	22e3487b9e	test_node.py: improve test coverage of node.py	2007-10-31 00:54:42 -07:00
Brian Warner	256ef1bf53	mutable slots: add some test coverage for lease-addition	2007-10-31 00:38:30 -07:00
Brian Warner	68d3d62002	mutable slots: finish up basic coding on server-side containers, add some tests. Remove all caching from MutableShareFile.	2007-10-31 00:10:40 -07:00
Brian Warner	b24c2925e8	checkpointing mutable-file work. Storage layer is 80% in place.	2007-10-30 19:47:36 -07:00
Brian Warner	b257f905a0	webish.py: handle asynchronous checker results. ... Thanks to robk for pointing out that Nevow will accept a Deferred almost everywhere. In this case, we just pass a Deferred into ctx.fillSlots(). One quirk: nevow doesn't evaluate all rows of the table in parallel: using a slow Deferred in a slot in one row seems to stall the next row until that one has fired, probably to simplify the flattening of the HTML.	2007-10-30 17:00:37 -07:00
Brian Warner	5491339912	'tahoe stop': use SIGKILL, clean up twistd.pid ourselves, wait forever	2007-10-29 11:43:35 -07:00
Brian Warner	3d0ed4b0a9	checker.checker_results_for: ignore uris of 'None'	2007-10-26 18:38:37 -07:00
Brian Warner	42bd2bcf8e	webish: tolerate not having a checker, since some unit tests don't make one	2007-10-23 17:35:31 -07:00
Brian Warner	046bda2b47	webish: add checker results and a 'Check' button to the web interface	2007-10-23 17:23:57 -07:00
Brian Warner	afca99c485	interfaces: clarify IChecker.checker_results_for a bit	2007-10-22 18:10:46 -07:00
Brian Warner	57f994fb02	checker: remember checker results, but only in ram for now	2007-10-22 17:46:24 -07:00
Brian Warner	81c818f90c	checker.py: rearrange classes a little bit	2007-10-22 16:19:18 -07:00
Brian Warner	e992fed1f5	cli: improve test coverage	2007-10-21 12:33:17 -07:00
Brian Warner	afcc8107af	tahoe_ls: list individual files	2007-10-21 12:33:06 -07:00
Brian Warner	aff059dd4b	checker: return more information per CHK file, including the shareholder list	2007-10-17 02:25:50 -07:00
Brian Warner	d1f13fd117	add an equally-simple file-verifier	2007-10-16 12:25:09 -07:00
Brian Warner	6160af5f50	encode.py: update comments, max_segment_size is now 2MiB	2007-10-16 11:00:29 -07:00
Brian Warner	9da1d70676	add a simple checker, for both files and directories	2007-10-15 16:16:39 -07:00
Zooko O'Whielacronx	81c56b8e53	shush, pyflakes	2007-10-26 18:53:21 -07:00
Zooko O'Whielacronx	d7af8ae677	improve code coverage by removing untested raise RuntimeError and replacing it with tested assert	2007-10-26 18:37:40 -07:00
Zooko O'Whielacronx	81e0820904	cli: use urllib.quote() on vdrive-path arguments before passing them through HTTP	2007-10-26 18:30:44 -07:00
czooko	d589c385e6	make the anchor text in the WUI be descriptive of the target -- not "Click Here!"!	2007-10-22 16:56:30 -07:00
czooko	5f9f54c11c	trivial change to comment -- sentences start with a capital letter and end with a punctuation mark ... I've been volunteering in Irby's first-grade class, and you do NOT get a star on your Power Sentences worksheet unless you follow this rule.	2007-10-22 16:55:20 -07:00
czooko	3847228444	patch the LogObserver in a more modern, forward-compatible way and update the in-line comments about it	2007-10-22 16:52:55 -07:00
Zooko O'Whielacronx	74f52d79f2	tests: make test_encode specify the erasure coding params it wants instead of expecting the defaults to be what it wants	2007-10-15 20:07:42 -07:00
Zooko O'Whielacronx	426721f3f2	update a few documents, comments, and defaults to mention 3-of-10 instead of 25-of-100	2007-10-15 19:53:59 -07:00
Zooko O'Whielacronx	bba7b81456	startstop_node: use the new find_exe module	2007-10-15 10:25:15 -07:00
Zooko O'Whielacronx	5b476d73ee	setup: generalize the kludge of finding an executable (i.e. trial or twistd) when there might be only a .py script version of it available	2007-10-15 10:25:04 -07:00
Zooko O'Whielacronx	fbc5bdf607	remove unused imports (thanks, pyflakes)	2007-10-15 08:32:21 -07:00
Zooko O'Whielacronx	83d7eb23e6	remove a file accidentally left over from the experiment in using sqlite to maintain lease information	2007-10-14 20:57:59 -07:00
Zooko O'Whielacronx	ed84534c5b	a slightly nicer method of computing our timestamp format	2007-10-14 20:46:51 -07:00
Zooko O'Whielacronx	240de64598	install our custom timestamp formats in a less disruptive way ... The unit tests on Windows fail because trial is attempting to remove its own log observer during teardown. This patch customizes the extant log observer object by replacing its formatTime method with our own. I first tried the approach of storing their log observer object and putting it back during teardown, but it didn't work (perhaps because our node object doesn't get a chance to do its deferred stopService behavior in time), and anyway I generally prefer the "fail-safe", or "crash-only" design.	2007-10-14 20:43:11 -07:00
Brian Warner	07b03bd55b	test_runner.RunNode: pass an explicit webport, to avoid using 8123 (which might be in used by a running node). Closes #175.	2007-10-13 16:06:39 -07:00
Zooko O'Whielacronx	af0edec753	filter out "0.0.0.0" from detected IP addresses	2007-10-13 00:38:16 -07:00
Brian Warner	f0e727867a	cli: simplify code by using stdlib's httplib module	2007-10-11 22:29:23 -07:00
Brian Warner	4361b32f2d	cli: implement 'mv'. Closes #162.	2007-10-11 20:31:48 -07:00
Brian Warner	3709b343ab	tahoe_get.py: remove unused import	2007-10-11 19:47:40 -07:00
Brian Warner	8985b6565d	cli: add test coverage	2007-10-11 19:20:41 -07:00
Brian Warner	654bbbed8f	node.py: fix timestamps (add ms and Z) by replacing the FileLogObserver. #171.	2007-10-11 17:30:07 -07:00
Brian Warner	479abae92f	cli: fix usage to refer to 'tahoe', not 'allmydata'. Closes #154.	2007-10-11 15:37:52 -07:00
Brian Warner	e1f604f7cd	change our default HTTP port to 8123	2007-10-11 13:17:23 -07:00
Brian Warner	dc573554c8	rename bin/allmydata-tahoe to bin/tahoe. Closes #155.	2007-10-11 03:38:24 -07:00
Brian Warner	ee72a5adfd	rename client.tac to tahoe-client.tac, so that 'ps ax|grep tahoe' works. Closes #156.	2007-10-11 02:48:06 -07:00
Brian Warner	34a25b9b28	node.py: don't append 'Z' to the timestamp, since it's really localtime. We need deeper changes to make it be UTC	2007-10-11 02:24:17 -07:00
Brian Warner	6ed8664c7f	check_memory.py: don't use self.keepalive_file until it's been initialized	2007-10-11 02:20:20 -07:00
Brian Warner	a76898244b	create_node.py: allow config['webport'] to be missing, for check_memory	2007-10-11 02:19:59 -07:00
Brian Warner	1b406f3c32	node.py: set logging timestamp to '2007-10-11 02:11:14.000Z', per ticket #171. No milliseconds yet, though	2007-10-11 02:13:05 -07:00
Brian Warner	0f000784ce	create_node: use a webport by default, on localhost:8011	2007-10-11 02:01:23 -07:00
Brian Warner	8ce93b5e27	test_cli.py: hush pyflakes with a dummy usage, until we get some real CLI tests	2007-10-11 01:55:29 -07:00
Brian Warner	7fd1964c6f	runner: make most commands use ~/.tahoe by default (create-client, start/stop/restart, all CLI tools, but *not* create-introducer	2007-10-11 01:54:23 -07:00
Brian Warner	a29ab33379	webish: write node.url, for the benefit of CLI tools	2007-10-11 01:38:04 -07:00
Brian Warner	a470947cc3	cli: use urllib.escape on all URIs	2007-10-11 01:34:44 -07:00
Brian Warner	5b28411b04	cli: add --node-directory and --root-uri to all commands	2007-10-11 00:30:36 -07:00
Brian Warner	ec8781e587	'tahoe restart': re-enable --force option, which was accidentally lost	2007-10-10 20:10:03 -07:00
Zooko O'Whielacronx	021b5d4f2a	more information SkipTest for XMLRPC	2007-10-04 11:07:46 -07:00
Brian Warner	8fa19fa273	check_speed: average multiple pings when measuring RTT	2007-09-26 18:16:15 -07:00
Brian Warner	51b59b1219	dump-share: clarify the label on the size of the original file	2007-09-26 15:00:59 -07:00
Brian Warner	3fcee101ee	check_speed: measure RTT, report per-file times as a multiple of RTT	2007-09-26 13:07:33 -07:00
Brian Warner	c301b41f50	control: add measure_peer_response_time(), to estimate RTT for the mesh	2007-09-26 12:21:15 -07:00
Brian Warner	c10da1e1ef	client.py: add the missing remote_get_nodeid() method claimed in interfaces.py	2007-09-26 12:20:48 -07:00
Brian Warner	0596b9fe19	test_system.py minor typo	2007-09-26 12:07:37 -07:00
Brian Warner	0cfed96796	test_system.py: add coverage for allmydata.control	2007-09-26 12:06:55 -07:00
Brian Warner	aafccf7008	test_client.py: add TODO to test the hotline-timeout file	2007-09-25 21:03:54 -07:00
Brian Warner	c14349f695	check_speed.py: minor comment	2007-09-25 20:07:03 -07:00
Brian Warner	e05311f69a	client.py: increase hotline timeout, the check_memory test is failing	2007-09-25 19:22:33 -07:00
Brian Warner	1691524044	check_speed.py: use more small-file tests to improve accuracy of per-file time	2007-09-25 18:57:36 -07:00
Brian Warner	50415cf9c4	check_memory: more keepalive-file debug messages	2007-09-25 18:47:48 -07:00
Brian Warner	8d7368f748	check_memory: put framework log in a stable place, add debug code to figure ... out why the keepalive-file isn't doing its job	2007-09-25 18:26:54 -07:00
Brian Warner	9b09fd3bf1	test_system.py: do one upload, then test debug scripts, then do other uploads	2007-09-24 18:12:37 -07:00
Zooko O'Whielacronx	e3d782da52	cli: use port 8123 as the example port number	2007-09-24 13:17:27 -07:00
Zooko O'Whielacronx	93efd2902e	startstop_node.py: use SIGINT instead of SIGTERM to stop the node (fixes #143)	2007-09-23 07:06:54 -07:00
Zooko O'Whielacronx	a18f7aa6d5	fix test_vdrive (fixes #144) ... It turns out that we actually have *two* files in our storage servers at the time that test_vdrive asserts things about the shares. I suppose that test_vdrive happens to pass on all other operating systems because the filesystem happens to return the right share as the first one in a "listdir()". The fix in this patch is slightly kludgey -- allow either share to pass -- but good enough.	2007-09-22 15:26:27 -07:00
Brian Warner	ff07ce6234	check_speed: add optional 100MB test, if the 10MB test finished fast enough	2007-09-22 00:04:46 -07:00
Brian Warner	325c7a7bfa	node.py: chmod the foolscap private key (node.pem) to 0600, since it's secret	2007-09-21 16:52:55 -07:00
Zooko O'Whielacronx	594a104c71	startstop_node.py: fix bug with options passed to restart and enable restart --profile	2007-09-21 15:48:15 -07:00
Zooko O'Whielacronx	3e4616f4ad	cli: add --profile flag	2007-09-21 15:03:47 -07:00
Zooko O'Whielacronx	20079e3019	startstop_node.py: don't pass config to do_start() and do_stop()	2007-09-21 15:03:15 -07:00
Zooko O'Whielacronx	28b2c19f1e	startstop_node.py: find twistd.py in sys.prefix/scripts if there isn't a twistd executable on the path	2007-09-21 14:45:20 -07:00
Brian Warner	1c29065f1c	check_memory: touch the hotline file more often, to avoid accidental timeout	2007-09-20 19:35:31 -07:00
Brian Warner	d58d794757	check_speed: do both upload and download tests	2007-09-20 18:52:44 -07:00
Brian Warner	2632c0ad8c	check_speed: upload multiple files, measure Ax+B	2007-09-20 16:55:33 -07:00
Zooko O'Whielacronx	f0b427485b	startstop_node.py: remove debug printout which breaks unit tests	2007-09-20 17:02:36 -07:00
Zooko O'Whielacronx	c2e310825d	startstop_node.py: improve the hack of launching twistd ... Thanks to Brian for helping me figure out the cleaner way to do this: take the first result from which("twistd"), and if it has the extension ".bat" or ".exe" then execute it, else execute python and give it as the first argument.	2007-09-20 12:37:50 -07:00
Brian Warner	3774ce59ea	check_memory: fix race condition for startup of in-process server nodes	2007-09-20 15:33:58 -07:00
Brian Warner	77fae0b338	check_memory.py: record initial memory usage (before any connections are made)	2007-09-20 12:36:27 -07:00
Brian Warner	b9d5a4ead4	check_speed.py: run two 1MB uploads and measure the time it takes	2007-09-19 18:40:18 -07:00
Brian Warner	f6be35e122	started framework for an automated speed-checking tool. Doesn't do much yet.	2007-09-19 18:27:47 -07:00
Brian Warner	2b93164a03	test_runner: try harder to work on slow buildslaves and cygwin	2007-09-19 13:56:00 -07:00
Brian Warner	bede39a156	check-memory: add 'receive' mode, for #97 (consumption during share receive	2007-09-19 12:59:32 -07:00
Brian Warner	0cbf2cca08	introducer.py: add test coverage of _disconnected()	2007-09-19 11:50:13 -07:00
Brian Warner	e2f19825d9	startstop_node.py: improve test coverage a little bit	2007-09-19 01:50:27 -07:00
Brian Warner	d31d7ff941	test_system.py: do a large-file test (1.5MB) to trigger pauseProducing	2007-09-19 01:43:44 -07:00
Brian Warner	1340c484c6	download.py: use producer/consumer to reduce memory usage, closes #129. ... If the DownloadTarget is also an IConsumer, give it control of the brakes by offering ourselves to target.registerProducer(). When they tell us to pause, set a flag, which is checked between segment downloads and decodes. webish.py: make WebDownloadTarget an IConsumer and pass control along to the http.Request, which already knows how to be an IConsumer. This reduces the memory footprint of stalled HTTP GETs to a bare minimum, and thus closes #129.	2007-09-19 00:34:47 -07:00
Brian Warner	9c9a793540	upload.py: fix signature of NonConvergentUploadMixin.get_encryption_key	2007-09-18 23:38:11 -07:00
Brian Warner	ee9b7fd678	check_memory: oops, silly bug make 'upload' push to ourselves, raising the usage	2007-09-19 04:14:48 -07:00
Brian Warner	08cc32bb77	check_memory: add download-GET-slow, to simulate memory usage of a node feeding downloaded data via HTTP GET to a slow client	2007-09-18 20:35:27 -07:00
Brian Warner	a5fab9935b	test_runner: better diagnostics in case test_client fails	2007-09-18 20:03:18 -07:00
Brian Warner	2886c5aeae	check_memory: don't accept shares for download/download-GET test, since that hits bug #97	2007-09-18 19:40:29 -07:00
Brian Warner	f6acf5a0c3	check_memory: add download, download-GET	2007-09-18 18:56:05 -07:00
Brian Warner	38946439f3	test_runner.py: fix race conditions in start/stop node, should run on cygwin now	2007-09-18 15:17:26 -07:00
Brian Warner	645927ca73	test_runner.py: skip the start/stop test when sys.platform is win32/cygwin. ... The previous twisted.python.runtime.platformType approach didn't catch cygwin for some reason.	2007-09-17 21:56:08 -07:00
Brian Warner	f72ef4d8f9	test_runner.py: add tests for startstop_node.py	2007-09-17 02:25:31 -07:00
Brian Warner	cf69accfb6	startstop_node.py: use SIGINT instead of SIGTERM, to allow the node a chance to clean up	2007-09-17 02:24:21 -07:00
Brian Warner	b7c18d6505	webish: improve test coverage	2007-09-17 01:53:46 -07:00
Brian Warner	6cd11037c6	debug: remove the 'dump-share-leases' command, it is subsumed by 'dump-share'	2007-09-17 01:44:39 -07:00
Brian Warner	79bd7d422d	provisioning.py: get full test coverage	2007-09-17 01:38:54 -07:00
Brian Warner	311ed144f8	uri.py: improve test coverage a bit	2007-09-17 01:09:47 -07:00
Brian Warner	8451b485a4	storage: fill alreadygot= with all known shares for the given storageindex, not just the ones they asked about	2007-09-17 00:48:40 -07:00
Brian Warner	808f851589	upload: make peer-selection a bit more uniform. Closes #132.	2007-09-16 17:08:34 -07:00
Brian Warner	24e6ccddce	peer-selection: if we must loop, send a minimal number of queries (by asking for more than one share per peer on the second pass)	2007-09-16 01:53:00 -07:00
Brian Warner	8a251d8670	upload: remove Tahoe3 peer-selection algorithm	2007-09-16 01:26:11 -07:00
Brian Warner	baa16087cd	upload: switch to Tahoe2, add test for uniform share allocation	2007-09-16 01:25:03 -07:00
Brian Warner	979d12cd42	upload.py: implement Tahoe2 peer-selection algorithm	2007-09-16 01:24:07 -07:00
Brian Warner	f24d7450a7	vdrive.py: oops, forgot an import	2007-09-15 15:21:57 -07:00
Brian Warner	d37d74fa1d	vdrive: log an error if we weren't able to use the vdrive the way we wanted to	2007-09-15 15:17:29 -07:00
Brian Warner	d628d5f503	storage: remove the leftover incoming/XYZ/ directory when we're done with it	2007-09-15 14:34:04 -07:00
Brian Warner	e9d32f14b9	check_memory.py: preserve client.log and stats.out in _test_memory/ ... Put the nodes in _test_memory/test/, which is clobbered on each test. Also kill the client with SIGINT instead of SIGKILL. Also don't daemonize the client, since we're going to kill it at the end of the test anyways: this cleans up shutdown a bit.	2007-09-15 20:53:06 -07:00
Brian Warner	d5ed60e4ad	webish: upload: when the localdir= doesn't exist, say so in the HTTP response	2007-09-15 12:49:07 -07:00
Brian Warner	c7986389e5	check_memory.py: have all clients write their logs to _test_memory/client.log instead of a separate file per client	2007-09-15 12:34:05 -07:00
Brian Warner	e7f61a3a6c	check_memory: oops, only bail if the process ended abnormally	2007-09-15 11:47:29 -07:00
Brian Warner	32f25f33e3	check-memory: if the child process fails to start, shut down instead of hanging forever	2007-09-14 20:16:57 -07:00
Brian Warner	df49fa55e0	test_client.py: remove unused 'stat' import	2007-09-13 20:17:42 -07:00
Zooko O'Whielacronx	8fd98624a3	don't try to use bindann ... It causes a mysterious misbehavior in Python import which causes the previous patch to fail (the patch to not run trial tests if dependencies can't be imported)	2007-09-13 19:14:46 -07:00
Zooko O'Whielacronx	f443f8adbc	remove test in test_client -- we can't easily assert that files aren't readable by others, on Windows	2007-09-13 20:12:26 -07:00
Brian Warner	e1e037e9b5	storage: always record lease expiration times as integers	2007-09-11 14:53:31 -07:00
nejucomo	96f55be844	Add some passing unit tests for testutil.PollMixin.	2007-09-07 16:15:41 -07:00
Brian Warner	89a25f1987	provisioning: include the cost of replacing failed hard drives	2007-09-10 15:46:45 -07:00
Brian Warner	0497d62939	provisioning: add cost information	2007-09-10 15:29:33 -07:00
Zooko O'Whielacronx	09ff5af99c	loosen constraint on share data size	2007-09-11 11:29:18 -07:00
Brian Warner	3eaa888a0e	provisioning: oops, fix test_provisioning too	2007-09-07 15:01:17 -07:00
Brian Warner	0ae7d31d33	provisioning.py: update test with new encoding-parameters format	2007-09-07 14:45:43 -07:00
Brian Warner	2a48283e53	provisioning: require at least one drive per server	2007-09-06 23:08:21 -07:00
Brian Warner	5561d3afe4	provisioning: explain the 'survival' number better	2007-09-06 23:01:07 -07:00
Brian Warner	421277cafa	provisioning: more repair/survivability data	2007-09-06 22:54:53 -07:00
Brian Warner	69d680d820	provisioning: add some drive failure and repair rate info	2007-09-06 18:47:41 -07:00
Brian Warner	849dbb4950	provisioning.py: add file/server availability numbers	2007-09-05 18:16:21 -07:00
Brian Warner	76be4a582c	webish: implement 'PUT /uri?t=mkdir' (to create anonymous dirnodes)	2007-09-05 17:23:06 -07:00
Brian Warner	db3f11648c	webish: implement 'PUT /uri' (to create anonymous files)	2007-09-05 17:12:27 -07:00
Brian Warner	77394da447	client.py: only import webish.py if 'webport' is set, to save 3MB of footprint	2007-09-04 16:33:06 -07:00
Brian Warner	277e720f7c	storage: add version number to share data. Closes #90.	2007-09-04 09:00:24 -07:00
Brian Warner	fb65aadd82	storage: don't add a duplicate lease, renew the old one instead	2007-09-02 21:39:47 -07:00
Brian Warner	89c7f27572	storage: remove get_or_add_owner, since I don't know what we need yet	2007-09-02 15:03:40 -07:00
Brian Warner	94233b8813	storage: remove unused delete_bucket() method, lease-cancellation covers it	2007-09-02 15:00:29 -07:00
Brian Warner	85f3107b12	storage: handle simultanous uploads: add a lease for the pre-empted client	2007-09-02 14:57:49 -07:00
Brian Warner	dbcabc1142	debug: 'dump-uri-extension' command becomes 'dump-share', add 'dump-share-leases'. Both display leases.	2007-09-02 14:48:20 -07:00
Brian Warner	0fe1205789	storage: replace sqlite with in-share lease records	2007-09-02 14:47:15 -07:00
Brian Warner	c7288286ae	test_uri.py: remove tiny whitespace	2007-08-28 23:40:03 -07:00
wilcoxjg	33bae16ed2	created DirnodeURI schema	2007-08-28 23:28:26 -07:00
Brian Warner	a605fe5cad	storage: use sqlite from either python2.5's stdlib or the pysqlite2 package	2007-08-28 23:28:52 -07:00
Brian Warner	2a63fe8b01	deletion phase3: add a sqlite database to track renew/cancel-lease secrets, implement renew/cancel_lease (but nobody calls them yet). Also, move the shares from BASEDIR/storage/* down to BASEDIR/storage/shares/*	2007-08-27 23:41:40 -07:00
Brian Warner	850bc9da02	deletion phase2b: create renew/cancel secrets for real.	2007-08-27 19:30:26 -07:00
Brian Warner	1428079a84	node: change get_or_create_config to strip whitespace and accept a filemode= argument	2007-08-27 19:23:50 -07:00
Brian Warner	89ceb49309	node.py: change get_or_create_config() to accept a function	2007-08-27 19:07:12 -07:00
Brian Warner	56afda11d1	deletion phase2a: improve creation of renew/cancel secrets. Still fake though.	2007-08-27 19:00:18 -07:00
Brian Warner	77a7232867	node.py: refactor config-file getting and setting	2007-08-27 18:58:39 -07:00
Brian Warner	739ae1ccde	deletion phase1: send renew/cancel-lease secrets, but my_secret is fake, and the StorageServer discards them	2007-08-27 17:28:51 -07:00
Brian Warner	ed525f7478	test_web: improve provisioning.py test coverage a bit by using a live web hit	2007-08-27 17:26:39 -07:00
Brian Warner	5946985c07	add a provisioning utility page which shows necessary storage space and transfer rates for grids of various sizes	2007-08-26 23:44:24 -07:00
Brian Warner	b620d0955b	debug.py: add share-overhead size info to dump-uri-extension	2007-08-26 23:42:39 -07:00
Brian Warner	0a39430730	webish: strip leading/tailing whitespace from user-provided filenames, to avoid confusion	2007-08-25 12:05:24 -07:00
Brian Warner	44db5fd8a2	webish: strip leading/tailing whitespace from user-provided URIs	2007-08-25 12:05:06 -07:00
Zooko O'Whielacronx	427bf2cdc9	change name of the query "allow_local_access()?" to "local_access_is_allowed()" ... So as to avoid confusing it with the setter method "allow_local_access(allowed=True)".	2007-08-22 10:32:00 -07:00
Brian Warner	ac358fc32f	tahoe_put.py: hush pyflakes by removing unused 'sys' import	2007-08-17 16:29:50 -07:00
tahoe	f49194cb5e	trying to introduce old style humanreadablied logs hopefully without breaking the existing ones	2007-08-11 14:52:37 -07:00
Brian Warner	664a0e6af4	crypto: fix compiler warnings in the .c files	2007-08-15 17:48:52 -07:00
Brian Warner	6c24848797	copy RSA from PyCrypto into the allmydata/ tree, we'll use it eventually	2007-08-15 17:36:39 -07:00
Brian Warner	6ac237b845	hush pyflakes warning in Crypto.Util.number	2007-08-14 14:17:12 -07:00
Brian Warner	b7703df6a5	remove PyCrypto, copy AES/SHA256/Util.number into the allmydata/ tree	2007-08-14 13:57:41 -07:00
Brian Warner	3e1c417f81	webish: modify JSON to match zooko's proposed API changes in #118	2007-08-23 13:00:39 -07:00
Zooko O'Whielacronx	0c22044f7e	command-line: remove the --vdrive option (it is now hardcoded to "global")	2007-08-23 13:27:00 -07:00
Brian Warner	f3353ee5e4	web: replace FILE links with /uri -based ones, to prevent an XSS attack against the secret vdrive URI contained in the current URL	2007-08-22 17:35:01 -07:00
Brian Warner	930618fba2	web: oops, forgot to add start.html	2007-08-22 15:08:00 -07:00
Brian Warner	4de5767c98	web: remove /vdrive/private, replace with a start.html file that points at the /uri/PRIVATE_URI, to prevent XSRF attacks	2007-08-22 14:54:34 -07:00
Zooko O'Whielacronx	8717905e35	refactor reading of configuration files in client.py ... This makes it so that an optional file which is unreadable or is rm'ed at the wrong moment will be ignored instead of raising an exception. It also bums out a couple of unnecessary lines of code (the explicit ".close()" call).	2007-08-22 10:29:57 -07:00
Brian Warner	d5262ec7e1	client.py: add a newline to "myself.furl" contents	2007-08-17 23:28:22 -07:00
Zooko O'Whielacronx	9996836d23	command-line: fix ticket #111 by requiring input to be a local file and sending Content-Length header	2007-08-17 14:59:49 -07:00
Zooko O'Whielacronx	594912e334	command-line: fix a few bugs in the "execute this python file" way to execute rm	2007-08-17 14:17:31 -07:00
Brian Warner	3e2d1e28eb	tahoe_put-web2ish.py: clean up imports to be pyflakes-compliant, add newline to output	2007-08-16 17:47:52 -07:00
Brian Warner	2014662b47	cli.py: fix typo in synopsis	2007-08-16 17:47:24 -07:00
Brian Warner	75056b6222	test_system: weaken the assertion about the 410 error message when downloading a bogus URI, since some systems emit error messages differently	2007-08-16 17:05:26 -07:00
Brian Warner	6b5e05cc67	test_system.py: verify that we can replace files in place	2007-08-16 17:03:50 -07:00
Brian Warner	fe06b3be8b	dirnode: change the defined behavior of RIVirtualDriveServer.set to allow replace-in-place without raising an exception	2007-08-16 17:03:19 -07:00
Brian Warner	be0ab3b1f1	test_system.py: many (failing) web tests were accidentally bypassed, fix those. Add some PUT tests.	2007-08-16 16:49:40 -07:00
Zooko O'Whielacronx	a12ef1ae82	command-line: add "rm", and tidy-up variable names, and make it so "allmydata-tahoe spam" prints a usage message instead of returning silently	2007-08-17 13:23:16 -07:00
Zooko O'Whielacronx	e554962ac3	command-line: remove some redundant options checking	2007-08-17 13:06:43 -07:00
Zooko O'Whielacronx	bffd0c97f5	command-line: fix all three commands and all two ways to invoke them to require node-url and give a useful usage string if node-url is absent or of the wrong form	2007-08-17 12:54:47 -07:00
Zooko O'Whielacronx	e4f07a975f	tahoe_put.py: don't treat "^HTTP 200 OK" appearing somewhere in the body as if it appeared in the header	2007-08-17 12:06:41 -07:00
Zooko O'Whielacronx	7e8c6eb50b	tahoe_put.py: require node-url to be provided	2007-08-17 12:06:11 -07:00
Zooko O'Whielacronx	922663a7e2	cmdline: change "--server" to "--node-url" and make it have no default value	2007-08-16 16:53:27 -07:00
Zooko O'Whielacronx	1564987e6c	put now exits after doing its work, and it prints a terse message if 200 or 201, and a full dump of response elsewise	2007-08-16 16:30:39 -07:00
Zooko O'Whielacronx	6ed3a23c62	incomplete version of tahoe-put.py ... It doesn't exit properly afterward, and it might not do the best things with non-success responses from the server. (See tahoe-put-web2ish.py for an example of better response handling.)	2007-08-16 16:01:01 -07:00
Zooko O'Whielacronx	9ad02dbe80	fix version class to preferred format and correct parsing	2007-08-16 15:38:01 -07:00
Zooko O'Whielacronx	5a1d4aa8b7	version_class.py: if you don't have pkg_resources for comparing version numbers, use distutils.version.LooseVersion	2007-08-16 16:16:41 -07:00
Brian Warner	ceef80bee6	scripts: rearrange Options, make --version behave the same for all commands	2007-08-16 12:50:19 -07:00
Zooko O'Whielacronx	4ed7f0ac2a	fix small bug in unit tests which caused spurious failures on Windows	2007-08-16 14:14:41 -07:00
Zooko O'Whielacronx	a431811011	in-line doc edit: s/pyutil/allmydata.util/	2007-08-16 14:10:48 -07:00
Zooko O'Whielacronx	094b687d6e	import version class and make-version script from pyutil -- fixes win32 build, improves error handling, and eliminates unused features	2007-08-16 14:09:30 -07:00
Zooko O'Whielacronx	6c4fb6fd93	a first crack at the "put" command-line ... There are actually two versions in this patch, one of which requires twisted.web2 and the other of which uses the Python standard library's socket module. The socketish one doesn't know when the web server is done so it hangs after doing its thing. (Oh -- maybe I should add an HTTP header asking the web server to close the connection when finished.) The web2ish one works better in that respect. Neither of them handle error responses from the server very well yet. After lunch I intend to finish the socketish one. To try one, mv src/allmydata/scripts/tahoe_put-{socketish,web2ish}.py src/allmydata/scripts/tahoe_put.py . If you want to try the web2ish one, and you can't find a web2 package to install, you can get one from: http://allmydata.org/~zooko/repos/twistedweb2snarf/	2007-08-16 12:15:38 -07:00
Brian Warner	ee6a6c79fa	webish.py: allow users to delete (but not create) empty-named files. Closes #94.	2007-08-15 23:14:05 -07:00
Brian Warner	c1da0c11bc	webish: implement replace= for POST commands	2007-08-15 15:21:38 -07:00
Brian Warner	1752c9e29e	webish: implement replace= for PUT commands	2007-08-15 13:22:23 -07:00
Brian Warner	42dcc3088e	IDirectoryNode: add has_child() method	2007-08-15 13:22:01 -07:00
Brian Warner	244471dcc4	check_memory.py: Disable the 100MB test for now: our buildslave can't currently handle it because the testnet/framework processes uses something like 600M of RSS.	2007-08-15 12:55:11 -07:00
Brian Warner	a0c16f1a36	client.py: make a note in the logs when the auto-shutdown feature is in use	2007-08-14 02:12:30 -07:00
Brian Warner	179d5462c0	webish: look for when_done= in POST fields as well as queryargs. Closes #101. ... We need to look in the fields because that's how we build the mkdir/upload forms. Without this, uploading or creating directories would leave us on a page that had just a URI, instead of something actually useful to a human.	2007-08-13 17:45:02 -07:00
Brian Warner	6c6c62037d	fix pyflakes warnings from recent b32decode change	2007-08-12 16:33:51 -07:00
Brian Warner	0332e94656	test_system.py: add coverage for get_permuted_peers()	2007-08-12 16:29:34 -07:00
Brian Warner	1ac83637a6	Client.get_permuted_peers: use self.nodeid now that it's fixed	2007-08-12 16:24:51 -07:00
tahoe	53349c571f	shorten ids	2007-08-12 11:53:18 -07:00
Zooko O'Whielacronx	42f8e57416	don't over-encode the nodeid many times with ascii-encoding	2007-08-12 10:29:38 -07:00
Brian Warner	2f5a27316f	webish: localfile=/localdir= are now disabled by default, a special switch is required to enable them	2007-08-10 18:21:22 -07:00
Brian Warner	9281a0aedc	webish: reduce POST memory footprint by overriding http.Request ... The original twisted.web.http.Request class has a requestReceived method that parses the form body (in the request's .content filehandle) using the stdlib cgi.parse_multipart() function. parse_multipart() consumes a lot of memory when handling large file uploads, because it holds the arguments entirely in RAM. Nevow's subclass of Request uses cgi.FieldStorage instead, which is much more memory-efficient. This patch uses a local subclass of Request and a modified copy of the requestReceived method. It disables the cgi.parse_multipart parsing and instead relies upon Nevow's use of FieldStorage. Our code must look for form elements (which come from the body of the POST request) in req.fields, rather than assuming that they will be copied into req.args (which now only contains the query arguments that appear in the URL). As a result, memory usage is no longer inflated by the size of the file being uploaded in a POST upload request. Note that cgi.FieldStorage uses temporary files (tempfile.TemporaryFile) to hold the data. This closes #29.	2007-08-10 17:25:33 -07:00
Brian Warner	b6ff38e4c7	webish.POSTHandler: fix typo that meant we didn't look for 'name' in req.fields	2007-08-10 17:25:28 -07:00
Brian Warner	998802fd6d	#96: add flag to enable pushing data to ourselves, defaulting to False	2007-08-09 18:30:24 -07:00
Brian Warner	8b14ad1673	encode.py: log a percentage complete as well as the raw byte counts	2007-08-09 18:28:45 -07:00
Brian Warner	08ee80176a	Encoder.__repr__: mention the file being encoded	2007-08-09 18:26:56 -07:00
Brian Warner	684966d103	encode.py: add a reactor turn barrier between segments, to allow Deferreds to retire and free their arguments, all in the name of reducing memory footprint	2007-08-09 18:26:17 -07:00
Zooko O'Whielacronx	531109e803	fix test_web refactoring so that the WebMixin class isn't a TestCase ... Thanks, Brian.	2007-08-10 08:40:02 -07:00
wilcoxjg	e13783cd6a	If do_stop attempts to stop a nonexistent process the OSError exception is caught (by errno) and handled tersely.	2007-08-09 08:57:45 -07:00
Zooko O'Whielacronx	39f218ab90	refactor test_web so that other tests can use the part of test_web that sets up a simple filesystem	2007-08-09 13:08:42 -07:00
Brian Warner	9ac5c14037	introducer: don't log.err() an initial connection failure, since that flunks tests. Use self.log() instead. Also improve test_client.py to always trigger this case, before it was finishing to quickly to always hit the problem.	2007-08-09 12:53:44 -07:00
Brian Warner	84d41adfcc	check_memory.py: include a single 100MB test	2007-08-09 11:30:33 -07:00
Zooko O'Whielacronx	9b08691f51	fix iputil so that it doesn't launch dozens of processes when you give it a full path and so that it tries executables in preference order and stops as soon as one gives out a dotted-quad string	2007-08-09 10:56:47 -07:00
Zooko O'Whielacronx	524c9f1fc2	startstop_node now works if the logs directory already exists ... Thanks to Arno and Arch for pointing this out.	2007-08-08 21:43:48 -07:00
Brian Warner	57df32aac5	now that the buildslave is moved to a new machine, enable the 50MB test	2007-08-09 01:32:52 -07:00
Zooko O'Whielacronx	42d3afffa2	refactor test_uri and add a test of the empty file	2007-07-26 10:48:30 -07:00
Brian Warner	2ba415bd3e	move nodelogs from BASEDIR/twistd.log* to BASEDIR/logs/twistd.log*, closes #61	2007-07-24 20:44:33 -07:00
Brian Warner	8de93635af	test_web.py: oops, fix syntax error caused by removing comments	2007-07-25 03:21:30 -07:00
Brian Warner	6aff425fd2	test_web.py: remove spurious '# YES' lines, now that all the tests pass	2007-07-24 20:16:21 -07:00
Brian Warner	f664d4d80d	vdrive: eventually create public/private dirnodes event if the vdrive server isn't available at start up	2007-07-24 19:44:25 -07:00
Brian Warner	c3293f4f37	interfaces: move signatures into docstrings, to reduce lines of code and improve code-coverage numbers	2007-07-24 19:43:21 -07:00
Brian Warner	e3eba2d6c5	remove unused imports: hush pyflakes warnings	2007-07-24 18:13:58 -07:00
Zooko O'Whielacronx	32c90c6468	switch from rfc 3548 base-32 to z-base-32 except for tubids/nodeids	2007-07-24 13:46:06 -07:00
Brian Warner	e6e9ddc588	refactor upload/encode, to split encrypt and encode responsibilities	2007-07-23 19:31:53 -07:00
Brian Warner	9c5ab89afe	truncate storage index to 128 bits, since it's derived from a 128 bit AES key	2007-07-22 19:48:44 -07:00
Brian Warner	81a9904455	CHK: remove the storage index from the URI, deriving it from the key instead	2007-07-21 18:23:15 -07:00
Brian Warner	2bc3c163b6	uri.py: get keys and index from the URI instance	2007-07-21 17:45:00 -07:00
Brian Warner	1d9a58977f	uri: implement URI-processing classes, IFileURI/IDirnodeURI, use internally	2007-07-21 15:40:36 -07:00
Brian Warner	32fcf0b405	dirnode.build_manifest(): tolerate cycles in the directory graph	2007-07-21 15:40:13 -07:00
Brian Warner	e3a57fca98	upload: finish refactoring, all unit tests pass now	2007-07-19 22:53:29 -07:00
Brian Warner	9af506900b	upload: refactor to enable streaming upload. not all tests pass yet	2007-07-19 18:21:44 -07:00
Brian Warner	a7f04616bf	webish: improve test coverage further	2007-07-17 12:28:29 -07:00
Brian Warner	3197e1d39b	webish: improve test coverage of WebDownloadTarget	2007-07-17 12:16:45 -07:00
Brian Warner	d927d371b7	check_memory.py: test POST-based uploads as well as from-disk ones	2007-07-16 20:17:51 -07:00
Brian Warner	4db205b8b3	vdrive.py: log a note when we successfully retrieve the global root directory	2007-07-16 20:17:06 -07:00
Brian Warner	0826b1bd34	introducer.py: minor rearrangement of methods	2007-07-16 19:47:42 -07:00
Brian Warner	0269a6229b	check_memory.py: fix benign-but-noisy vdrive.furl handling bug	2007-07-16 19:34:52 -07:00
Brian Warner	0fd5dcacf6	check_memory: update it, write stats to a file, disable 50MB test for now	2007-07-16 18:08:55 -07:00
Brian Warner	1aa22b9abd	client.py: add a 'debug_no_storage' option to throw out all share data	2007-07-16 18:07:03 -07:00
Brian Warner	e91288d9ed	client.py: write control.furl in the right place	2007-07-16 18:06:27 -07:00
robk-org	920fed7f2a	added unit test to webish's rename function ... added unit tests to test various permutations of the rename function, and some sanity checks on the rename-form function. also added a guard to prevent '/' in from_/to_name in rename calls.	2007-07-16 17:05:01 -07:00
Brian Warner	9be3da03f6	runner.py: make 'allmydata-tahoe --version' emit version numbers of everything	2007-07-16 14:58:36 -07:00
Brian Warner	20c980d02b	reduce MAX_SEGMENT_SIZE from 2MB to 1MB, to compensate for the large blocks that 3-of-10 produces	2007-07-16 13:48:34 -07:00
Brian Warner	56dcb814a8	webish: test error cases more thoroughly by looking inside the response text	2007-07-16 12:01:19 -07:00
Brian Warner	3f294d5597	webish: oops, handle POST without localfile= too	2007-07-16 12:00:54 -07:00
Brian Warner	f207f4a199	webish.py: disallow slashes in POSTed filenames. Closes #75.	2007-07-16 11:53:12 -07:00
Brian Warner	7bc031a81d	webish.Directory: add a 'text/plain' link for files, to download as plaintext. ... This is useful to view (e.g.) .py files that would otherwise be served as some weird text/x-python that browsers will just want to save to disk.	2007-07-13 23:45:35 -07:00
Brian Warner	225fbfee57	storage.py: add a test for the next_power_of_k fix I made a few hours ago, basically do an upload with a non-power-of-two number of segments	2007-07-13 22:24:06 -07:00
Brian Warner	6e1ad64c9d	webish.py: test that _get_or_create_directories fix I made a few hours ago	2007-07-13 22:19:16 -07:00
Brian Warner	a290205ade	webish: handle PUTs to direct children of the root	2007-07-13 19:31:52 -07:00
Brian Warner	8a39ee9034	storage.py: turn some assertions into preconditions	2007-07-13 19:30:48 -07:00
Brian Warner	9fc687cdfc	storage.py: handle num_segments != power-of-two without an assertion	2007-07-13 19:30:21 -07:00
Brian Warner	5c08be170e	interfaces: increase ShareSize now that our default k is smaller (hence blocks are bigger)	2007-07-13 19:29:31 -07:00
Brian Warner	c6f52e379a	rename storageserver.py to just storage.py, since it has both server and client sides now	2007-07-13 17:25:45 -07:00
Brian Warner	0468448b85	welcome.xhtml: add a link to allmydata.org	2007-07-13 17:15:44 -07:00
Brian Warner	d206843625	encode.py: minor typo	2007-07-13 17:00:06 -07:00
Brian Warner	4d868e6649	fix dump-uri-extension	2007-07-13 16:58:08 -07:00
Brian Warner	35117d77a0	storageserver.ReadBucketProxy: break out _parse_offsets, for debug tools	2007-07-13 16:52:17 -07:00
Brian Warner	7589a8ee82	storage: we must truncate short segments. Now most tests pass (except uri_extension)	2007-07-13 16:38:25 -07:00
Brian Warner	1f8e407d9c	more #85 work, system test still fails	2007-07-13 15:09:01 -07:00
Brian Warner	cd8648d39b	storage: use one file per share instead of 7 (#85). work-in-progress, tests still fail	2007-07-13 14:04:49 -07:00
Brian Warner	6ff94541a8	control.py: make get_memory_usage() callable from elsewhere	2007-07-08 19:07:54 -07:00
robk-org	d65d02fa58	add a 'rename' button to the webish dir view ... alongside the 'del' button is now presented a 'rename' button, which takes the user to a new page, the 't=rename-form' page, which asks ther user for the new name of the child and ultimately submits a POST request to the dir for 't=rename' to perform the actual rename i.e. an attach followed by a delete of children.	2007-07-12 16:53:54 -07:00
robk-org	15c3055176	fix bug in arg handling around 'delete' button in directory view ... the code composing the form providing the 'delete' button in a dir view was broken in that it tried to put some of the arguments into the url query, rather than as the form's post args. worse, nevow was kind enough to escape the querystring making it invalid.	2007-07-12 16:46:54 -07:00
robk-org	cacc9ac7b1	include the filename on URI-link urls for files ... this is likely to induce browsers to do more useful things with the result than they would when given neither content-type nor filename. (i.e. they can guess that a .jpg file is an image, even with a bogus content type)	2007-07-11 15:26:47 -07:00
Brian Warner	798b714bef	webish.DirnodeWalkerMixin: comment out the code that demonstrates what we ... would do if we could just use generators, since we don't use it.	2007-07-12 23:28:09 -07:00
Brian Warner	b0c65ed232	note that setting k=1 is equivalent to replication	2007-07-12 16:22:12 -07:00
Brian Warner	ba7e14a870	fix several methods to handle LIT URIs correctly, rather than assuming that all filenodes are CHK URIs	2007-07-12 16:17:49 -07:00
Brian Warner	53cf757be3	make it possible to download LIT uris. oops.	2007-07-12 16:16:59 -07:00
Brian Warner	5399395c27	allow the introducer to set default encoding parameters. Closes #84. ... By writing something like "25 75 100" into a file named 'encoding_parameters' in the central Introducer's base directory, all clients which use that introducer will be advised to use 25-out-of-100 encoding for files (i.e. 100 shares will be produced, 25 are required to reconstruct, and the upload process will be happy if it can find homes for at least 75 shares). The default values are "3 7 10". For small meshes, the defaults are probably good, but for larger ones it may be appropriate to increase the number of shares.	2007-07-12 15:33:30 -07:00
Brian Warner	def63d193e	implement URI:LIT, which stores small (<55B) files inside the URI itself. Fixes #81.	2007-07-12 13:22:36 -07:00
Brian Warner	b62252183d	interfaces.py: increase RIVirtualDriveServer.list constraint from 100 entries to 1000, for now	2007-07-12 13:23:52 -07:00
Brian Warner	9238c61224	cli: implement 'get'	2007-07-11 10:26:19 -07:00
Zooko O'Whielacronx	32166c360a	tahoe_ls: remove a couple of vestigial or debugging bits	2007-07-10 19:46:40 -07:00
Brian Warner	b419b853ea	cli: implement 'allmydata-tahoe ls'	2007-07-10 19:37:37 -07:00
Brian Warner	d8a878da86	startstop_node.py: refactor find_twistd() out so it is only run when you need to start a node	2007-07-10 19:13:55 -07:00
Brian Warner	f22801aa33	runner.py: further refactoring	2007-07-10 19:05:18 -07:00
Brian Warner	4a16d20d85	runner.py: start to refactor into separate files	2007-07-10 18:41:52 -07:00
Brian Warner	944583a599	tahoe-ls.py: hush a pyflakes warning	2007-07-10 16:26:05 -07:00
Brian Warner	ea3c3e6780	import simplejson directly, and remove our local copies of the component .py files from util/json*	2007-07-10 15:49:32 -07:00
Brian Warner	dec729028f	tahoe-ls.py: initial version of an 'ls' CLI utility	2007-07-10 15:34:55 -07:00
Brian Warner	13e84526a1	webapi: normalized API: use t=upload or t=download when providing localdir= or localfile=	2007-07-10 13:24:10 -07:00
Brian Warner	ad038497db	import the decoder portion of simplejson-1.7.1 too	2007-07-10 13:12:40 -07:00
Brian Warner	5264559561	webish: cosmetic: fix missing whitespace in HTML	2007-07-10 10:33:19 -07:00
Zooko O'Whielacronx	2a4423ba14	tahoe-get.py: fix bug: actually respect the optional server argument	2007-07-09 18:47:16 -07:00
Zooko O'Whielacronx	9bc87f610c	tahoe-get.py: add optional target-local-file argument	2007-07-09 18:27:11 -07:00
Zooko O'Whielacronx	9641a6df22	tahoe-get.py: accept vdrive and server options (using optparse)	2007-07-09 18:20:02 -07:00
Zooko O'Whielacronx	8769f2eeba	tahoe-get.py: the first, most primitive command-line client	2007-07-09 18:19:53 -07:00
Brian Warner	dce1dc2730	storage: wrap buckets in a local proxy ... This will make it easier to change RIBucketWriter in the future to reduce the wire protocol to just open/write(offset,data)/close, and do all the structuring on the client end. The ultimate goal is to store each bucket in a single file, to reduce the considerable filesystem-quantization/inode overhead on the storage servers.	2007-07-08 23:27:46 -07:00
Brian Warner	8891544d6c	directory.xhtml: oops, missed a comma	2007-07-08 00:44:08 -07:00
Brian Warner	295c3a75c2	directory.xhtml: remove the leftover XML link	2007-07-08 00:33:20 -07:00
Brian Warner	0395ad76eb	web: indent the JSON to make it easier for humans to read, but slightly larger	2007-07-08 00:23:23 -07:00
Brian Warner	72fc8c5cb8	web: use real JSON instead of the fake stubs ... Also include the encoder portion of Bob Ippolito's simplejson-1.7.1 as allmydata.util.json_encoder . simplejson is distributed under a more liberal license than Tahoe (looks to be modified BSD), so redistributing it should be ok.	2007-07-08 00:17:11 -07:00
Brian Warner	2d06a66ec1	test_web.test_welcome: give the rooturl a trailing slash, otherwise older versions of twisted complain	2007-07-07 23:22:52 -07:00
Brian Warner	bd8625076a	web: remove t=XML, and other dead code	2007-07-07 22:55:15 -07:00
Brian Warner	159f8bfd50	web: improve test coverage	2007-07-07 22:47:18 -07:00
Brian Warner	e2f99247f2	web: make sure we present read-only directories properly	2007-07-07 22:21:20 -07:00
Brian Warner	135f08a518	web: remove more dead code	2007-07-07 22:20:03 -07:00
Brian Warner	8952114442	webish.py: remove dead code	2007-07-07 22:12:46 -07:00
Brian Warner	62e8528cc6	web: /uri/ must escape slashes, we use bangs for this	2007-07-07 22:06:52 -07:00
Brian Warner	fb40b55bc3	web: replace welcome-page download-URI form with new version	2007-07-07 22:06:22 -07:00
Brian Warner	5b0c9df14c	web: survive bogus URIs when displaying sizes	2007-07-07 21:31:18 -07:00
Brian Warner	fe3c002842	web: show the root name of the vdrive	2007-07-07 21:31:02 -07:00
Brian Warner	9e2ce51de7	web: change title to say 'Tahoe', not tahoe2	2007-07-07 21:30:38 -07:00
Brian Warner	e5482b5254	web: add a 'return to welcome page' link	2007-07-07 21:30:04 -07:00
Brian Warner	2adb773210	web: add when_done to all POST operations, use it from upload/mkdir/mount/delete forms	2007-07-07 21:17:48 -07:00
Brian Warner	0cb154f402	web: make 'delete' buttons work again	2007-07-07 20:46:30 -07:00
Brian Warner	b3f0293d50	web: change per-directory forms to match new POST scheme	2007-07-07 20:35:47 -07:00
Brian Warner	617a85df67	web: remove debug prints	2007-07-07 20:11:30 -07:00
Brian Warner	464f25e5f2	web: more test work, now all tests pass, POST too, only XMLRPC left to implement	2007-07-07 20:06:58 -07:00
Brian Warner	f35c9c6540	test_system.py: change/remove the tests that currently fail due to web changes	2007-07-07 20:06:44 -07:00
Brian Warner	7d92b8a123	webish.py: add links to JSON/etc representations of directory contents to the listing	2007-07-07 11:31:07 -07:00
Brian Warner	5fbdce155c	runner.py: add --force flag to restart, to restart a not-already-running node	2007-07-07 11:17:32 -07:00
Brian Warner	10a5ff3767	webish: fix ?t=manifest, ?t=xml so they don't throw exceptions, prune directory.xhtml	2007-07-07 11:15:31 -07:00
Brian Warner	0cd730a7b3	web: more test work, now all tests either pass or are skipped (POST, XMLRPC, and URI/)	2007-07-07 10:34:05 -07:00
Brian Warner	d501984eba	webapi: checkpointing more test progress	2007-07-07 00:16:36 -07:00
Brian Warner	6570253d6b	checkpointing new webapi: not all tests pass yet	2007-07-06 19:43:55 -07:00
Brian Warner	21e12f383d	web: missed a IndexError-to-KeyError conversion	2007-07-06 19:43:03 -07:00
Brian Warner	71c04fc2e7	web: use KeyError (rather than IndexError) to signal a missing child	2007-07-06 19:40:08 -07:00
Brian Warner	a34040df99	interfaces: remove spurious 'self' from interface declarations	2007-07-06 19:39:47 -07:00
Brian Warner	9e42dda6a4	add IDirectoryNode.get_child_at_path	2007-07-06 19:38:37 -07:00
Brian Warner	a1c97aa0be	storageserver: include metadata in the sizelimit, at least after the bucket has been closed	2007-07-03 17:38:49 -07:00
Brian Warner	94e6e6160b	activate storage size limits in the client. Closes #34. ... To use this, write a number like 10MB or 5Gb or 5000000000 to a file named 'sizelimit' in the client's base directory. The node will not grant leases for shares that would take it much beyond this many bytes of storage. Note that metadata is not included in the allocation count until a restart, so the actual space consumed may grow beyond the limit if the node is not restarted very frequently and the amount of metadata is significant.	2007-07-03 17:27:07 -07:00
Brian Warner	c80ea7d693	storageserver: implement size limits. No code to enable them yet, though	2007-07-03 17:08:02 -07:00
Brian Warner	9ddb929651	fileutil: add du() function	2007-07-03 15:49:45 -07:00
Brian Warner	c4a8db3eb2	webish: provide a valid Content-Length header on downloads	2007-07-03 15:09:00 -07:00
Brian Warner	622acc690a	webish: improve reporting of web download errors that occur early enough ... If the error occurs before any data has been sent, we can give a sensible error message (code 500, stack trace, etc). This will cover most of the error cases. The ones that aren't covered are when we run out of good peers after successfully decoding the first segment, either because they go away or because their shares are corrupt.	2007-07-03 13:47:37 -07:00
Brian Warner	f15bb302a1	webish.py: handle errors during download better. Addresses #65. ... Previously, exceptions during a web download caused a hang rather than some kind of exception or error message. This patch improves the situation by terminating the HTTP download rather than letting it hang forever. The behavior still isn't ideal, however, because the error can occur too late to abort the HTTP request cleanly (i.e. with an error code). In fact, the Content-Type header and response code have already been set by the time any download errors have been detected, so the browser is committed to displaying an image or whatever (thus any error message we put into the stream is unlikely to be displayed in a meaningful way).	2007-07-03 13:18:14 -07:00
Brian Warner	7e32d3c8af	test_util: improve test coverage of allmydata.util.fileutil	2007-07-03 11:15:05 -07:00
Brian Warner	6427b44b42	webish: make each link in the directory name an href to that directory	2007-06-29 11:17:18 -07:00
Brian Warner	9d47c2524e	test_system: stall for a second while bouncing the client, it might help windows	2007-06-28 19:20:28 -07:00
Brian Warner	9ecde0d23e	webish: fix link to private_vdrive	2007-06-28 19:07:52 -07:00
Brian Warner	5ffc66ad7b	client.py: remove unused import	2007-06-28 18:01:00 -07:00
Brian Warner	4196c9ff66	client.py: remove unused code	2007-06-28 17:55:13 -07:00
Brian Warner	a8bf0645dc	interfaces.py: remove some unused 'pass' lines	2007-06-29 00:50:00 -07:00
Brian Warner	efb99078b6	change IVirtualDrive.get_node_at_path to accept either a list or a single slash-separated string	2007-06-28 17:46:14 -07:00
Brian Warner	5854916299	test_system.py: more vdrive coverage	2007-06-28 15:05:33 -07:00
Brian Warner	d6e732b65f	vdrive: add get_node_at_path(pathlist)	2007-06-28 11:00:03 -07:00
Brian Warner	13400e24ac	vdrive: make sure we really record global/private root directory uris	2007-06-28 10:59:51 -07:00
Brian Warner	b06c74c2a1	Add the 'vdrive' service, for clients to access the public/private root dirs. ... These allow client-side code to conveniently retrieve the IDirectoryNode instances for both the global shared public root directory, and the per-user private root directory.	2007-06-27 17:11:06 -07:00
Brian Warner	1d86cd161b	webish.py: add a web page to display the manifest for any particular directory	2007-06-26 19:55:21 -07:00
Brian Warner	18ab5ce837	dirnode: add build_manifest() and introduce 'refresh capabilities'	2007-06-26 19:41:20 -07:00
Brian Warner	2766b7988b	remove unused/obsoleted workqueue.py	2007-06-26 17:25:23 -07:00
Brian Warner	b11fa20191	merge vdrive.py and filetable.py into a single dirnode.py	2007-06-26 17:16:58 -07:00
Brian Warner	78c2376fa3	more runner.py test coverage: don't bypass argv parsing	2007-06-26 16:51:38 -07:00
Brian Warner	912fc5a77e	runner.py: improve test coverage further: implement --quiet with StringIOs	2007-06-26 16:19:18 -07:00
Brian Warner	de24d3cd94	dump_uri_extension: improve test coverage of runner.py	2007-06-26 15:55:00 -07:00
Brian Warner	a505560bbe	improve test coverage of runner.py	2007-06-26 15:36:46 -07:00
Brian Warner	b6162681d5	webish: mark read-only directories as such when listing their parent	2007-06-26 12:37:00 -07:00
Brian Warner	bc2603c818	vdrive: protect dirnode contents with an HMAC	2007-06-26 12:36:21 -07:00
Brian Warner	4a0682874b	remove old filetree code	2007-06-25 20:34:19 -07:00
Brian Warner	fb02488a8e	vdrive: switch to URI:DIR and URI:DIR-RO, providing transitive readonlyness	2007-06-25 13:23:51 -07:00
Brian Warner	107207e09d	webish: log dirname in mkdir	2007-06-15 02:48:19 -07:00
Brian Warner	a00bde050b	webish: when mounting a shared directory, don't automatically move to it	2007-06-15 02:34:24 -07:00
Brian Warner	2066e8ddfc	webish: add option to export/import shared directories (always read-write)	2007-06-15 02:31:23 -07:00
Brian Warner	53da82ec04	test_system.py: match change to /global_vdrive URL	2007-06-15 01:38:55 -07:00
Brian Warner	20dcda4ca7	create a personal (non-shared) vdrive, in addition to the global shared one	2007-06-15 01:33:24 -07:00
Brian Warner	51809ce341	webish: enable deletion of directories	2007-06-15 01:32:57 -07:00
Brian Warner	1eba30cd14	webish.py: put the URI in a narrower auto-scrolling box (with CSS)	2007-06-15 01:32:20 -07:00
Brian Warner	1cb1faf11d	runner.py: add 'dump-directory-node' command	2007-06-15 00:47:05 -07:00
Brian Warner	82c38d370a	make new vdrive work, implement convenience wrapper, passes all tests	2007-06-15 00:37:32 -07:00
Brian Warner	5e5347a1bc	test_filetable: fix to match new vdrive approach	2007-06-15 00:01:01 -07:00
Brian Warner	ec77a227be	revamp vdrive: nodes with furls. tests still fail.	2007-06-14 20:14:34 -07:00
Brian Warner	106177a7f2	filetable: switch to new approach with anonymous nodes	2007-06-14 17:24:56 -07:00
Brian Warner	a182edf4de	upload.py: minor refactoring	2007-06-11 19:21:51 -07:00
Brian Warner	e0dfedb0a6	add 'allmydata-tahoe dump-uri-extension' utility command	2007-06-11 18:38:21 -07:00
Brian Warner	382888899b	refactor URI_extension handlers out of encode/download and into uri.py	2007-06-11 18:25:18 -07:00
Brian Warner	48a8c32ed7	webish: display program/library versions on the welcome page	2007-06-11 10:51:11 -07:00
Brian Warner	363ed1264a	webish: show vdrive and introducer connectedness separately ... Also don't offer a link to the vdrive webpages if we don't have a vdrive.furl	2007-06-09 21:03:57 -07:00
Brian Warner	956d5ae256	rename fileid/verifierid to plaintext_hash/crypttext_hash	2007-06-09 20:46:04 -07:00
Brian Warner	584dc4ae94	handle uri_extension with a non-bencode serialization scheme	2007-06-08 16:17:54 -07:00
Brian Warner	c9ef291c02	rename thingA to 'uri extension'	2007-06-08 15:59:16 -07:00
Brian Warner	72bc8627de	consolidate multiple definitions of NotEnoughPeersError	2007-06-07 22:20:55 -07:00
Brian Warner	f62a544b93	remove several leftover defintions of netstring()	2007-06-07 22:13:18 -07:00
Brian Warner	6df5c856ac	test_client.py: improve test coverage a bit	2007-06-07 22:09:02 -07:00
Brian Warner	c049941529	move almost all hashing to SHA256, consolidate into hashutil.py ... The only SHA-1 hash that remains is used in the permutation of nodeids, where we need to decide if we care about performance or long-term security. I suspect that we could use a much weaker hash (and faster) hash for this purpose. In the long run, we'll be doing thousands of such hashes for each file uploaded or downloaded (one per known peer).	2007-06-07 21:47:21 -07:00
Brian Warner	cabba59fe7	test_encode.py: even more testing of merkle trees, getting fairly comprehensive now	2007-06-07 21:24:39 -07:00