5198 Commits

Author	SHA1	Message	Date
meejah	d0141b2f95	better docstring from exarkun	2018-09-10 21:58:28 -06:00
meejah	edb581b434	better docstring from exarkun	2018-09-10 21:58:28 -06:00
meejah	507c2db649	better utest docstring, cleanup	2018-09-10 21:58:28 -06:00
meejah	376a1dcbd5	turn XXX into a ticket	2018-09-10 21:58:28 -06:00
meejah	8b5e988755	use docstring to mark method as async	2018-09-10 21:58:28 -06:00
meejah	2b5d3be3c8	convert XXX comment to ticket	2018-09-10 21:58:28 -06:00
meejah	7de94f623c	clarify Introducer imports	2018-09-10 21:58:28 -06:00
meejah	9a8ef9512e	clarify comments	2018-09-10 21:58:28 -06:00
meejah	f488b79c71	self._portnumfile -> config.portnum_fname	2018-09-10 21:58:28 -06:00
meejah	692128ad1d	better defaults handling	2018-09-10 21:58:28 -06:00
meejah	2e2bdf1ea1	test for EnvironmentError that's not ENOENT	2018-09-10 21:58:28 -06:00
meejah	ab812ca75f	fix utests for Deferred/Failure-returning methods	2018-09-10 21:58:28 -06:00
meejah	f6b62ca192	import	2018-09-10 21:58:28 -06:00
meejah	a344bb24dc	immutable default	2018-09-10 21:58:27 -06:00
meejah	dee8b72206	dead code	2018-09-10 21:58:27 -06:00
meejah	9aae4bd459	only handle ENOENT	2018-09-10 21:58:27 -06:00
meejah	a5287add80	clarify comments	2018-09-10 21:58:27 -06:00
meejah	efce7b1f6a	use Failure for all errors from async methods	2018-09-10 21:58:27 -06:00
meejah	676a9efc23	docstring	2018-09-10 21:58:27 -06:00
meejah	714b0887dc	more docstrings	2018-09-10 21:58:27 -06:00
meejah	2d55b61dd2	fixups after rebase	2018-09-10 21:58:27 -06:00
meejah	c22d7c632e	docstring improvements	2018-09-10 21:58:27 -06:00
meejah	3974af6660	redundant imports	2018-09-10 21:58:27 -06:00
meejah	7632504373	create_client is async	2018-09-10 21:58:27 -06:00
meejah	14a66a54f0	flake8	2018-09-10 21:58:27 -06:00
meejah	0cfbdeb634	dead code	2018-09-10 21:58:27 -06:00
meejah	767f4ddd5d	reorder args to match other method	2018-09-10 21:58:27 -06:00
meejah	045af64c2b	dead code	2018-09-10 21:58:27 -06:00
meejah	06de4c88c4	codecleanup (incorrect merge?)	2018-09-10 21:58:27 -06:00
meejah	edc50f655b	get rid of is_tub_listening	2018-09-10 21:58:27 -06:00
meejah	a82aa4ba2c	code cleanup, docs	2018-09-10 21:58:27 -06:00
meejah	955d7abfa3	move validation code to parser-helper	2018-09-10 21:58:27 -06:00
meejah	2517535d2b	test for 'listen on port 0'	2018-09-10 21:58:27 -06:00
meejah	9e34d15b90	simplify _tub_portlocation helper	2018-09-10 21:58:27 -06:00
meejah	053b494054	comment	2018-09-10 21:58:27 -06:00
meejah	e74d2a7d01	get rid of redundant 'introducer_clients' var	2018-09-10 21:58:27 -06:00
meejah	c84e50baf0	post-rebase fixup: _client_factory and docs	2018-09-10 21:58:27 -06:00
meejah	c7b47f44d2	flake8	2018-09-10 21:58:27 -06:00
meejah	17fa32633c	post-rebase fixups; needs review	2018-09-10 21:58:27 -06:00
meejah	23d304814f	not required for tests	2018-09-10 21:58:27 -06:00
meejah	283be23e3a	basedir fixup, more tests	2018-09-10 21:58:27 -06:00
meejah	a628411988	basedir fixup	2018-09-10 21:58:27 -06:00
meejah	c2d1cf4400	correct service-parent in tests	2018-09-10 21:58:27 -06:00
meejah	136495e405	basedir fixup	2018-09-10 21:58:27 -06:00
meejah	2d044e1324	remove unused method	2018-09-10 21:58:27 -06:00
meejah	bb332fecf7	basedir fixup	2018-09-10 21:58:27 -06:00
meejah	548d9cbbcc	basedir fixup	2018-09-10 21:58:27 -06:00
meejah	5d6a76ffee	proper daemonize error-handling	2018-09-10 21:58:27 -06:00
meejah	08e0c3b7e2	get rid of 'add_service' (just an alias to setServiceParent anyway)	2018-09-10 21:58:27 -06:00
meejah	ea99915af6	basedir/config fixup	2018-09-10 21:58:27 -06:00
meejah	7685fb34cd	pull 'basedir' entirely into _Config ... Put all config-related methods into _Config; change code to ask config for paths instead of using basedir; add some better docstrings	2018-09-10 21:58:27 -06:00
meejah	f80e61fef6	put setServiceParent calls in create_*	2018-09-10 21:58:27 -06:00
meejah	26007f363b	pull 'StorageFarmBroker' out of __init__ ... This means also pulling out introducer-clients and some related utility methods	2018-09-10 21:58:27 -06:00
meejah	42c39d435a	imports	2018-09-10 21:58:27 -06:00
meejah	465bfb8053	cleanup	2018-09-10 21:58:27 -06:00
meejah	48f0df627e	cleanup	2018-09-10 21:58:27 -06:00
meejah	739aaa3ef9	put create() methods in i2p_, tor_provider ... Also Provider -> _Provider, improve docs and update tests	2018-09-10 21:58:27 -06:00
meejah	903d4afaa4	handle None for providers	2018-09-10 21:58:27 -06:00
meejah	793827f8a6	put providers back in service tree	2018-09-10 21:58:27 -06:00
meejah	e7a8d39fbc	split client, introducer READMEs	2018-09-10 21:58:27 -06:00
meejah	c2946cc2e3	remove unused code	2018-09-10 21:58:27 -06:00
meejah	f0c3db0e5a	pyflakes	2018-09-10 21:58:27 -06:00
meejah	0ff21ea7d5	all tests pass. wooo	2018-09-10 21:58:27 -06:00
meejah	c7515b5d8b	more working test	2018-09-10 21:58:27 -06:00
meejah	92ca04d629	fix more tests	2018-09-10 21:58:27 -06:00
meejah	0e51bb183e	fix more things	2018-09-10 21:58:27 -06:00
meejah	279bd814fc	remember info for init_client_storage_broker temporarily	2018-09-10 21:58:27 -06:00
meejah	3c4e065e63	more tests work	2018-09-10 21:58:27 -06:00
meejah	d1307cf05c	fix one more	2018-09-10 21:58:27 -06:00
meejah	43d857a0bd	more refactor + fix some node tests	2018-09-10 21:58:27 -06:00
meejah	71484b4a12	upgrade create_introducer	2018-09-10 21:58:27 -06:00
meejah	4f2d45626c	no more reveal_ip	2018-09-10 21:58:27 -06:00
meejah	d8b432700e	more tests work	2018-09-10 21:58:27 -06:00
meejah	3d5f8becb5	more tests work	2018-09-10 21:58:27 -06:00
meejah	d1a83e9be0	more tests work	2018-09-10 21:58:27 -06:00
meejah	329ef1256a	refactor create_client to be async (works to run, some unit-test fails still)	2018-09-10 21:58:27 -06:00
meejah	41cfd8fb16	change imports; introducer client.read_config	2018-09-10 21:58:27 -06:00
meejah	eab56276e5	introduce create_node_dir	2018-09-10 21:58:27 -06:00
meejah	0f22b9bad0	fixups after rebase	2018-08-25 02:23:58 -06:00
meejah	65ebde6f9d	fix incorrect rebase resolutions	2018-08-25 02:12:02 -06:00
meejah	91517bbad0	unused import	2018-08-25 02:12:02 -06:00
meejah	cce4a92900	fix path-creation cases after merge	2018-08-25 02:12:02 -06:00
meejah	dab291c72f	docstring	2018-08-25 02:12:02 -06:00
meejah	74b560c3c2	failUnless -> assert	2018-08-25 02:10:44 -06:00
meejah	7db48e3677	skip using decorator	2018-08-25 02:10:44 -06:00
meejah	89872b832c	docstring, naming improvements	2018-08-25 02:10:44 -06:00
meejah	38dac24b2b	use skipIf decorator, not inline logic	2018-08-25 02:10:44 -06:00
meejah	d478cf3831	add some docstrings, fix comments	2018-08-25 02:10:44 -06:00
meejah	536ccf8b6d	better 'file not found' handling	2018-08-25 02:10:44 -06:00
meejah	c0772cdd5f	improve docstring	2018-08-25 02:10:44 -06:00
meejah	37d4b59a39	modern syntax	2018-08-25 02:10:44 -06:00
meejah	1b30e9edfc	consistently use 'config' not 'self.config' in __init__	2018-08-25 02:10:44 -06:00
meejah	a432fc35da	docstring improvements	2018-08-25 02:10:44 -06:00
meejah	629185d98f	skip some tests on windows (permissions)	2018-08-25 02:10:44 -06:00
meejah	5000787c18	flake8	2018-08-25 02:10:44 -06:00
meejah	27ea11d164	dead code	2018-08-25 02:10:44 -06:00
meejah	abd7b638b8	basic stfpd setup test	2018-08-25 02:10:44 -06:00
meejah	dd2209a96c	tests to cover config changes	2018-08-25 02:10:44 -06:00
meejah	6c388b9d58	test for authtoken API	2018-08-25 02:10:44 -06:00
meejah	ab947704f0	replace PortLocations tests ... The old tests were hard to read due to all the "if" statements; these might be slightly more verbose but also more explicit	2018-08-25 02:10:44 -06:00
meejah	4d7f8ec9dd	keep clientdir for now, improve how we access it	2018-08-25 02:10:44 -06:00
meejah	2937c729e3	change imports; introducer client.read_config	2018-08-25 02:10:44 -06:00
meejah	4c7f60f42a	test fixups from review	2018-08-25 02:10:44 -06:00
meejah	107ddcd1ba	get rid of get_clientdir	2018-08-25 02:10:44 -06:00
meejah	d544284f92	introduce create_node_dir	2018-08-25 02:10:44 -06:00
meejah	c93ee4f867	document a test-method	2018-08-25 02:10:44 -06:00
meejah	f7f3c54f93	dead code	2018-08-25 02:10:44 -06:00
meejah	86a9ce5793	create _NoNetworkClient using same code as _Client	2018-08-25 02:10:44 -06:00
meejah	22e2d0a417	re-expand path because user input	2018-08-25 02:10:44 -06:00
meejah	f68a0ab74c	remove debug	2018-08-25 02:10:44 -06:00
meejah	79756c088e	split client, introducer READMEs	2018-08-25 02:10:44 -06:00
meejah	990f23d5c7	_Config does this for us	2018-08-25 02:10:44 -06:00
meejah	38063037c1	add documentation	2018-08-25 02:10:44 -06:00
meejah	f37ab3b12d	get_app_version -> global function	2018-08-25 02:10:44 -06:00
meejah	35810a5692	pull 'basedir' entirely into _Config ... Put all config-related methods into _Config; change code to ask config for paths instead of using basedir; add some better docstrings	2018-08-25 02:10:44 -06:00
Jean-Paul Calderone	292448a423	Merge pull request #518 from LeastAuthority/test_upload_and_download_random_key ... Avoid EADDRINUSE from allmydata.test.test_system.SystemTest Fixes ticket:2933	2018-08-22 09:31:48 -04:00
Jean-Paul Calderone	74420fb764	Try to preserve the working directory	2018-08-17 15:53:59 -06:00
meejah	1af71e6ba3	confirm .stop() called	2018-08-17 15:53:59 -06:00
meejah	03712c9cca	add docstrings	2018-08-17 15:53:59 -06:00
meejah	0607b7331f	fix tests by overriding/patching reactor.stop in tests	2018-08-17 15:53:59 -06:00
meejah	423208f391	add a 'tahoe run' unit-test for config errors	2018-08-17 15:53:59 -06:00
meejah	e9879abc93	add unit-tests	2018-08-17 15:53:59 -06:00
meejah	bd63a4354b	fixup for errors	2018-08-17 15:53:59 -06:00
meejah	4aec12a92f	whitespace on long lines	2018-08-17 15:53:59 -06:00
meejah	8e0e96da01	improve user experience on config errors	2018-08-17 15:53:59 -06:00
meejah	3723945296	Actually validate configs when loading them ... An indenting problem meant the validators weren't being called, which revealed some follow-on errors.	2018-08-17 15:53:59 -06:00
Jean-Paul Calderone	f9d527f418	Avoid the chance of providing a real endpoint parser ... This endpoint is test-grade.	2018-08-08 12:01:20 -04:00
Jean-Paul Calderone	0ca6b8ed52	An adopted-port-endpoint-based fix for the collision	2018-08-07 15:26:35 -04:00
Jean-Paul Calderone	9ec8ec814a	Some comment improvements	2018-08-07 15:26:19 -04:00
Jean-Paul Calderone	b1520c8d71	Some improved logging while waiting for connections	2018-08-07 15:22:19 -04:00
Jean-Paul Calderone	d8dfcc9de5	There are no such subclasses.	2018-08-07 14:05:16 -04:00
Jean-Paul Calderone	f576575f11	system test setup broken into manageable pieces ... `set_up_nodes` and `_set_up_nodes_2` now a little easier to understand	2018-08-07 14:04:20 -04:00
Jean-Paul Calderone	284ea9c3b9	not gonna import from Crypto at all	2018-07-19 14:48:12 -04:00
Jean-Paul Calderone	33485c184a	No longer needed for SFTP. ... Twisted's SFTP is now based on ``cryptography``.	2018-07-19 14:44:18 -04:00
Jean-Paul Calderone	814203961d	Merge pull request #504 from exarkun/2929.circleci ... Configure CircleCI for much of the Linux testing	2018-07-03 12:32:51 -04:00
Jean-Paul Calderone	53adf5083d	Shorten the long path used by the long path test	2018-06-15 08:44:52 -04:00
Jean-Paul Calderone	b31b78e6cf	Merge pull request #501 from exarkun/1595.address-already-in-use-test_introducer-create_tub ... Fix "Address already in use" errors from test_introducer on POSIX	2018-06-08 16:21:34 -04:00
Jean-Paul Calderone	1911b35499	Get the fcntl import up to the top as well	2018-06-08 13:15:19 -04:00
Jean-Paul Calderone	97e0ad627b	explain the buried reactor import, just in case	2018-06-08 13:13:26 -04:00
Jean-Paul Calderone	2c38b148bf	Move the safe interface import to the top	2018-06-08 13:12:46 -04:00
Jean-Paul Calderone	d25693145c	Factor a little more duplication out of the tests	2018-06-08 11:21:25 -04:00
Jean-Paul Calderone	c491b1a7d4	bring some imports up to the top	2018-06-08 10:36:45 -04:00
Jean-Paul Calderone	28a2e6e557	Merge pull request #502 from exarkun/2926.cli-status-exceptions ... Fix several cases where `tahoe status` rendering raises an unhandled exception. Closes ticket:2926 Some possible operation states were not accounted for.	2018-05-28 08:38:42 -04:00
Jean-Paul Calderone	bfedd79633	Merge pull request #487 from tahoe-lafs/1455.x-frame-options.2 ... Set `X-Frame-Options: DENY` for all web status pages. This prevents attackers from loading web status pages in a frame as a way to trick users into interactions which attackers are restricted from performing unaided.	2018-05-28 08:12:35 -04:00
Jean-Paul Calderone	20084506b3	Handle all the status cases	2018-05-23 13:59:42 -04:00
Jean-Paul Calderone	318eea05e3	docs	2018-05-23 10:59:42 -04:00
Jean-Paul Calderone	8a5e2edb91	Also CLOEXEC the descriptor ... This avoids leaking it into any child processes that the tests might launch.	2018-05-23 10:56:26 -04:00
Jean-Paul Calderone	be6e458770	Ensure the fd will be valid by the time we listen	2018-05-23 10:50:54 -04:00
Jean-Paul Calderone	62836b6858	Switch another test to the adoption method	2018-05-23 10:47:41 -04:00
Jean-Paul Calderone	711d63960d	Switch another test to the adoption method	2018-05-23 10:47:30 -04:00
Jean-Paul Calderone	277cd16594	Avoid race-prone allocate_tcp_port for some Tubs when possible ... create_tub on POSIX can pre-allocate a port safely instead.	2018-05-23 10:45:15 -04:00
meejah	c219102afc	Merge pull request #492 from exarkun/2879.magic-folder-create-and-umask.0 ... Create the magic-folder local directory if necessary	2018-05-07 09:49:39 -06:00
meejah	cec3746682	explicit umask and skip part of test on windows	2018-05-01 16:45:50 -06:00
meejah	c9e00a988a	Merge pull request #475 from meejah/2909.backup-behavior.0 ... #2909 fix .backup file behavior and (some of) the incorrect .conflict cases (#2911)	2018-05-01 15:52:10 -06:00
Jean-Paul Calderone	e0c73e0ea1	Merge remote-tracking branch 'origin/master' into 1455.x-frame-options.2	2018-05-01 14:39:48 -04:00
Jean-Paul Calderone	4cc1c2171d	Merge remote-tracking branch 'origin/master' into 2879.magic-folder-create-and-umask.0	2018-05-01 14:24:54 -04:00
meejah	458889682c	Merge pull request #493 from exarkun/lgtm-configuration ... Add recommended lgtm configuration.	2018-05-01 11:09:29 -06:00
meejah	5bbf0abe9d	Merge pull request #490 from exarkun/pending-deprecations ... Avoid pending Twisted deprecations	2018-05-01 11:07:09 -06:00
meejah	5414930184	not sure why we assert this	2018-05-01 10:52:12 -06:00
meejah	b54d5d88f8	unused var	2018-04-30 14:55:52 -06:00
meejah	158498f61a	'positive' version of assert	2018-04-30 14:55:52 -06:00
meejah	6352db9c01	comment + logging cleanup	2018-04-30 14:55:51 -06:00
meejah	c71116d45e	keep original exception behavior	2018-04-30 14:55:51 -06:00
meejah	b53ea0a11c	redundant	2018-04-30 14:55:51 -06:00
meejah	04734dcfcb	redundant test code	2018-04-30 14:55:51 -06:00
meejah	c2aec93209	fix windows move_into_place logic	2018-04-30 14:55:51 -06:00
meejah	fbc142d346	re-jig 'the logic'	2018-04-30 14:55:51 -06:00
meejah	6d5355fda3	moar unit-tests	2018-04-30 14:55:51 -06:00
meejah	1f529567ca	fix commented code	2018-04-30 14:55:51 -06:00
meejah	7c71194eea	utest fixup	2018-04-30 14:55:51 -06:00
meejah	5f417c5179	objects_conflicted fixup	2018-04-30 14:55:51 -06:00
meejah	99c3d7f23f	this shouldn't have gotten removed	2018-04-30 14:55:51 -06:00
meejah	927ee097e5	remove prints	2018-04-30 14:55:51 -06:00
meejah	65ceda3a32	fix conflict behavior	2018-04-30 14:55:51 -06:00
meejah	7975efbe8d	whitespace and a comment	2018-04-30 14:55:51 -06:00
meejah	cc93a12a85	Test fixups	2018-04-30 14:55:51 -06:00
meejah	435b30c7aa	use local last_uploaded_uri, last class of conflict from spec	2018-04-30 14:55:51 -06:00
meejah	affb80e39e	test fixups	2018-04-30 14:55:51 -06:00
meejah	47b1787633	stop writing .backup files for normal overwrites	2018-04-30 14:55:51 -06:00
meejah	a1db8f4708	some logging fixups and notes on backup behavior	2018-04-30 14:55:51 -06:00
Jean-Paul Calderone	b623a4a199	Remove dead Tor TCP control port setup code. ... If someone wants this I bet they can figure it out.	2018-04-26 15:32:27 -04:00
Jean-Paul Calderone	87daa3ec5a	Remove dead debug logging code.	2018-04-26 15:32:02 -04:00
Jean-Paul Calderone	3705264740	Use preferred exception raising syntax. ... Also, make the `WindowsError` class "reachable".	2018-04-26 15:27:00 -04:00
Jean-Paul Calderone	b6d33c92ff	Remove disabled ad hoc debug logging	2018-04-26 15:26:17 -04:00
Jean-Paul Calderone	8d4d000132	Fix pre-release matching regex character class ... Previously matched any single character from `abc|r` (with duplicate specification of `c`). Now matches any single character from `abc` or the two character sequence `rc`. I guess this was the intent, anyway.	2018-04-26 15:20:27 -04:00
Jean-Paul Calderone	6b16afaa2e	Avoid using the list comprehension loop variable ... It works fine but it relies on leaky scopes.	2018-04-26 15:16:00 -04:00
Jean-Paul Calderone	9f8c90393f	Remove dead synopsis definition	2018-04-26 15:10:02 -04:00
Jean-Paul Calderone	7609fd1861	Remove impossible third codepath	2018-04-26 15:09:01 -04:00
Jean-Paul Calderone	64243527eb	Remove the strange option to not use flog	2018-04-26 15:08:14 -04:00
Jean-Paul Calderone	6d9f0c59b7	Remove pointless conditional	2018-04-26 14:59:47 -04:00
Jean-Paul Calderone	da9d0ded94	Remove pointless conditional	2018-04-26 14:59:18 -04:00
Jean-Paul Calderone	a7218cb16f	Some nice documnetation and proper exception types	2018-04-23 18:46:12 -04:00
Jean-Paul Calderone	169ccbfbca	directory creation fixes	2018-04-23 18:46:07 -04:00
Jean-Paul Calderone	035dc6dc76	reduce fragility of tests .. maybe? ... only trivially, at best, of course.	2018-04-23 11:41:36 -04:00
Jean-Paul Calderone	e1c469e3b6	make sure we pass the client node	2018-04-23 11:41:29 -04:00
Jean-Paul Calderone	cfa33332a5	Add missing information/import	2018-04-23 11:09:24 -04:00
Jean-Paul Calderone	8d104dab1c	Move the complicated MagicFolder constructor ... All that complexity can be part of MagicFolder itself.	2018-04-23 10:59:33 -04:00
Jean-Paul Calderone	ac6269dd2d	Only read magic-folder config from config reader ... Also, fix the umask feature which was completely broken previously due to failure to parse the umask string into an integer.	2018-04-23 10:41:48 -04:00
Jean-Paul Calderone	0bdabacce3	document the node_directory parameter	2018-04-23 10:41:48 -04:00
Jean-Paul Calderone	8eb83bbfb9	avoid about-to-be-deprecated getClientIP if we can ... Use the replacement, getClientAddress. But have a fallback to getClientIP to keep supporting older versions of Twisted.	2018-04-20 16:03:19 -04:00
Jean-Paul Calderone	edf3c7aac7	reformat to fit within 80 cols	2018-04-20 15:59:35 -04:00
Jean-Paul Calderone	06a1ada624	Remove double-encoding of magic-folder params	2018-04-20 14:43:34 -04:00
Jean-Paul Calderone	234db487cd	Add a unit test for X-Frame-Options being set	2018-04-13 12:56:50 -04:00
Jean-Paul Calderone	7bc207cdcb	Shift this out of the way	2018-04-13 12:52:12 -04:00
Jean-Paul Calderone	36c7756890	Reference X-Frame-Options documentation	2018-04-13 12:48:03 -04:00
Daira Hopwood	718fa4493c	Add "X-Frame-Options: DENY" header to all pages. refs #1455 ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2018-04-13 12:43:18 -04:00
Jean-Paul Calderone	dc4d30f7c2	Switch to TAHOE_LAFS prefix	2018-04-03 14:07:12 -04:00
Jean-Paul Calderone	6bca749592	this is still needed	2018-04-03 13:59:35 -04:00
Jean-Paul Calderone	f6f617c33c	Teach it about a CI profile ... This profile does not have time-based warnings.	2018-04-03 13:15:54 -04:00
Jean-Paul Calderone	da2e4a80cd	Get rid of these individual suppressions	2018-04-03 13:15:54 -04:00
Chris Wood	c850638537	Fix PyInstaller builds ... This commit contains a few small changes to fix PyInstaller frozen builds (which were recently broken in a few ways by changes introduced with `tahoe invite`, `tahoe daemonize`, and the addition of "setuptools >= 28.8.0" to setup_requires) and removes a couple of hacks that are no longer necessary to create working frozen tahoe executables with PyInstaller.	2018-03-29 14:11:15 -04:00
Brian Warner	ce473bd5f4	cli.test_alias: move skip-unless utility to common_util.py ... next to the other skip-unless function	2018-03-28 15:30:24 -07:00
Brian Warner	0616aa7de7	test_absolute_storage_dir: don't use uSNOWMAN on non-unicode platforms ... Exercising the unicode possibilities are nice, but not critical to this test, so let's just avoid the non-ascii characters when the filesystem encoding can't handle them	2018-03-28 15:17:23 -07:00
Brian Warner	0964bc0d05	test_alias: skip unicode tests on non-unicode platform ... This was flunking the OS-X buildbot, which runs in an environment without $LANG being set, and thus encodingutil deduces (correctly but unhelpfully) that we're limited to ASCII. Other tests detect this situation and raise SkipTest, so let's do that here too.	2018-03-27 15:56:10 -07:00
Jean-Paul Calderone	6ec5005ccd	Merge pull request #474 from exarkun/1587.basic-progress-report ... Basic progress reporting for `tahoe backup` Fixes ticket:1587	2018-03-27 07:47:01 -04:00
Jean-Paul Calderone	fa567958c3	2 blank between top-level; 1 blank between methods ... Just like PEP8 says	2018-03-26 20:15:45 -04:00
Jean-Paul Calderone	bafe043b73	Explicit new-style class	2018-03-26 20:12:47 -04:00
Jean-Paul Calderone	b78c6cc5ed	Implement the progress reporting	2018-03-26 11:34:31 -04:00
Jean-Paul Calderone	6690aa7337	restore, with tests, checked counters	2018-03-26 10:27:19 -04:00
Jean-Paul Calderone	c55d2823ae	first pass refactoring ... now collect backup work up-front instead of mixed with processing	2018-03-26 10:02:42 -04:00
Brian Warner	38da8f471c	test_web: appease pyflakes	2018-03-21 00:22:31 -07:00
Brian Warner	e6ddd03338	test_web: remove noisy print statement	2018-03-21 00:13:52 -07:00
meejah	544f87a318	need setuptools for PEP440 identifiers ... (needs fixup, probably, just depending on latest setuptools)	2018-03-20 17:45:08 -07:00
Jean-Paul Calderone	3bb4c979c8	Revert "Merge PR470" ... This reverts commit 7960a1b2a4, reversing changes made to 3fa74d860f.	2018-03-20 18:10:44 -04:00
meejah	4eac3caa77	need setuptools for PEP440 identifiers ... (needs fixup, probably, just depending on latest setuptools)	2018-03-20 14:15:33 -07:00
meejah	643e2f9107	fix windows, maybe	2018-03-20 11:40:50 -06:00
meejah	fcfcbaa6a2	fix behavior of fake fileops test-helper	2018-03-20 11:40:49 -06:00
meejah	2fd2a9b2c7	nicer assert	2018-03-20 11:40:49 -06:00
meejah	9df84e71fe	use better error-testing idiom and don't tweak internal method	2018-03-20 11:40:49 -06:00
meejah	57cc877c57	add some context to logging	2018-03-20 11:40:49 -06:00
meejah	ee0f548aa7	describe test	2018-03-20 11:40:49 -06:00
meejah	a06fe375ec	describe test	2018-03-20 11:40:49 -06:00
meejah	c7cbb4a6b3	better var names	2018-03-20 11:40:49 -06:00
meejah	30811d8818	typo	2018-03-20 11:40:49 -06:00
meejah	9032539860	flake8; naming	2018-03-20 11:40:49 -06:00
meejah	797932244d	log failure properly, add test	2018-03-20 11:40:49 -06:00
meejah	39080852cc	ensure 'in progress' items return status	2018-03-20 11:40:49 -06:00
meejah	1f495ec41f	cover uploads and downloads in tests	2018-03-20 11:40:49 -06:00
meejah	dbc9f7e06c	remove code that sets item size (from review)	2018-03-20 11:40:49 -06:00
meejah	75e97e6c6d	use QueuedItem directly instead of FakeStatusItem	2018-03-20 11:40:49 -06:00
meejah	1b16e1f792	Some documentation for QueueMixin	2018-03-20 11:40:49 -06:00
meejah	472d6ba78b	cover some of get_status	2018-03-20 11:40:49 -06:00
meejah	bfabfeba64	add 'size' to tests, print unicode	2018-03-20 11:40:49 -06:00
meejah	0ebb587666	report download sizes in magic-folder status	2018-03-20 11:40:49 -06:00
meejah	0431be14fa	log inotify callback errors	2018-03-20 11:40:49 -06:00
meejah	42766da553	add a 'size' to pending uploads	2018-03-20 11:40:49 -06:00
meejah	8ac63cf5bd	only keep history for items that are processed	2018-03-20 11:40:49 -06:00
meejah	bcdd15394b	ensure there are no gaps in magic-folder status	2018-03-20 11:40:49 -06:00
Jean-Paul Calderone	371965f7d4	Specify the new optFlag correctly	2018-03-19 15:12:59 -04:00
Jean-Paul Calderone	1f92879788	improve test coverage	2018-03-19 14:21:28 -04:00
Ruben Pollan	136de7d7f7	Add storage_dir config field ... On the [storage] section of the tahoe.cfg now there is a field 'storage_dir' where the path to the storage folder can be configured.	2018-03-19 11:20:04 -04:00
meejah	a1cb401f06	Merge pull request #452 from meejah/list-aliases-json.1 ... Add --json option to 'tahoe list aliases'	2018-03-13 17:05:20 -06:00
tpltnt	a41d827f79	_None() as new style class	2018-03-02 17:17:30 +01:00
tpltnt	1967cd7357	removed unnecessary after return	2018-03-02 16:45:11 +01:00
tpltnt	1a14d5218b	added some docstrings	2018-03-02 16:45:11 +01:00
tpltnt	3f99eceb36	minor whitespace fixes	2018-03-02 16:45:11 +01:00
tpltnt	0461cc0148	only one module import per line	2018-03-02 16:45:11 +01:00
tpltnt	f7bbc9dffe	made mkstemp() call compatible with mktemp()	2018-03-02 16:45:11 +01:00
tpltnt	710ba84a02	switched to secure mkstemp()	2018-02-27 18:35:08 +01:00
meejah	0b25cad71e	documentation fixups from daira's review	2018-02-06 15:28:02 -07:00
meejah	1171bf13af	ticket #2882: preserve user mtime	2018-02-06 15:27:56 -07:00
meejah	68783d8b8a	directories get +x, files do not	2018-01-09 01:26:37 -07:00
meejah	3466088717	add --json option for 'tahoe list-aliases' ... some new tests to cover previously-uncovered code that changed	2018-01-05 18:07:52 -07:00
meejah	187eb562f2	Merge pull request #447 from tpltnt/ticket-1307 ... Ticket 1307	2018-01-04 17:50:21 -07:00
tpltnt	95acd57fdd	removed exact version info from header	2018-01-04 22:29:57 +01:00
tpltnt	48b442f8c1	mentioned ctx	2018-01-04 22:29:57 +01:00
tpltnt	6d4f4f4a65	adjusted doc string	2018-01-04 22:29:57 +01:00
tpltnt	68b1271190	added docstring	2018-01-04 22:29:57 +01:00
tpltnt	005bcf5012	added code as described in ticket 1307	2018-01-04 22:29:57 +01:00
tpltnt	c5e8d83b73	removed old test/trial helper module	2018-01-04 22:09:52 +01:00
tpltnt	c7cf7c3956	make twisted >= 16.4.0 a requirement	2018-01-04 22:09:52 +01:00
meejah	c351a1d327	Merge pull request #453 from meejah/magic-folder-test-time ... use '5 years ago' instead of absolute timestamp	2018-01-03 15:41:32 -07:00
meejah	5f00842532	use exarkun's suggestion from review	2018-01-03 14:07:38 -07:00
Brian Warner	5e62118e14	hush somewhat-real problems found by static analysis ... These would have caused problems if a certain unlikely code path was taken: mostly error-handling pathways.	2017-12-31 00:23:43 +01:00
Brian Warner	da4e7dcfbe	hush a bunch of not-really-problems caught by the lgtm.com static analyzer ... This is all minor stuff: unreachable debug code (that should be commented-out instead of in an 'if False:' block), unnecessary 'pass' and 'global' statements, redundantly-initialized variables. No behavior changes. Nothing here was actually broken, it just looked suspicious to the static analysis at https://lgtm.com/projects/g/tahoe-lafs/tahoe-lafs/alerts/?mode=list .	2017-12-31 00:19:41 +01:00
meejah	e5d6272b62	use '5 years ago' instead of absolute timestamp	2017-12-20 13:17:54 -07:00
meejah	6c0c90829f	cheezy test for more coverage	2017-12-05 12:02:16 -07:00
meejah	87ad3cb8e9	fix 2880 ... - remember upload URI so we don't re-download - account for empty files in "should_download" - fix 'conflicted' asserts in tests	2017-12-05 12:02:16 -07:00
meejah	672475cb2b	Multiple magic-folders ... This moves all magic-folder configs to a single YAML file. We load legacy config fine and don't mess with legacy config unless you use a magic-folder command that changes the config. Increase test coverage	2017-12-05 10:34:48 -07:00
meejah	1b6f477549	Pull "config" handling out of Node and hide node-derivitives ... This includes: - refactor to create_client and _Client - refactor to _IntroducerNode and create_introducer	2017-11-20 12:57:20 -07:00
meejah	50f8397c99	Merge branch 'invalid-pidfile' into delete-invalid-pidfile--lpirl ... Conflicts: src/allmydata/scripts/startstop_node.py src/allmydata/test/cli/test_cli.py	2017-11-08 14:49:29 -07:00
Brian Warner	a1711088ed	test_i2p_provider: exercise escaping of apiEndpoint attribute	2017-11-03 00:24:46 -07:00
Brian Warner	1f1afe65ec	tor_provider.get_listener(): return a real endpoint, instead of a descriptor ... get_listener() is allowed to return either, and the Tor provider is currently simple enough to not really need more than a basic descriptor, but have it return a full Endpoint for use as an example of what I2P can do later.	2017-11-03 00:24:46 -07:00
Brian Warner	d713e0e57d	address review feedback: docstrings, better test class name	2017-11-03 00:24:46 -07:00
Brian Warner	097abb42fa	tahoe.cfg: add tub.port=listen:i2p (and/or listen:tor) ... This delegates the construction of the server Endpoint object to the i2p/tor Provider, which can use the i2p/tor section of the config file to add options which would be awkward to express as text in an endpoint descriptor string. refs ticket:2889 (but note this merely makes room for a function to be written that can process I2CP options, it does not actually handle such options, so it does not close this ticket yet)	2017-11-03 00:24:46 -07:00
Brian Warner	d1fd43aa4f	tor/i2p: rename create_onion/create_dest to create_config	2017-11-03 00:24:46 -07:00
meejah	fd90346c88	Wait up to 60s for node start; improve messaging; look for errors	2017-09-20 00:55:19 -06:00
meejah	263a3f43c1	stop chdir	2017-09-19 10:39:19 -06:00
meejah	9375056b61	Split up startstop_node and add 'tahoe daemonize' ... This sets the stage for further changes to the startup process so that "async things" are done before we create the Client instance while still reporting early failures to the shell where "tahoe start" is running Also adds a bunch of test-coverage for the things that got moved around, even though they didn't have coverage before	2017-09-19 10:39:19 -06:00
Lukas Pirl	89bacbb70a	Merge branch 'master' into invalid-pidfile	2017-08-24 18:40:00 +02:00
Lukas Pirl	46305c74e1	added test for (ignoring an) invalid PID file when starting a node	2017-08-24 17:56:57 +02:00
Jean-Paul Calderone	86f79e8111	Add a test for the non-numeric case	2017-08-24 16:46:46 +02:00
Jean-Paul Calderone	354567cb85	Add a test for the non-numeric case	2017-08-24 10:34:58 -04:00
Lukas Pirl	1bf032959f	delete invalid PID file on `tahoe (re)start`	2017-08-24 15:22:25 +02:00
Lukas Pirl	dd9b951c4b	fix crash when stopping/restarting with an invalid pidfile ... in node directory	2017-08-23 20:23:15 +02:00
Brian Warner	8dd0a5f772	dictutil: remove unused move, subtract, del_if_present	2017-08-12 21:48:05 -07:00
Brian Warner	61b85dbf78	dictutil: remove unused ValueOrderedDict	2017-08-12 21:34:24 -07:00
Brian Warner	4f493b1a03	remove unused UtilDict	2017-08-12 21:31:47 -07:00
Brian Warner	a76d9e084f	dictutil: remove unused NumDict	2017-08-12 21:21:04 -07:00
Brian Warner	3afa38adb4	NummedObj: stop using dictutil.NumDict	2017-08-12 21:02:32 -07:00
Brian Warner	3f2f7dfb05	dictutil: fix bug in str(ValueOrderedDict), and improve test coverage ... It looks like str() was meant to truncate the dict, but a missing i+=1 meant that it never actually did. I also changed the format to include a clear "..." in case we truncate it, to avoid confusion with a non-truncated dict of the same size. This also improves test coverage in subtract() and NumDict.item_with_largest_value(). refs ticket:2891	2017-08-10 10:27:02 -07:00
Brian Warner	95ac5494ff	test_backupdb: tolerate newer sqlite-3.20.0 error messages	2017-08-10 10:21:39 -07:00
meejah	798bf57e28	Add 'tahoe invite' and 'tahoe create-node --join' commands ... This opens a wormhole and sends appropriate JSON down it to a tahoe-gui using a wormhole server running on tahoe-lafs.org The other end uses the 'tahoe create-node' command (with new --join option) to read the configuration JSON from a 'tahoe invite' command	2017-08-08 18:27:06 -06:00
meejah	e2a5751d02	at least use random port	2017-08-08 10:42:11 -06:00
Brian Warner	a4be2dce71	avoid variable coverage by using a defaultdict ... refs ticket:2891	2017-07-27 18:19:57 -07:00
Brian Warner	05317d4d12	test_web: replace all shouldHTTPError with assertHTTPError ... and update to inlineCallbacks while we're in there	2017-07-27 16:31:44 -07:00
Brian Warner	2687ee90c5	test_web: remove last traces of deprecated client.getPage	2017-07-27 16:31:44 -07:00
Brian Warner	965e974ce8	test_web: fix POST(url,body) calls that should have been POST2() ... and remove the followRedirect= argument from POST	2017-07-27 16:31:44 -07:00
Brian Warner	0deb903888	test_web: rewrite all POST(followRedirect=True) calls with do_http ... Since POST() is about to lose followRedirect=True	2017-07-27 16:31:44 -07:00
Brian Warner	3f03367d2f	test_web: remove all other uses of shouldRedirect	2017-07-27 16:31:44 -07:00
Brian Warner	76063b1c12	test_web: replace some instances of shouldRedirect with shouldRedirectTo	2017-07-27 16:31:44 -07:00
Brian Warner	6be91e369c	test_web: replace shouldReplace2 with shouldReplaceTo ... This removes many uses of self.POST (since shouldReplaceTo does its own HTTP), which will make it easier to remove client.getPage from POST.	2017-07-27 16:31:44 -07:00
Brian Warner	4745239c2c	test_web: use inlineCallbacks in many functions ... specifically everywhere we use self.shouldRedirect2	2017-07-27 16:31:44 -07:00
Brian Warner	73d09082d7	test_web.web.Web.POST: split out build_form() helper function	2017-07-27 16:31:44 -07:00
Brian Warner	202a9714c4	test_web test_bad_method: remove a client.getPage ... Add WebErrorMixin.assertHTTPError, to replace (getPage + shouldHTTPError)	2017-07-27 16:31:44 -07:00
Brian Warner	bee05e883c	test_web: remove HTTPClientGETFactory ... This also changes the tests to handle the new API: URL can be unicode, and the returned Headers object is not a dictionary.	2017-07-27 16:31:44 -07:00
Brian Warner	5a895b5fb6	GridTestMixin.GET: use treq instead of HTTPClientGETFactory	2017-07-27 16:31:44 -07:00
Brian Warner	3311f9520d	test.cli.test_status: replace getPage with treq	2017-07-27 16:31:44 -07:00
Brian Warner	bd1e4507ee	remove unused HTTPClientHEADFactory	2017-07-27 16:31:44 -07:00
Jean-Paul Calderone	e32f581d07	Get rid of the reliance on rref ... It provides information more correctly obtained from get_connection_status().	2017-07-27 16:31:41 -04:00
Jean-Paul Calderone	1f16a173f0	flatten the result structure ... flat is better than nested	2017-07-27 15:32:47 -04:00
Jean-Paul Calderone	7027547ecc	fix buggy test; restructure data ... scared about duplicate nodeids in the data getting lost, switch to a list	2017-07-27 15:26:30 -04:00
Jean-Paul Calderone	754304e4d5	Switch Root over to MultiFormatPage	2017-07-27 15:26:30 -04:00
Jean-Paul Calderone	f518c2a521	Switch over to stdlib json	2017-07-27 15:26:30 -04:00
Jean-Paul Calderone	e339130e63	Just make an assertion against the full value	2017-07-27 15:26:30 -04:00
Jean-Paul Calderone	e3e49632a5	use preferred spelling for new assertions	2017-07-27 15:26:30 -04:00
David Stainton	cd97f8519b	web test: get_version should use a version string	2017-07-27 15:26:29 -04:00
David Stainton	923363c0d6	web test: test for server key in json welcome	2017-07-27 15:26:29 -04:00
David Stainton	16e56611f3	web: set json welcome content type to json utf8	2017-07-27 15:26:29 -04:00
David Stainton	ecb6651926	Add basic unit test for json welcome page	2017-07-27 15:26:29 -04:00
David Stainton	876a7b6602	Fix indention	2017-07-27 15:26:29 -04:00
David Stainton	ad66caeaff	Make sure server.rref is not None	2017-07-27 15:26:29 -04:00
David Stainton	180975b0af	Make sure version is not None	2017-07-27 15:26:29 -04:00
David Stainton	b2d4374976	Add more server info to welcome json page	2017-07-27 15:26:29 -04:00
David Stainton	32110bfc68	Add storage and introducer summaries to json welcome output	2017-07-27 15:26:29 -04:00
David Stainton	822ed78b13	Add rough sketch of json welcome page	2017-07-27 15:26:29 -04:00
Jean-Paul Calderone	1e885fa40b	Convert storage.py over to MultiFormatPage	2017-07-27 13:57:02 -04:00
Jean-Paul Calderone	6e4c4b7abc	Convert status.py over to MultiFormatPage	2017-07-27 13:57:02 -04:00
Jean-Paul Calderone	5a30bd6169	Convert introweb.py over to MultiFormatPage	2017-07-27 13:57:02 -04:00
Jean-Paul Calderone	0581ce7270	Convert directory.py over to MultiFormatPage	2017-07-27 13:57:02 -04:00
Jean-Paul Calderone	7c8c63a01f	a helper for the ?t=json resources	2017-07-27 13:57:02 -04:00
Brian Warner	ad92b2b9b2	test_web: replace HEAD/PUT/DELETE with treq	2017-07-12 17:10:49 -07:00
Brian Warner	486fca0d5c	web.test_introducer: replace getPage with treq	2017-07-12 17:10:49 -07:00
Brian Warner	8be5072fca	test_system: replace getPage with treq	2017-07-12 17:10:49 -07:00
Brian Warner	2ec3791411	test_deepcheck: replace t.w.client.getPage with treq	2017-07-12 17:10:46 -07:00
Brian Warner	74e7ef4b98	test_happiness: limit string sizes to 30 chars ... I think these are supposed to represent share ids or server ids, so we don't need to exercise them being too huge.	2017-07-12 16:37:44 -07:00
Brian Warner	11c39325fb	test_happiness: suppress the hypothesis 'too_slow' check ... I was seeing buildbot failures where the host machine was too busy, and hypothesis was spooked into thinking that data generation was too slow.	2017-07-12 16:27:13 -07:00
meejah	0977e52cbf	Add a test for 'tahoe list-aliases --readonly-uri'	2017-06-06 18:03:05 +01:00
Daira Hopwood	d3acf82776	Rename FilesystemOptions to FileStoreOptions. refs #2345 ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2017-06-06 11:20:49 +01:00
Daira Hopwood	2055a66cf2	Doc changes that require more careful review. refs #2345 ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2017-06-06 11:20:49 +01:00
Daira Hopwood	4723af6e2f	LAFS now stands for "Least-Authority File Store" rather than "Least-Authority File System". refs #2345 ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2017-06-06 11:20:49 +01:00
meejah	05f48c3601	Various cleanups, fixes and improvements ... Squashed all commits that were meejah's between 30d68fb499f300a393fa0ced5980229f4bb6efda and 33c268ed3a8c63a809f4403e307ecc13d848b1ab On the branch meejah:1382.markberger-rewrite-rebase.6 as per review	2017-06-05 16:31:41 -06:00
David Stainton	19c5bbb43b	Fix test test_lost_servers ... Remove old hypothesis tests Fix allmydata.test.cli.test_cli.Errors.test_get this was broken due to differing share placements whereas we need to allow this. Fix test_5_overdue_immutable This change makes the test not depend on the value of PYTHONHASHSEED. Revert "Fix test_5_overdue_immutable" This reverts commit 5f3696d9a53e7df8781a2c463c7112282397cd69. fix test to actually hang the first 5 *servers* sort keys for stable output use file-context-managers remove probably-unneeded assert (that fails sometimes) another non-deterministic test?	2017-06-05 16:26:46 -06:00
meejah	56f6dbd363	distribute only to read/write peers ... correctly calculate happiness guard with except fix tests, and happiness calculation remove debug fix placements to None happiness calc shouldn't have to filter None WIP fixing some tests etc	2017-06-05 16:26:46 -06:00
David Stainton	a611673934	Make a correction to a hypothesis test comment ... Comment out all debug print statements Add hypothesis tests for the old servers of happiness implementation Attempt to speed up meejah's servers of happiness WIP Fix test_calc_happy WIP	2017-06-05 16:26:46 -06:00
meejah	b6d9945b95	default answer for every share ... refactor hypothesis to be 'pytest style' and add another one get rid of 'shares->set(1 thing)' in generate_mappings return Add a unittest hypothesis came up with fix tests since we return peers, not sets-of-1-peer add more debug add a unit-test that's like test_problem_layout_ticket_1128 fix bug add a note fix utest unit-test for bigger numbers re-insert markberger code for testing results of pairing with david	2017-06-05 16:26:46 -06:00
David Stainton	e68b331bb1	Add servers of happiness hypothesis testing ... Fix happiness test var names Remove unused imports Get rid of trailing whitespace	2017-06-05 16:26:46 -06:00
meejah	ef17ef2c62	fix happiness calculation ... unit-test for happiness calculation unused function put old servers_of_happiness() calculation back for now test for calculate_happiness remove some redundant functions	2017-06-05 16:26:46 -06:00
David Stainton	42011e775d	Make correction to docstring for Tahoe2ServerSelector's _handle_existing_response ... Add comments 10 and 8 from the servers of happiness spec Fix bug in _filter_g3 for servers of happiness Remove usage of HappinessUpload class here we modifying the PeerSelector class. we make sure to correctly calculate the happiness value by ignoring keys who's value are None... Remove HappinessUpload and tests Replace helper servers_of_happiness we replace it's previous implementation with a new wrapper function that uses share_placement	2017-06-05 16:26:46 -06:00
meejah	adb9a98383	WIP: refactoring (squahshed all meejah's commits)	2017-06-05 16:26:46 -06:00
Mark Berger	17cff7a176	Implements 'Servers of Happiness' algorithm for upload ... This is Mark Berger's original commits, from ticket #1382	2017-06-05 16:26:46 -06:00
Brian Warner	bf0149f7d7	Merge PR412	2017-04-10 11:59:12 -07:00
Brian Warner	f24681bfdc	simplify rendering of path to tahoe.cfg	2017-04-10 11:56:33 -07:00
David Stainton	366fe039e8	magic-folder: change stopService to call MultiService.stopService	2017-04-10 11:36:58 -07:00
tpltnt	63bcc07e27	explicit path to tahoe.cfg in creation message	2017-04-07 16:59:17 +02:00
meejah	4f1dd95963	Add a 'tahoe status' command ... This required also fixing the JSON output from the "recent and active operations" page.	2017-04-04 12:48:41 -06:00
Brian Warner	2ceea6df0d	Merge PR406: use @implementer instead of implements ... This also updates several classes to be new-style (inheriting from 'object').	2017-03-23 19:25:56 -07:00
Brian Warner	0abcbe3339	update some comments where plain cut+paste was confusing	2017-03-23 19:19:00 -07:00
meejah	49ae2c5eb7	use @implementer instead of implements ... Also, derive some more things from 'object'	2017-02-27 11:01:30 -07:00
meejah	031a88c25f	simplejson -> json	2017-02-23 15:21:36 -07:00
Brian Warner	bb782b0331	Merge PR400: add "tahoe list-aliases --readonly-uri" ... Closes tahoe-lafs/tahoe-lafs:400	2017-02-21 10:09:31 -08:00
meejah	093945db22	set Referrer-Policy in one place	2017-02-13 16:07:41 -07:00
meejah	a9521b89a6	--readonly-uri for 'tahoe list-aliases'	2017-01-19 15:40:09 -07:00
Brian Warner	be6e0a9c29	deps: we need foolscap-0.12.6 for the new I2P API	2017-01-17 20:13:05 -05:00
Brian Warner	6db7686a8f	update tests to match	2017-01-17 20:12:55 -05:00
str4d	998af5c26e	Pass I2P keyfile to foolscap ... If no session management is performed, txi2p starts a process-wide session the first time a connection (client or server) is opened; all subsequent connections use that session and its configuration properties. This commit ensures that the same properties are passed to both client and server endpoints, so that the correct I2P Destination is started regardless of whether the first connection made by Tahoe-LAFS is for a client or server. Closes #2858.	2017-01-17 20:12:32 -05:00
Brian Warner	ee58637813	fix syntax (tcp=disabled, not =none), add test	2017-01-10 16:30:04 -08:00
David Stainton	3c22a3ce50	Set tcp = tor only if txtorcon is imported ... for i2p this means tcp = none as long as txtorcon is not imported	2017-01-10 16:30:04 -08:00
Brian Warner	64e344b03a	introducer.yaml: ascii-fy furls before use ... This fixes the foolscap-rejects-unicode-furls problem which prevented a correctly-formatted introducers.yaml from working. closes ticket:2862	2017-01-08 22:59:51 -08:00
Brian Warner	582931b1c2	docs: recommend proper introducers.yaml syntax ... add a test to exercise that this syntax is parseable, and another to point out that the previously-recommended syntax was not refs ticket:2862	2017-01-08 22:59:51 -08:00
Brian Warner	79ff7ae5cb	move 'shutilwhich' dependency out to the [test] extra ... This is only used by integration/conftest.py, so we don't need an unconditional dependency on it. closes ticket:2856	2016-12-25 14:15:52 -05:00
Brian Warner	a06cf2ea4c	express pypiwin32 dep via a setup.py extra ... Previously this looked at sys.platform to decide what dependencies to include. The problem with that approach is that wheels built on a unix box won't work on windows (and vice versa), when the (pure-python) Tahoe wheels aren't supposed to be platform-dependent. setup.py provides a syntax to express this properly, so wheels created on either platform will include the pypiwin32 dependency in the metadata, but marked as only being relevant when installing on a win32 platform. closes ticket:2763	2016-12-24 18:00:13 -05:00
Stefan Lew	481a02aee7	made tests work with new attribute rel="noreferrer"	2016-12-23 23:20:52 -05:00
Stefan Lew	12844220ec	use rel="noreferrer" to prevent leakage of dircap	2016-12-23 23:20:52 -05:00
Vladimir Rusinov	0d47e12e40	Add some docstrings.	2016-12-23 22:50:48 -05:00
Vladimir Rusinov	7faec2e10a	Adjust formatting to be pep8-compliant.	2016-12-23 22:50:48 -05:00
Vladimir Rusinov	80fe154634	DeepStats: Initialize all class arguments in __init__ (good practice; makes pylint happy).	2016-12-23 22:50:48 -05:00
Vladimir Rusinov	a95f2743ef	Add test for deep-stats api version.	2016-12-23 22:50:48 -05:00
Vladimir Rusinov	195d94cf2c	Add api-version to DeepStats.	2016-12-23 22:50:48 -05:00
Vladimir Rusinov	8d63208edd	Convert DeepStats to a new-style class.	2016-12-23 22:50:48 -05:00
Vladimir Rusinov	672efa7d17	Add a docsrting to dirnode.py.	2016-12-23 22:50:48 -05:00
Vladimir Rusinov	73fabaec19	Move DeepStats to separate file as dirnode is a bit too large to work with comfortably.	2016-12-23 22:50:48 -05:00
Brian Warner	5651994501	magic_folder_cli: remove unnecessary conditional ... (this increases our branch coverage by one)	2016-12-14 21:20:45 -08:00
Brian Warner	9303d20ed7	test_magic_folder: improve coverage	2016-12-14 21:20:37 -08:00
meejah	e32b664b2b	Adjust default poll_interval ... Also adds a --poll-interval option to both 'magic-folder join' and 'magic-folder create' so that the integration tests can pass something "very short".	2016-12-14 20:29:41 -07:00
meejah	37c7d54bd3	keeps defaults in CreateClientOptions but makes shares-* required?	2016-12-13 19:50:19 -07:00
meejah	5b8be255d7	Set total/needed/happy from command-line	2016-12-13 18:52:45 -07:00
Brian Warner	5cbe580d90	improve ConnectionStatus and welcome-page display ... * replace "last_details" with "non_connected_statuses" dict * rename "last_connection_summary" to just "summary" * for connected servers, show other hints in a tooltip * for not-yet-connected servers, show all hints in a list * build the list (in STAN) on the server side, not using IContainer	2016-12-09 18:41:38 -08:00
Brian Warner	70db0db5bd	interface.py: cosmetic, rewrite docstrings as attributes	2016-12-09 18:41:38 -08:00
meejah	74209e643e	Show the connection-status of all hints until we're connected to one	2016-12-09 18:41:38 -08:00
Brian Warner	8b35fe9efe	tests: exercise connection_status.py	2016-12-09 11:46:26 -08:00
Brian Warner	70a616826c	welcome.xhtml: update copyright year ... Not sure why we have this in there, but it might as well be current.	2016-12-09 11:19:26 -08:00
Brian Warner	38935bbca6	remove old unused connection-status APIs	2016-12-08 16:25:00 -08:00
Brian Warner	77fd41b66e	update WUI welcome page with new connection-status info ... This shows current-connection info, and provides per-hint status details in a tooltip. The "Connection" section no longer shows seconds-since-loss when the server was not connected (previously it showed seconds-since-connect when connected, and flipped to seconds-since-loss when disconnected). We already have the "Last RX" column, which is arguably more meaningful (and I can't think of a good case when these would differ), so we don't really need seconds-since-loss, and the new ConnectionStatus doesn't track it anyways. So now the "Connection" timestamp for non-connected servers is just "N/A" (both the main text and the tooltip). The "Introducers" section was changed the same way. This moves the per-server connection timestamp out of the nickname/serverid box and over into the Connection box. It also right-floats all timestamps, regardless of which box they're in, which makes them share the box with connection_status more politely. Internally, this adds code to create ConnectionStatus objects when necessary.	2016-12-08 16:24:32 -08:00
Brian Warner	48fc14bd30	add IConnectionStatus and implementation ... also a function to build one from a foolscap.Reconnector	2016-12-08 15:41:37 -08:00
Brian Warner	8d008967e7	tor_provider: use new Foolscap API to provide better status	2016-12-08 15:41:37 -08:00
Brian Warner	de4295ae60	require foolscap >= 0.12.5, for ReconnectionInfo	2016-12-08 15:41:37 -08:00
Brian Warner	6879622894	create-node: avoid introducer.furl=None ... Previously, "tahoe create-node" without an --introducer= argument would result in the literal string "None" being written into tahoe.cfg: [client] introducer.furl = None We were using config.get("introducer",""), but that didn't suffice because the key was actually present: it just had a value of None, which then got stringified into "None" when writing out tahoe.cfg. This briefly caused test/cli/test_create to fail, as the startup code tried to parse "None" as a FURL. This only happened against a development version of Foolscap which accidentally became sensitive to unparseable FURLs in started Reconnectors. I fixed that in the final foolscap-0.12.5 release, so we shouldn't hit this bug, but I wanted to fix it properly in the tahoe-side source.	2016-12-08 15:37:49 -08:00
Brian Warner	d5f3d47483	test/cli/test_create: remove errant print()	2016-12-08 14:59:11 -08:00
Brian Warner	5a5da0de2a	magic_folder_cli.py: fix typo	2016-12-06 11:01:51 -08:00
Brian Warner	250c6f6054	test_magic_folder: mark failing test as .todo ... As discussed in this week's meeting, since we don't yet know why some flavors of linux have slightly different inotify behavior than others, and since we believe the actual functionality is not significantly impacted, and since the red buildbot is reducing our confidence that the other tests are passing, and since we have a release coming up: we're marking the one troublesome test as ".todo". We expect that the test will be fixed soon (perhaps to accept either 3 or 4 events), but not necessarily before the 1.12 release. refs ticket:2834	2016-11-02 13:31:22 -07:00
meejah	783449b4f3	When tor or i2p options are present, confirm we have txi2p or txtorcon ... We mock out some import methods and provide tests for providing server-side or client-side tor/i2p options	2016-10-26 13:41:10 -06:00
meejah	0636d44cc1	add poll-interval config for magic-folder	2016-10-25 12:38:57 -07:00
meejah	da4e1589bb	Rename _turn_delay and friends to make things more clear ... This forces the Uploader and Downloader to implement a _scan_delay method and makes the naming more consistent with what's actually happening. Also, fix a few "bugs" in the names of args in the mocks for some tests.	2016-10-25 12:38:57 -07:00
meejah	ddaa29ce39	refactor/white-space MagicFolder creation	2016-10-25 12:38:57 -07:00
Brian Warner	203067810d	magic_folder: announce debug on/off to log, not stdout	2016-10-24 09:37:35 -07:00
Brian Warner	4d003c026e	test_magic_folder: remove unused debug code	2016-10-24 09:23:26 -07:00
Brian Warner	ae5637dc9c	test_magic_folder: turn off debugging when done	2016-10-24 09:23:16 -07:00
David Stainton	d6cbe44786	Enable debug logging for test_move_tree magic-folder unit test	2016-10-24 09:15:09 -07:00
str4d	6071c2b6f8	Implement i2p_provider and --listen=i2p ... Closes ticket:2838	2016-10-23 20:30:01 -05:00
Lukas Pirl	1491e9fab1	added /sbin/ip as possible binary to detect local IP address ... fixes IP detection for RHEL flavors Signed-off-by: Lukas Pirl <git@lukas-pirl.de>	2016-10-12 12:46:12 +02:00
Brian Warner	6d3a20e78b	test_tor_provider: accept windows pathname separators	2016-10-09 01:49:32 -04:00
Brian Warner	ea1c1d27de	fix tests to match	2016-10-09 01:22:17 -04:00
meejah	2ecb8171eb	flip remote/local around for Onion-service 'ports' lines	2016-10-09 01:21:52 -04:00
meejah	eb6c6bee9e	tor's output, for debugging	2016-10-09 01:21:52 -04:00
meejah	646fad69ed	ControlPort needs "unix:" prefix for Tor, and allocate SOCKSPort	2016-10-09 01:21:52 -04:00
Brian Warner	89eb86c5a4	node.py: use tor_provider at startup ... We use it for two things: to create the foolscap connection handler, and to possibly start an .onion listener at startup. This also updates node._common_config_sections to accept the new tor settings written by create-node/create-introducer.	2016-10-09 01:21:51 -04:00
Brian Warner	6b9218ff22	create_node.py: use tor_provider to handle --listen=tor ... This adds tor-related CLI arguments to "create-node" and "create-introducer", to control exactly how we should be using Tor. * --tor-launch * --tor-executable= * --tor-control-port= I went with "--tor-launch" instead of "--launch-tor" for consistency. I don't particularly like the grammatical flow of it, and it doesn't actually put all the tor-related arguments next to each other in the --help output (the flags are put in one block, then the parameters in the next). But it seems slightly more consistent to start all the tor-related argument names with a "--tor*" prefix.	2016-10-09 01:21:51 -04:00
Brian Warner	a1741ce4dc	rewrite tor_provider ... This uses a unix-domain control port, and includes test coverage. create_onion() displays pacifier messages, since the allocate-onion step takes around 35 seconds	2016-10-09 01:21:47 -04:00
Brian Warner	8f9ad009bb	move torutil.py to util/tor_provider.py	2016-10-09 00:57:02 -04:00
David Stainton	897079105a	Code refactor TorProvider from Daira's code review	2016-10-09 00:35:03 -04:00
David Stainton	e02d21aed1	Add tor listening options to create-node	2016-10-09 00:34:57 -04:00
David Stainton	4a1d1a5541	Add helper function CreateOnion	2016-10-09 00:30:33 -04:00
David Stainton	b3b80d376b	Use a TorProvider to get a Tor handler	2016-10-09 00:30:33 -04:00
David Stainton	993d91ec0d	Implement TorProvider class as described by Brian Warner ... https://tahoe-lafs.org/trac/tahoe-lafs/ticket/2490#comment:10	2016-10-09 00:30:33 -04:00
Brian Warner	f549000121	test_create: get full coverage on create_node.py	2016-10-09 00:29:29 -04:00
Brian Warner	1e418ade83	create_node: slight refactoring ... This prepares for various listeners to contribute values to tub.port and tub.location, which will be helpful once Tor support is added.	2016-10-09 00:24:19 -04:00
Brian Warner	b5aaafbe68	create_node: add cosmetic newline in tahoe.cfg	2016-10-09 00:24:19 -04:00
Brian Warner	0eda544a73	create-node: introducer needs private/ too ... All server-like nodes (storage servers and introducers both) will need this for the tor state directory and .onion private key file, and it needs to exist before the config is written, so tor onion-service private keys can be placed there. Also remove a redundant import.	2016-10-09 00:24:19 -04:00
Brian Warner	395f7cd4f0	create_node: prep for slow write_node_config() ... This puts the right inlineCallbacks in place to allow write_node_config() to return a Deferred. The upcoming Tor support will need this (since it must wait for an .onion address to be allocated before it can write tahoe.cfg's tub.port and tub.location lines).	2016-10-09 00:24:11 -04:00
Brian Warner	0e3225ac27	set up logging before init_connections	2016-10-09 00:11:17 -04:00
Daira Hopwood	4d18425ed4	Fix for Magic Folder tests if the I/O encoding is borked on Unicode platforms (OS X, Windows). ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-09-27 18:05:09 +01:00
Daira Hopwood	64841e339d	Remove redundant use of NonASCIIPathMixin. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-09-27 16:16:19 +01:00
Daira Hopwood	c9d73a936a	Fix Unicode-related Magic Folder test failures. fixes #2807 ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-09-27 16:09:15 +01:00
Brian Warner	5ef04ef59e	add hashutil.permute_server_hash ... which uses SHA1 to combine the file's storage index (known as "peer selection index" in this context) and each server's "server permutation seed". This is the only thing in tahoe that uses SHA1. With this change, we stop importing sha1 from anywhere else.	2016-09-26 20:42:42 -07:00
Brian Warner	b00c2d21b7	test tub.port with multiple endpoints, add docs ... I think the preferred way to listen on both IPv4 and IPv6 will be to use "--port=tcp:PORT,tcp6:PORT". This is now reflected in the docs. refs ticket:867	2016-09-20 13:04:06 -07:00
David Stainton	e6d7895865	Support comma separated endpoint list in tub.port ... fixes ticket #867	2016-09-20 10:20:28 -07:00
Brian Warner	b9b731e3e2	update test to match new output	2016-09-20 10:14:35 -07:00
str4d	e82e2c31e2	Update error message to match implementation and docs ... Closes ticket:2824	2016-09-20 10:02:05 -07:00
Brian Warner	a638a97806	implement connections:tcp=disabled ... This enables an I2P-only node, which disables TCP entirely (instead of mapping TCP to Tor, which was the only other option that reveal-IP-address=False would allow). closes ticket:2824	2016-09-14 16:27:12 -07:00
Brian Warner	02ba2a05c3	implement --listen=none, use it for create-client ... Improve docs on server configuration to explain --listen options.	2016-09-14 16:12:32 -07:00
Brian Warner	aa162f5a90	close more filehandles with context managers	2016-09-13 22:17:22 -07:00
Brian Warner	a8f02d6684	enhance tests	2016-09-13 22:05:48 -07:00
David Stainton	9943a268ce	Make the create-node --hide-ip option set [connections]tcp=tor	2016-09-13 10:15:45 +00:00
Brian Warner	a37b93ead9	create_node.py: refactor options	2016-09-13 01:14:48 -07:00
Brian Warner	d802135db1	test introducerless config ... refs ticket:68	2016-09-12 17:45:17 -07:00
Brian Warner	4ae574cca0	test_multi_introducers: add more tests	2016-09-12 17:30:06 -07:00
David Stainton	4f717ba808	Add a multi-introducer implementation ... this is based the previous work of warner, killyourtv and leif fixes ticket #68	2016-09-13 00:15:52 +00:00
Brian Warner	3b17289569	create_node: simplify validation, clean up tests	2016-09-09 18:53:22 -07:00
Brian Warner	229e306e9d	test_runner: fix to use new conventions ... create-node needs --hostname=, and we can't expect the node to write client.port at startup (because tahoe.cfg now has tub.port= set properly)	2016-09-09 18:53:22 -07:00
Brian Warner	38ebdfac20	create-node: reject --listen=tcp without --hostname=	2016-09-09 18:53:22 -07:00
Brian Warner	9a1a186197	simplify tests: use parse_cli, assertRaises, assertFailure ... parse_cli() got added during the async-CLI-dispatch work assertRaises/assertFailure have been in Twisted for a while, but I only learned about them recently. Over time I'm looking forward to changing all tahoe tests to use them (and getting rid of ShouldFailMixin/etc).	2016-09-09 18:52:01 -07:00
Brian Warner	7d3b4149ae	remove leftover debug prints	2016-09-09 18:51:57 -07:00
Brian Warner	09ce7963c6	fixes	2016-09-09 17:29:20 -07:00
David Stainton	e9c1075ac5	Make corrections from Daira's code review	2016-09-09 17:29:16 -07:00
David Stainton	31dfee2dcd	fix cli test create tests	2016-09-09 17:10:22 -07:00
David Stainton	83db7e8b43	Add create node args: listen, port, hostname, location ... fixes ticket: 2773	2016-09-09 17:10:22 -07:00
Brian Warner	802cfc87fe	CLI: allow dispatch functions to return Deferred ... In addition, CLI functions are allowed to use sys.exit() instead of always needing to return the exit code as an integer. runner.py now knows about the blocking httplib calls in scripts/cli and scripts/magic_folder, and uses deferToThread() to invoke them. Those functions cannot return a Deferred: when rewrite them to use twisted.web or treq, we'll remove this deferToThread call. Option parsing was split out to a separate function for testing. We now use twisted.internet.task.react() to start the reactor, which required changing the way runner.py is tested. closes ticket:2826	2016-09-09 15:52:42 -07:00
Brian Warner	97c29a3c0b	test_runner: factor out parse_cli() helper	2016-09-09 15:41:00 -07:00
Brian Warner	7193bff48b	tests: use shared run_cli()/do_cli() ... A couple of test classes which defined their own flavors were changed to use the common one.	2016-09-09 15:41:00 -07:00
Brian Warner	07e4c491f5	move run_cli() up to test/common_util.py	2016-09-09 15:40:01 -07:00
Brian Warner	442468f599	do_cli(): split out run_cli() ... The main part of CLITestMixin.do_cli() was split into a standalone function named run_cli(), leaving do_cli() as a method which includes a nodedir in the arguments (for use by GridTestMixin tests which do a lot of CLI operations against one of their client nodes, for which adding the extra --nodedir argument would be ugly).	2016-09-09 15:40:01 -07:00
Brian Warner	1877bd38b9	consolidate skip_if_cannot_represent_filename() ... Remove duplicate copies of this utility, move it from a mixin/test-class method to being a simple function in common_util.py	2016-09-09 15:40:01 -07:00
Brian Warner	d85d1ea499	CLITestMixin: move into common.py ... Also move parse_options(). It was kind of awkward having other test files import these from test_cli.py.	2016-09-09 15:40:01 -07:00
Brian Warner	45e5d5b891	test_configutil doesn't need CLITestMixin	2016-09-09 15:40:01 -07:00
Brian Warner	720aa1a51f	unify signature of all CLI dispatch functions ... Now they all take a single 'config' argument, instead of some also taking stdout/stderr args.	2016-09-09 15:40:01 -07:00
Brian Warner	57bed47495	runner.py: remove unused arguments	2016-09-09 15:40:01 -07:00
Brian Warner	8d9afdc27e	CLI: remove 'debug trial', 'debug repl' ... These are obsolete. Tests are run with 'tox', or by running 'trial allmydata' from a populated virtualenv. A populated virtualenv is also the right way to get a repl: just run 'python'. refs ticket:2735	2016-09-09 15:37:28 -07:00
meejah	de8e61ddf1	it's okay if we already cancelled it	2016-09-08 12:18:58 -07:00
meejah	622ae646b1	fix shutdown	2016-09-08 12:18:58 -07:00
Brian Warner	74dc7194b6	validate_config: improve tests ... closes ticket:2809	2016-09-06 19:23:51 -07:00
meejah	2732c37941	Check for unknown config options ... The list of valid sections + config-items came from grep'ing the source for `.get_config`	2016-09-06 15:05:46 -06:00
Brian Warner	ed91398a3f	WUI: disable google timing chart on mapupdate page ... The google image chart API has been deprecated since 2012, sending the URL to google leaks server IDs and the client's IP address (especially important when the client is otherwise behind Tor), and the X-axis has no units anyways. refs ticket:1942 , which is both about removing the URL-based chart, and eventually replacing it with a browser-rendered d3.js-based one	2016-09-02 16:30:21 -07:00
David Stainton	f88ae3861a	Make the grid status display the location hints ... Instead of displaying what appears to be the remote peer address we display the list of connection hints. closes ticket:2818	2016-09-02 11:09:16 -07:00
Brian Warner	0116c965fa	private-mode: reject legacy host:port locations	2016-09-02 09:25:26 -07:00
Brian Warner	d0da17adeb	create-node: add --hide-ip, update docs ... So "tahoe create-node --hide-ip" causes "reveal-IP-address = false" to get written into tahoe.cfg . This also changes the default tahoe.cfg to include "reveal-IP-address = true", for clarity. refs ticket:1010	2016-09-01 23:24:38 -07:00
Brian Warner	57e7f7bb7c	import/delegate-to foolscap's allocate_tcp_port ... (instead of using a copy). Foolscap-0.12.3 fixes a problem with allocate_tcp_port() that was causing intermittent test failures. I think it makes more sense to use Foolscap's copy (and fixes) than to keep re-copying it into Tahoe each time it changes. If/when we manage to stop depending upon foolscap for server RPC, we can re-copy this back into tahoe's source tree. refs ticket:2795	2016-09-01 22:39:03 -07:00
Brian Warner	076b3895dc	config: change syntax of no-listen mode ... We now use:: tub.port = disabled tub.location = disabled instead of using an empty value (but the key still being present, since if the key is missing entirely, that means "be automatic"). closes ticket:2816	2016-09-01 21:26:48 -07:00
Brian Warner	a1594df0a6	introclient: only give ascii key_s to storagebroker ... This was triggered when the initial Introducer connection failed, so the node read the introducer_cache.yaml from disk. That always returns unicode strings, and the StorageFarmBroker insisted that it's server-IDs (aka "key_s") were bytestrings. The tests were extended to exercise the code that loads from disk and delivers to the StorageFarmBroker, and more preconditions were put in place to catch this sort of thing earlier next time. closes ticket:2817	2016-09-01 20:29:52 -07:00
Brian Warner	7f9b715b3e	bump foolscap dep to 0.12.3, for the new tor API ... The current version of Foolscap is compatible with the current version of txtorcon, but not with tahoe. This fixes that.	2016-09-01 20:26:08 -07:00
David Stainton	1307c7262d	Use the new Foolscap Tor handler, pass an endpoint for SOCKS connectivity ... closes ticket:2813	2016-09-02 03:17:45 +00:00
Brian Warner	a03f68b787	tahoe.cfg: fix spelling of 'enabled' for tor/i2p ... We should use "enabled = true", instead of "enable = true", since the rest of tahoe.cfg uses "enabled". refs ticket:2788	2016-08-31 15:23:47 -07:00
Brian Warner	d47fc0fd27	config: add reveal-IP-address=False ... This adds a safety flag named `[node] reveal-IP-address`, for which the default value is True. When this is set to False, any configuration that might reveal the node's IP address (to servers, or the external network) will cause a PrivacyError to be raised at startup, terminating the node before it gets a chance to betray the user's privacy. It also adds docs and tests. refs ticket:1010	2016-08-31 02:44:27 -07:00
Brian Warner	325028c967	error if tcp=tor is requested but tor is unimportable ... This only catches txtorcon not being installed (which should be fixed by doing `pip install tahoe-lafs[tor]`). It doesn't notice that the Tor daemon is not running (which we can't detect during startup, only afterwards, when it's harder to notify the user), in which case Tor connections (and all connections when "tcp = tor" is enabled) will just fail silently.	2016-08-31 01:50:13 -07:00
meejah	72f17afa76	Move check_magicfolder_smoke.py to proper integration tests ... This introduces a py.test-based integration suite (currently just containing magic-folder end-to-end tests). Also adds a tox environment ("integration") to run them. The test setup is: - a "flogtool gather" instance - an Introducer - five Storage nodes - Alice and Bob client nodes - Alice and Bob have paired magic-folders	2016-08-30 20:47:47 -06:00
meejah	0466522868	fix _should_download similar to other place	2016-08-30 16:44:52 -06:00
Daira Hopwood	bd62eba934	Work in progress on Magic Folder concurrent write problem. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-08-30 16:44:26 -06:00
Brian Warner	fea8b627dd	if tub.port is empty, don't listen ... Updated config docs. Added errors if we're not listening but were told to enable storage, helper, or if we're the Introducer server. closes ticket:2816	2016-08-29 22:28:55 -07:00
Brian Warner	2d358f2e35	Disable foolscap "gifts" ... "gifts" are third-party references (where Alice sends Bob a reference to Carol, and Carol lives in a different Tub than either Alice or Bob). Tahoe doesn't currently use these, and they make it possible for e.g. a server to provoke a client into making a connection to an IP address of the server's choosing. If the client isn't configured otherwise, this would use raw TCP (revealing the client's IP address) even if the connection to the server used Tor. A connection policy of `tcp: tor` would block this, but it seems like a good idea to disable gifts entirely. closes ticket:2810	2016-08-28 18:15:30 -07:00
Brian Warner	a099b9237d	tor: socks.port is now a (restricted) endpoint string ... Foolscap has limitations that prevent us from accepting anything but a TCP endpoint, but that will change in the future, so make the tahoe.cfg syntax accept an endpoint, but then reject non-TCP ones. See the ticket for details: refs ticket:2813. This depends upon the new `foolscap.connections.tor.socks_port(host, port)` API in foolscap-0.12.2, so it bumps the dependency to that (the previous commit depended upon 0.12.1, but I hadn't gotten around to updating the dep before now).	2016-08-28 16:30:31 -07:00
Brian Warner	bc079a71eb	implement+test I2P setup ... Note that many of the Foolscap handler-creation functions are still stubbed out, so Tahoe won't be able to honor the full range of config syntax until foolscap support is complete.	2016-08-28 03:16:43 -07:00
Brian Warner	15e5ca0e99	exercise TCP/Tor-related tahoe.cfg parsing ... This exercises everything about _make_tcp_handler() and _make_tor_handler() except for when txtorcon cannot be imported.	2016-08-28 03:16:40 -07:00
Brian Warner	c56a0a4ba9	fix unicode handling in server_id from YAML ... YAML, like JSON, is all-unicode. StorageFarmBroker.set_static_servers() is defined to take an all-unicode dictionary (the "storage:" key from private/servers.yaml), so the server_id keys it gets will be unicode. NativeStorageServer is defined to accept server_ids which are bytes (at least it is now). The tests were only passing bytes into set_static_servers(), whereas a real launch passed unicode in, causing a problem when NativeStorageServer tried to base32.a2b() the pubkey and choked on the unicode it received. This fixes set_static_servers() to convert the server_id to bytes, and changes NativeStorageServer to assert that it gets bytes. It also fixes the test to match real usage more closely.	2016-08-28 02:42:51 -07:00
Brian Warner	3cc8daf5bd	implement full range of Tor handlers	2016-08-28 02:42:47 -07:00
Brian Warner	73d5376b6a	Implement basic connection handlers (tor/i2p) ... The node now attempts to create Tor/I2P connection handlers (if the right libraries are available), and will use them for tor/i2p FURL hints by default. For now it only creates default handlers: there is not yet any code to interpret the `[tor]`/`[i2p]` sections of tahoe.cfg which would let you override that process. The node also parses the `[connections]` section, allowing `tcp: tor` to use Tor for all outbound TCP connections. It defaults to `tcp: tcp`, of course. Static storage-server connections will now honor the `connections:` overrides in `servers.yaml`, allowing specific servers to use TCP where they would normally be restricted to Tor. refs ticket:2788 refs ticket:517	2016-08-28 02:41:44 -07:00
Brian Warner	30b421d48b	test_client: improve coverage slightly	2016-08-28 02:39:39 -07:00
Brian Warner	cac99569e9	factor out "Tub maker" ... This adds Node._create_tub(), which knows how to make a Tub with all the right options and connection handlers that were specified in tahoe.cfg (the connection handlers are disabled for now, but they'll get implemented soon). The new Node.create_main_tub() calls it. This main Tub is used: * to connect to the Introducer * to host the Helper (if enabled) * to host the Storage Server (if enabled) Node._create_tub() is also passed into the StorageFarmBroker, which passes it into each NativeStorageServer, to create the (separate) Tub for each server connection. _create_tub knows about the options, and NativeStorageServer can override the connection handlers. This way we don't need to pass tub options or default handlers into Client, StorageFarmBroker, or NativeStorageServer. A number of tests create NativeStorageServer objects: these were updated to match the new arguments. test_storage_client was simplified because we no longer need to mock out the Tub() constructor.	2016-08-27 17:36:55 -07:00
Brian Warner	8cf53d2d12	derive permutation seed from pubkey/server_id	2016-08-27 13:08:18 -07:00
Brian Warner	663e39593b	static servers: tolerate missing nickname/versions ... A minimally-defined static server only specifies server_id, anonymous-storage-FURL, and permutation-seed-base32. But the WUI Welcome page wouldn't render (it raised an exception) without also defining nickname and version. This allows those values to be missing.	2016-08-27 11:34:32 -07:00
Brian Warner	d75b9f822a	Improve loading of static servers ... This follows the latest comments in ticket:2788, moving the static server definitions from "connections.yaml" to "servers.yaml". It removes the "connections" and "introducers" blocks from that file, leaving it responsible for just static servers (I think connections and introducers can be configured from tahoe.cfg). This feeds all the static server specs to the StorageFarmBroker in a single call, rather than delivering them as simulated introducer announcements. It cleans up the way handlers are specified too (the handler dictionary is ignored, but that will change soon).	2016-08-26 18:05:03 -07:00
Brian Warner	f23660e178	NativeStorageServer: create with server_id, not key_s ... They're the same thing, but knowing that is the responsibility of the caller, not NativeStorageServer. Try to normalize on "server_id" as the spelling. Remove support for missing key_s, now that we require V2 introductions.	2016-08-26 18:04:31 -07:00
Brian Warner	54f974d44c	make IServer.get_serverid() use pubkey, not tubid ... This is a change I've wanted to make for many years, because when we get to HTTP-based servers, we won't have tubids for them. What held me back was that there's code all over the place that uses the serverid for various purposes, so I wasn't sure it was safe. I did a big push a few years ago to use IServer instances instead of serverids in most places (in #1363), and to split out the values that actually depend upon tubid into separate accessors (like get_lease_seed and get_foolscap_write_enabler_seed), which I think took care of all the important uses. There are a number of places that use get_serverid() as dictionary key to track shares (Checker results, mutable servermap). I believe these are happy to use pubkeys instead of tubids: the only thing they do with get_serverid() is to compare it to other values obtained from get_serverid(). A few places in the WUI used serverid to compute display values: these were fixed. The main trouble was the Helper: it returns a HelperUploadResults (a Copyable) with a share->server mapping that's keyed by whatever the Helper's get_serverid() returns. If the uploader and the helper are on different sides of this change, the Helper could return values that the uploader won't recognize. This is cosmetic: that mapping is only used to display the upload results on the "Recent and Active Operations" page. I've added code to StorageFarmBroker.get_stub_server() to fall back to tubids when looking up a server, so this should still work correctly when the uploader is new and the Helper is old. If the Helper is new and the uploader is old, the upload results will show unusual server ids. refs ticket:1363	2016-08-26 14:29:50 -07:00
Brian Warner	a970828b94	dicts don't have .haskey() ... Apparently we don't have test coverage for this yet.	2016-08-26 13:56:50 -07:00
David Stainton	140c9a92cd	Make StorageFarmBroker use specified or default handler	2016-08-26 11:17:56 +00:00
David Stainton	de61cd260c	Add got_static_announcement and unit test	2016-08-24 21:11:58 +00:00
David Stainton	61eb839843	Add tub_handlers arg to NativeStorageServer constructor ... Here we also define tub_handlers as a node attribute and pass it all the way down to the NativeStorageServer via the Client and StorageFarmBroker	2016-08-22 09:15:25 -07:00
Brian Warner	32974775ff	fix codechecks: trailing whitespace, suspicious variable captures	2016-08-19 17:42:05 -07:00
Brian Warner	2e7c7d30c9	run_trial: don't do anything on import ... That broke check-interfaces.py, which imports everything.	2016-08-19 17:42:05 -07:00
meejah	5e011e65d3	more iterate()s for windows	2016-08-18 16:39:45 -07:00
meejah	c7dda9d4fc	windows needs more iterate()s	2016-08-18 11:41:10 -07:00
meejah	054efe055c	Fix file-operations/inotify testing ... This re-factors the magic-folder tests to abstract the whole "do a file operation" so we can properly send fake (or wait for real) inotify events to the uploader/downloader. This speeds up the tests quite a bit and makes test_alice_bob reasonable again (at about 1.5s instead of over 30s).	2016-08-18 10:41:40 -06:00
Brian Warner	b637636ef9	remove _appname.py file ... We no longer need the complexity of choosing the application name at runtime. This removes the setup.py code which populates the _appname.py file, and the code in __init__.py which reads it. It does not yet remove the tests which compare the output of e.g. `tahoe --version` against `allmydata.__appname__`, which I think could be removed, but that's more invasive than I want to do right now. closes ticket:2754	2016-08-11 23:24:11 -07:00
meejah	c3035dc409	remove unnecessary error-handling	2016-08-10 19:44:26 -06:00
meejah	75a218d80f	Show last time for successful scans too	2016-08-10 19:44:14 -06:00
meejah	8a33fc612a	Add magic-folder indicator to main welcome page ... This doesn't reveal very much information, but does tell you if magic-folder is currently working and if not it will indicate when the last attempt to do a remote scan was.	2016-08-09 18:07:00 -06:00
meejah	a6920d9799	Improve error-reporting from JSON APIs, and use it better in CLI ... Improve error-handling for directories if you ask for JSON from the /uri endpoint, but an error occurs (you get a proper HTTP status code and a valid JSON object). For 'tahoe magic-folder status' e now retrieve *all* the remote data required in the CLI before doing anything else so that errors can be shown immediately. Use the improved JSON endpoints to print better errors.	2016-08-09 14:36:09 -06:00
Brian Warner	af1e2a1e2f	node.py: advise twistd.log readers about flogtool	2016-08-09 15:27:33 -04:00
meejah	69b86ebe9a	Improve startup error-reporting ... This keeps re-trying the initial magic-folder scan and alerts the user (via logs only :/) until it succeeds at least once. After this happens and the node has started up, it will continue to re-try if enough storage servers go away later such that the remote collection can't be retrieved.	2016-08-09 14:16:11 -04:00
Brian Warner	8ca653c30d	no_network: turn off control_tub and log_tub ... These aren't needed for most tests, and saves about 7% of the total test runtime (on my laptop, it drops from 844s to 786s).	2016-08-05 19:33:49 -04:00
Brian Warner	a35073cb3d	test.common: allow test_repairer to be run independently ... Previously, this file importing "allmydata.immutable" but assuming that "allmydata.immutable.upload" was available, which only worked if some other file had imported upload.py . This didn't affect running the entire test suite (something imported upload.py before anything else needed it), but caused errors when running specific tests like test_repairer.py .	2016-08-05 18:26:18 -04:00
Brian Warner	cd083b9a04	move SystemTestMixin from common.py to test_system.py ... It's only being used in test_system.py anyways, so having it live in a "common" file didn't provide much value.	2016-08-04 01:09:12 -04:00
Brian Warner	4f0e71db4a	split test_web.py into smaller files ... there's more to be done, but this is a start	2016-08-03 20:46:12 -04:00
Brian Warner	421520ade4	test_magic_folder: use oneshare=True too ... I can't currently test this (my OS-X laptop can't run those tests), but based on how much time test_magic_folder takes on the buildbots, I expect oneshare=True to help considerably.	2016-08-03 19:11:01 -04:00
Brian Warner	6dd58ceb2a	tests: use oneshare=True on more test cases ... This saves more time (as measured on my laptop): * test_sftp: 17.7s -> 13s * test_dirnode: 26.5s -> 20s * test_ftp, test_configutil, test_web show negligible speedups As before, some tests care about the number of shares, generally ones which delete or corrupt shares and then expect to see the errors get noticed or fixed. Those tests continue to use k=3/N=10.	2016-08-03 19:11:01 -04:00
Brian Warner	22d032ef2e	test.cli: use oneshare=True, reduce runtime by 50% ... Most of the CLI tests don't care about the actual shares. Configuring the test client to use k=N=1 reduces the runtime from 180s to 90s on my laptop. A few tests *do* care, like test_check (which delete some shares, then assert that 'tahoe check' shows the damage). These still use k=3/N=10.	2016-08-03 19:11:01 -04:00
Brian Warner	50df45e3a0	add GridTestMixin.set_up_grid(oneshare=True) ... This configures client[0] to use k=N=1, which is a lot faster for tests that don't care about the actual shares (e.g. most of the CLI tests).	2016-08-03 19:11:01 -04:00
Brian Warner	e431faf58c	test_cli_*: move files into test/cli/*	2016-08-03 19:10:53 -04:00
Brian Warner	f11769560c	test.mutable.Update: stop exercising max_shares ... Many of the test cases would exercise two copies of each file: one with k=3/N=10, and a second with k=127/N=255 (255 being the maximum supported by zfec). Large number of shares increases the overhead of the testing apparatus, which is pushing those shares to lots of local servers. I don't think the "max_shares" case is necessary, and it takes forever. Because of it, "mutable.Update" was consuming 15% of the total test runtime, and a third of that was just a single function (test_replace_locations_max_shares, now deleted). On a Raspberry Pi 3 (our "slow computer" benchmark), including branch coverage, this one class took 42 minutes to complete, and requires disabling a bunch of timeouts to finish at all. The total number of shares in a file ("N") affects one thing: the width (and thus height) of the share hash tree. This should be exercised in test_hashtree. The number of required shares ("k") affects one thing: the segment size must be a multiple of k. I don't think we need to exercise this, but if so, it could be exercised by a few small values for k, rather than 127. Removing the max_shares cases saves 82% of the mutable.update runtime (on top of the previous three-segment fix), reducing it from 64s to 11.3s on my laptop.	2016-08-03 15:32:01 -04:00
Brian Warner	1bdae637e1	test.mutable.update: reduce data size in half ... We only need three segments to exercise everything. This saves about 15% of the test.mutable runtime (from 76s to 64s on my laptop).	2016-08-03 15:08:10 -04:00
Brian Warner	dbd78f19d2	test.mutable.update: minor refactoring	2016-08-03 15:07:51 -04:00
Brian Warner	4f56c4b053	split test_mutable.py into smaller files	2016-08-03 14:54:32 -04:00
Brian Warner	6f8c96e262	oops, fix NoNetworkStorageBroker to match	2016-07-21 18:13:28 -07:00
Brian Warner	1b64ab5e85	simplify when_connected_enough() ... This seems happier as a method on StorageBroker, rather than a completely separate helper class.	2016-07-21 17:23:22 -07:00
meejah	88cd4cb6c2	skip all tests w/o inotify support	2016-07-21 12:35:59 -07:00
meejah	87acfe968e	pyflakes warnings	2016-07-21 12:35:59 -07:00
meejah	4e0a30ab91	get rid of more win32 special-casing	2016-07-21 12:35:59 -07:00
meejah	1ac6c9bdf6	remove special-case win32	2016-07-21 12:35:59 -07:00
meejah	161c876f2f	make tests reflect reality	2016-07-21 12:35:59 -07:00
meejah	46f1f952e9	tighten delay on windows tests	2016-07-21 12:35:59 -07:00
meejah	e182562dbe	remove tub.location from smoke_test config for storage nodes	2016-07-21 12:35:59 -07:00
meejah	041aabd8c4	Error-handling and --debug option to print stack-trace for all magic-folder commands	2016-07-21 12:35:59 -07:00
meejah	4509c7dafd	Fix magic-folder 'status' command	2016-07-21 12:35:59 -07:00
meejah	db40417886	make check_magicfolder_smoke easier to run, cross-platform	2016-07-21 12:35:59 -07:00
meejah	1b72683741	get rid of more spew	2016-07-21 12:35:59 -07:00
meejah	e9343c8715	fix typo	2016-07-21 12:35:59 -07:00
meejah	efef4ae632	only use long delay on windows	2016-07-21 12:35:59 -07:00
meejah	fc5f820202	whitespace	2016-07-21 12:35:59 -07:00
meejah	516af88252	put longer timeout for test_alice_bob back in	2016-07-21 12:35:59 -07:00
meejah	a533aef960	remove delayedcall debugging	2016-07-21 12:35:59 -07:00
meejah	304da362f7	remove prints	2016-07-21 12:35:59 -07:00
meejah	ec11551404	slightly better way to find tahoe binary	2016-07-21 12:35:59 -07:00
meejah	18f799bdc8	remove incorrect override	2016-07-21 12:35:59 -07:00
meejah	34aed4bbad	fixup remove debug	2016-07-21 12:35:59 -07:00
meejah	e060c584d7	fixup for pending-call calls	2016-07-21 12:35:59 -07:00
meejah	3de2f9c756	get rid of a bunch of debug	2016-07-21 12:35:59 -07:00
meejah	db6717951f	properly cancel pending-call things in windows inotify	2016-07-21 12:35:59 -07:00
meejah	fce5b789dc	fixup rebase conflict mis-resolutions	2016-07-21 12:35:59 -07:00
Daira Hopwood	ba9abe6a2b	Repair fileutil tests. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	12773ea034	Increase timeout for test_alice_bob to 5 minutes. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	504e0d02f3	refactor to use context managers	2016-07-21 12:35:59 -07:00
meejah	ec37d52942	more	2016-07-21 12:35:59 -07:00
meejah	d41477bd5f	fix/change some assertions	2016-07-21 12:35:59 -07:00
meejah	575c00113a	fix typo for .conflict files	2016-07-21 12:35:59 -07:00
Daira Hopwood	c978db4c90	more fixes	2016-07-21 12:35:59 -07:00
meejah	c087a87b4a	fixes to test_alice_bob	2016-07-21 12:35:59 -07:00
Daira Hopwood	5c01d4781f	Temporarily increase delay to make sure we receive notifications. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	874b4cf343	Debug prints in Windows inotify emulation. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	e845f7ff65	Filter out FILE_ACTION_MODIFIED for directories. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	23f4b9dbfd	Repair tests after changing the semantics of objects_succeeded. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	8a83cff14d	Make _process return a boolean to indicate whether processing succeeded, ... and only increment objects_succeeded if that is the case. Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	0abe5baaf2	WIP: temporarily work around the AccessDenied errors after a test failure. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	afc7ab43a1	Fix a subtle typo. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	b949082f8e	Change type of mtime and ctime stored in magicfolderdb to integer nanoseconds. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
meejah	7c85de813f	(some) fixes for smoke_magicfolder	2016-07-21 12:35:59 -07:00
Daira Hopwood	237f4719ee	Remove the flush_volume call in RealTest as well as in RealTestAliceBob. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	b05b701532	Try to avoid spurious failures when shutting down the INotify loop on Windows. ... This only partially solves the problem. Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	23a1f311af	Fix pyflakes errors, add a comment about a suspicious test. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	a78e9c750a	WIP: add close method for MagicFolderDB and make MagicFolderDbTests use it. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
meejah	a50d0e5c8b	WIP and debugging things ... Conflicts: src/allmydata/test/test_magic_folder.py	2016-07-21 12:35:59 -07:00
meejah	58f9641179	Correct magic-folder use of StorageFarmBroker ... When landing the "connected enough" changes on master, we changed the API; this fixes up usage of it in magic-folder.	2016-07-21 12:35:59 -07:00
Daira Hopwood	e67d23b8fa	Fix negative counts for object_queued. fixes ticket:2731 ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
meejah	625c5a42cd	daira points out bug	2016-07-21 12:35:59 -07:00
meejah	5f073c6987	more startup waiting	2016-07-21 12:35:59 -07:00
meejah	faed2574d6	possible less race-y startup	2016-07-21 12:35:59 -07:00
meejah	378ce27cbd	Make retry_download test work, and some bonus comments	2016-07-21 12:35:59 -07:00
Daira Hopwood	9e5b01b3ae	Swallow the error from Downloader._when_queue_is_empty. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	811d597cd8	Add WIP test_download_retry. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
meejah	e6104cd1a2	Replace _lazy_tail with iterative loop, cleanup tests (#2412) ... * uses @inlineCallbacks to turn the _lazy_tail recursion into a "real" looking loop; * remove the need for "immediate" vs delayed iteration of said loop; * make it easier for the unit-tests to control the behavior of the uploader/downloader; * consolidates (some) setup/teardown code into the setUp and tearDown hooks provided by unittest so unit-tests aren't doing that themselves * re-factors some of the unit-tests to use an @inlineCallbacks style so they're easier to follow and debug This doesn't tackle the "how to know when our inotify events have arrived" problem the unit-tests still have, nor does it eliminate the myriad bits of state that get added to tests via all the MixIns.	2016-07-21 12:35:59 -07:00
Daira Hopwood	e03e243c67	Expand test for Magic Folder statistics.	2016-07-21 12:35:59 -07:00
Daira Hopwood	37d22efaea	Improve test for Magic Folder statistics and move it from test_system.py to test_magic_folder.py.	2016-07-21 12:35:59 -07:00
Daira Hopwood	dd0597f869	Work in progress to fix incorrect statistics output for Magic Folder. refs ticket:2709	2016-07-21 12:35:59 -07:00
Daira Hopwood	e3dfc8f3f4	Fix test_magic_folder.py to use absolute unicode paths when calling write_downloaded_file.	2016-07-21 12:35:59 -07:00
David Stainton	d2ae4255c8	Add daira's implementation of make_dirs_with_absolute_mode	2016-07-21 12:35:59 -07:00
David Stainton	2963e9e50c	Use os.path.dirname instead of split and rename var to initial_path_u	2016-07-21 12:35:59 -07:00
David Stainton	cd390b90e4	remove superfluous trailing comma from make_dirs_with_absolute_mode def	2016-07-21 12:35:59 -07:00
David Stainton	ba35d7262b	Add unit test and make corrections to make_dirs_with_absolute_mode	2016-07-21 12:35:59 -07:00
David Stainton	ee44732d03	Break out our chmod while loop into fielutils.py	2016-07-21 12:35:59 -07:00
David Stainton	51ccec7a33	Use chmod instead of changing umask ... Conflicts: src/allmydata/frontends/magic_folder.py	2016-07-21 12:35:59 -07:00
David Stainton	3d909682a4	Add comment about FUDGE_SECONDS and refer to our design doc	2016-07-21 12:35:59 -07:00
Daira Hopwood	807cfbf0dc	Fix pyflakes warnings. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
meejah	9cfcb1b3aa	remove humanize, use internal method, teach internal method to understand timedelta	2016-07-21 12:35:59 -07:00
meejah	573a5481d1	add Bob leaving to smoke-tests	2016-07-21 12:35:59 -07:00
meejah	c217d1f8f3	fix return-valid of leave	2016-07-21 12:35:59 -07:00
meejah	bf3bf70814	add docstrings	2016-07-21 12:35:59 -07:00
meejah	8ba2912ff4	make history longer	2016-07-21 12:35:59 -07:00
meejah	7e84b0cf57	flesh out IQueuedItem	2016-07-21 12:35:59 -07:00
meejah	05ec62c4b1	humanize doesn't report a proper version	2016-07-21 12:35:59 -07:00
meejah	5ed24913d2	add humanize dependency	2016-07-21 12:35:59 -07:00
meejah	658470ba6e	Add some magicfolder database tests	2016-07-21 12:35:59 -07:00
meejah	2573cf18ef	Add simple auth-token to get JSON data	2016-07-21 12:35:59 -07:00
meejah	86abe56d91	Flesh out "tahoe magic-folder status" command ... Adds: - a JSON endpoint - CLI to display information - QueuedItem + IQueuedItem for uploader/downloader - IProgress interface + PercentProgress implementation - progress= args to many upload/download APIs	2016-07-21 12:35:59 -07:00
David Stainton	3df0a82a38	Add cli stub for magic-folder status command	2016-07-21 12:35:59 -07:00
David Stainton	af10ab43ee	Daira's fix during session with David and Meejah.	2016-07-21 12:35:59 -07:00
David Stainton	71e3164e92	Further work in progress refinements to unit tests ... wip from pairing with Daira and Meejah	2016-07-21 12:35:59 -07:00
David Stainton	9483d24f67	Fix cleanup after test_periodic_full_scan.	2016-07-21 12:35:59 -07:00
David Stainton	38dd49ee5a	Add logging for Downloader.stop.	2016-07-21 12:35:59 -07:00
David Stainton	2dabd8a10b	Work in progress on fixing test_periodic_full_scan.	2016-07-21 12:35:59 -07:00
David Stainton	b4153659d3	Minor cleanup and logging fix.	2016-07-21 12:35:59 -07:00
David Stainton	5380ff8f5a	Clean up queue loging and add a clock advance in unit test ... not yet working	2016-07-21 12:35:59 -07:00
David Stainton	e1822c0518	WIP	2016-07-21 12:35:59 -07:00
David Stainton	96b846f762	Add ignore pending argument to _periodic_full_scan	2016-07-21 12:35:59 -07:00
David Stainton	cf462d8b1f	Attempt to test periodic uploader full scan	2016-07-21 12:35:59 -07:00
David Stainton	a5163c32cc	Only perform full scan if pending set is empty	2016-07-21 12:35:59 -07:00
David Stainton	e9e74c43e1	Naive periodic full scan	2016-07-21 12:35:59 -07:00
David Stainton	9b4b43cf19	Fix uploader's _process to extend queue after scan and count stats properly	2016-07-21 12:35:59 -07:00
David Stainton	d2971fbabe	Daira's excellent fixes to the uploader from our pairing session	2016-07-21 12:35:59 -07:00
David Stainton	70f66677be	Teach uploader's _scan to use the deque	2016-07-21 12:35:59 -07:00
David Stainton	6b32841b10	Remove UNIQUE constraint from magicfolder db schema	2016-07-21 12:35:59 -07:00
Daira Hopwood	9c9a4e7aa2	Add get_pathinfo. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	278762200d	Teach magic-folder join to use configutil. ... Author: David Stainton <david@leastauthority.com> Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
David Stainton	cc721505e7	Add magic-folder test_scan_once_on_startup	2016-07-21 12:35:59 -07:00
Daira Hopwood	cb6291e93c	Various test cleanups. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
David Stainton	3074b687be	Teach join twice test to test for proper stdout and stderr values	2016-07-21 12:35:59 -07:00
David Stainton	74ad656e39	More thorough checks to the join leave join test	2016-07-21 12:35:59 -07:00
David Stainton	966c556562	Fix test_join_leave_join	2016-07-21 12:35:59 -07:00
David Stainton	1c3c9df492	Add more magic-folder join leave tests	2016-07-21 12:35:59 -07:00
David Stainton	24aa74bf53	Add unit test for join twice failure	2016-07-21 12:35:59 -07:00
Daira Hopwood	2043bed3f1	Delay only the download scan not the turning of our event queue. ... Author: David Stainton <david@leastauthority.com> Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
David Stainton	9d7785f57b	Fix magic folder constructor signature in test	2016-07-21 12:35:59 -07:00
David Stainton	68539e3fe0	Fix tests usage of umask	2016-07-21 12:35:59 -07:00
David Stainton	0273034b0c	Fix umask again	2016-07-21 12:35:59 -07:00
David Stainton	ae59b17e86	Fix umask usage bit fiddling and use try finally blocks	2016-07-21 12:35:59 -07:00
David Stainton	f2a7978b29	Add download.umask config option with default of 077	2016-07-21 12:35:59 -07:00
David Stainton	623ed3516e	Add umask to Downloader	2016-07-21 12:35:59 -07:00
David Stainton	7b9c89c6f4	remove magic_folder from config in test	2016-07-21 12:35:59 -07:00
David Stainton	189b1b2dc2	Remove magic_folder subsection from default tahoe.cfg	2016-07-21 12:35:59 -07:00
David Stainton	79f3c5d81a	Remove magic-folder exclude stat	2016-07-21 12:35:59 -07:00
Daira Hopwood	cd1c166790	Improve the error reporting for 'tahoe magic-folder join/leave'. refs #2568 ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
David Stainton	b4b5590b00	teach leave to: remove magic_folder section of config	2016-07-21 12:35:59 -07:00
David Stainton	d6e68129ab	Add simple magic-folder leave command	2016-07-21 12:35:59 -07:00
David Stainton	8ff7642fba	Teach join to also check for existing magicfolder db file	2016-07-21 12:35:59 -07:00
David Stainton	aea2cf2890	Prevent magic-folder join if already joined.	2016-07-21 12:35:59 -07:00
Daira Hopwood	6b4d3b8c6d	Refactor is_new_file. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	02dacbc067	Minor simplification. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
David Stainton	e990483750	Simply conflict detection by removing nested if statements	2016-07-21 12:35:59 -07:00
David Stainton	6ed4480eb9	Scan our own dmd upon start only	2016-07-21 12:35:59 -07:00
meejah	0aaabe0ebe	remove print	2016-07-21 12:35:59 -07:00
meejah	ed4e4d053e	Add a unit-test and correct the code for "already deleted" ... If a Downloader decides that it needs to delete a file, but that file is already gone locally, the exeption is caugt and a log message produced.	2016-07-21 12:35:59 -07:00
David Stainton	49b5661347	Fix test: previously we accounted for the propagation of the conflict ... because alice scaned her own dmd... whereas now she does not.	2016-07-21 12:35:59 -07:00
David Stainton	eab24d022b	Fix test helper _check_version_in_local_db	2016-07-21 12:35:59 -07:00
Daira Hopwood	9d59ef493f	WIP: Refactoring to get db fields in one query. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
David Stainton	bca4c87e47	WIP	2016-07-21 12:35:59 -07:00
Daira Hopwood	42f535edf9	Fix pending upload conflict detection. ... Author: David Stainton <david@leastauthority.com> Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
David Stainton	6eeab03b70	Create test for last upload uri conflict	2016-07-21 12:35:59 -07:00
Daira Hopwood	e8b6b31373	Detect remote conflict by checking for pending upload. ... Author: David Stainton <david@leastauthority.com> Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
David Stainton	c7f38a496d	Add debugging print statement for timestamp comparison	2016-07-21 12:35:59 -07:00
David Stainton	edc671f577	Add last uploaded timestamp comparison for remote conflict detection	2016-07-21 12:35:59 -07:00
Daira Hopwood	6402159106	Fix an unused import. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	c7f298bce5	Fix some miscapture bugs. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	868c38d228	Workaround. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	b21a3dc070	Simplify _notify and improve logging. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	17384ac512	Simplify _scan_remote_* and remove Downloader._download_scan_batch attribute. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	d686575393	Cosmetics. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	4f41be0e70	Downloader doesn't need the pending set. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	017d63b47b	Delete redundant is_ready attribute from MagicFolder. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	3fab5caa02	test_encodingutil: fixes for Unix. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	5f98d73216	Add precondition to Uploader._process. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	583066b364	Fix test_errors. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	9646a3ce3e	Fix test_move_tree. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	0c12162bc8	Debugging WIP. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	b53efdbf8b	Fix a corner case for to_filepath on Windows to make it consistent with Unix. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	b9eb8932b6	test_encodingutil: add tests for FilePath-related functions. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	b6b638efd1	Depend on FilePath.asTextMode(). ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	84af199d51	WIP: exclude own dirnode from scan. This is not quite right; we shouldn't exclude it on startup. refs #2553 ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	ea14b1bebe	Don't add subdirectory watches if the platform's notifier doesn't require them. refs #2559 ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	6bb0963197	Improve reporting of assertion failures. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	a5b9cf36b8	Fix unused import. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	f77c593177	Add test for 'tahoe create-node/client/introducer' output. closes ticket:2556 ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	d1eb64c01b	bin\tahoe can't be run directly on Windows. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	3647ffc3a7	Correct type for Windows BOOL. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	9dafa2e759	Disable precondition that autoAdd == recursive. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	a6842946ca	Fix a type error. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	0e19636146	Flush handling WIP. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	5e9792f8d7	Use fileutil.write for magic folder tests. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	f3803c058d	Improve all of the Windows-specific error reporting. ... Also make the Windows function declarations more readable and consistent. Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	c20cf843eb	Fix fileutil tests. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	dacf6ef633	More path fixes. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	f12ac3d94e	Fix a test broken by the last commit. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	a622109a87	Don't include [magic_folder]enabled and local.directory fields by default. ... Add a comment reminding to do the field modification properly. Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	6fb7f86855	Don't use a long path for the [magic_folder]local.directory field. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	2a79252a72	Fix some path Unixisms. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	45adc30ed6	Fix test_alice_bob. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	e591c3aa81	Refactor _check_up/downloader_count. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	1abc2c39c8	Don't download the deletion marker file unnecessarily. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	c6191257ea	Distinguish deletion of directories. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	d6bb49b1a6	Rename deleted files to .backup rather than unlinking them. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
David Stainton	a043a31860	Add file conflict unit test	2016-07-21 12:35:59 -07:00
David Stainton	99d1339d38	Add basic bob upload test and fix conflict detect	2016-07-21 12:35:59 -07:00
David Stainton	ee3d9b19be	Fix bob's uploading test...	2016-07-21 12:35:59 -07:00
David Stainton	890e4622b9	Attempt to teach bob to upload a file	2016-07-21 12:35:59 -07:00
David Stainton	9ed61f1eb4	Count conflicted objects	2016-07-21 12:35:59 -07:00
Daira Hopwood	7a3b35e29c	Basic remote conflict detection based on ancestor uri ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	f75dda7635	Avoid .format, since it is inconsistent between Python 2.6 and 2.7 (and the rest of Tahoe-LAFS doesn't use it). ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	5a2b2e61c7	Fix test_alice_bob. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	e4082a45d3	Add counter for uploader.objects_not_uploaded. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	95e2961a2e	Advance Bob's clock after notifying. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	6e5e8ddd80	test_alice_bob: use magic= argument to notify, rather than self.magicfolder. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
meejah	0d466ec1f9	add excluded check	2016-07-21 12:35:59 -07:00
meejah	c883977eb6	add the 'spurious' notifies	2016-07-21 12:35:59 -07:00
Daira Hopwood	c156581b73	Fix a pyflakes warning and check existence of file in Bob's local dir. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
meejah	d61b8ed39e	implement 'delete' functionality, with tests	2016-07-21 12:35:59 -07:00
meejah	946656b249	smoketest for magic-folder functionality	2016-07-21 12:35:59 -07:00
Daira Hopwood	cec315da49	Add test that we don't write files outside the magic folder directory. refs ticket:2506 ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	567668eb8c	Fix infinite loop in should_ignore_path for absolute paths. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	5d5d804719	More debug logging. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	57753a62bc	Unicode fix for do_join. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	7d2f30643e	Minor cleanups to tests. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	f754938cc9	Ensure that errors from Alice-and-Bob tests are reported correctly if setup fails. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	b56c3e06e4	Cosmetics. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	2a7bb6f444	Eliminate duplicate parsing of invite code. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	c28594268e	Remaining test fixes. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	a5874070e3	Make sure that do_cli is only called with strs, and avoid unnecessary use of attributes in tests. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	ed2d27e712	Cosmetics. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	643a105991	URIs are strs. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	59eede8e88	Aliases and nicknames are Unicode. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	bb596b017d	Fix call to argv_to_abspath. Also rename localdir to local_dir. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
David Stainton	b32b61e727	Fix magic-folder cli tests ... convert path to abs path when matching strings in the generated config file.	2016-07-21 12:35:59 -07:00
David Stainton	a1beef34a7	Attempt to fix cli tests	2016-07-21 12:35:59 -07:00
Daira Hopwood	b2d494837c	Better but still broken tests. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00
Daira Hopwood	903c6f1d6c	Fix check for initial '-' in argv_to_abspath. ... Signed-off-by: Daira Hopwood <daira@jacaranda.org>	2016-07-21 12:35:59 -07:00